System and method for predicting treatment device weaning

Abstract

A system and method are disclosed for predicting dropout related to a patient's use of a therapeutic device. A communication interface collects event data related to the operation of the therapeutic device to deliver medication to the patient. The patient's event data and clinical data are stored. A dropout analysis module inputs the event data and clinical data and applies a machine learning model to determine the likelihood of patient dropout over a predetermined period of time from the input event data and clinical data. The machine learning model is trained from a machine learning pipeline having inputs of event data and clinical data from a population of patients using the therapeutic device to determine at least one trigger for patient dropout. It is determined whether the likelihood of dropout exceeds a threshold. If the likelihood of dropout exceeds the threshold, an action is triggered for the patient.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit and priority of U.S. Provisional Patent Application No. 63 / 086,924, filed on October 2, 2020, which is hereby incorporated by reference in its entirety.

[0002] The present disclosure generally relates to methods for improving adherence to applications that execute medical devices, and more particularly, to determining and predicting factors that affect the churn of users of drug devices.

Background Art

[0003] Respiratory diseases such as asthma and chronic obstructive pulmonary disease (COPD) remain significant and costly public health problems. For example, in the United States, more than 22 million people suffer from asthma. Worldwide, the World Health Organization estimates that the population with asthma could be 300 million and predicts it will increase to 400 million by 2025. Similarly, in a recent study by the U.S. Centers for Disease Control and Prevention, COPD is ranked as the third leading cause of death in the United States, and it is estimated that nearly 13 million people may have lung function impairment due to COPD.

[0004] Despite the development of new drugs, the rates of hospital admissions and emergency department visits have not decreased. In the United States, each year, approximately 2 million people come to the emergency department due to asthma, 500,000 are hospitalized, and 5,000 die. In addition, asthma is responsible for an estimated 15 million school days missed and 12 million work days missed. The total annual cost to U.S. health insurance companies and employers exceeds $18 billion. Due to COPD, each year, approximately 715,000 people are hospitalized and 134,000 die. In addition, the cost to the country due to COPD is recently expected to be approximately $49.9 billion, which includes $29.5 billion in direct medical costs, $8 billion in indirect morbidity costs, and $12.4 billion in indirect mortality costs.

[0005] While the majority of asthma exacerbations can be prevented with currently available treatments, only one in five asthma sufferers have their disease under control. Such treatments often depend on identifying the conditions that trigger asthma episodes and administering appropriate medications and other therapies. One mechanism for patients to self-administer medication is the inhaler. When respiratory symptoms occur, patients can take medication via puffs from the inhaler.

[0006] Similarly, COPD patients often carry inhaled medications to immediately relieve symptoms whenever and wherever they occur. The frequency and location of a patient's use of these medications are important indicators of how well the disease is managed and treated. Currently, inhalers may include either built-in or attached sensors to collect information about inhaler use. This information may then be transmitted to a central server to determine adherence. [Overview of the project] [Problems that the invention aims to solve]

[0007] However, many patients do not adhere to the correct use of inhaler and sensor combinations, resulting in high churn rates. Lack of adherence means that patients are not receiving adequate treatment for their respiratory illnesses. Many digital health applications and devices suffer from higher churn rates compared to other consumer applications and devices. Payers, providers, and healthcare professionals currently have limited ability to know, predict, and intervene when users are likely to discontinue treatment with therapeutic devices such as inhalers. Lack of adherence and high churn rates reduce the cost-effectiveness of such treatments and the ability to track and know about inhaler use.

[0008] There is a need for a system that enables analysis to reduce the high discontinuation rate of therapeutic devices. There is a need for a system that collects various data to determine the factors that determine the likelihood of discontinuation in patients using therapeutic devices. There is a further need for a system that identifies patients at high risk of discontinuation and provides preventive measures and treatment to those patients. [Means for solving the problem]

[0009] The disclosed analytical system enables the curation of patient behavioral and static data to train and deploy predictive models to increase adherence. This analysis will allow for the analysis of patient dropouts from therapeutic devices.

[0010] One example disclosed is a system and method for predicting patient churn regarding the use of a therapeutic device. The system includes a communication interface for collecting event data related to monitoring the operation of the therapeutic device. A storage device stores the patient's event and clinical data. A churn analysis module takes the event and clinical data as input and applies a machine learning model to determine the likelihood of patient churn over a given period of time from the input event and clinical data. The machine learning model is trained from a machine learning pipeline with event and clinical data input from a population of patients using therapeutic devices to determine at least one trigger for patient churn. The analysis module determines whether the likelihood of churn exceeds a threshold. If the likelihood of churn exceeds the threshold, the module triggers an action on the patient.

[0011] A further implementation of the exemplary system is an embodiment in which the therapeutic device is a rescue inhaler or control inhaler. In another implementation, the communication interface communicates with the patient's mobile device. Sensors communicating with the therapeutic device are synchronized with an application run on the mobile device to provide event data. In another implementation, the machine learning pipeline is trained from customer service data about the patient population. In another implementation, the action is one of the following: a patient alert, a healthcare provider alert, or a therapeutic device manufacturer alert. In another implementation, the machine learning pipeline responds with a predictive endpoint that includes the determined likelihood of churn, the values ​​of the processed clinical and event data, and the quantified impact of how these values ​​affect the determined likelihood of churn.

[0012] Another disclosed example is a method for determining patient churn of a therapeutic device operated by a patient. Clinical data of the patient is collected. Event data related to monitoring the operation of the therapeutic device is collected via a communication interface. A machine learning model is applied to determine the likelihood of patient churn over a given period of time from the input event data and clinical data. The machine learning model is trained from a machine learning pipeline with input event data and clinical data from a population of patients using therapeutic devices to determine at least one trigger for patient churn. It is determined whether the likelihood of churn exceeds a threshold. If the likelihood of churn exceeds the threshold, an action on the patient is triggered.

[0013] A further implementation of the exemplary method is an embodiment in which the therapeutic device is a rescue inhaler or a control inhaler. In another implementation, the exemplary method includes communicating with a patient's mobile device via a communication interface and synchronizing sensors communicating with the therapeutic device with an application run on the mobile device to provide event data. In another implementation, the machine learning pipeline is trained from customer service data about a patient population. In another implementation, the action is one of a patient alert, a healthcare provider alert, or a therapeutic device manufacturer alert. In another implementation, the machine learning pipeline responds with a predictive endpoint that includes a determined likelihood of churn, values ​​of processed clinical and event data, and a quantified impact of how these values ​​affect the determined likelihood of churn.

[0014] Another disclosed example is a pipeline method for training a machine learning model to predict patient dropout of therapeutic devices. Data input is collected from a population of patients. Data input includes clinical and event data related to monitoring the operation of therapeutic devices. Data input is curated to generate a data table. A machine learning model is trained from the data table to weight multiple input factors influencing dropout, determine the impact of the input factors in determining the likelihood of dropout, and generate a predictive endpoint. The trained machine learning model is stored.

[0015] A further implementation of the exemplary method includes, via a preprocessing module, refining the data table to exclude data relating to patients who have had insufficient interaction with the therapeutic device, and transforming the data for input to a training module, in embodiments where the therapeutic device is a rescue inhaler or a control inhaler. In another implementation, the data input includes data from a mobile device operated by a population of patients, data from an application running on the mobile device which interfaces with the therapeutic device, and customer service data. In yet another implementation, the stored model is run by a predictive endpoint module to provide a prediction of the likelihood of dropout for a single patient, based on clinical and event data of the therapeutic device from that single patient.

[0016] Another disclosed example is a machine learning pipeline system for training a machine learning model to predict patient dropouts from therapeutic devices. A data input interface collects data inputs from a population of patients. The data inputs include clinical and event data related to monitoring the operation of therapeutic devices. A curation module curates the data inputs and generates a data table. A training module trains a machine learning model from the data table, weights multiple input factors that influence dropout, determines the influence of the input factors in determining the likelihood of dropout, and generates a predictive endpoint. A storage device stores the trained machine learning model.

[0017] A further implementation of the exemplary machine learning pipeline system is an embodiment that includes a preprocessing module that refines the data table to exclude data about patients who have had insufficient interaction with the therapeutic device and transforms the data for input to the training module, wherein the therapeutic device is a rescue inhaler or a control inhaler. Another implementation includes data input from a mobile device operated by a population of patients, data from an application running on the mobile device which interfaces with the therapeutic device, and customer service data. Another implementation is in which a stored model is run by a predictive endpoint module to provide a prediction of the likelihood of dropout for a single patient based on clinical and event data of the therapeutic device from that single patient.

[0018] The above summary is not intended to illustrate any or all embodiments of the present disclosure. Rather, the above summary merely provides some examples of novel embodiments and features described herein. The above features and advantages of the present disclosure, as well as other features and advantages, will be readily apparent from the following detailed description of typical embodiments and modes for carrying out the invention, in conjunction with the accompanying drawings and claims.

[0019] This disclosure will be better understood from the following exemplary embodiments and with reference to the accompanying drawings. [Brief explanation of the drawing]

[0020] [Figure 1] This document presents a dropout analysis system for predicting patient dropout rates from therapeutic devices. [Figure 2] This is a high-level block diagram showing an example of a computing device used as either a client device, an application server, and / or a database server. [Figure 3] This is an illustrative block diagram of a machine learning pipeline. [Figure 4] An example of a curated data table generated by the exemplary machine learning pipeline of FIG. 3. [Figure 5] A block diagram of the model training and development module of the machine learning pipeline of FIG. 3. [Figure 6] A service endpoint generated by the machine learning pipeline of FIG. 3.

Best Mode for Carrying Out the Invention

[0021] This disclosure has room for various modifications and alternative forms. Some representative embodiments are shown by way of example in the drawings and will be described in detail herein. However, it should be understood that the present invention is not intended to be limited to the particular forms disclosed. Rather, this disclosure encompasses all modifications, equivalents, and alternative forms that fall within the spirit and scope of the present invention as defined by the appended claims.

[0022] The present invention can be embodied in many different forms. Representative embodiments are shown in the drawings and will be described in detail herein. This disclosure is an example or illustration of the principles of this disclosure and is not intended to limit the broad aspects of this disclosure to the embodiments shown. To that extent, for example, elements and limitations disclosed in, for example, the "Summary", "Abstract of the Invention", and "Detailed Description of the Invention" but not explicitly stated in the "Claims" should not be incorporated into the "Claims" singly or collectively by implication, inference, or otherwise. For the purposes of the "Best Mode for Carrying Out the Invention", unless otherwise specified, the singular form includes the plural form, the plural form includes the singular form, and the term "comprising" means "including but not limited to". Further, terms indicating approximation such as "about", "substantially", "essentially", "approximately", etc. can be used herein, for example, to mean "in", "near", "substantially therein", or "within 3 - 5% thereof", or "within acceptable manufacturing tolerances", or any logical combination thereof.

[0023] This disclosure relates to a system for collecting relevant data on the abandonment of treatment devices such as drug inhalers. Factors related to the abandonment rate are determined from data collected via a machine learning pipeline.

[0024] One advantage of abandonment analysis is that the treatment device is improved. This is because the analysis enables the product team and the engineering team to design functions to address situations and behaviors related to future abandonment. Another advantage is that the identified factors related to abandonment can be applied to more personalized communication, content, and functions related to individual patients to reduce the abandonment rate. The ability to predict the probability that a user will abandon within the next predetermined period, for example, 30 days, and the factors affecting that probability enables proactive communication with patients who are more likely to be affected. This makes it possible to solve problems in advance or provide specific incentives for behavior change, leading to a higher retention rate and thus better long-term clinical outcomes. Better adherence to personal treatment devices such as inhalers can significantly reduce emergency department visits and hospitalizations over 12 months. Therefore, by improving the patient retention rate with this service, there is a very high likelihood that medical utilization will be reduced. This analysis enables a significant improvement in adherence. Therefore, retaining patients leads to an increase in the refill volume of medications for pharmaceutical partners. These medications have been proven to improve clinical outcomes.

[0025] FIG. 1 shows an abandonment analysis system 100 for collecting data based on monitoring accurate and real-time drug device events according to one embodiment, performing an analysis on the data, and providing an output based on identifying signs of high abandonment of a particular treatment device for a general patient population.

[0026] The analysis system 100 includes a client computing device 110, a drug device sensor 120, a drug device 160, an application server 130, a database server 140, and a network 150. Figure 1 shows only single instances of most of the components of the analysis system 100, but in practice, each component may exist in two or more instances, or additional or fewer components may be used.

[0027] The client devices 110 interact with the analysis system 100 via the network 150 at the request of their users. For explanation and clarification, it is useful to identify at least two different types of users. Patient 111 is a user suffering from a respiratory disease such as asthma or COPD, in this example, the patient uses the respiratory disease analysis system 100 to obtain, at least in part, personalized rescue event risk notifications provided by the server 130 and administrative notifications created by the patient's healthcare provider 112. Such notifications may be provided in exchange for the user's permission, which allows the respiratory disease analysis system 100 to monitor the patient's use of the medication device 160. As described below, medication events are detected by sensors 120 associated with the medication device 160 and the user's client device 110, and these events are then reported to the application server 130. In this embodiment, patient 111 represents a patient population, for which individual data is obtained for each patient in the group. Another type of user may use the drug device 160 for control medication to control symptoms of respiratory illness.

[0028] The client device 110 is a computer system. An exemplary physical implementation is described more fully below with reference to Figure 2. The client device 110 is configured to communicate wirelessly with the respiratory disease analysis system 100 via network 150. By accessing network 150, the client device 110 transmits to the system 100 information describing the user's geographical location and the time of rescue or control medication events, as well as events received from associated drug device sensors 120 (hereinafter referred to as "sensor 120").

[0029] Regarding user location and event time, the client device 110 can determine the geographical location and time of a rescue or control event using information about the cellular or wireless network 150 to which it is connected. For example, the current geographical location of the client device 110 can be determined by directly querying the software stack providing the network 150 connection. Alternatively, geographical location information may be obtained by pinging an external web service (not shown in Figure 1) that becomes accessible via network 150. The time of the event may be provided by the sensor 120 as part of the event data, or it may be added to the event data by querying an appropriate software routine available as part of the client device's native operating system.

[0030] Application 115 provides a user interface (hereinafter referred to as the “dashboard”) that is displayed on the screen of the client device 110 and allows the user to enter commands to control the operation of Application 115. The dashboard is a mechanism for healthcare providers 112 and patients 111 to access resources managed by System 100. For example, the dashboard allows patients 111 and providers 112 to interact with each other, receive asthma rescue event risk notifications, exchange messages about treatment, provide and receive additional event and non-event data, etc. Application 115 may be coded as a web page, a set of web pages, or content otherwise coded to be rendered within an internet browser. Application 115 may also be coded as a proprietary application configured to run on the native operating system of the client device 110.

[0031] In addition to providing a dashboard, application 115 may also use the resources of client device 110 to perform some data processing on rescue or control event data locally, and then transmit the processed data over network 150. Thus, in this embodiment, application 115 enables the communication of event data from sensor 120 in relation to the operation of drug device 160. The event data transmitted over network 150 is received by application server 130, where it is analyzed and processed in cooperation with database server 140 for storage and retrieval. Application server 130 may direct retrieval and storage requests to database server 140 as needed by client application 115. As described, this analysis may be extended to include event data from multiple patients 111.

[0032] The client device 110 communicates with the sensor 120 using a network adapter and either a wired or wireless communication protocol, one example being the Bluetooth® Low Energy (BTLE) protocol. BTLE is a short-range, low-power protocol standard for wirelessly transmitting data over a wireless link in a short-range wireless network. After the sensor 120 and the client device 110 are paired with each other using a BTLE passkey, the sensor 120 automatically synchronizes and communicates information regarding the use of the drug device to the client device 110. If the sensor 120 has not been paired with the client device 110 prior to a rescue medication event, the information is stored locally until such pairing occurs. During pairing, the sensor 120 communicates any stored event records to the client device 110. Other implementations may use other types of wireless connections (e.g., infrared or IEEE 802.11).

[0033] Although the client device 110 and the drug device 160 are described above as separate physical devices (e.g., a smartphone and an inhaler, respectively), in the future, the drug device 160 may include not only a sensor 120 integrated with the device 160 in a single housing, but also embodiments of the client device 110. For example, the drug device 160 may include an audiovisual interface, including a display or other lighting element, as well as a speaker, for presenting audiovisual information. In such an implementation, the drug device 160 itself may directly present the content of notifications provided by the server 130, instead of presenting it via the client device 110, or in addition to doing so.

[0034] An exemplary drug device 160 is a medical device used to deliver a drug to the lungs of a user experiencing a rescue event of respiratory illness. Various drugs are used for asthma and COPD. Various drugs can be used for the rescue and control of asthma and COPD. Drug devices (e.g., inhalers) are typically portable and small enough to be carried by hand for easy access when treating respiratory symptoms. In one embodiment, the drug is delivered in aerosol form via a drug device 160, such as a metered-dose inhaler. The metered-dose inhaler includes a pressurized spray canister for aerosol drug, a throttling valve for delivering a predetermined dose of drug, and a plastic holder that holds the pressurized canister and also forms a mouthpiece for delivering the drug. In another embodiment, the drug is delivered in dry powder form via a drug device 160, such as a dry powder inhaler. The dry powder inhaler may have a Cartesian oval-shaped body housing a wheel and gear mechanism, allowing the user to split and feed strips of dry powder drug. The body of the dry powder inhaler also includes a manifold and mouthpiece for delivering the dry powder to the user. While a drug device is used in this embodiment, it should be understood that withdrawal analysis may be applied to any personal therapeutic device that requires the user to use the device regularly. Furthermore, although the above example relates to respiratory disorders, the principles described herein may be applied to treatment plans for other types of illnesses.

[0035] Each patient may be associated with multiple drug devices 160. For example, a patient may have a rescue drug device 160 for distributing rescue medication and a controller drug device 160 for distributing controller medication. Similarly, each patient may be associated with two or more sensors 120, each sensor may be selected to work with one of the patient's drug devices 160.

[0036] Generally, the sensor 120 is a physical device that monitors the use of the drug dispenser 160. The sensor 120 can either be detachably attached to the drug dispenser without interfering with the operation of the drug dispenser, or the sensor 120 is an integrated component that is a unique part of the drug dispenser 160 and is made available by the manufacturer of the drug dispenser 160.

[0037] The sensor 120 includes a network adapter (not shown) of the sensor itself that communicates with the client device 110 via either a wired connection or, more typically, a wireless high-frequency connection. In one embodiment, the network adapter is a Bluetooth® Low Energy (BTLE) wireless transmitter, but in other embodiments, other types of wireless communication may be used (e.g., infrared, IEEE 802.11).

[0038] The sensor 120 may also be configured to communicate more directly with the application server 130. For example, if the network adapter of the sensor 120 is configured to communicate by a wireless standard such as IEEE 802.11 or LTE, the adapter may exchange data with a wireless access point such as a wireless router, which may then communicate with the application server 130 without necessarily involving the client device 110 in all data exchanges. These two methods of communication are not mutually exclusive, and the sensor 120 may be configured to communicate with both the client device 110 and the application server 130, for example using redundant transmission, to ensure that event data reaches the application server 130, or to directly provide information to the client device 110 while the application server 130 decides what notification to provide in response to the event.

[0039] As described above, the sensor 120 captures data about the use of the drug device 160. Specifically, each sensor 120 captures rescue medication events, i.e., the time and geographical location of the patient 111's use of the rescue drug device 160. Each sensor 120 transmits event data automatically in real time or as soon as network connectivity is established, without input from the patient 111 or healthcare provider 112. The medication event information is sent to the application server 130 for use in analysis, generation of asthma rescue event notifications, and aggregate analysis of event data across multiple patients for the purpose of determining withdrawal.

[0040] To achieve this goal, there are several different ways to construct the sensor 120, and this structure will depend in part on the structure of the drug device 160 itself. Generally, all sensors 120 will include the onboard processor, persistent memory, and network adapter described above, which work together to record, store, and report medication event information to the client device 110 and / or server 130. The sensor 120 may also include a clock to record the time and date of the event.

[0041] Regarding a particular sensor 120 structure, conventional inhalers such as mechanical dose counters are not designed with the sensor 120 in mind, and therefore the sensor 120 may be constructed accordingly. In this way, several implementations include mechanical, electrical, or optical sensors for detecting the movement of the device 160, the priming of the device, the activation of the device, and inhalation by the user. In contrast, recent inhalers such as deflectable membrane dose counters include an electrical circuit that may report event information as electrical data signals designed for the sensor 120 to receive and interpret, for example, the drug device 160 itself may report movement, priming, and activation to the sensor 120.

[0042] Further information relating to the hardware and software components for the sensor 120 and the drug device 160, as well as the interaction between them for recording rescue medication events, can be found in U.S. Patent Application No. 12 / 348,424, filed January 1, 2009, and International Patent Application No. PCT / US2014 / 039014, filed May 21, 2014, both of which are incorporated herein by reference in their entirety.

[0043] The application server 130 is a computer or a network of computers. While a simplified example is shown in Figure 2, typically the application server is a server-class system using a more powerful processor, larger memory, and faster network components compared to a typical computing system used, for example, as a client device 110. The server typically has large-scale secondary storage, such as using a RAID (Independent Disk Redundancy Array) array and / or establishing a relationship with an independent content delivery network (CDN) contracted to store, exchange, and transmit data such as the asthma notifications considered above. In addition, the computing system includes an operating system, such as a UNIX® operating system, a LINUX® operating system, or a WINDOWS® operating system. The operating system manages the hardware and software resources of the application server 130 and also provides various services, such as process management, data input / output, and peripheral device management. The operating system provides various functions for managing files stored on the device, such as creating new files, moving or copying files, and transferring files to remote systems.

[0044] The application server 130 includes a software architecture to support access to and use of the analysis system 100 by many different client devices 110 via the network 150, and can therefore be characterized at a high level as a cloud-based system. The application server 130 generally provides a platform for patients 111 and healthcare providers 112 to report data recorded by sensors associated with their drug devices 160, including both rescue medication events and controller medication events, to collaborate in respiratory disease treatment planning, to view and retrieve information about their condition and geographical location, and to utilize various other functions.

[0045] Generally, the application server 130 is designed to handle a wide variety of data. The application server 130 includes logical routines that perform various functions, including checking the validity of incoming data, parsing and formatting the data as needed, passing the processed data to the database server 140 for storage, and verifying that the database server 140 has been updated.

[0046] The application server 130 stores and manages data on a patient-by-patient basis, at least partially. For this purpose, the application server 130 creates a patient profile for each user. The patient profile is a set of data that characterizes the patient 111 of the respiratory disease analysis system 100. The patient profile may include identifying information about the patient, such as age, sex, current rescue medication, current controller medication, notification preferences, controller medication adherence plan, relevant medical history, and a list of non-patient users authorized to access the patient profile. The profile may further specify a device identifier, such as a unique Media Access Control (MAC) address, that identifies one or more client devices 110 or sensors 120 authorized to submit data for the patient (e.g., controller and rescue medication events).

[0047] The profile may specify which of the different types of notifications will be provided to patient 111 and the patient's personal healthcare provider 112, and how often those notifications will be provided. For example, patient 111 may authorize healthcare provider 112 to receive notifications indicating rescue events. Patient 111 may also authorize their healthcare provider 112 to access the patient profile and rescue event history. If healthcare provider 112 is given access to patient 111's patient profile, the healthcare provider may specify a controller adherence or rescue medication plan for the corresponding COPD or asthma condition. The medication plan may include a prescribed number of daily doses of the controller medication.

[0048] The application server 130 also creates a profile of the healthcare provider 112. The healthcare provider profile may include identifying information about the healthcare provider 112, such as the location of the business, qualification certificates, and certifications. The healthcare provider profile also includes information about the healthcare provider's patient population. The provider profile may include access to all of the provider's patient profiles, as well as data derived from those profiles, such as aggregate demographic information, rescue and controller medication event patterns. This data may be further subdivided according to any type of data stored in the patient profiles, for example, by geographical area (e.g., neighborhood, city) or by time interval (e.g., weekly, monthly, yearly).

[0049] The application server 130 receives rescue medication event information from the client device 110 or sensor 120 and triggers various routines on the application server 130. In the exemplary implementation described below, the data analysis module 131 executes routines to access event data, synchronization data, and other data including patient profiles, analyzes the data, and outputs the results of the analysis to both the patient 111 and the healthcare provider 112. This process is commonly referred to as churn risk analysis. In this embodiment, the churn risk analysis is performed in relation to the use of a therapeutic device such as an inhaler 160. Alternatively, the churn risk analysis may be performed for a specific patient or a subgroup of patients.

[0050] Other types of analysis may include daily / weekly adherence trends, changes in adherence over time, adherence comparisons with other relevant populations (e.g., all patients, patients receiving specific rescue medications or controller medications or combinations thereof), identification of (spatial, temporal, and environmental) triggers, trends in rescue medication use over time, and comparisons of rescue medication use with other relevant populations.

[0051] In response to any analysis performed, the application server 130 prepares and delivers push notifications to the patient 111, authorized healthcare providers 112, and / or other users granted access to the patient's profile. The notifications may provide details about the timing, location, and affected patient 111 related to the medication rescue event. The notifications may also further include a distress or emergency signal requesting emergency assistance, which will be delivered to the emergency assistance provider 112. The notifications may also include the results of the withdrawal analysis performed by the data analysis module 131. Further information regarding the types of notifications that may be sent and the content that notifications may include is described below.

[0052] In addition to providing push notifications in response to asthma or COPD exacerbation risk analyses, notifications may also be provided as pull notifications at specific time intervals. Furthermore, some notifications (whether push or pull) may be triggered not in response to asthma or COPD exacerbation risk analyses conducted in response to rescue medication events, but rather in response to risk analyses conducted in response to a change in one of the underlying factors in the asthma or COPD exacerbation risk analysis, ensuring that notifications are updated. For example, if weather conditions indicate that an increase in air pollution is occurring or likely to occur, this may trigger the execution of an asthma exacerbation risk analysis for all patients located in a specific geographical area where the pollution is occurring.

[0053] The notification is provided to the client application 115 via the network 150 in a data format specifically designed for use by the client application, and may also be provided in other data formats communicated using short message service (SMS) messages, email, telephone calls, or other communication media.

[0054] The database server 140 stores relevant patient and provider data, such as profiles, medication events, and patient medical history (e.g., electronic medical records). Patient and provider data is encrypted for security purposes, at least password-protected, and otherwise protected to meet all the requirements of the Health Insurance Portability and Accountability Act (HIPAA). Data from multiple patients (e.g., aggregated rescue medication event data) is incorporated, and any analysis provided to users (e.g., asthma risk analysis) is anonymized to remove personally identifiable information in order to protect patient privacy. The database server 140 may also include customer service information about patients from healthcare providers 112, such as inquiries and requests for assistance.

[0055] The database server 140 may also store non-patient data. This data may include local data relating to several geographical areas, such as public spaces in residential or commercial districts where patients are physically located and may be exposed to contaminants. This data may, among other things, include, or be processed to include, the patient's proximity to green spaces (areas containing a concentrated number of trees and plants). An example of local data may include georeferenced meteorological data, such as temperature, wind patterns, humidity, and air quality indices. Another example is georeferenced pollution data, such as particulate counts of various contaminants measured at a given point in time or empirically. The local data may include information about current meteorological conditions relating to the time and location of a rescue event, such as temperature, humidity, and air quality indices. All of the above data items may change over time, and therefore the data itself may be indexed by time, for example, with separate data points available by time (including minutes or hours) or over longer periods, such as days, weeks, months, or seasons. In Figure 1, the database server 140 is shown as a separate entity from the application server 130. However, the database server 140 may instead be a hardware component that is part of another server, such as server 130. The database server 140 may be implemented as one or more persistent storage devices, and the software application layer for interfacing with the data stored in the database may be part of such another server 130.

[0056] The database server 140 stores data according to a defined database schema. Typically, data storage schemas across different data sources vary significantly, even when storing the same type of data, including cloud application event logs and log metrics, due to differences in the implementation of the underlying database structure. The database server 140 may also store different types of data, such as structured data, unstructured data, or semi-structured data. Data within the database server 140 may be associated with patients, groups of patients, and / or entities. The database server 140 manages the database objects represented by the database server 140 and provides support for database queries in query languages ​​(e.g., SQL for relational databases, JSON for NoSQL databases, etc.) that specify instructions for reading information from or writing to the database server 140.

[0057] Network 150 represents various wired and wireless communication paths between client devices 110, sensors 120, application servers 130, and database servers 140. Network 150 uses standard Internet communication technologies and / or protocols. Therefore, Network 150 may include links using technologies such as Ethernet®, IEEE 802.11, Integrated Services Digital Network (ISDN), and Asynchronous Transfer Mode (ATM). Similarly, networking protocols used in Network 150 may include Transmission Control Protocol / Internet Protocol (TCP / IP), Hypertext Transfer Protocol (HTTP), Simple Mail Transfer Protocol (SMTP), and File Transfer Protocol (FTP). Data exchanged through Network 150 can be represented using technologies and / or formats including Hypertext Markup Language (HTML) and Extended Markup Language (XML). In addition, all or part of the links may be encrypted using conventional encryption technologies such as Secure Sockets Layer (SSL), Secure HTTP (HTTPS), and / or Virtual Private Network (VPN). In another embodiment, the entity may use custom and / or proprietary data communication technology instead of or in addition to the technologies described above.

[0058] Figure 2 is a high-level block diagram showing the physical components of an exemplary computer 200, which may be used as part of a client device 110, an application server 130, and / or a database server 140 from Figure 1, according to one embodiment. A chipset 210 coupled to at least one processor 205 is shown. The chipset 210 is coupled to volatile memory 215, a network adapter 220, an input / output (I / O) device 225, a storage device 230 representing non-volatile memory, and a display 235. In one embodiment, the functionality of the chipset 210 is provided by a memory controller 211 and an I / O controller 212. In another embodiment, the memory 215 is directly coupled to the processor 205 instead of the chipset 210. In some embodiments, the memory 215 includes high-speed random access memory (RAM), such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices.

[0059] The storage device 230 is any non-temporary computer-readable storage medium such as a hard drive, compact disc read-only memory (CD-ROM), DVD, or solid-state memory device. The memory 215 holds instructions and data used by the processor 205. The I / O device 225 may be a touch input surface (capacitive or otherwise), a mouse, trackball, or other type of pointing device, a keyboard, or another form of input device. The display 235 displays images and other information from the computer 200. The network adapter 220 connects the computer 200 to the network 150.

[0060] As is known in the art, the computer 200 may have components different from and / or other components shown in Figure 2. In addition, the computer 200 may lack certain components shown. In one embodiment, the computer 200, which functions as a server 140, may lack dedicated I / O devices 225 and / or displays 218. Furthermore, the storage device 230 may be local and / or remote from the computer 200 (such as being implemented within a storage area network (SAN)), and in one embodiment, the storage device 230 is not a CD-ROM or DVD device.

[0061] Generally, the specific physical components used in the client device 110 differ in size, power requirements, and performance from those used in the application server 130 and the database server 140. For example, the client device 110, often a home computer, tablet computer, laptop computer, or smartphone, has relatively small storage capacity and processing power, but includes input devices and displays. These components are suitable for user data input, as well as receiving, displaying, and interacting with notifications provided by the application server 130. In contrast, the application server 130 may include a number of physically isolated and locally networked computers, each with a substantial amount of processing power to perform the churn risk analysis described above. In one embodiment, the processing power of the application server 130 is provided by a service such as Amazon Web Services®. Also in contrast, the database server 140 may include a number of physically isolated computers, each with a substantial amount of persistent storage capacity to store data associated with the application server.

[0062] As is known in the art, the computer 200 is adapted to run a computer program module for providing the functions described herein. The module can be implemented in hardware, firmware, and / or software. In one embodiment, the program module is stored in a storage device 230, loaded into memory 215, and executed by a processor 205.

[0063] As an example of the process for capturing such an event for a specific device 160 / sensor 120 combination, at the onset of symptoms, the sensor 120 may detect whether the cover of the drug dispenser 160 is open. When the cover of the drug dispenser is open, the sensor 120 can detect acceleration associated with priming of the dispenser 160. In some types of drug dispensers, "priming" involves activating a mechanism that releases a dose of the drug from the package. In other types of drug dispensers, "priming" involves rapidly shaking the drug canister.

[0064] After a priming action is detected, the sensor 120 is configured to store data associated with the rescue or control event in its active memory. The rescue or control event data may include information describing the time and date associated with the rescue event, the status or condition of the drug device 160 (e.g., battery level), the number of remaining doses (before or after the event), self-test results, and physiological data of the patient being treated with the drug device 160, as measured by the sensor 120. As soon as the sensor establishes a network connection with either the client device 110 or the network 150, the sensor sends the locally stored rescue or control event data to the client device 110 or the application server 130. If the event data is first sent to the client device 110, the client device 110 then sends the rescue or control event data to the application server 130 as soon as the client device 110 establishes a network connection with the network 150. Depending on the implementation, either the client device 110 or the sensor 120 adds the geographical location where the event occurred to the event data sent to the application server 130.

[0065] When the application server 130 receives and stores rescue use event data, it may request further information from the patient describing the rescue or control use event. To obtain this information, the application server 130 generates a push notification containing a question to be asked, which is sent to the patient's client device 110. The patient may respond to the request by providing input in response to the application. Alternatively, the patient 111 may choose not to respond to the request, which is permissible. Any gaps in the information may be obtained via subsequent push notifications or when the provider 112 provides input after a meeting with the patient 111. In one implementation, the remaining analysis does not stop even if additionally requested data is not received in response to a request.

[0066] In addition to requesting additional event data, the application server 130 accesses context data stored from the database server 140. Generally, context data refers to data other than event data, and includes, but is not limited to, atmospheric conditions, weather conditions, air quality conditions, pollen data, patient data recorded from past rescue use events, and any other considerations not directly detected by the drug device at the time of the rescue use event. In contrast, event data refers to any parameters related to a rescue event and reported by the drug device, such as dosage, time of event, location of event, and associated adherence data. Both forms of data may include temporally current information as well as stored historical data. Thus, as part of obtaining context data, the application server 130 also accesses the patient 111's rescue use event data history. This historical data may include all data from past controller or rescue medication event data from the patient's history across various past timeframes, and each historical event may include all the same information items reported for that event, along with any data collected in subsequent follow-up.

[0067] During normal course, database 140 receives data on rescue use events as they occur and updates the primary patient data store and the general patient population data store to reflect contextual data associated with recent current rescue use events for all patients in the primary patient and patient population. The frequency with which data stores are updated with new rescue use events may vary depending on many factors, including but not limited to the patient's condition, the patient's adherence regimen to controller medication, and environmental conditions. Patient adherence to prescribed controller medication therapy is a comparison of the frequency per day the patient uses the controller inhaler unit to the frequency per day the patient was instructed to use the controller inhaler unit, which may be measured based on the use of the controller inhaler unit.

[0068] This system includes an application 115 that runs on a portable computing device 110, which includes different interfaces. A data collection survey may be displayed to collect subjective patient data regarding the operation of the inhaler 160. The system stores the survey data entered by the patient.

[0069] In one embodiment, the process of predicting patient or treatment device churn is performed by the data analysis module 131 of the analysis system 100 in Figure 1, based on an analysis of primary patient context data, reported patient outcomes, and other relevant data. The analysis system 100 includes the data analysis module 131 on the application server 130, which analyzes various data collected by the system introduced above and further discussed below, to predict patient or device churn rates.

[0070] In this embodiment, the analysis system based on the data analysis module 131 uses the Spark data processing engine via AWS Glue Extract, Transform, and Load Services to curate hundreds of millions of distinct data points from service data about sensors, phones, applications, healthcare provider or device manufacturer contacts, and clinical / static patient data to create a comprehensive view of patients on a daily basis.

[0071] For the purposes of exemplary predictive modeling, a withdrawal event is defined as the first day of a consecutive 30-day (or longer) period in which the sensor is not synchronized with the application 115 on the mobile device 110, and therefore the sensor 120 is not communicating with the application 115. This can be considered event data indicating that the patient is not using the drug dispenser 160 because sensor data has not been received from the application. Synchronization would enable the sensor 120 to collect additional data regarding the use of the drug inhaler 160. Different lengths of consecutive days may be selected. The goal of this exemplary analysis system is to predict the probability that a withdrawal event will occur within the next 30 days. Other periods other than 30 days may be used to predict the probability.

[0072] As described above, the churn probability threshold can be set to any level to trigger actions such as text messages, emails, or push notifications. Such a threshold will trigger an action whenever the probability score exceeds that threshold. Trigger accuracy can be characterized in terms of churn event recall, defined as the percentage of events for which a trigger was created at least once in the 30 days prior to the churn event, and trigger / alert accuracy, defined as the percentage of triggers for which an event occurred in the following 30 days. For example, the prediction engine in this embodiment has a churn event recall of over 75% and a trigger / alert accuracy of over 30%.

[0073] In this embodiment, machine learning generation preferably includes best practices. These practices include complete version control of data, models, and code to enable reproducibility / analysis. Another best practice is rich job, model, variable, and sample metadata to enable reproducibility / analysis. Performance monitoring is readily available in this embodiment from the Glue tables generated by the Extract, Transform, Load (ETL) Glue service. Retraining, readjustment, and variable changes are part of the normal release cycle.

[0074] The machine learning model in this embodiment uses Shapley additive explanations, a cutting-edge technique for model-independent (in this case, gradient boosting tree) interpretability, to reveal not only the characteristics or behaviors present in patients over the past 30 days, but also how individual behaviors affect the probability of withdrawal. These explanations can help to better understand the impact of behaviors both collectively across the entire population and at the patient's daily level. The technology stack in the exemplary service (in addition to the data) may be used in future machine learning pipelines.

[0075] The data includes the breadth of data sources, the depth of analysis, and modularity to various scenarios. In this embodiment, in terms of source breadth, billions of data points from analysis systems 100 are curated into more than 50 variables per patient per day, and then further preprocessed into more than 500 preprocessed behavioral variables summarizing the past 30 days. For example, the variable to be curated may be "number of app launches today." The variable to be preprocessed may be "change in average app launches per week."

[0076] In terms of depth, key responses from the predictive endpoint include the probability of dropout in the next 30 days, pre-processed input variables, and the impact of these pre-processed input variables. Regarding modularity, the data sources, pipelines, and services themselves each support multiple use cases. Data sources include causal analysis (data science) and behavioral dashboards (data analytics). Machine learning pipelines can predict any behavior, such as application usage or adherence, with very limited code changes. Services include customer / patient service workflows, conditional marketing content, personalized product features, and notifications.

[0077] Figure 3 shows an example of a machine learning pipeline 300. The machine learning pipeline includes a set of data inputs 310, a data curation module 312, a preprocessing module 314, a model training / development module 316, a predictive endpoint module 318, and a predictive cron module 320. The outputs include an input / output table 330 and a service endpoint 332. In this embodiment, the data input 310 includes a sensor data input 340, an application collection data input 342, a clinical data input 344, a mobile device data input 346, and a customer service data input 348. In this embodiment, the data curation module 312, the predictive cron module 320, and outputs 330 and 332 are part of Glue ETL. In this embodiment, the preprocessing module 314, the model training / development module 316, the predictive endpoint module 318, and the predictive cron 320 are part of a SageMaker service.

[0078] The data curation module 312 enables the curation of data from different sources. Such sources may include, but are not limited to, product backends, third-party product analytics services, third-party customer experience services, existing tables in a data lake, and existing tables in a data warehouse. These sources may be curated using multiple extract-transform-load (ETL) jobs with parallel processing, e.g., Apache Spark and AWS Glue. The final ETL job transforms all the data from the different sources into a single table, where each row represents one day for one patient, and each column represents either metadata about the patient-day (e.g., a user ID or date field) or data points summarizing specific characteristics or behaviors of the patient on that day.

[0079] In this embodiment, the input data to the data curation module 312 includes the user ID and date, as well as data input 310. In this embodiment, the sensor data input 340 is derived from event data collected by the application from either the drug inhaler 160 or the sensor 120. Examples of sensor data points include the number of days since the rescue sensor first synchronized with application 115, the number of days since the controller sensor first synchronized with application 114, the number of synchronizations, the number of synchronization retries, the number of controller doses administered, controller adherence (percentage of prescribed doses taken), the number of rescue uses, and whether the day included a withdrawal event (based on subsequent use and synchronization patterns).

[0080] Application data input 342 is a further analysis of event data from application 115 and subjective data collected from patients. In this embodiment, such inputs may include the number of days since the application was first launched, the number of times the application has been launched, the duration of the application launch, the number of interactions with various functions such as adding triggers, browsing the trends tab, and using the inhaler location search function, the duration of the interactions with various functions, the number of responses to in-application feedback, the number of push notifications received, and the number of push notifications opened.

[0081] Mobile device data input 346 is data derived from the mobile device, such as the patient's time, location, and movement. In this embodiment, data input 346 may include the operating system (i.e., Android®, iOS, and which version), phone model, percentage of application heartbeats with Bluetooth® enabled, percentage of application heartbeats with location enabled, and location variability.

[0082] Clinical data entry 344 and customer service data entry 348 may be derived from patient-related health and service records in an exemplary database 140. In this embodiment, clinical data entry may include disease, disease status, prescribed medications, dosages, and clinical metrics such as CAT or ACT scores. In this embodiment, customer service data entry may include user interaction data with provider 112, the number of emails sent to the user by provider, the number of emails opened by user, and the number of times the user interacted with the support team, for example, via phone, email, text, or chat.

[0083] Figure 4 shows an example of a curated data table generated by an exemplary data curation module 312. The data table includes a user ID and date in each row. Each column contains various events recorded for a particular user ID and date. For example, the first row includes the user ID and date, the number of times the application was launched (4), the time the application was launched (300 seconds), the number of times the rescue inhaler's drug sensor was synchronized (8), the number of days since the controller inhaler's first synchronization (10), and whether today represents a withdrawal event (1 if true, 0 if false, in this case 0).

[0084] The preprocessing module 314 in Figure 3 consumes the final table from the data curation module 312 and further preprocesses the data so that it can be used directly by the machine learning model for training, cross-validation, and general development. This preprocessing includes, but is not limited to, 1) excluding users who have not sufficiently interacted with the product or service (e.g., users who have not synchronized sensors or launched the app); 2) inferring time variables from date fields (e.g., year, month, day of the week); 3) one-hot encoding categorical variables into binary arrays that can be interpreted by the machine learning model; 4) copying specific variables into a model input table where the only relevant data point is the current value (e.g., number of days since initial sensor synchronization, day of the week, mobile platform); 5) creating input variables that summarize dynamic behavior over a defined trailing window (e.g., 30 days); 6) creating binary target variables based on a predefined forward window (e.g., 30 days) and whether a dropout event (defined as the first of at least 30 consecutive days in which the sensor is not synchronized) occurs within that window; and 7) user-day exclusion according to rules.

[0085] The creation of input values ​​summarizing dynamic behavior may include, but is not limited to, a) the difference in moving averages for 1-day, 2-day, 3-day, 7-day, 14-day, and 30-day combinations; b) the difference in moving standard deviations for 1-day, 2-day, 3-day, 7-day, 14-day, and 30-day combinations; c) the difference in sequential averages for 1-day, 2-day, 3-day, 7-day, and 14-day combinations (e.g., the difference between the past 7 days and the 7 days before that); d) the difference in sequential standard deviations for 1-day, 2-day, 3-day, 7-day, and 14-day combinations; e) the average patient-day values ​​over 2-day, 3-day, 7-day, 14-day, and 30-day combinations; and f) the standard deviation values ​​per patient-day over 2-day, 3-day, 7-day, 14-day, and 30-day combinations.

[0086] Rules for excluding user days may include, but are not limited to, a) the user day being more than 30 days since the last departure event for that user, b) there being an incorrect data point in the trailing or forward window, such as a null value or a significantly outlier value, or c) there being a change in the user ID in the forward or trailing window. The preprocessing module 314 then stores the final input and output arrays, along with other relevant metadata, in a data store such as an AWS S3 bucket.

[0087] The model training and development module 316 utilizes data from the preprocessing module 314 and develops a predictive model by performing standard machine learning practices. Development may include, but is not limited to, 1) assigning each user-day sample to a different cross-validation fold separated based on user ID, date, or both (rather than random sample assignment); 2) selecting variables / features based on heuristic or more complex methods, including but not limited to correlation-based, information-gain-based, and cluster-based methods; 3) training the machine learning model; 4) selecting model hyperparameters and / or model classes based on cross-validation performance; and 5) generating other relevant job data and metadata, such as training time and predictive performance for each fold. The model training and development module 316 then stores the model artifact in a data store, such as an AWS S3 bucket. This model artifact includes the model file itself, along with all relevant job, sample, and variable metadata.

[0088] Figure 5 is a block diagram of the model training and development module 316. Module 316 includes a cross-validation fold assignment submodule 510, in which each user-day sample is assigned to a different grouping for the purpose of cross-validation. The output of the submodule is sent to: a model selection submodule 512, in which a model class such as gradient boosting tree or random forest is selected; a hyperparameter selection submodule 514, in this embodiment, in which hyperparameters are selected, including but not limited to maximum depth, learning rate, splitting method, intensity of L1 or L2 regularization, maximum number of leaves, and subsample ratio; and a feature selection submodule 516, in which candidate input variables are selected based on a feature selection method such as a correlation-based method, an information-gain-based method, or a cluster-based method. The outputs of the selection submodules 512, 514, and 516 are sent to the training and cross-validation submodule 520. The output of submodule 520 is sent to calibration submodule 522, which corrects any overconfidence or underconfidence in the predictive scores to ensure that such scores represent true probability scores. The output of calibration submodule 522 is sent to model description submodule 524, which, in this embodiment, uses Shapley sums to collectively analyze the impact of each input variable for different user groups, individual users, or individual user-days. The output is a set of model artifacts 530 that store all relevant information associated with the creation of the model.

[0089] The prediction endpoint module 318 accepts input that is aligned with the schema of the final curated table and contains data across the same trailing window defined by the preprocessing module 314. The endpoint may also be in the form of preprocessing code and model artifacts generated by the model training and development module 316, which can be consumed and used directly by the prediction cron job. The endpoint may also be in the form of an API to which input data across the trailing window can be posted, and the endpoint will preprocess the data in the same way as the preprocessing module and respond with relevant data related to creating and understanding predicted probabilities of departure.

[0090] The relevant data points to which the prediction endpoint module 318 responds include, but are not limited to, 1) user ID; 2) date; 3) predicted probability or likelihood of an exit event within a defined forward window (e.g., 30 days); 4) processed input variable values ​​(e.g., daily values ​​and trend values, i.e., moving average and sequential mean, and standard deviation); and 5) a quantified impact of these values ​​on the predicted probability, inferred using a model explanation method such as Shapley summation. An exemplary prediction endpoint response can be seen in Figure 6. It should be understood that the example pipeline module is not limited to the variables in Figure 6, and many other variables not shown in Figure 6 may exist.

[0091] The Predictive Cron Module 320 runs at predefined times and frequencies (for example, at 2 a.m. every day), iteratively processing each user data from a curated table to extract the nearest trailing window for the behavior defined in the Preprocessing Module. The Predictive Cron Module 320 then uses this data to interact with the Predictive Endpoint to generate Predictive Probabilities and other relevant information, and stores this information in one or more data stores, such as an AWS S3 bucket or an AWS Glue table.

[0092] The exemplary input / output table 330 could be an AWS S3 bucket or an AWS Glue table. Different stores or tables may serve different purposes. In this embodiment, multiple tables may contain different information and serve different purposes, including, but are not limited to, the following: 1) a table where each row represents one user-day, with one column containing the user ID, one column containing the entire pre-processed input response as a string, and one column containing the probability of exit; 2) a table where each row represents a specific pre-processed input value for a particular user-day, with one column containing the user ID, one column containing the predicted date, one column containing the pre-processed input variable name, and one column containing the value; and 3) a table where each row represents a specific Shapley sum for a particular user-day, with one column containing the user ID, one column containing the predicted date, one column containing the pre-processed input variable name, and one column containing the Shapley sum for that particular variable-user-day combination.

[0093] Service endpoint 332 makes up-to-date information on each user's churn probability available to other parts of the product or organization, including but not limited to customer experience products and services, and marketing automation tools that drive product personalization tools for customizing the user experience. The service endpoint may take the form of direct queries against AWS Glue tables via another product, such as AWS Athena. The service endpoint may be accompanied by an API layer that enables end-user queries for information including, but not limited to, the probability of a user ID churn, other relevant data on a user ID's behavior and churn probability, all user IDs within a specific probability range, and all user IDs within a specific percentile range of churn probability. These queries may include other conditions about a user ID, including but not limited to specific programs, regions, diseases, age ranges, and clinical status.

[0094] As used in this application, the terms “component,” “module,” and “system” generally refer to computer-related entities, including hardware (e.g., circuits), combinations of hardware and software, software, or entities relating to an operational machine having one or more specific functions. For example, a component may be, but is not limited to, a process, processor, object, executable file, execution thread, program, and / or computer running on a processor (e.g., a digital signal processor). Exemplarily, an application running on a controller, and both the application and the controller, may be components. One or more components may reside in a process and / or execution thread, and components may be localized to one computer and / or distributed across two or more computers. Furthermore, “device” may take the form of specially designed hardware; general-purpose hardware that is specialized by the execution of software on the hardware, enabling the hardware to perform specific functions; software stored on a computer-readable medium; or a combination thereof.

[0095] The technical terms used herein are for the sole purpose of describing specific embodiments and are not intended to limit the invention. Where used herein, the singular forms “a,” “an,” and “the” may also include the plural form unless otherwise explicitly indicated by the context. Furthermore, where the terms “including,” “includes,” “having,” “has,” “with,” or variations thereof are used in either “Modes for Carrying Out the Invention” and / or “Claims,” such terms are intended to be as comprehensive as the term “comprising.”

[0096] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as those generally understood by those skilled in the art. Furthermore, terms as defined in commonly used dictionaries should be interpreted to have a meaning consistent with their meaning in the context of the relevant art, and not to be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0097] One or more elements, aspects, or steps, or parts thereof from any one or more of the following claims 1 to 20 can be combined with one or more elements, aspects, or steps, or any part thereof, or combination thereof from any one or more of the other claims 1 to 20 to form one or more further implementations and / or claims of the present disclosure.

[0098] While various embodiments of the present invention have been described above, it should be understood that these are not limitations but merely examples. Although the present invention is illustrated and described in relation to one or more implementation forms, those skilled in the art will be able to conceive of equivalent changes and modifications by reading and understanding this specification and the accompanying drawings. In addition, certain features of the present invention may be disclosed in relation to only one of several implementation forms, but such features may be combined with one or more other features of other implementation forms that may be desirable and advantageous for any given or particular application. Therefore, the scope and breadth of the present invention should not be limited by any of the embodiments described above. Rather, the scope of the present invention should be defined by the following claims and equivalents.

Claims

1. A system for predicting patient discontinuation of the use of therapeutic devices, wherein the system is A communication interface for collecting event data relating to monitoring the operation of the therapeutic device that delivers drugs to the patient, A storage device for storing the event data and clinical data of the aforementioned patient, This is an exit analysis module, The event data and clinical data are input into a machine learning model to determine the likelihood of patient dropout over a predetermined period from the input event data and clinical data, and the machine learning model is trained to determine at least one trigger for patient dropout through a pipeline having event data and clinical data input from a population of patients using the therapeutic device. To determine whether the likelihood of withdrawal exceeds a threshold, A withdrawal analysis module is operable to trigger an action relating to the patient if the likelihood of withdrawal exceeds the threshold, Equipped with, The pipeline responds with a predictive endpoint that includes the determined likelihood of withdrawal, the processed clinical data and event data values, and a quantified impact indicating how the values ​​affect the determined likelihood of withdrawal. system.

2. The system according to claim 1, wherein the treatment device is a rescue inhaler or a control inhaler.

3. The system according to claim 1 or 2, wherein the communication interface communicates with the patient's mobile device, and a sensor that communicates with the treatment device is synchronized with an application run on the mobile device to provide event data.

4. The system according to any one of claims 1 to 3, wherein the pipeline trains the machine learning model using customer service data relating to the patient population.

5. The system according to any one of claims 1 to 4, wherein the action is one of a patient alert, a healthcare provider alert, or a therapeutic device manufacturer alert.

6. A method for determining the withdrawal of a therapeutic device operated by a patient, wherein the system, Collect the clinical data of the aforementioned patient, Event data related to monitoring the operation of the aforementioned treatment device is collected via a communication interface. The event data and clinical data are input into a machine learning model to determine the likelihood of patient dropout over a predetermined period from the input event data and clinical data, and the machine learning model is trained to determine at least one trigger for patient dropout through a pipeline having event data and clinical data input from a population of patients using the therapeutic device. Determine whether the likelihood of withdrawal exceeds the threshold. If the likelihood of withdrawal exceeds the threshold, trigger an action relating to the patient. Includes, The pipeline responds with a predictive endpoint that includes the determined likelihood of withdrawal, the processed clinical data and event data values, and a quantified impact indicating how the values ​​affect the determined likelihood of withdrawal. method.

7. The method according to claim 6, wherein the treatment device is a rescue inhaler or a control inhaler.

8. The system The system communicates with the patient's mobile device via the aforementioned communication interface. To provide the event data by synchronizing the sensor that communicates with the treatment device with an application run on the mobile device, The method according to claim 6 or 7, further comprising:

9. The method according to any one of claims 6 to 8, wherein the pipeline trains the machine learning model using customer service data relating to the patient population.

10. The method according to any one of claims 6 to 9, wherein the action is one of a patient alert, a healthcare provider alert, or a therapeutic device manufacturer alert.

11. A method for training a machine learning model for predicting patient dropout of a therapeutic device using a pipeline system, wherein the pipeline system is The data input is collected from a group of patients, and the data input includes clinical data and event data relating to monitoring the operation of the therapeutic device. The aforementioned data inputs are curated and a data table is generated. The machine learning model is trained using the data in the data table, the machine learning model determines the impact of the data on the likelihood of churn, and generates a predictive endpoint that includes the impact. To save the trained machine learning model, Methods that include...

12. The pipeline system The data table is modified via a preprocessing module to exclude data relating to patients who have insufficient interaction with the treatment device. Converting data for input to the training module, The method according to claim 11, further comprising:

13. The data input includes data from a mobile device operated by the patient group, data from an application running on the mobile device, the application interface with the treatment device, and customer service data. The method according to claim 11 or 12.

14. The stored machine learning model is executed by the predictive endpoint module to provide a prediction of the likelihood of discontinuation for a single patient, based on clinical and event data of the treatment device from that single patient. The method according to any one of claims 11 to 13.

15. A pipeline system for training a machine learning model to predict patient dropout while using a therapeutic device, A data input interface for collecting data input from a group of patients, wherein the data input includes clinical data and event data relating to the operation of the therapeutic device. A curation module that curates the aforementioned data inputs and generates a data table, A training module that trains the machine learning model using the data in the data table, the machine learning model determines the impact of the data on the likelihood of churn, and generates a predictive endpoint that includes the impact, A storage device for storing the trained machine learning model, A system that includes this.

16. The preprocessing module further comprises modifying the data table to exclude data relating to patients who have insufficient interaction with the treatment device and transforming the data for input to the training module. The system according to claim 15.

17. The data input includes data from a mobile device operated by the patient group, data from an application running on the mobile device, the application interface with the treatment device, and customer service data. The system according to claim 15 or 16.

18. The stored machine learning model is executed by the predictive endpoint module to provide a prediction of the likelihood of discontinuation for a single patient, based on clinical and event data of the treatment device from that single patient. The system according to any one of claims 15 to 17.

Images