System and method for predicting continuous glucose monitoring results

Abstract

The method and device include predicting the user's future glucose levels and engagement levels by receiving a user's glucose values ​​collected by a continuous glucose monitoring (CGM) device over a predetermined period, receiving engagement data associated with the user, the engagement data relating to the user's drug intake, diet, physical activity, test results, and educational activities, determining a first glycememia risk index (GRI) value, and using a machine learning model to determine one or more predictions about the user's future glucose levels in accordance with the user's glucose values ​​and engagement data collected over a predetermined period, the predictions including the prediction that the future GRI value will be greater than or less than the first GRI value, and using a machine learning model to determine one or more predictions about the user's future engagement levels in accordance with the user's engagement data collected over a predetermined period.

Description

Related Applications

[0001] This application claims priority based on U.S. Provisional Application No. 63 / 495,468, filed on April 11, 2023, the entire disclosure of which is incorporated herein by reference in its entirety.

Technical Field

[0002] This disclosure generally relates to predicting a user's future glucose values and engagement levels, and in some embodiments, more specifically relates to using one or more machine learning models to determine predictions of health and engagement.

Background Art

[0003] The increasing cost of healthcare is limiting users' access to appropriate care. At the same time, healthcare-related companies are increasing the workload of providers and limiting the interaction between physicians and users. Diabetes treatment often relies on sporadic measurements (e.g., blood glucose measurements), and these sporadic measurements do not provide sufficient data to effectively offer treatment options or to predict health and / or engagement outcomes. Such measurements are often used alone, and changes are recommended based on one or two measurements. The medical, dietary, and / or lifestyle changes recommended as a result of a given measurement are limited because the data obtained from sporadic measurements is limited.

[0004] The introductory part described in this specification aims to generally present the context of the disclosure. Unless specifically stated otherwise, the materials described in this section are not prior art to the claims of this application, and it is not admitted that they are prior art or suggestions of prior art by virtue of the description in this section.

Summary of the Invention

[0005] This disclosure relates to a computer implementation method for predicting a user's health and engagement level. The method includes receiving a user's glucose values ​​collected by a continuous glucose monitoring (CGM) device over a predetermined period. The method further includes receiving user-associated engagement data collected by a computing device over a predetermined period, wherein the engagement data relates to the user's drug intake, diet, physical activity, test results, and educational activities. The method also includes determining a first glycememia risk index (GRI) value based on a first time period during which the user was hypoglycemic and a second time period during which the user was hyperglycemic. The method further includes determining one or more predictions about the user's future glucose values ​​using a machine learning model and in response to the user's glucose values ​​and engagement data collected over a predetermined period, the predictions including a prediction that the future GRI value will be greater than or less than a first GRI value, and determining one or more predictions about the future engagement level using a machine learning model and in response to the user's engagement data collected over a predetermined period.

[0006] This disclosure relates to a computer implementation method for training a machine learning model to predict a user's health and engagement levels. The method includes receiving a first glucose value of a user collected by a continuous glucose monitoring (CGM) device over a first period, receiving a second glucose value of a user collected by the CGM device over a second period following the first period, receiving a first engagement data associated with a user collected by a computing device over a first period, and receiving a second engagement data associated with a user collected by a computing device over a second period, wherein the first and second engagement data relate to one or more of the user's drug intake, diet, physical activity, test results, and educational activities. The method further includes training a machine learning model based on a machine learning algorithm and using training data including the first glucose value, the second glucose value, the first engagement data, and the second engagement data to produce a trained machine learning model, and determining one or more patterns in the training data.

[0007] The disclosure also relates to a system for predicting glucose levels and engagement, the system comprising a memory storing processor-readable instructions and a processor configured to access the memory and execute the processor-readable instructions, the instructions being configured to cause the processor to execute a method when executed by the processor. The method includes receiving a user's glucose levels over a predetermined period using a continuous glucose monitoring (CGM) device configured to acquire glucose levels using components that penetrate the user's skin, and receiving engagement data associated with the user collected by one or more electronic sensors via a computing device over a predetermined period, wherein the engagement data relates to one or more of the user's drug intake, diet, physical activity, test results, and educational activities. The method further includes determining a first glycemia risk index (GRI) value based on a first time period during which the user was hypoglycemic and a second time period during which the user was hyperglycemic. The method also includes determining one or more predictions about a user's future glucose level based on a machine learning model and user glucose level and engagement data collected over a predetermined period, wherein the predictions include a prediction that the future GRI level is greater than or less than a first GRI level, and determining one or more predictions about the future engagement level based on a machine learning model and user engagement data collected over a predetermined period. [Brief explanation of the drawing]

[0008] The accompanying drawings are incorporated herein and constitute part of this specification, illustrating embodiments of the disclosure and illustrating the principles of the disclosure together with the description. [Figure 1] This is a schematic diagram of a health management system according to one or more embodiments. [Figure 2] This is a schematic diagram of a part of the health management system shown in Figure 1, according to one or more embodiments. [Figure 3A] This is a schematic diagram of another part of the health management system of Figure 1, according to one or more embodiments. [Figure 3B] This is a schematic diagram illustrating the training of an exemplary machine learning model according to one or more embodiments. [Figure 4] This is a plot showing the GRI zone according to one or more embodiments. [Figure 5] This is a flowchart for determining one or more predictions regarding future glucose values ​​and engagement levels, according to one or more embodiments. [Figure 6] This is a flowchart for training a machine learning model to predict health and engagement outcomes, according to one or more embodiments. [Figure 7A] This figure shows the use of glucose and engagement data to predict future health and engagement outcomes according to one or more embodiments. [Figure 7B] This figure shows the use of glucose and engagement data to predict future health and engagement outcomes according to one or more embodiments. [Figure 7C] This figure shows the use of glucose and engagement data to predict future health and engagement outcomes according to one or more embodiments. [Figure 8A] This figure shows feature selection during the initial phase according to one or more embodiments. [Figure 8B] This figure shows feature selection during the initial phase according to one or more embodiments. [Figure 8C] This figure shows feature selection during the initial phase according to one or more embodiments. [Figure 8D] This figure shows feature selection during the initial phase according to one or more embodiments. [Figure 9A] This chart shows the results of a machine learning model according to one or more embodiments. [Figure 9B] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9C] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9D] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9E] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9F] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9G] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9H] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9I] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9J] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9K] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9L] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9M] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9N] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9O] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9P] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9Q]A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9R] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9S] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9T] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9U] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9V] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9W] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 9X] A chart showing the results of a machine learning model according to one or more embodiments. [Figure 10] Figures 10A - 10D are exemplary plots of the average GRI values over time in a population according to one or more embodiments. [Figure 11] Figures 11A - 11C are diagrams showing the relationships of precision, recall, specificity, and sensitivity according to one or more embodiments. [Figure 12] A simplified functional block diagram of a computer according to one or more embodiments.

Best Mode for Carrying Out the Invention

[0009] Embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. As far as possible, the same reference numbers are used throughout the drawings to refer to the same or similar members.

[0010] In the following descriptions, relative terms such as “approximately,” “substantially,” and “roughly” are used to indicate that there may be a ±10% variation in the stated values. It should be noted that the descriptions in this specification are merely illustrative and are not intended to limit the embodiments covered herein or the application and use of such embodiments. Embodiments described as illustrative in this specification should not be construed as superior or advantageous to other embodiments. Rather, as stated above, the term “exemplary” is used in the sense of example or “explanatory,” not “ideal.” “Include,” “contain,” “have,” “equip,” and their variations are used synonymously to indicate non-exclusive inclusion. Thus, processes, methods, articles, or apparatus using these terms do not include only these processes, structures, or elements, but may include other processes, structures, or elements not expressly described in such processes, methods, articles, or apparatus, or that are not specific to such processes, methods, articles, or apparatus. Furthermore, terms such as “first,” “second,” etc., in this specification do not indicate order, quantity, or importance, but are used simply to distinguish one element from another. In addition, in this specification, the terms "a" and "an" do not indicate a limit on quantity, but rather indicate that there is at least one item being referenced.

[0011] Medical and computing environments Figure 1 is a block diagram of a health management system 100 according to an example of the present disclosure. A user (e.g., a patient, consumer, etc.) 8 having an electronic device 19 (e.g., a mobile device, computer, medical device, or any other electronic device configured to access an electronic network 32 such as the Internet) can communicate with or access the mobile health (mHealth) application 1. In some examples, the network 32 may include wireless or wired links (e.g., a cellular network, Wi-Fi, LAN, WAN, Bluetooth, near-field communication (NFC), or other appropriate network communication methods). Multiple electronic devices 19 may be configured to access the electronic network 32. The user 8 can access the mHealth application 1 with a single account linked to multiple electronic devices 19 (e.g., by one or more of a mobile phone, tablet, and laptop computer). The electronic device 19 may include, but is not limited to, a mobile health device, a desktop computer or workstation, a laptop computer, a mobile handset, a personal digital assistant (PDA), a mobile phone, a network appliance, a camera, a smartphone, a smartwatch, a High-Speed ​​General-Package Radio Service (EGPRS) mobile phone, a media player, a navigation device, a game console, a set-top box, a biometric sensor device with communication capabilities, a smart TV, or any combination thereof, or other computing devices having at least one processor, local memory, a display (e.g., a monitor or touchscreen display), one or more user input devices, and a network communication interface.The electronic device 19 may include any type or combination of input / output devices, such as a display monitor, keyboard, touchpad, accelerometer, gyroscope, mouse, touchscreen, camera, projector, touch panel, pointing device, scroll device, button, switch, motion sensor, acoustic sensor, pressure sensor, temperature sensor, and / or microphone. The electronic device 19 may also communicate with each other by any suitable wired or wireless means (e.g., Wi-Fi, radio frequency (RF), infrared (IR), Bluetooth, near-field communication, or other suitable means) for sending and receiving information.

[0012] The mHealth application 1 may be implemented to communicate with other entities or networks and send and receive information. In some examples, the mHealth application 1 may communicate with one or more applications associated with user 8 (e.g., exercise tracking (e.g., step tracking) applications and / or other health-related applications). The mHealth application 1 may be configured to import data from other applications and analyze and use that data in generating a treatment plan for user 8. For example, the mHealth application 1 may import activity tracking data from another application and use that data to identify patterns between user 8's exercise and glucose values ​​collected before using the mHealth application 1. The mHealth application 1 may also import other appropriate data from other mobile health applications, such as blood pressure, body mass index (BMI), glycated hemoglobin (A1C), exercise type, exercise time, exercise distance, calories burned, total steps, exercise days, exercise start and end times, and sleep. The mHealth application 1 may also export data to other mobile applications, such as other mobile health applications with social or interactive features. A healthcare provider 7 (e.g., a physician) may prescribe the application. However, it should also be considered that the mHealth application 1 may not require a prescription; that is, it may be a commercially available consumer application accessible without a prescription from a digital distribution platform for computer software. The mHealth application 1 may be customized for a specific user 8 and activated in person by the user 8 visiting a pharmacy 9 or other authorized authority. For example, the user 8 may receive an access code from the pharmacy that authorizes them to access the mHealth application 1.User 8 can receive training on how to use mHealth Application 1 from the mHealth support system 25 and / or application trainer 24. mHealth Application 1 may include various programming 28, such as machine learning programming algorithms 26. The user's treatment plan includes prescriptions (for example, for drugs, devices, and / or treatments) issued by pharmacy 9. Pharmacy 9 may authorize the re-dispensing of prescribed products / treatments after receiving authorization based on the user's adherence to the treatment plan. Authorization may be received by pharmacy 9 through communication from mHealth Application 1, for example, via network 32 and various servers 29. Information on the use of drugs or other medical products / treatments can also be transmitted to manufacturer 37 via network 32 to inform manufacturer 37 of the amount of medical products or treatments used by user 8. This information may help manufacturer 37 assess the demand for medical products or treatments and develop supply plans. Healthcare provider 7 may also receive reports based on user information received by mHealth Application 1 and update the user's treatment plan based on this information. The user's electronic medical record (EMR) 14 may also be automatically updated via the network 32 based on user information, including electronically transmitted user 8 feedback regarding the application, received by the mHealth application 1. The healthcare provider 7 may be any appropriate healthcare provider, such as a physician, specialist, nurse, educator, social worker, medical assistant (MA), physician's assistant, or physician associate (PA).

[0013] Figure 2 is a schematic diagram of an additional embodiment of system 100. For example, system 100 can access decision models stored in the decision model database 270 via the network 32. The acquired decision models may be used for display and / or processing by one or more electronic devices 19, such as a mobile device 215, a tablet device 220, a computer (e.g., a laptop or desktop) 225, a kiosk 230 (e.g., a kiosk, pharmacy, clinic, or hospital with medical and / or prescription information), and / or any device connected to the network 32.

[0014] In the example shown in Figure 2, the mobile device 215, the tablet device 220, and the computer 225 may each be equipped with or have a Global Positioning System (GPS) receiver for acquiring and reporting location information (e.g., GPS data) from, for example, one of the servers 29 and / or one or more GPS satellites 255 via the network 32.

[0015] Each of the electronic devices 19, including a mobile device 215, a tablet device 220, a computer 225, and / or a kiosk 230, may be configured to send and receive data (e.g., clinical information) to and from the server 29 system via the network 32. Each of the devices 19 can receive information such as clinical data from the server 29 via the network 32. The server 29 may include a clinical data server 240, an algorithm server 245, a user interface (UI) server 250, and / or any other suitable server. The electronic devices 19 may have a user interface that communicates data with the UI server 250 via the network 32. Each server can access the decision model database 270 to obtain decision models. Each server may include memory, a processor, and / or a database. For example, the clinical data server 240 may have a processor configured to retrieve clinical data from a provider's database and / or a patient's electronic medical record. The algorithm server 245 may have a database containing various algorithms and a processor configured to process clinical data. The UI server 250 may be configured to receive and process input from user 8, such as preferences for clinical decisions. The satellite 255 may be configured to send and receive information between the server 29 and the device 19.

[0016] The clinical data server 240 can receive clinical data, such as user data, from the electronic device 19 via the network 32 or indirectly via the UI server 250. The clinical data server 240 can store the information in memory (e.g., computer-readable memory).

[0017] The clinical data server 240 may communicate with one or more other servers, such as the algorithm server 245 and / or external servers. Server 29 may include data on provider preferences and / or user 8's health history. In addition, the clinical data server 240 may include data from other users. The algorithm server 245 may include machine learning and / or other appropriate algorithms. The algorithm server 245 may communicate with other external servers and be updated as needed. For example, the algorithm server 245 may be updated with new algorithms, more powerful programming, and / or more data. The clinical data server 240 and / or the algorithm server 245 can process the information and send the data to the model database 270 for processing. In one example, the algorithm server 245 can obtain pattern definitions in a simple format, predict multiple future time steps using models such as Markov models, Gaussian models, Bayesian models, principal component analysis (PCA), multivariate linear or nonlinear regression, and / or classification models such as linear discriminant functions, nonlinear discriminant functions, synthetic discriminant functions, and random forest algorithms, optimize results based on these predictions, detect transitions between patterns, obtain abstract data and extract information for inferring higher-order knowledge, combine higher-order and lower-order information to understand user 8 and clinical behavior, infer from multitemporal (e.g., different time scales) data and related information, and reduce noise over time using variable-order Markov models and / or clustering algorithms such as slope-based and curve-smoothing algorithms and k-means clustering.

[0018] Each server in the Server 29 system, including the Clinical Data Server 240, Algorithm Server 245, and UI Server 250, may represent any of various types of servers, including, but not limited to, web servers, application servers, proxy servers, network servers, or server farms. Each server in the Server 29 system may be implemented using any general-purpose computer capable of providing data to, for example, device 19 or any other computing device (not shown) via network 32, but not limited to these. Such a general-purpose computer may include, but not limited to, a server device having a processor and memory for executing and storing instructions. Memory may include any type of random access memory (RAM) or read-only memory (ROM) implemented on a physical storage medium such as magnetic storage (floppy disk, hard disk, or magnetic tape), semiconductor storage (solid-state disk (SSD) or flash memory), optical disk storage, or magneto-optical disk storage. Software may include one or more applications and an operating system. Hardware may include, but not limited to, a processor, memory, and a graphical UI display. Each server may have multiple processors and multiple shared or isolated memory components, configured to function collaboratively, for example, within a cluster computing environment or server farm.

[0019] Figure 3A is another representation of a part of system 100 showing additional details of electronic device 19 and server 29. Electronic device 19 and server 29 may each comprise one or more processors, such as processor 301-1 and 304-1. Processors 301-1 and 304-1 may each be a central processing unit, a microprocessor, a general-purpose processor, an application-specific processor, or any device that executes instructions. Electronic device 19 and server 29 may each comprise one or more memories, such as memories 301-2 and 304-2, that store one or more software modules. Memories 301-2 and 304-2 may be implemented using any computer-readable storage medium, such as a hard drive, CD, DVD, flash memory, RAM, ROM, etc. Memory 301-2 may store module 301-3 executed by processor 301-1. Similarly, memory 304-2 may store module 304-3 executed by processor 304-1.

[0020] The electronic device 19 may further include one or more UIs. The UI can provide one or more interfaces for presenting information such as plans or interventions to the user 8. The UI may be web-based, such as a web page, or a standalone application. The UI may be configured to accept information about the user 8 (e.g., data input and user feedback). The user 8 may input the information manually or automatically. In one example, the user 8 (or the user's caregiver) may input information such as the time when medication was taken or the food and beverages consumed by the user 8. The electronic device 19 may also include test equipment (not shown) or an interface for receiving information from test equipment. Test equipment may include, for example, a blood glucose meter, glucose meter, heart rate monitor, scale, blood pressure cuff, etc. The electronic device 19 may also include one or more sensors (not shown), such as a camera, microphone, or accelerometer, for collecting feedback from the user 8. In one example, the device may include a glucose meter for reading and automatically reporting the user's glucose level.

[0021] The electronic device 19 may include a presentation layer. The presentation layer may be a web browser, an application, a messaging interface (e.g., email, instant messaging, SMS, etc.). The electronic device 19 can present notifications, alerts, reading material, reference materials, guides, reminders, or suggestions to the user 8 via the presentation layer. For example, the presentation layer may present articles deemed relevant to the user 8, reminders for medication purchases, tutorials on a topic (e.g., a tutorial on carbohydrates), testimonials from others with similar symptoms, and / or one or more goals (e.g., carbohydrate counting goals). The presentation layer may also present information such as tutorials (e.g., user guides or explanatory videos) and / or enable communication between the healthcare provider and the user 8 (e.g., the patient). Communication between the healthcare provider and the user 8 (e.g., the patient) may be via electronic messages (e.g., email or SMS), voice, or real-time video. One or more of these items may be presented based on a treatment plan or an updated treatment plan, as described below. The presentation layer may also be used to receive feedback from users.

[0022] System 100 may include one or more databases, such as database 302. Database 302 may be implemented using any database technology known to those skilled in the art, such as relational database technology or object-oriented database technology. Database 302 may store data 302-1. Data 302-1 may include a knowledge base for inference, statistical models, and / or user information. Data 302-1 or a portion thereof may be stored in server 29 or electronic device 19, either alternatively or concurrently.

[0023] System 100 can be used for a wide range of applications, including, for example, monitoring and tracking a user's healthcare efforts, maintaining their finances, and monitoring and tracking their nutrition and / or sleep. In some embodiments of System 100, any received data may be stored in a database in an encrypted format that enhances security against unauthorized access to the data and complies with HIPAA privacy and / or other legal, medical, financial, or other regulations.

[0024] Any server or server system 29 shown in system 100 may have one or more databases. In one example, a database may be any type of data store or recording medium used to store any type of data. For example, database 302 may store data received or processed by server 29, including information related to the user's treatment plan (including timing and dosage related to each prescription drug in the treatment plan). Database 302 may also store information related to user 8, including literacy levels related to each of several prescription drugs.

[0025] As further disclosed herein, one or more components of the disclosed subject matter may be implemented using machine learning models. Figure 3B shows an example of a training module 310 for training one or more of the machine learning models disclosed herein. It will be understood that different training modules may be used to train each of the machine learning models disclosed herein, and / or a single training module 310 may be used to train two or more machine learning models.

[0026] As shown in Figure 3B, the training data 312 may include one or more stage inputs 314 and known outcomes 318 related to the machine learning model being trained. The stage inputs 314 may be from any applicable source, including healthcare providers 7, one or more servers 29, electronic devices 19, EMR 14, and outputs from steps (e.g., one or more outputs from steps in flowchart 500 in Figure 5 or flowchart 600 in Figure 6, time-in-range (TIR) ​​values, time-above-range (TAR) values, time-below-range (TBR) values, severity scores, continuous glucose monitoring (CGM) classifications, GRI values, engagement data, etc.). The known outcomes 318 may be included in the machine learning model generated based on supervised or semi-supervised training. Unsupervised machine learning models may not be trained with known outcomes 318. The known outputs 318 may include known or desired outputs for future inputs of the same or identical category as stage inputs 314 that do not have corresponding known outputs.

[0027] The training data 312 and the training algorithm 320 may be provided to a training component 330 that applies the training data 312 to the training algorithm 320 to generate a machine learning model. In one embodiment, the training component 330 is provided with a comparison result 316 that compares the previous output of the corresponding machine learning model, and the machine learning model can be retrained by applying the previous result. The comparison result 316 may be used by the training component 330 to update the corresponding machine learning model. The training algorithm 320 may utilize machine learning networks and / or models, including but not limited to deep learning networks such as deep neural networks (DNNs), convolutional neural networks (CNNs), fully convolutional networks (FCNs), and recurrent neural networks (RCNs), probabilistic models such as Bayesian networks and graphical models, and / or discriminative models such as decision forests and maximum margin methods.

[0028] Health condition Diabetes mellitus (commonly called diabetes) is a chronic, lifelong metabolic disorder (or condition) in which a person's body is unable to produce any or any insulin, or is unable to utilize the insulin that is produced (insulin resistance), resulting in elevated blood glucose levels. The three most easily identifiable types of diabetes to be diagnosed include prediabetes, type 1 diabetes, and type 2 diabetes. Prediabetes is a condition in which blood glucose levels are high, but not as high as in type 2 diabetes. Type 2 diabetes is a chronic condition that affects how the body processes blood glucose. Finally, type 1 diabetes is a chronic condition in which the pancreas produces little to no insulin.

[0029] Diabetes is generally diagnosed in several ways. Diagnosing diabetes may require repeated testing over several days for a definitive diagnosis of the type of diabetes. Health parameters used by physicians or other appropriate healthcare providers to confirm a diagnosis of diabetes include glycated hemoglobin (A1C) levels, fasting plasma glucose (FPG) levels, oral glucose tolerance tests, and / or random plasma glucose tests. Generally, healthcare providers are interested in a patient's A1C levels to aid in diagnosing diabetes. Glycated hemoglobin is a form of hemoglobin that can be measured to determine the mean plasma glucose concentration over the past three months, and can be used by physicians and / or other appropriate healthcare providers. Other parameters include weight, age, nutritional intake, exercise activity, cholesterol levels, triglyceride levels, obesity, smoking, and family history.

[0030] Once a type of diabetes is diagnosed by a physician or other appropriate healthcare provider, a patient may receive treatment to manage their diabetes. Patients whose diabetes is being tracked or monitored by a physician or other healthcare provider may be treated with a combination of diet, exercise, oral medications, and / or insulin therapy to control blood glucose levels. Some patients also require regular screening for complications. Depending on how long the patient has been diagnosed with diabetes, the mHealth application 1 may suggest a specific treatment plan to manage the condition. Oral medications typically include tablets taken orally to reduce glucose production by the liver and increase the sensitivity of muscles to insulin. In other cases, if the diabetes is more severe, additional medications (including injections) may be required for the patient's diabetes treatment. Injections of basal insulin, also known as background insulin, may be used by healthcare providers to maintain stable glucose levels during fasting. During fasting, a patient's body steadily releases glucose into the bloodstream to supply energy to cells. Therefore, injections of basal insulin are needed to control glucose levels and allow cells to take up glucose for energy. Basal insulin is usually administered once or twice daily, depending on the type of insulin. Because basal insulin acts for a relatively long period, it is considered long-acting or intermediate-acting insulin. In contrast, bolus insulin may be used to act rapidly. For example, a bolus dose of insulin may be given specifically at mealtimes to control glucose levels after a meal. In some cases, when a physician or healthcare provider creates a treatment plan for managing a patient's diabetes, they may create a basal-bolus insulin regimen, which may include multiple injections per day, for example. A basal-bolus insulin regimen may include injections at each mealtime and attempts to roughly mimic how the body of a non-diabetic person delivers insulin. Basal-bolus insulin regimens are applicable to both type 1 and type 2 diabetes patients.In addition to basal and bolus insulin regimens requiring insulin injections, treatment plans may be supplemented by the use of prescribed oral medications. Patient adherence to the treatment plan is crucial for managing the patient's disease state. If a patient has been diagnosed with diabetes for more than six months, for example, to achieve healthy or favorable glucose levels, they may need to adhere to a very specific treatment regimen. Ultimately, the weekly pattern of these drug therapies may be important in diabetes management. The mHealth application may recommend a treatment plan to help patients manage their diabetes.

[0031] Exemplary Method Diabetes mellitus is a chronic disease resulting from an inability to maintain glucose levels within a normal or recommended target range. Such fluctuating glucose levels (i.e., outside the normal or recommended target range) can lead to serious complications. Sporadic blood glucose monitoring (BGM) provides only intermittent measurements a few times a week, making it difficult to gain meaningful insights and predictions about health and engagement outcomes because it does not provide a basis for understanding patterns and the underlying causes of those patterns (e.g., determining elevated BGM based on the type of meal). Similar problems exist with flash glucose monitoring (FGM), where measurements are sporadic, irregular, or non-continuous.

[0032] Continuous glucose monitoring (CGM) offers the possibility of automatically collecting high-density data (e.g., data based on a collection frequency of every 5 minutes or less) through a wearable sensor (e.g., a subcutaneous sensor) that provides periodic glucose values ​​(e.g., User 8's glucose value). CGM can improve diabetes care by providing User 8 or other entities (e.g., Healthcare Provider 7) with glucose data readout information continuously (e.g., approximately every 5 minutes or less) or semi-continuously (e.g., at intervals longer than approximately 5 minutes) so that User 8 or other entities (e.g., Healthcare Provider 7) can have a better understanding of User 8's glucose levels at any time of day. Such data can be used to train machine learning models to predict future health and engagement levels and outcomes based on glucose value and engagement data inputs.

[0033] A CGM monitor may be a continuous analyte sensor system having any sensor configuration that provides an output signal indicating the concentration of an analyte. The CGM monitor may sense the concentration of an analyte based on body fluids (e.g., interstitial fluid) to determine, for example, a glucose value. Body fluids may be collected through the user's skin. The output signal may be in the form of, for example, sensor data (raw data stream, filtered data, smoothed data, and / or other transformed sensor data, etc.) and may be transmitted to a receiver. The receiver may be connected to the CGM monitor by a wired or wireless connection and may be located locally or away from the sensor. According to several embodiments, the CGM monitor may include a transcutaneous glucose sensor, a subcutaneous glucose sensor, a continuously refillable subcutaneous glucose sensor, a continuous intravascular glucose sensor, etc. The CGM monitor may be a compact medical system having one or more sensors inserted into the user's abdomen and having a small cannula that penetrates the user's skin. An adhesive patch may hold the monitor in place. The sensors may continuously or semi-continuously sense glucose measurements in the interstitial fluid.

[0034] A transmitter may be connected to the sensor, allowing the CGM monitor to wirelessly transmit glucose measurements to a monitoring device. The monitoring device may be a dedicated monitoring device for the CGM monitor, or it may be a third-party device, an electronic device 19, or other applicable device. The monitoring device may be a dedicated monitoring device, or it may be an electronic device 19 that provides one or more functions other than CGM monitoring. An application or other software may be used to facilitate the analysis and / or display of glucose measurements and related data via the monitoring device. The monitoring device may be used for the analysis and / or display of data related to glucose measurements. Alternatively, or in addition, the CGM monitor may have a display for displaying glucose measurements and / or related data. The CGM monitor and / or external device may be configured to generate and / or provide alerts based on glucose data (e.g., if blood glucose levels are too high or too low, or if there is an unfavorable trend).

[0035] By using CGM data, the Time-in-Range (TIR) ​​value can be determined, where the TIR value is based on the amount of time over a reference period during which the user's glucose value is within a threshold band. The threshold band may be predetermined, user-specific, or dynamically determined.

[0036] The threshold band may be a predetermined value based, for example, on a patient cohort. The lifestyle, habits, and medical test results of each patient within the cohort may be used to determine the predetermined value. For example, one or more patient cohorts may be determined based on the patients' lifestyles, habits, demographics, etc., and a threshold band may be generated for each of the one or more cohorts. The threshold band may be determined based on the optimal result (e.g., a preferred A1C value) based on an analysis of glucose levels over a certain period.

[0037] The GRI value may be determined using CGM data, where the GRI value is a composite index from CGM tracking indicating the quality of user 8's glycemia and may support the basic clinical interpretation of CGM data. "Quality of glycemia" may be characterized by the proportion of time spent in both low / very low and high / very high glucose concentrations. The GRI value is based on the hypoglycemic and hyperglycemic components. The hypoglycemic component may relate to the amount of time user 8 was in a hypoglycemic state during a given period, and the hyperglycemic component may relate to the amount of time user 8 was in a hyperglycemic state during a given period.

[0038] The following formula may be used to determine the GRI value. In the following formula, "VLow" represents the percentage of time the user experiences very low glucose hypoglycemia, "Low" represents the percentage of time the user experiences low glucose hypoglycemia, "VHigh" represents the percentage of time the user experiences very high glucose hyperglycemia, and "High" represents the percentage of time the user experiences high glucose hyperglycemia. "HypoComponent" represents the hypoglycemic component, and "HyperComponent" represents the hyperglycemic component.

[0039] Low blood sugar component=VLow+(0.8×Low)

[0040] High blood sugar component=VHigh+(0.5×High)

[0041] GRI=(3.0×HypoComponent)+(1.6×HyperComponent)

[0042] Another equivalence equation for GRI is as follows:

[0043] GRI=(3.0×VLow)+(2.4×Low)+(1.6×VHigh)+(0.8×High)

[0044] Figure 4 shows a plot 400 illustrating five GRI zones according to one or more embodiments. Since blood glucose control is of a two-dimensional nature, the hypoglycemic and hyperglycemic components of GRI may be displayed on a two-dimensional plot as shown in plot 400. In plot 400, the hypoglycemic component is shown on the horizontal axis and the hyperglycemic component is shown on the vertical axis. However, it is understood that the hypoglycemic and / or hyperglycemic components may be visualized in any applicable manner. A series of diagonal lines divide plot 400 into five glycememia risk zones labeled 401, 402, 403, 404, and 405. Each of these zones corresponds to a quintile of overall glycememia quality (from best (0th to 20th percentile, zone A401) to worst (81st to 100th percentile, zone E405)). A diabetic patient may receive a GRI score for a given period, which may be mapped to one of the five GRI zones. Additional information regarding a composite index of glycemia quality from CGM to support the basic clinical interpretation of CGM data is provided in Klonoff et al. (Klonoff DC, Wang J, Rodbard D et al. A glycemia risk index (GRI) of hypoglycemia and hyperglycemia for continuous glucose monitoring validated by clinician ratings. J Diabetes Sci Technol. 2022 Mar 29.), which is incorporated herein by reference.

[0045] Figure 5 shows a flowchart 500 for determining one or more predictions regarding future glucose levels and engagement levels according to one or more embodiments. As used herein, “engagement level” may correspond to the level of user input or interaction related to adherence to medical care, diet, exercise, etc. In 502, the blood glucose level of user 8 may be received. The blood glucose level may be provided continuously or semi-continuously by a CGM monitor as disclosed herein. The blood glucose level may be provided by a standard blood glucose monitor (BGM), a flash glucose monitor (FGM), or a combination of CGM, BGM, and / or FGM. The blood glucose level may be received by a component of the CGM monitor itself, or by a local or remote component (e.g., an electronic device 19, mHealth application 1, one or more servers 29, etc.). The blood glucose level may be automatically provided from the CGM monitor to one or more components, pushed when blood glucose levels are collected, or the CGM monitor may be instructed to transmit one or more collected blood glucose levels.

[0046] For example, user 8 may wear a CGM monitor on their body, and the CGM monitor may collect blood glucose values ​​approximately every 5 minutes. The CGM monitor may be connected to user 8's mobile device (e.g., via a network connection, local area network connection, wide area network connection, WiFi connection, Bluetooth® connection, etc.). According to the first embodiment, the CGM monitor may automatically transmit the blood glucose values ​​to user 8's mobile device each time a measurement is collected (e.g., every 5 minutes). Alternatively, or in addition, the CGM monitor may store one or more blood glucose values ​​and transmit those one or more blood glucose values ​​to user 8's mobile device as a group of measurements and / or when user 8's mobile device or another component requests the transmission of one or more blood glucose values.

[0047] In Figure 5, step 504, engagement data associated with the user may be received. Engagement data may be collected by a computing device, such as electronic device 19, over a predetermined period (e.g., the same period as glucose value collection in step 502). Engagement data may include, and / or relate to, the user 8's drug intake, diet, physical activity, test results, and / or educational activities (e.g., "MEDAL" activities), which are described in further detail herein. Engagement data may also include CGM engagement associated with user 8. CGM engagement may include blood glucose activity collected using a CGM device.

[0048] In some cases, engagement data may be collected through user input via applications such as the mHealth application 1. For example, user 8 can use the mHealth application 1 to record each time user 8 takes medication. User 8 can record the time and date of medication intake, as well as the name, type, and / or dosage of the medication. User 8 can also record the method of medication intake, such as whether the medication was taken orally or by another method of administration (e.g., injection, inhalation, topical application, etc.). User 8 can record the reason for taking the medication (e.g., to treat pain or other symptoms). User 8 can further record whether the medication relieved any of the symptoms and can also record any side effects from taking a particular medication.

[0049] As another example, user 8 may receive a prompt via mHealth application 1 to take a specific medication at a specific time. mHealth application 1 can send a push notification at the prescribed time to take the medication to remind the user to take it. In some cases, mHealth application 1 may remind user 8 in advance (e.g., one hour before) to take a specific medication at a specific time. After the medication prompt (e.g., push notification), mHealth application 1 may follow up by asking user 8 to input that they have actually taken the medication. This may be done by asking for confirmation of drug intake through the user interface on the electronic device 19 via mHealth application 1. In some cases, there may be no prescribed time to take a specific medication, and user 8 may not receive a prompt to take a specific medication. In these cases, user 8 can periodically (e.g., once a week, once a day, or multiple times a day) access mHealth application 1 to enter details of drug intake.

[0050] In some embodiments, drug administration may refer to insulin administration. As described herein, insulin administration may refer to basal insulin injection or insulin bolus administration, which may help the user's body regulate blood glucose levels (i.e., glucose levels). In some cases, user 8 may manually administer insulin using a syringe, insulin pen, insulin pump, or insulin inhaler. User 8 can manually calculate and determine when to administer basal and bolus insulin using any of these methods. User 8 may also use the mHealth application 1 for reminders and calculations regarding insulin administration timing and dosage. A combination of manual and automatic reminders and calculations may also be used. For example, a user may receive basal insulin automatically and regularly (e.g., hourly) via an insulin pump, but may need to manually calculate the dosage of bolus insulin at meal times (often via the same insulin pump). A continuous glucose monitor (CGM) can be used in conjunction with an insulin pump to function as an "artificial pancreas," where the CGM device monitors blood glucose levels and determines basal or bolus insulin dosages based on current and predicted glucose levels. When the CGM device and insulin pump work together to administer insulin to user 8, the time of each administration, the amount administered, the glucose level, and the type of insulin may be recorded and stored for future analysis by user 8 or user 8's physician.

[0051] Other devices and sensors may also be used to measure and record user 8's drug intake. For example, user 8 may place a certain amount of tablets into a pillbox or pill dispenser, which may be connected to electronic device 19 and / or network 32. The pill dispenser may have one or more sensors to automatically monitor whether tablets (e.g., tablets for a particular day) have been removed from the dispenser, thereby indicating that user 8 has taken their medication. For example, the pill dispenser may have a weight sensor that can detect minute weight differences to determine whether tablets remain in the compartment or have been removed. The pill dispenser may also have a light sensor that is blocked by one or more tablets, thereby indicating that tablets are still present. There may also be a light that turns on periodically, and if the light sensor receives enough light, the pill dispenser may determine that the tablets are gone because one or more tablets block at least some of the light. The light sensor may also determine the color of the tablets by measuring the wavelength of light reflected from the tablets. A camera may also be used to identify the shape and color of the tablets to determine which tablets are present and which tablets have been taken. The pill dispenser may include an automatic dispenser programmed to dispense tablets to user 8 at the time they should be taken. The pill dispenser can record the time when the tablets should be dispensed and dispense them automatically for user 8. The pill dispenser can record the time when the tablets are taken and transmit these records to an electronic device 19 for analysis.

[0052] User 8's meals may be tracked through user input via the mHealth application 1. User 8 can input the foods consumed during and between meals into the mHealth application 1. In some cases, User 8 calculates some or all of the nutritional information, such as calories, fat content, and sugar content. In other cases, the mHealth application 1 estimates the nutritional information based on a food database containing nutritional information. The mHealth application 1 may prompt User 8 with a reminder to record meal information at a specified time of day. In one or more embodiments, the mHealth application 1 can interact with a CGM device to determine that User 8 has eaten a meal. If the glucose level rises above a threshold or rises at a rate exceeding the threshold rate, the mHealth application 1 and / or the CGM device can determine that User 8 has eaten a meal. Based on this rise in glucose level, the mHealth application 1 may provide a prompt to User 8 to record meal information. In one or more embodiments, the user can take a picture of the food before eating and provide it as input to the mHealth application 1.

[0053] According to one embodiment, the mHealth application 1 can use food intake machine learning to determine what a food is and its nutritional details. The food intake machine learning model may be trained using historical or simulated food items and their respective nutritional details. Based on the training, the food intake machine learning model can generate nutritional details based on food intake information such as the type of food, cooking method, ingredients, and quantity.

[0054] Similar to drug intake or meals, physical activity can also be manually recorded by User 8 via the mHealth application 1. After any type of physical activity (e.g., a workout or exercise), User 8 can record about the activity, including notes on the type of activity, the intensity of the activity, feelings and mood before, during, and after the activity, and / or other unexpected difficulties. User 8 can also record workout details related to a specific physical activity. For example, if the activity was running, User 8 can record the distance run, running time, percentage of walking during the run, etc.

[0055] Physical activity may be automatically collected and recorded using one or more activity trackers (e.g., smartwatches, smart rings, activity bands, smartphones, etc.), which measure one or more indicators related to physical activity using one or more sensors. For example, an activity tracker may track a user's steps, elevation change, heart rate, body temperature, etc. An activity tracker may be equipped with one or more electronic sensors, such as an accelerometer, gyroscope, altimeter, photoplethysmography (PPG) sensor, pulse oximeter sensor (e.g., SpO2 monitor), bioimpedance sensor (e.g., one or more electrodes or EKG / ECG sensor), skin potential sensor, GPS sensor, light / optical sensor, compass, UV sensor, magnetometer, gesture sensor, temperature sensor, microphone, and / or skin conduction sensor. One or more of these sensors may operate independently or in combination to detect motion, movement, acceleration, elevation, heart rate and / or other cardiac activity, blood oxygen levels, direction, position, device orientation and rotation, etc. Raw electronic data can be acquired from one or more of these electronic sensors, analyzed, and integrated to obtain indicators useful for determining and analyzing physical activity. For example, data related to motion or acceleration may be used to determine the number of steps taken, the intensity of walking (e.g., whether the user was walking, jogging, or sprinting). Data related to elevation changes can be integrated with motion data to derive inferences such as the number of floors climbed or descended, or the distance and elevation gain during a hike.

[0056] Other health-related data may be collected automatically through user input or via connected electronic devices, and may include, for example, blood pressure and weight. For example, blood pressure is a useful indicator that provides information about the user's cardiovascular system and may be measured manually or automatically using an inflatable blood pressure cuff. When reading blood pressure manually using a standard blood pressure cuff, user 8 can enter the blood pressure into the user interface associated with mHealth application 1. Electronic blood pressure monitor devices can measure blood pressure and transmit the value to mHealth application 1 for storage and analysis. Other methods of measuring blood pressure include arterial tonometry and oscillometric blood pressure measurement. Similarly, the user's weight may be measured using a standard (e.g., non-electronic) scale, and the result may be manually entered by user 8 into mHealth application 1. Weight may also be measured using an electronic device 19 or an electronic scale ("smart scale") connected to network 32. The measured weight may be automatically transmitted to mHealth application 1.

[0057] The results of clinical or other medical tests may be collected in the mHealth application 1 via user input, or they may be automatically transmitted via network 32 directly from the laboratory where the clinical test was performed or from the healthcare facility or clinic where the medical test was performed. Clinical or other medical tests may include A1C tests, lipid profile tests, renal function tests (e.g., urinary albumin tests), ophthalmological tests, foot tests, liver function tests, thyroid function tests, C-peptide tests, and / or hemoglobin tests.

[0058] Educational activities can include any activities that users undertake to learn about their disease, disease management, and how to prevent or manage complications. For example, diabetes-related educational activities may take various forms, such as one-on-one counseling with a healthcare provider or diabetes instructor, group classes or support groups, online resources and apps, and written or audio / video materials. Diabetes education can cover a wide range of topics, including understanding the causes and symptoms of diabetes, methods for monitoring blood glucose levels and interpreting the results, the role of pharmacotherapy and insulin therapy in diabetes management, methods of diet and nutrition management to control blood glucose levels, the importance of physical activity and exercise in diabetes management, strategies for preventing and managing diabetes-related complications such as neuropathy, kidney disease, and visual impairment, and coping with the emotional and psychological challenges of living with diabetes.

[0059] Educational activities may be facilitated through mHealth Application 1. For example, User 8 can engage with mHealth Application 1 by viewing articles on the role of pharmacotherapy and insulin therapy in diabetes management. Another example is receiving counseling from a healthcare provider or diabetes instructor through mHealth Application 1, which may include video conferencing. Engagement with mHealth Application 1, such as viewing articles or participating in video conferences with instructors or providers, may be recorded as completed educational activities. User 8 can also manually enter completed educational activities into mHealth Application 1 without using it.

[0060] As described above, engagement data related to drug intake, diet, physical activity, test results, and educational activities (e.g., "MEDAL" data) may be aggregated and integrated to generate an engagement score (e.g., engagement status) for user 8 over a predetermined period. Engagement as used herein refers to engaging with mHealth application 1 by manually entering data into the mHealth application 1, directly interacting with the mHealth application 1, and / or engaging with one or more entities that submit data to the mHealth application 1 on behalf of the user (e.g., automated engagement). For example, one or more entities may include labs, support groups, devices, platforms, activity trackers, pill dispensers, online calorie trackers, etc. Therefore, engagement data refers to any data that indicates engagement with the mHealth application 1, such as manual user input from user 8 in the mHealth application 1, automatic input (using one or more electronic devices 19), drug intake, diet, physical activity, test results, and communications received from third parties regarding any interaction with user 8 regarding educational activities.

[0061] In some embodiments, the engagement state may be binary, classifying it as either high engagement or low engagement. If user 8 engages with mHealth application 1 or entities that submit data to mHealth application 1 to a threshold level or higher, the engagement state may be classified as high engagement. If user 8 engages with mHealth application 1 or entities that submit data to mHealth application 1 to a threshold level or lower, the engagement state may be classified as low engagement. In some embodiments, the engagement state may be determined based on the number of “engagement activities” in which user 8 participated over a given period. An engagement activity may be a single activity or interaction with application 1 or entities that submit data to mHealth application 1 on behalf of user 8. For example, an engagement activity may include a single user input or entry to mHealth application 1 about medications taken, meals and their nutritional information, workouts, test results, and / or educational activities. As a further example, engagement activities may include communications received by the mHealth application 1 from an activity tracker regarding user 8's activities, from a networked pill dispenser, from a third-party laboratory with test results, and / or from a networked educational website. In one embodiment, each interaction between user 8 and the mHealth application 1 may be considered an engagement activity. Educational activities facilitated through the mHealth application 1 or other applicable platforms may be recorded as engagement activities. As an example of a binary engagement state, the high engagement threshold could be approximately five engagement activities in a 10-day period. Using this example threshold, if user 8 has performed approximately five or more engagement activities in a 10-day period, user 8's engagement state may be determined to be "highly engaged."If user 8 engages in engagement activities less than approximately 5 times in a 10-day period, their engagement status may be determined to be "low engagement."

[0062] There may be more than two engagement states. In one example, there may be three engagement states: "high engagement," "low engagement," and "no engagement." In this example, the high and low engagement states may be the same as in the example of the two engagement states mentioned above. The difference is that the no engagement state is recorded when the number of engagement activities during a given period is zero. Engagement may have a scale, where the number of engagement activities is recorded and used for analysis and forecasting instead of a discrete number of engagement states. Furthermore, in some embodiments, different weights may be assigned to engagement activities, with some activities counting as higher engagement levels and others as lower engagement levels. The weight of each engagement activity may be considered when determining the engagement state.

[0063] The engagement data received by 504 may also include measurements of user 8's use of the CGM device ("CGM engagement"). For example, if user 8 uses (wears) the CGM device continuously day and night with only minimal intervals between sensor changes, the received engagement data may indicate high CGM engagement. In some embodiments, the state of CGM engagement is determined based on a threshold number of collected CGM measurements. For example, if user 8's CGM device records more than approximately 70% of the CGM measurements that can be obtained during a given period, the CGM engagement state may be classified as "high engagement." If user 8's CGM device records less than approximately 70% of the CGM measurements that can be obtained during a given period, the CGM engagement state may be classified as "low engagement." The number of CGM measurements obtained during a given period is approximately 288 per day (e.g., one measurement approximately every 5 minutes), which may refer to the frequency at which the CGM device is configured to measure blood glucose levels.

[0064] In Figure 5, at 506, the GRI value may be determined based on hypoglycemic and hyperglycemic components according to one or more techniques disclosed herein. The hypoglycemic component may relate to the amount of time that user 8 was hypoglycemic during a given period (e.g., the period during which glucose values ​​and engagement levels are collected in steps 502 and 504, respectively). The hyperglycemic component may relate to the amount of time that user 8 was hyperglycemic during the same period. The GRI zone may also be determined based on the GRI value as well as the hypoglycemic and hyperglycemic components.

[0065] In some embodiments, a time-in-range (TIR) ​​value related to blood glucose measurement is determined. The range-in-range blood glucose value may correspond to the amount of time the blood glucose measurement is within a predetermined range, the ratio of blood glucose measurements within the range to those outside the range, or the number of blood glucose measurements within and outside the range. For example, the TIR value may be based on the case where the blood glucose measurement is between approximately 70 mg / dL and approximately 180 mg / dL. The TIR value can distinguish between the time when user 8's blood glucose value is within the range and the time when it is outside the range. The determined TIR value may be based on the amount of time when user 8's blood glucose value is within the threshold band during a reference period. The reference period may be a single 24-hour day or different reference periods. The reference period may be predetermined (by user 8, healthcare provider 7, pre-programmed, etc.) or may be dynamically determined based on one or more factors. One or more factors may include patient vectors, patient attributes, current or past TIR status, etc.

[0066] According to one embodiment, the TIR value may be relative to a reference period, or it may be a TIR value associated with a patient over multiple reference periods. For example, the TIR value for user 8 may be determined for each day over a total of 10 days. The TIR values ​​for each day over the 10 days are aggregated using any applicable technique (e.g., mean value), and the 10-day TIR value associated with user 8 becomes the aggregated TIR value.

[0067] In Figure 5, at 508, one or more predictions for future glucose values ​​are determined based on the glucose value and engagement data of user 8 received in steps 502 and 504. One or more predictions are based on the glucose value and engagement data of user 8 over a predetermined period (e.g., 10 days), which may be considered the initial period. One or more predictions may be determined using a machine learning model trained to determine predictions for glucose values ​​and / or engagement levels, in accordance with one or more techniques disclosed herein. The predictions may be estimates or calculations based on current or historical values ​​(e.g., values ​​determined during the initial period).

[0068] Predicted values ​​may include quantitative measurements such as blood glucose levels, or qualitative indicators derived from quantitative measurements such as GRI values. Therefore, one or more predictions may include predictions of one or more future GRI values. Future GRI values ​​may be predicted based on GRI values ​​received and / or determined in the present and past. In addition, it may be determined whether future GRI values ​​are greater than or less than the first GRI value determined in step 506. Future GRI zones may also be determined based on future GRI values. Future GRI zones may be compared to currently determined GRI zones to determine whether future GRI zones are the same as or different from currently determined GRI zones.

[0069] In some embodiments, the prediction of user 8's future glucose levels is based on both the user's current and past glucose levels and engagement level. In another embodiment, the prediction of user 8's future glucose levels is based only on the user's current and past glucose levels, without considering the engagement level.

[0070] One or more predictions regarding future glucose values ​​in step 508 may also include predictions regarding one or more future TIR values. The predictions may include whether the future TIR values ​​are greater than or less than currently determined TIR values, depending on a comparison between current / past TIR values ​​and predicted future TIR values. In some embodiments, predictions regarding future TIR values ​​may include whether the future TIR values ​​are within a certain percentage of currently determined TIR values, or greater than or less than a certain percentage. For example, future TIR values ​​may be predicted to be within about 5% of currently determined or past TIR values, greater than about 5% or less than about 5%.

[0071] In Figure 5, at 510, one or more predictions regarding future engagement levels (e.g., future MEDAL levels) are determined based on received engagement levels collected over a predetermined period. According to one embodiment, one or more predictions regarding future engagement levels may be based on received glucose values ​​collected over a predetermined period. One or more predictions regarding future engagement levels may be determined using a trained machine learning model that takes current and / or past engagement levels and outputs predictions regarding future engagement levels. In some embodiments, predictions regarding future engagement levels may be based on current and past values ​​of glucose values ​​and engagement levels, or in some cases, on current and past values ​​of engagement levels only, or on current and past values ​​of glucose values ​​only. Predictions regarding future engagement levels may include predictions of future engagement states. For example, predicted future engagement states may be high engagement, low engagement, no engagement, engagement state values, or hierarchies. Furthermore, predicted future engagement states may be compared to currently received or determined engagement states to determine whether they are different or the same. Predictions regarding future engagement levels may include predictions of how user 8 will interact with and / or engage with mHealth application 1, tracking devices, etc., as described herein. For example, a machine learning model may receive input engagement data (e.g., MEDAL values) that reflect how user 8 has engaged with mHealth application 1 in a particular way, and predict how user 8 will engage with mHealth application 1, tracking devices, etc., for example, how user 8 will engage with specific foods, specific drugs, specific exercise habits, etc.

[0072] Future engagement level predictions may also include one or more predictions regarding future CGM engagement. For example, if User 8's CGM engagement was classified as high engagement during the engagement data collection period, it may be predicted that User 8 will continue to show high engagement or use of CGM devices in the future, or, additionally based on other factors, it may be predicted that User 8 will show low engagement in the future.

[0073] According to one or more embodiments of this disclosure, one or more feature sets may be derived from the user's glucose values ​​and / or engagement data. Details regarding the features derived from the CGM data and engagement data are disclosed herein in relation to Figures 8A–8D. The derived feature sets may be provided as input to a machine learning model to determine predictions about future glucose values ​​and future engagement levels. For example, a machine learning model may be trained to extract features derived based on glucose values ​​and / or engagement data and / or to make feature-based predictions based on the derived features.

[0074] Figure 6 shows a flowchart 600 for training a machine learning model to predict user health and engagement levels according to one or more embodiments. In 602, a first set of glucose values ​​collected by a CGM device over a first period is received. In one or more embodiments, two or more first sets of glucose values ​​are received. For example, a first set of glucose values ​​may be received for multiple users, including user 8. The two or more first sets of glucose values ​​may refer to multiple iterations in which glucose values ​​are collected for the same user (e.g., user 8) over multiple fixed periods. According to one embodiment, the glucose values ​​received in 602 may be simulated (using a simulation model configured to output representative glucose values).

[0075] In 604, a second set of glucose values ​​collected by the CGM device over a second period is received. The second period follows the first period and may be the same length as the first period or of a different length. In one or more embodiments, two or more second sets of glucose values ​​are received, and each glucose value set is associated with the same user (e.g., user 8) or possibly different users. Each first glucose value set and each second glucose value set corresponding to the same user are related to each other, and patterns and relationships between the two may be analyzed and determined.

[0076] According to one embodiment, the CGM device used to collect glucose values ​​during the first period and the CGM device used to collect glucose values ​​during the second period may be the same. According to another embodiment, the CGM device used to collect glucose values ​​during the first period and the CGM device used to collect glucose values ​​during the second period may be different. According to this embodiment, the differences in CGM devices may be taken into consideration when training a machine learning model. For example, the machine learning model may be provided with technical information related to each CGM device (e.g., drift value, calibration index, etc.) and configured to normalize the CGM values ​​output by each CGM device. The first and second glucose value sets may contain the same or nearly the same number of glucose value measurements, but in some cases, the first and second glucose value sets may contain different numbers of glucose value measurements.

[0077] In 606, a first engagement dataset is received. In some embodiments, two or more first engagement datasets are received, each set associated with one user (e.g., user 8). Additional sets may be associated with multiple different users or may be engagement data collected multiple times over multiple fixed periods for the same user. The first engagement dataset is collected by a computing device (e.g., electronic device 19) over a first period. The first period in which the first engagement dataset is collected may correspond to the first period (e.g., initial period) in which the first glucose value set is collected.

[0078] In 608, a second engagement dataset is received. In some embodiments, two or more second engagement datasets are received, each set associated with one user (e.g., user 8). Additional sets may be associated with multiple different users or may be engagement data collected multiple times over multiple fixed periods for the same user. The second engagement dataset is collected by a computing device (e.g., electronic device 19) over a second period. The second period in which the second engagement dataset is collected may correspond to the second period in which the second glucose value set was collected. Both the first and second engagement datasets may be associated with one or more of the user's drug intake, diet, physical activity, test results, and educational activities (e.g., MEDAL activities).

[0079] In step 610, one or more machine learning models (e.g., glucose and engagement machine learning models) are trained based on a machine learning algorithm. The machine learning models may be trained using training data that includes a first glucose value received in step 602, a second glucose value received in step 604, first engagement data received in step 606, and second engagement data received in step 608. According to one embodiment, the first machine learning model may be trained specifically for patients with type 1 diabetes, and the second machine learning model may be trained specifically for patients with type 2 diabetes. In some cases, a single machine learning model may be trained for all diabetic patients, regardless of the type of diabetes.

[0080] The feature set may be derived from a first glucose value set received in step 602 and first engagement data received in step 606. In some cases, the features may also be derived from a second glucose value set received in step 604 and second engagement data received in step 608. The derived features may be included as part of the training data, or they may be determined by one or more machine learning models based on the input training data from steps 602-608.

[0081] In one embodiment, the data received in steps 602–608 may be cleaned up before being input as training data. Data cleanup includes identifying, correcting, or removing errors, inconsistencies, or irrelevant information from the dataset. Missing data may be handled by identifying missing values ​​in the dataset and determining data corrections such as deleting rows or columns with missing values, replacing missing values ​​with estimates, or imputing missing values ​​using machine learning algorithms. Duplicates are removed to avoid bias in results. If data is received from multiple different sources, standardization is performed, including converting the data to a consistent format and / or units. Outliers (e.g., extreme values ​​that significantly impact training) may be corrected (e.g., removed or corrected). The data received in steps 602–608 is inspected for human or technical errors, and any errors identified are corrected. Irrelevant data may be removed to focus training on the most relevant and important data for predicting future health and engagement outcomes.

[0082] As described herein, relevant features are identified and selected. According to one embodiment, relevant features may include those most important for predicting future health and engagement outcomes (e.g., glucose levels and engagement levels). Irrelevant or redundant features may be removed, as they may negatively impact the performance of the machine learning model and increase complexity. The selection of relevant features may include exploratory data analysis, correlation analysis, feature importance ranking, and domain knowledge.

[0083] Exploratory data analysis may involve visualizing training data and analyzing relationships between different variables, and may be used to identify features that are highly correlated with the target variable and those that are weakly correlated. Correlation analysis may involve calculating the correlation coefficient between each feature and the target variable. Features with high correlation coefficients may be more relevant than features with low correlation coefficients. Feature importance ranking may involve ranking the importance of each feature based on its contribution to the model's accuracy, using statistical algorithms such as decision trees or random forests. Domain knowledge may be used to identify relevant features that cannot be captured by statistical methods. For example, in this disclosure, it is known to those skilled in the art that food and drug features are most relevant to glucose value prediction, while comment features (e.g., user input comments to mHealth application 1) and educational activity features are less relevant and may only increase model complexity and decrease usefulness. Once the most relevant features are selected, the training dataset may be transformed to include only the features ranked as most important to the model. This reduces model complexity, decreases the risk of overfitting, and improves the overall accuracy of the model in predictions. In some embodiments, feature selection is performed automatically, and the most relevant features may be selected without manual intervention.

[0084] In one embodiment, applicable features may be identified in such a way that related features are not extracted. Therefore, such features may include not only known related features but also features whose relevance is unknown. One or more machine learning models may be trained to apply all or some of these features based on applicable weights, layers, biases, synapses, training algorithms, etc. In this embodiment, these applicable features may be provided to one or more machine learning models for training or for generating predictive outputs, or one or more machine learning models may determine these applicable features.

[0085] According to embodiments disclosed herein, features may be determined based on any applicable attributes relating to a given data, such as categories, variances, segments, etc. Such attributes may be based on, but are not limited to, time or time range (e.g., time, hours, days, etc.), data type (e.g., CGM data, MEDAL data, TIR data, TBR data, GRI data, glucose management index (GMI) data, etc.), type of analysis relating to the data (e.g., mean, sum, value, etc.), and others.

[0086] The model is trained using a training dataset in accordance with one or more techniques of this disclosure, including those described in relation to Figure 3B. Training involves using statistical algorithms to find patterns and relationships (e.g., associations, correlations, dependencies, connections, similarities, etc.) in the data available for prediction. For example, the model is trained with inputs including first and second glucose value sets and first and second engagement datasets, as described in relation to steps 602–608. The model may be trained using statistical algorithms to determine patterns and relationships between the first glucose value set received in step 602 and the second glucose value set received in step 604. Alternatively, or in addition, the model may be trained to determine patterns and relationships between the first engagement dataset received in step 606 and the second engagement dataset received in step 608.

[0087] In step 612, one or more patterns in the training data are determined. For example, the model may be trained to determine patterns and relationships between a first glucose value and engagement dataset and a second glucose value and engagement dataset. Determining patterns in the training data may include determining the relationship between the first glucose value dataset received in step 602 and the second glucose value dataset received in step 604. Determining patterns may also include determining the relationship between the first engagement dataset received in step 606 and the second engagement dataset received in step 608. Weights, layers, biases, and / or synapses associated with the machine learning model may be updated based on the determined patterns. Furthermore, test data may be provided to the machine learning model, its accuracy determined, and the model may be retrained, updated, or adjusted based on the results of one or more accuracy tests.

[0088] In one or more embodiments disclosed herein, a machine learning model trained based on the method shown in flowchart 600 may be implemented to make one or more predictions about a new unobserved dataset. For example, a third glucose value set may be received and collected by a CGM device over a third period following a second period. A third engagement dataset may also be received and collected by a computing device over a third period. Both the third glucose value set and the third engagement dataset may be associated with the same user, such as user 8. Using the third glucose value set and the third engagement dataset as input, the trained machine learning model may be used to determine one or more predictions about future glucose values ​​or engagement levels (e.g., future "fourth" and subsequent periods).

[0089] Figure 7A shows the prediction of future health and engagement outcomes using glucose ("CGM use") and engagement data ("MEDAL use") according to one or more embodiments. Diagram 700 is a timeline including an "initial phase" period 702 (e.g., initial period, about 10 days) and a "future period" 704. The initial phase period 702 may be a predetermined period during which glucose and engagement data are collected to predict glucose values ​​and engagement levels during the future period 704. In some embodiments, both the initial phase period 702 and the future period 704 may be the same period, but may be of different lengths in some cases. For example, the initial phase period 702 may range from 10 to 30 days, and the future period 704 may range from about 5 to 90 days. The initial phase period 702 may be immediately preceding the future period 704, or there may be a time gap between the initial phase period 702 and the future period 704. Patterns shown during the initial phase period 702 may indicate behavior and results during the future period 704.

[0090] Figure 7B is Table 710, which shows embodiments of future outcome prediction according to one or more embodiments. Column 712 contains questions about how initial data is used to predict different types of future health and engagement outcomes. Column 714 contains outcome variables observed to determine each outcome prediction. Column 716 contains numbers assigned to each question asked and the various scenarios that can be output. Column 718 contains descriptions of the various possible outcome predictions for each observed variable. As shown in Figure 7B, in some embodiments, there may be nine different outcome prediction categories based, for example, on three different possible outcomes (e.g., as shown in the questions in column 712). Outcome predictions may be based on two-state and three-state directional or value predictions based on the relative difference between historical inputs (e.g., based on CGM data, compliance data, MEDAL data, time information, etc.) and predicted outcomes, or on two-state predictions based on value predictions based on metric values ​​(e.g., TIR value, GRI value, MEDAL engagement value, etc.).

[0091] Figure 7C shows examples of MEDAL and CGM data collected for an exemplary user according to one or more embodiments. Each entry on the horizontal axis represents a 5-day period, and each entry on the vertical axis represents a 45-minute interval of a day. For example, at index point 726, values ​​for a past period 722 of 30 days (e.g., three 10-day periods) are input into a machine learning model to determine a prediction for a future period 724 of 30 days. Diagram 720 also shows MEDAL data, including exercise / activity, height, steps, and weight inputs, input into the mHealth application 1. The top 728 of diagram 720 shows TIR, and the dots indicate that the user used (e.g., wore) the CGM device for more than 70% of the day.

[0092] Figures 8A to 8D show feature selection during an initial phase period according to one or more embodiments of the present disclosure. Records may be data extracted based on glucose values ​​(using CGM) and / or engagement data (using MEDAL) 810. Glucose values ​​and / or engagement data 810 may be divided into known and unknown data 808. Records may be aggregated and represented as a single value or record, represented in Figure 8A as the overall record 802, which may represent the average of glucose values ​​and other potential features. Records may be sliced ​​into smaller, isolated sections (e.g., separated by time) and represented by aggregated slices such as MAEN values ​​of the record (e.g., average values ​​for morning, afternoon, evening, and night) 804 and hourly records 806. From each record, one or more features 820 may be extracted, as shown in Figure 8A, such as CGM features, food features, drug features, comment features, etc.

[0093] In some embodiments, a single feature may include multiple attributes, and these attributes themselves may be used as features, as shown in Figures 8B–8D. This feature extension may be useful for improving the accuracy of machine learning models used to predict future health and engagement outcomes. For example, as shown in Figure 8B, metadata related to CGM glucose values ​​and / or mean values ​​may become attributes or sub-features 812 of a single CGM glucose value feature 802A. A single CGM glucose value feature 802A may include several sub-features 812 of the feature, such as meanBGValue (mean blood glucose value over a given period), meanVeryLow (mean percentage of time with very low blood glucose value), meanLow (mean percentage of time with low blood glucose value), meanTIR (mean percentage of time with within range), meanHigh (mean percentage of time with high blood glucose value), meanVeryHigh (mean percentage of time with very high blood glucose value), meanGRI (mean GRI value over a given period), meanTBR (mean percentage of time below range), meanTAR (mean percentage of time above range), meanGMI (mean glucose management index value), and wearableTimeRate (percentage of time the wearable CGM device was worn). In some embodiments, the sub-features are predefined and classified under supervision. A machine learning model may rank each feature and determine which has the greatest impact on the results. In some embodiments, a machine learning model may determine the features and sub-features without manual intervention. For example, a machine learning model might provide predicted glucose values ​​and / or engagement levels based on historical glucose values ​​and / or engagement levels (e.g., those collected during the initial period). The machine learning model might further output a ranking of features (or sub-features) based on their dependence on each feature in determining the predicted glucose values ​​and / or engagement levels. The feature ranking might be provided, for example, as an ordered list via a graphical user interface (GUI). The list might be ordered based on the ranking and updated based on the updated machine learning output.

[0094] According to one embodiment, an instance of the mHealth application 1 for user 8 may be automatically updated based on a ranking of features associated with user 8. For example, the feature ranking may be used to identify one or more features that meet thresholds affecting a favorable predicted glucose value and / or engagement for that user. Therefore, an instance of the mHealth application 1 for user 8 may be automatically updated to generate prompt alerts (e.g., alerts based on medication, diet, activity, etc.) based on one or more features that meet thresholds affecting a favorable predicted glucose value and / or engagement. For example, the mHealth application 1 may be updated to generate alerts and / or notifications, or more frequent alerts and / or notifications, based on one or more features that meet thresholds affecting a favorable predicted glucose value and / or engagement.

[0095] Figure 8C shows a sub-feature 814 that can be extracted from a single exemplary dietary feature 802B. For example, a single meal feature might be a user entry stating that user 8 ate a sandwich. Multiple sub-features can be extracted from a user entry for meal intake. For instance, calorie count and other nutritional indicators (e.g., carbohydrates, fiber, fat, protein, sodium, various types of lipids, vitamins, minerals, sugars, etc.) may be used as features in a machine learning model.

[0096] Figure 8D shows sub-features 816 that can be extracted from a single exemplary drug feature 802C. For example, a single drug feature may include a use entry indicating that user 8 took a particular drug. The user entry may include multiple sub-features, such as average dose, time of administration, type of drug, and prescription class. Similar feature expansion principles can be applied to other features such as educational activities, physical activities, and / or test results.

[0097] Figures 9A–9E are charts showing experimental results of applying machine learning models according to one or more embodiments disclosed herein. Figure 9A shows the results of several different machine learning models trained to predict the difference in TIR. In this example, samples from three 10-day periods [1,2,3] (including a total of over 2880 CGM measurements and at least one engagement activity) were input into various machine learning models. The model that showed the highest accuracy in this experiment is shown in Figure 9A, and several less accurate models are not shown. Table 902 shows the models that predicted three different states of TIR (e.g., TIR worsened by more than about 5% from the input period, within about 5% of the measured TIR, and TIR improved by more than 5%), separated by diabetes type and the number of future periods predicted. For explanation purposes, "DT1" and "DT2" refer to type 1 diabetes and type 2 diabetes, respectively. "AUC" refers to the area under the curve and may indicate the class-discriminating ability of the model. As shown in Table 902, the accuracy in this example experiment ranged from 43% to approximately 55%. Other metrics, recall, precision, and F1 score, are included in Table 902 and all subsequent tables in Figures 9A-9E. Recall is a performance metric used to measure the model's ability to correctly identify all positive instances in the dataset. This is the proportion of true positive instances correctly identified by the model out of all positive instances present in the dataset. Precision is a performance metric used to measure the model's ability to correctly identify positive instances correctly identified by the model out of all instances identified as positive. This is the proportion of true positive instances correctly identified by the model out of all instances identified as positive by the model. The F1 score is a performance metric that combines precision and recall into a single metric of overall model performance. Table 904 shows the model predicting two different states (e.g., better or worse TIR for the measured TIR over the input period). The accuracy in these predictions improved from 58.33% to 64.33%.

[0098] Figure 9B shows the results of the predictive ability of multiple models to predict engagement outcomes. In this example, samples from three 10-day periods [1,2,3] (including at least some records of CGM activity and other engagement (MEDAL) activity) were input into various machine learning models to predict different engagement outcomes for three 10-day periods [7,8,9]. Table 910 shows the models that predicted future CGM engagement (e.g., high or low engagement) with accuracy of 76.25% for type 1 diabetes and 79.29% for type 2 diabetes. Table 912 shows the models that predicted future engagement (e.g., high or low engagement) in a mobile application (e.g., mHealth application 1) with accuracy of approximately 90% for both type 1 and type 2 diabetes.

[0099] Figure 9C shows the results of the predictive ability of multiple models to predict health outcomes (e.g., TIR values ​​and their changes). In this example, samples from three 10-day periods [1,2,3] (including a total of over 2880 CGM measurements and possibly at least one engagement activity) were input into various machine learning models. As shown in Table 920, including engagement data (MEDAL data) improved the accuracy of the model output in 3-state TIR prediction from 47.58% to 51.59%. As shown in Table 922, including engagement data (MEDAL data) improved the accuracy of the model output in 2-state TIR prediction from 61.21% to 62.95%.

[0100] Figure 9D shows the results of the predictive ability of multiple models to predict health outcomes (TIR values ​​and their changes). The main difference between the examples in Figure 9C and Figure 9D is that the test input in Figure 9D includes more than five engagement activities, indicating a high level of engagement with the mobile application (e.g., mHealth application 1). Table 930 shows that when user 8 has a high initial engagement status with mHealth application 1 and high CGM engagement, the prediction accuracy in 3-state TIR prediction improves by more than approximately 68%. Table 932 shows that when user 8 has a high initial engagement status with mHealth application 1 and high CGM engagement, the prediction accuracy in 2-state TIR prediction improves by more than approximately 61%.

[0101] Figure 9E shows further results of the predictive ability of multiple models to engage in engagement (CGM engagement and mHealth application 1 engagement). Unlike the experiment shown in Figure 9B, this experiment includes more than five engagement activities in the test input in Figure 9E. Table 940 shows that the model accuracy for predicting two-state CGM engagement is approximately 78%, and Table 942 shows that the model accuracy for predicting two-state engagement with mHealth application 1 is approximately 68-69%.

[0102] Figures 9F and 9G show further results for the model's proficiency (accuracy, AUC, precision, recall, and F1) in predicting TIR values ​​based on an initial dataset (e.g., CGM data, compliance data, MEDAL data, etc.). Table 950 in Figure 9F shows the model's proficiency in determining whether future TIR values ​​will be in one of three states: worse (e.g., more than 5% lower than the current value), the same (e.g., within 5% of the current value), or improved (e.g., more than 5% higher than the current value). Table 952 in Figure 9G shows the model's proficiency in determining whether future TIR values ​​will be in one of two states: worse than current / past TIR values, or better than current / past TIR values.

[0103] Figure 9H is a diagram 954 showing Shapley additive explanation (SHAP) values ​​and feature values ​​for a set of features and / or sub-features used in a model to predict differences in TIR values, according to one or more embodiments. For example, the horizontal axis of diagram 954 shows SHAP values, which represent the value of the impact a given feature or sub-feature had on the model output. Each point in diagram 954 represents an individual (e.g., a user). The pattern of each point represents a feature value, with the first pattern indicating a small feature value, the second pattern indicating a medium feature value, and the third pattern indicating a large feature value. In the example of diagram 954, the features that had the greatest impact on the model output were related to CGM values. For example, if there are many high CGM glucose values ​​(e.g., above 180 mg / dL), the user is likely to have a better TIR value in the future compared to their current or past TIR value. As another example, if the historical data shows relatively low CGM glucose values ​​(e.g., below 54 mg / dL), the user is likely to have a worse TIR value in the future.

[0104] Figure 9I shows the results of the model's ability to predict absolute (e.g., not relative) TIR values ​​in the future. Table 956 in Figure 9I shows the model's proficiency in determining whether future TIR values ​​will be in one of two states: above the target TIR value (e.g., approximately 70% TIR) or below the target TIR value.

[0105] Figure 9J is a diagram 958 showing SHAP values ​​and feature values ​​for a set of features and / or sub-features used in a model to predict TIR values ​​according to one or more embodiments. For example, a high past TIR value 959A and high past exercise time 959E may have a positive impact on future period TIR values. However, a high past TIR value 959B, male status 959C, a high past GRI value 959D, a history of heavy sleep 959F, and type 1 diabetes status 959G may have a negative impact on future period TIR values.

[0106] Figures 9K and 9L show the results of the model's ability to predict the difference between current / past GRI values ​​and future GRI values. Table 960 in Figure 9K shows the model's proficiency in determining whether future GRI values ​​will be in one of two states (worse than current / past TIR values, or better than current / past TIR values). Table 962 in Figure 9L shows the model's proficiency in determining whether future GRI values ​​will be in one of three states (worsening (e.g., more than 5% lower than current values), similar (e.g., within 5% of current values), or improving (e.g., more than 5% higher than current values)).

[0107] Figure 9M shows the results of the model's ability to predict absolute (e.g., not relative) GRI values ​​in the future. Table 964 in Figure 9M shows the model's proficiency in determining whether future GRI values ​​will be in one of two states: above the target GRI value (e.g., a GRI value of approximately 40) or below the target GRI value.

[0108] Figure 9N shows the results of the model's ability to predict engagement with mHealth application 1 over a future period. Table 966 shows the model's proficiency in determining whether a user's future engagement with mHealth application 1 will fall into one of two states: high engagement (e.g., approximately 5 or more engagement activities) or low engagement (e.g., less than approximately 5 engagement activities).

[0109] Figure 90 is a diagram 968 showing SHAP values ​​and feature values ​​for a set of features and / or sub-features used in a model to predict engagement levels, according to one or more embodiments. For example, steps, sleep volume, and exercise have been found to influence the accurate determination of future engagement levels. A high step count and a high number of steps may have a positive impact on morning engagement, and wearing wearables more during the morning hours may have a positive impact on overall engagement.

[0110] Figure 9P shows the results of the model's ability to predict manual engagement with mHealth Application 1 over a future period. Manual engagement generally refers to engagement activities in mHealth Application 1 that require manual user input. Therefore, since many engagement activities are automatically measured by the wearable, activity engagements may be excluded when determining manual engagement. Table 970 shows the model's proficiency in determining whether a user's future manual engagement with mHealth Application 1 will fall into one of two states: high manual engagement (e.g., approximately 5 or more engagement activities including drug intake, educational activities, meals, and test results) or low manual engagement (e.g., fewer than 5 engagement activities including drug intake, education, meals, and test result activities).

[0111] Figure 9Q is a diagram 972 showing SHAP values ​​and feature values ​​for a set of features and / or sub-features used in the model to predict manual engagement levels, according to one or more embodiments. For example, as shown in diagram 972, medication management, weight input, and manually entered sleep records have been found to influence the accurate determination of future manual engagement levels.

[0112] Figure 9R shows the results of the model's ability to predict CGM engagement over a future period. Table 974 in Figure 9R shows the model's proficiency in determining whether future CGM engagement levels will fall into one of two states: above the target CGM device wear time (e.g., approximately 70% of the total 2880 measurements possible in a week) or below the target CGM device wear time.

[0113] Figure 9S is a diagram 976 showing SHAP values ​​and feature values ​​for a set of features and / or sub-features used in a model to predict CGM engagement levels according to one or more embodiments. For example, as shown in diagram 976, in M11 (e.g., November), a high number of nighttime CGM records, as well as a high number of evening, afternoon, morning, after breakfast, after lunch, and after dinner CGM records, may have a negative impact on CGM engagement. However, in M7 (July), a high number of CGM records may have a positive impact on CGM engagement. A high number of nighttime records may generally have a positive impact on CGM engagement, and individuals classified as male may show a positive impact in future periods.

[0114] Figure 9T shows the results of the model's ability to predict future CGM engagement based on historical engagement data. Table 978 in Figure 9T shows the model's proficiency in determining whether future CGM engagement levels, determined using engagement data as input, will fall into one of two states: above the target CGM device wear time (e.g., approximately 70% of the total 2880 measurements possible in a week) or below the target CGM device wear time.

[0115] Figure 9U is a diagram showing SHAP values ​​and feature values ​​for a set of features and / or sub-features used in a model to predict CGM engagement levels, according to one or more embodiments. For example, male status may have a negative relationship with CGM engagement, while patients over 65 years of age and with type 1 diabetes status may have a positive relationship with CGM engagement. Individuals with a high exercise record may have a positive relationship with CGM engagement. Other features that influence the model output include exercise distance, evening exercise, and exercise time, but demographic features tend to have a greater influence on the output.

[0116] Figure 9V shows a confusion matrix 982 based on predicted and actual CGM engagement levels according to one or more embodiments. As shown in the confusion matrix 982 of the random forest classifier, the model tends to make more positive predictions about the likelihood of users engaging with CGM devices in the future than the actual number of users who will engage with CGM devices.

[0117] Figures 9W and 9X are example plots showing the AUC of different models according to one or more embodiments. For example, plot 984 in Figure 9W shows the AUC of a random forest classifier model, and plot 986 in Figure 9X shows the AUC of an optical gradient boosting machine (LGBM) model. Both models have relatively similar precision and recall values, but the random forest classifier has a higher AUC, indicating higher class-specific ability and may be used as a summary of the receiver operating characteristic (ROC) curve.

[0118] In some embodiments, a TIR value of approximately 0.7 or higher can be a clinically significant target for optimal diabetes management. Therefore, the binary outcome variable of TIR can be set to a first state where TIR is approximately 0.7 or higher, and a second state where TIR is less than approximately 0.7. Various types of models, including, for example, LGBM, random forest classifiers, quadratic discriminant analysis, naive Bayes, and / or logistic regression models, can be implemented to predict whether the TIR for the prediction period will be above approximately 0.7. In experiments between different implemented models (e.g., five different models), test results from 304 individuals showed that the LGBM model had a predictive precision of 0.80 and an AUC of 0.88. The random forest classifier showed model performance with a precision of 0.77 and an AUC of 0.82. The quadratic discriminant analysis model had a recall score of 0.94. Regarding feature importance, it was shown that a high baseline TIR and longer exercise time increased the probability of a future TIR exceeding 0.7. High baseline TAR, male gender, and type 1 diabetes negatively impacted TIR, specifically reducing the probability of a TIR exceeding 0.7. Various model output statistics demonstrate that decision tree-based models, such as LGBM and random forest classifiers, are well-suited for predicting health outcome variables like TIR, such as being above or below clinically significant thresholds. Feature importance analysis can help understand which baseline CGM and MEDAL features are important and how they impact future health outcomes. These features can be used to design meaningful, individualized interventions for population and individual health management.

[0119] Figures 10A to 10D show exemplary plots of the time course of average GRI values ​​for various individual groups according to one or more embodiments. Figure 10A shows exemplary plot 1000 of the time course of average GRI values ​​for individual groups according to one or more embodiments. The GRI values ​​of CGM users are shown on the vertical axis. The number of days for which GRI values ​​were determined for CGM users is shown on the horizontal axis. Each line in exemplary plot 1000 represents the average GRI value for individual groups that started in one of five zones. As shown in the example in plot 1000, individuals whose baseline GRI values ​​started in zones A and B generally ended with higher GRI values ​​but remained in the same zones (zones A and B), respectively. However, individuals whose baseline GRI values ​​started in zones C, D, and E, respectively, generally showed improvement in GRI values, with each individual group improving to a higher zone at the end of 14 days. In other words, individuals who started in Zone C generally finished in Zone B, individuals who started in Zone D generally finished in Zone C, and individuals who started in Zone E generally finished in Zone D. In some cases, one or more improvements were the result of one or more of the technologies disclosed herein.

[0120] Figure 10B shows another exemplary plot 1010 of the time course of the mean GRI values ​​of individual groups according to one or more embodiments. Exemplary plot 1010 displays three different individual groups grouped by age. The top line 1012 represents individuals aged 18–39 years. The middle line 1014 represents individuals aged 40–64 years. The bottom line 1016 represents individuals aged 65 years and older. The mean GRI values ​​of individuals aged 65 years and older remained in zone B, while the mean GRI values ​​of individuals aged 18–64 remained in zone C. As the data obtained in plot 1010 shows, GRI values ​​tend to decrease with age.

[0121] Figure 10C shows another exemplary plot 1020 of the time course of the mean GRI values ​​of individual groups according to one or more embodiments. The exemplary plot 1020 displays two different groups of individuals grouped by sex. The upper line 1022 represents men and the lower line 1024 represents women. As the data obtained in plot 1020 shows, GRI values ​​were lower for women than for men.

[0122] Figure 10D shows another exemplary plot 1030 of the time course of mean GRI values ​​for individual groups according to one or more embodiments. Exemplary plot 1030 displays two different groups of individuals grouped by type of diabetes. Line 1032 represents individuals with type 2 diabetes, and line 1034 represents individuals with type 1 diabetes. As the data obtained in plot 1030 shows, there was no significant difference in GRI values ​​between types of diabetes.

[0123] Figures 11A to 11C show the relationships between precision, recall, specificity, and sensitivity according to one or more embodiments. Diagram 1102 in Figure 11A shows that sets of positive and negative values ​​can be classified and predicted based on their values.

[0124] Figure 11B is Chart 1104, which shows the classification results shown in Diagram 1102 of Figure 11A. The results shown in Chart 1104 indicate that there were two negative values ​​that were incorrectly predicted, one positive. Figure 11C shows Equation 1106, which shows how precision, recall, specificity, and sensitivity are determined. In this example, recall may be the same as sensitivity, but precision is not the same as specificity.

[0125] Figure 12 shows a high-level functional block diagram of an exemplary computer device or system in which embodiments or parts thereof of the present disclosure may be implemented, for example, as computer-readable code. Additionally, each of the exemplary computer servers, databases, user interfaces, modules, and methods described above with respect to Figures 1 to 11C may be implemented in the device 1200 and in one or more computer systems or other processing systems using hardware, software, firmware, tangible computer-readable media storing instructions, or a combination thereof. Hardware, software, or any combination thereof may implement each of the exemplary systems, user interfaces, and methods described above with respect to Figures 1 to 11C.

[0126] Where programmable logic is used, that logic may be executed on a commercially available processing platform or application-specific device. Those skilled in the art will understand that the embodiments of the subject of the disclosure can be implemented in a variety of computer system configurations, including multicore multiprocessor systems, minicomputers, mainframe computers, connected or clustered computers with distributed capabilities, and popular or small computers that can be incorporated into virtually any device.

[0127] For example, at least one processor device and memory may be used to implement the above embodiment. The processor device may be a single processor, multiple processors, or a combination thereof. The processor device may have one or more processor "cores".

[0128] The various embodiments of this disclosure described above in the examples in Figures 1 to 11C can be implemented using the apparatus 1200. After reading this description, it will be apparent to those skilled in the art how embodiments of this disclosure can be implemented using other computer systems and / or computer architectures. While operations may be described as sequential processes, in practice some operations may be performed in parallel, concurrent, and / or distributed environments, and program code may be stored locally or remotely and accessed by single or multiprocessor machines. In addition, in some embodiments, the order of operations may be rearranged without departing from the spirit of the disclosed subject matter.

[0129] For example, a device 1200, such as a computer or server, may have a data communication interface port 1260 for packet data communication. The device may also have one or more processor-based central processing units (CPUs) 1220 for executing program instructions. The platform typically includes an internal communication bus 1210, program storage, and data storage (e.g., ROM 1230 and RAM 1240, etc.) for various data files processed and / or communicated by the platform. The hardware elements, operating systems, and programming languages ​​of such a device are prior art and are presumably well-known to those skilled in the art. The device 1200 may also have input / output ports 1250 and communication ports 1260 for connecting to input and output devices such as keyboards, mice, touchscreens, monitors, and displays. Of course, various server functions may be implemented in a distributed manner across multiple similar platforms to distribute the processing load. Alternatively, a server may be implemented by appropriate programming on a single computer hardware platform.

[0130] It will be apparent to those skilled in the art that this disclosure can be implemented in many different examples of software, hardware, firmware, and / or entities shown in the drawings, as described herein. Actual software code, with specific hardware controls for executing the embodiments, is not limited to the detailed description. Given the level of detail described herein, the embodiments are described herein with the understanding that modifications and variations are possible. The embodiments of the subject matter described may typically be considered as “products” or “manufactured goods” carried on or embodied on some kind of machine-readable medium, in the form of executable code and / or associated data. “Storage” media include any or all of tangible memory such as computers and processors, or their associated modules (various semiconductor memories, tape drives, disk drives, etc.), which may provide non-temporary storage for software programming from time to time. The software, in whole or in part, may be communicated from time to time via the Internet or various other communication networks. For example, such communications may enable the loading of software from one computer or processor to another, for instance, from a management server or host computer in a mobile communication network to a server's computer platform, and / or from a server to a mobile device. Therefore, other types of media carrying software elements may include light waves, radio waves, and electromagnetic waves, such as those used via physical interfaces between local devices, wired and optical fixed-line networks, and various wireless links. Physical elements carrying such waves (wired or wireless links, optical links, etc.) may also be considered media carrying software. As used herein, computer or machine-readable media, etc., refer to any medium involved in providing instructions to a processor for execution, unless limited to non-temporary tangible "storage" media.

[0131] It should be understood that the above general description and subsequent detailed descriptions are merely illustrative and explanatory, and do not limit the claimed disclosures.

[0132] Other examples of the present disclosure will be apparent to those skilled in the art in consideration of the specifications and practices of the invention disclosed herein. The specifications and examples are illustrative only, and the true scope and spirit of the invention are indicated by the following claims.

Claims

1. A computer implementation method for predicting user health and engagement levels, Receiving the user's glucose values ​​collected by a continuous glucose monitoring (CGM) device over a predetermined period, Receiving engagement data associated with the user, collected by a computing device over the predetermined period, wherein the engagement data relates to the user's medication activities, eating activities, physical activities, test results, or educational activities. Based on the first amount of time during which the user was in a hypoglycemic state and the second amount of time during which the user was in a hyperglycemic state during the predetermined period, a first glycemic risk index (GRI) value is determined. Using a machine learning model and in accordance with the user's glucose values ​​and engagement data collected over a predetermined period, one or more predictions regarding the user's future glucose values ​​are determined, wherein the predictions include a prediction that the future GRI value is greater than or less than the first GRI value. Using the aforementioned machine learning model, and in accordance with the user engagement data collected over the predetermined period, one or more predictions regarding future engagement levels are determined. Methods that include...

2. The computer implementation method according to claim 1, further comprising determining a time-in-range (TIR) ​​value of the user's glucose value, wherein the determined TIR value is based on the amount of time over a predetermined period during which the user's glucose value is within a threshold band, and one or more predictions regarding the user's future glucose value include a prediction that the future TIR value will be within the threshold of the determined TIR value, greater than the threshold, or less than the threshold.

3. The computer implementation method according to claim 1, further comprising determining a time-in-range (TIR) ​​value of the user's glucose value, wherein the TIR value is based on the amount of time over a predetermined period during which the user's glucose value is within a threshold band, and one or more predictions regarding the user's future glucose value further include predictions that the future TIR value is greater than or less than the TIR value.

4. The computer implementation method according to claim 2, wherein one or more predictions of the user regarding the future glucose value include a prediction that the future TIR value is greater than or less than a threshold TIR value.

5. Deriving a feature set from the user's glucose value and the engagement data, The feature set is provided as input to the machine learning model to determine one or more predictions regarding the user's future glucose levels and one or more predictions regarding their future engagement levels. The computer implementation method according to claim 1, further comprising:

6. The method further includes determining a first GRI zone based on the first GRI value, The computer implementation method according to claim 1, wherein one or more predictions of the user regarding the future glucose value include a prediction that the future GRI value is in a GRI zone higher than the first GRI zone or in a GRI zone lower than the first GRI zone.

7. The computer implementation method according to claim 6, wherein one or more predictions of the user's future glucose value include a prediction that the future GRI value is in the first GRI zone, in a second GRI zone higher than the first GRI zone, or in a third GRI zone lower than the first GRI zone.

8. The computer implementation method according to claim 1, wherein one or more predictions of the user regarding the future glucose value include a prediction that the future GRI value is greater than or less than the threshold GRI value.

9. The computer implementation method according to claim 1, wherein one or more predictions regarding the future engagement level include a prediction that the future engagement level will be high-engagement or low-engagement.

10. The computer implementation method according to claim 1, wherein one or more predictions regarding the future engagement level include a prediction that the future manual engagement level is a first manual engagement state or a second manual engagement state.

11. The computer implementation method according to claim 1, wherein one or more predictions regarding the future engagement level include a prediction that the future CGM device engagement level will exceed or fall below a threshold amount for CGM device engagement, and CGM device engagement includes measuring the use of the CGM device by the user.

12. The computer implementation method according to claim 1, wherein at least a portion of the engagement data is collected through user input from one or more applications on the computing device.

13. The computer implementation method according to claim 1, wherein at least a portion of the engagement data is collected using one or more sensors associated with the user, and the one or more sensors include at least one of a weighing scale, a blood pressure monitor, an activity tracker, a heart rate monitor, a multipurpose wearable device, a blood glucose monitor, and a ketone tracking device.

14. A computer implementation method for training a machine learning model to predict user health and engagement levels, The first glucose value of the user collected by a continuous glucose monitoring (CGM) device over a first period, Receiving the user's second glucose value collected by the CGM device over a second period following the first period, Receiving first engagement data associated with the user, collected by a computing device over the first period, Receiving second engagement data associated with the user, collected by the computing device over the second period, wherein the first and second engagement data relate to one or more of the user's drug use, diet, physical activity, test results, and educational activities. The process involves training a machine learning model based on a machine learning algorithm, using training data that includes the first glucose value, the second glucose value, the first engagement data, and the second engagement data, and generating a trained machine learning model. To determine one or more patterns in the aforementioned training data. Methods that include...

15. The method further includes deriving one or more feature sets from the first glucose value and the first engagement data. The computer implementation method according to claim 14, wherein the training data includes the derived one or more feature sets.

16. The computer implementation method according to claim 15, further comprising extracting one or more partial features from each of the one or more feature sets.

17. Determining one or more patterns in the training data is: The relationship between the first glucose value and the second glucose value is determined, To determine the relationship between the first engagement data and the second engagement data. The computer implementation method according to claim 14, including the method described in claim 14.

18. Training the aforementioned machine learning model is The computer implementation method according to claim 14, further comprising updating the weights, layers, biases, or synapses of the machine learning model based on the determined pattern to generate the trained machine learning model.

19. Receiving the user's third glucose value collected by the CGM device over a third period following the second period, Receiving third engagement data associated with the user, collected by the computing device over the third period, wherein the third engagement data relates to the user's drug intake, diet, physical activity, test results, and educational activities. Using the trained machine learning model, and in accordance with the third glucose value and third engagement data collected over the third period, one or more predictions regarding the user's future glucose value are determined. Using the aforementioned trained machine learning model, and in accordance with the third engagement data of the user collected over the third period, one or more predictions regarding future engagement levels are made. The computer implementation method according to claim 14, further comprising:

20. A system for predicting future glucose levels and engagement, Memory containing processor-readable instructions, A processor configured to access the memory and execute processor-readable instructions, wherein the instructions are configured to cause the processor to perform the following actions when executed by the processor: Equipped with, The aforementioned method, The continuous glucose monitoring (CGM) device, configured to acquire glucose values ​​using components that penetrate the user's skin, receives the user's glucose values ​​over a predetermined period of time, Receiving engagement data associated with the user, collected by one or more electronic sensors via a computing device over the predetermined period, wherein the engagement data relates to one or more of the user's drug intake, diet, physical activity, test results, and educational activities. Based on the first amount of time during which the user was in a hypoglycemic state and the second amount of time during which the user was in a hyperglycemic state during the predetermined period, a first glycemic risk index (GRI) value is determined. Using a machine learning model and in accordance with the user's glucose values ​​and engagement data collected over a predetermined period, one or more predictions regarding the user's future glucose values ​​are determined, wherein the predictions include a prediction that the future GRI value is greater than or less than the first GRI value. Using the aforementioned machine learning model, and in accordance with the user engagement data collected over the predetermined period, one or more predictions regarding future engagement levels are determined. A system that includes this.

Images