Disease prediction using analyte measurement features and machine learning

A wearable analyte monitoring device with machine learning models addresses the inaccuracies and invasiveness of conventional diabetes tests by selecting robust features to predict health status accurately, enhancing early detection and compliance.

JP2024541790APending Publication Date: 2024-11-13DEXCOM INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024504558
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-10-27
Filing Date
2022-10-21
Publication Date
2024-11-13

AI Technical Summary

Technical Problem

Conventional diabetes tests are inaccurate, inconsistent, and cumbersome, often leading to incorrect diagnoses due to fluctuations in glucose levels and requiring invasive procedures, which can deter individuals from getting tested, especially pregnant women.

Method used

A wearable analyte monitoring device combined with machine learning models selects robust features from analyte measurements to predict health status classifications, minimizing sensor manufacturing variations and providing accurate health status predictions without invasive procedures.

Benefits of technology

The system enhances prediction accuracy by reducing inconsistencies and invasiveness, enabling early detection of diabetes and associated conditions, improving patient compliance and healthcare outcomes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024541790000001_ABST
    Figure 2024541790000001_ABST
Patent Text Reader

Abstract

Disease prediction using analyte measurements and machine learning is described. In one or more implementations, a combination of analyte measurement features may be selected from a plurality of analyte measurement features based on a robustness metric and a performance metric of the combination, and a machine learning model may be trained to predict a health condition classification using the combination. The performance metric may be related to the accuracy rate of predicting the health condition classification, and the robustness metric may be related to the insensitivity to analyte sensor manufacturing variations in the accuracy rate. Once trained, the machine learning model predicts a health condition classification for a user based on the user's analyte measurements collected by a wearable analyte monitoring device. A combination of features may be extracted from the user's analyte measurements and input into the machine learning model to predict the classification.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 63 / 263,106, entitled "Generalized Diagnostic Continuous Glucose Monitor," filed on October 27, 2021, the entire disclosure of which is incorporated herein by reference. [Background technology]

[0002] Several health conditions (e.g., medical conditions) produce or alter the levels of analytes in blood and / or interstitial fluid. As an example, diabetes is a metabolic condition that affects hundreds of millions of people that results in elevated blood glucose levels. Diabetes is one of the leading causes of death worldwide, but with early detection and proper treatment, damage to the heart, blood vessels, eyes, kidneys, and nerves caused by diabetes can be largely avoided.

[0003] Traditional tests for diabetes that are accepted by clinical and regulatory bodies include Hemoglobin A1c (HbA1c), Fasting Plasma Glucose (FPG), and 2-Hour Plasma Glucose (2Hr-PG). Both FPG and 2Hr-PG are part of the Oral Glucose Tolerance Test (OGTT), but FPG can be tested separately from the OGTT. In FPG testing, a blood sample is taken and the results are used to classify the person as "normal" (e.g., non-diabetic), having pre-diabetes, or having diabetes. Generally, a person is considered normal if their fasting blood glucose level is less than 100 milligrams per deciliter (mg / dL), whereas a person is classified as having prediabetes if their fasting blood glucose level is between 100 and 125 mg / dL, or as having diabetes if their fasting blood glucose level is greater than 126 mg / dL, in two separate tests.

[0004] After measuring a person's fasting blood glucose for the FPG test, the OGTT then requires the person to drink a sugar drink to cause a spike in the person's blood glucose level. Many people, especially pregnant women, have difficulty tolerating this sugar drink. The person's blood glucose level is then tested periodically using additional blood samples during the next two hours of the 2Hr-PG. Blood glucose levels below 140 mg / dL are considered "normal," whereas a reading greater than 200 mg / dL two hours after drinking the sugar drink indicates diabetes. A reading between 140 and 199 mg / dL indicates prediabetes.

[0005] Unlike the OGTT's FPG and 2Hr-PG tests, which each measure a person's blood glucose level at a single point in time, the HbA1c test measures a user's average blood glucose level over the preceding 2-3 months. However, the HbA1c test does not measure glucose directly, but rather measures the percentage of glucose attached to hemoglobin. When glucose accumulates in a person's blood, it attaches to hemoglobin, the oxygen-carrying protein in red blood cells. Red blood cells live in humans for approximately 2-3 months, and therefore the HbA1c test shows the average level of blood glucose over the preceding 2-3 months. As with the FPG and 2Hr-PG tests, a blood sample is taken from the person and used to measure the person's HbA1c level. However, unlike the FPG and 2Hr-PG tests, the person does not need to be in a fasting state when the HbA1c test is performed. An HbA1c level of 6.5 percent or higher on two separate tests indicates that a person has diabetes, while an HbA1c level of 5.7 to 6.4 percent generally indicates that a person has prediabetes. HbA1c levels below 5.7 percent are considered normal.

[0006] Each of these conventional tests performed to screen (or diagnose) diabetes has various drawbacks that often lead to an improper diagnosis. Conventional diabetes tests are often inaccurate because a given test performed on an individual on different days may result in inconsistent diagnoses due to various external factors that cause glucose levels to fluctuate, such as illness, stress, increased exercise, or pregnancy. In contrast, even though the HbA1c test measures the average glucose level over the previous 2-3 months, the HbA1c test result is heavily influenced by the user's blood glucose level in the weeks leading up to the test. Thus, the HbA1c test result may be heavily influenced by changes in blood characteristics during the 3-month period, such as due to pregnancy or illness. In addition, because the HbA1c test is not a direct measure of blood glucose, such tests may be inaccurate for people with various blood conditions, such as anemia, or for people with rare forms of hemoglobin.

[0007] In addition, such conventional tests often have poor agreement. In other words, these tests do not necessarily detect diabetes in the same individual. The lack of consistency between test types can lead to an inaccurate diagnosis or an inability to determine an appropriate treatment plan. For example, a user may have high fasting glucose but an HbA1c score within the normal range. In such a scenario, different physicians may reach different conclusions regarding whether the user has diabetes and the type of treatment plan for the user.

[0008] Finally, there are also various limitations and drawbacks to administering these tests to different people, such as pregnant women. For example, these conventional diabetes tests require the user to visit a clinic or laboratory to take a blood sample, which can be time-consuming, expensive, and painful for some users. Each of these factors, alone or in combination, can act to create psychological barriers that prevent users from getting tested for diabetes, thereby reducing the benefits associated with early detection. Moreover, many of these conventional tests require the user to be in a fasting state, which can be difficult or even dangerous for some users, including pregnant women. Summary of the Invention [Means for solving the problem]

[0009] To overcome these problems, health condition prediction, including diabetes prediction, using robustly accurate features extracted from analyte measurements and machine learning is utilized. In one or more implementations, a combination of analyte measurement features may be selected from a plurality of analyte measurement features based on a robustness metric and a performance metric of the combination, and a machine learning model may be trained to predict a health condition classification using the combination. The performance metric may be related to the accuracy rate of predicting the health condition classification, and the robustness metric may be related to the insensitivity to analyte sensor manufacturing variability in terms of the accuracy rate. Once trained, the machine learning model predicts a user's health condition classification based on the user's analyte measurements collected by a wearable analyte monitoring device. A combination of features may be extracted from the user's analyte measurements and input into the machine learning model to predict the classification.

[0010] This Summary introduces a selection of concepts in a simplified form that are further described below in the Detailed Description. As such, this Summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. [Brief description of the drawings]

[0011] The detailed description is given with reference to the accompanying drawings. [Figure 1] 1 is an illustration of an environment in one example implementation operable to employ the techniques described herein. [Diagram 2] 1 illustrates an example of a wearable analyte monitoring device in more detail. [Diagram 3] 1 illustrates an example of an implementation in which analyte data is routed to different systems in relation to health condition classification. [Figure 4] An example of an implementation of the predictive system is shown in more detail, in which a combination of extracted analyte features is selected that is both robust to analyte sensor variation and accurate for predicting health condition classification. [Diagram 5] 1 shows exemplary graphs illustrating analyte feature categorization and selection based on performance and robustness metrics. [Figure 6] An example of an implementation of the predictive system is shown in more detail, in which a machine learning model is trained to predict a health condition classification using a selected combination of extracted analyte features. [Figure 7] An example of an implementation of the prediction system is shown in more detail, in which a health condition classification is predicted using machine learning and a selected combination of extracted analyte features. [Figure 8] 1 illustrates an example implementation of a user interface that is displayed to inform a user regarding a health status prediction generated based on analyte measurements collected during an observation period. [Figure 9]1 illustrates an example implementation of a user interface that may be displayed to report a user's health status prediction along with other information generated in connection with the health status prediction. [Figure 10] 1 illustrates an example implementation of a user interface that is displayed to collect additional data that may be used as input to a machine learning model to generate a health state prediction. [Figure 11] 1 illustrates a procedure in an example implementation in which analyte features for robustly predicting health condition classifications are learned based on historical analyte measurement and outcome data for a user population. [Figure 12] 1 illustrates a procedure in an example implementation in which a machine learning model is trained to predict a health condition classification based on historical analyte measurement and outcome data for a user population. [Figure 13] 1 illustrates a procedure in an example implementation in which a machine learning model predicts a health condition classification based on a user's analyte measurements collected by a wearable analyte monitoring device over an observation period. [Figure 14] 1 shows an exemplary plot of percent time above a given glucose level versus interquartile range for participants with different diabetes classifications. [Figure 15] 1 illustrates an example of a system including various components of an example of a device that may be implemented as any type of computing device described and / or utilized with reference to FIGS. 1-13 for implementing embodiments of the techniques described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] overview As mentioned above, conventional tests for diabetes that are accepted by clinical and regulatory bodies include hemoglobin A1c (HbA1c), fasting plasma glucose (FPG), and 2-hour plasma glucose (2Hr-PG). Both FPG and 2Hr-PG can be part of an oral glucose tolerance test (OGTT), or FPG can be tested separately from OGTT. However, such conventional tests often have poor agreement in detecting diabetes in the same individual, or may produce different results on different days due to various factors that affect blood glucose levels. There are also various limitations and drawbacks in administering these tests to different people, such as pregnant women.

[0013] Thus, machine learning may be leveraged for health condition prediction, such as diabetes prediction. Health condition prediction may be performed by processing analyte measurements (e.g., glucose measurements) obtained by an analyte monitoring device through one or more machine learning models (e.g., regression models, neural networks, reinforcement learning agents). For example, the analyte monitoring device may include a needle configured to be inserted through a user's skin to contact blood and / or interstitial fluid and measure, via a sensor in the monitoring device, one or more analytes relevant to identifying a health condition over an observation period. The one or more machine learning models are generated using the historical analyte measurements and the historical outcome data of a user population to predict a health condition classification (e.g., a diabetes classification) for an individual user. The historical analyte measurements of the user population may be provided by analyte monitoring devices worn by users of the user population. Furthermore, the historical outcome data used for training may vary depending on the classification the machine learning model is configured to output. Generally, the historical outcome data includes one or more diagnostic measures obtained from a source independent of the analyte monitoring device. For example, the historical outcome data may indicate whether each user of the user population has been clinically diagnosed with diabetes based on one or more diagnostic measures such as HbA1c, FPG, or 2Hr-PG (or OGTT as a combination of FPG and 2Hr-PG), etc. Thus, the historical outcome data may indicate a clinically determined health status classification.

[0014] Unlike conventional tests that are traditionally administered in a laboratory or clinic, the use of wearable analyte monitoring devices allows analyte measurements to be collected remotely. For example, the analyte monitoring device may be mailed or otherwise provided to a user, e.g., by an analyte monitoring device provider, a pharmacy, a medical testing facility, a telemedicine service, etc. The user may then wear the analyte monitoring device over the course of an observation period, such as by wearing the device continuously at home and / or at work.

[0015] A user may insert a sensor of a wearable analyte monitoring device into the user's body, such as by using an automatic sensor applier. Unlike blood draws required by conventional tests such as HbA1c, FPG, and 2Hr-PG, user-initiated application of an analyte monitoring device is largely painless and does not utilize blood draws, consumption of sugary drinks, or fasting. Moreover, an automatic sensor applier generally allows a user to embed a sensor into the user's skin without the assistance of a clinician or health care provider. Although an automatic sensor applier is discussed, the analyte monitoring device may be applied to a human in other ways or otherwise worn by a human, such as without an automatic sensor applier, with the assistance of a health care worker (or the health care worker may simply apply the wearable to the human), or by peeling off a protective layer of an adhesive and applying the adhesive to the human, to name a few, without departing from the spirit or scope of the techniques described herein. Once the sensor is inserted into the user's skin, the analyte monitoring device monitors the human's analyte levels over an observation period that may span multiple days. It should also be understood that in some implementations, the sensor may not be inserted into the skin of a person. Instead, the sensor may simply be disposed against the skin of a person in such implementations, such as an adhesive patch. In any case, the sensor of the analyte monitoring device may continuously detect the analyte and enable the generation of an analyte measurement.

[0016] However, sensors in analyte monitoring devices may have manufacturing variations that cause sensor-to-sensor differences in the sensor's response to the analyte. For example, manufacturing variations may exist between different manufacturing lots, between different sensor brands, between different sensor models of the same brand, etc. These manufacturing variations may result in different sensors producing different analyte measurements for the same analyte level in blood or interstitial fluid due to sensor bias (e.g., positive or negative baseline offset), etc. As a result, one or more machine learning models may erroneously predict the state of some users' health conditions if the models do not take into account the manufacturing variations.

[0017] To overcome these problems, for example, features representing values ​​and patterns in the analyte measurements are extracted from the analyte measurements, and robust combinations of the features that are insensitive to sensor manufacturing variations are selected and used to predict the health condition via the machine learning model. In one or more implementations, the robust combinations of the features are selected based on a robustness metric and a performance metric for each of a plurality of candidate combinations of the features. The performance metric for each candidate combination of the features indicates the accuracy rate of predicting the health condition classification using the candidate combination, and the robustness metric for each candidate combination indicates the insensitivity of the accuracy rate to manufacturing variations.

[0018] To determine the robustness metric, the variability simulator adds various percentages of simulated variability to the historical analyte measurements, and the performance of each candidate combination is evaluated at each percentage of simulated variability. For example, the robustness metric can be a measure of the average percentage change in the performance metric per percentage change in the simulated variability. In some implementations, the simulated variability is a multiplicative percentage bias drawn from a normal distribution with a fixed standard deviation and a mean value that is swept between a first lower percentage and a second higher percentage. Candidate combinations with high robustness metrics exhibit lower changes in the performance metric per percentage change in the simulated variability. Robust combinations of features can be selected to effectively balance the performance metric and robustness metric, such as by selecting the candidate combination with the highest performance metric after filtering out candidate combinations with robustness metrics below a threshold.

[0019] Once the robust combination of features is selected, a machine learning model is trained to predict a health condition classification based on the robust combination of features extracted from the historical analyte measurements of the user population (e.g., without adding simulated variability) and the outcome data. The trained machine learning model processes new analyte measurements of the user collected by the wearable analyte monitoring device over the observation period to predict a health condition classification for the user. In particular, the trained machine learning model processes the robust combination of features extracted from new analyte measurements of the user to predict a health condition classification.

[0020] Generally speaking, a health condition classification describes the state (or status) of a user during an observation period with respect to a particular health condition classification for which a machine learning model is trained. In some implementations, this "health condition classification" may indicate whether the user has a health condition, whether they are at risk for developing a health condition, and / or whether they indicate a side effect that the user is predicted to experience. As an example, a user may have their glucose monitored via an analyte monitoring device to predict whether they have diabetes (e.g., type 1 diabetes, type 2 diabetes, gestational diabetes mellitus (GDM), cystic fibrosis diabetes, etc.), whether they are at risk for developing diabetes (e.g., prediabetes), and / or whether they are predicted to experience a side effect associated with diabetes (e.g., retinopathy, neuropathy, comorbidities, dysglycemia, macrosomia requiring cesarean section, and neonatal hypoglycemia, to name a few). In one or more implementations, the machine learning model may additionally or alternatively be configured to predict the type of prediabetes (e.g., impaired glucose fasting (IFG) or impaired glucose tolerance (IGT)). Alternatively or additionally, the health state classification may correspond to a risk level of having or developing an associated health state, such as high risk, low risk, or no risk of developing the health state. In operation, the health state classification predicted by the machine learning model may be used (e.g., by a health care practitioner) to treat a human or develop a treatment plan, similar to how a human would be treated if clinically diagnosed (e.g., as having a certain type of diabetes and / or prone to experiencing side effects) using traditional testing.

[0021] In particular, unlike conventional diabetes tests, the health status classification (e.g., diabetes classification) predicted by the machine learning model is based on analyte (e.g., glucose) values ​​observed over multiple days. Thus, the prediction is more accurate than tests that rely on blood samples collected at a single time point. Moreover, unlike HbA1c tests, which are an indirect measurement of blood glucose and may be affected by recent changes in glucose levels caused by external factors or conditions such as illness or pregnancy, the health status classification predicted by the machine learning model is based on glucose measurements taken directly during the current observation period.

[0022] The health condition classification prediction is then presented, such as by displaying an indication of the health condition classification to the user, a physician, or the user's guardian via a user interface. Other information may also be presented, such as a visualization of the analyte measurements and other statistics derived from the analyte measurements. In some cases, the health condition classification prediction is presented in an analyte observation report that may also include one or more treatment options for the user, a visual representation of the analyte measurements collected by the analyte monitoring device during the observation period, analyte level statistics for the user generated based on the collected analyte measurements, a severity level, next steps (e.g., for a physician, health care professional, or user), a follow-up request, a request to order more sensors for the analyte monitoring device, activity levels, trends in analyte levels or other markers, patterns of analyte levels or other markers, patterns of movement, interpretations of the analyte measurements, or activities related to the analyte levels. Thus, unlike traditional blood glucose test results for diabetes, the analyte observation report generated by one or more machine learning models may include a detailed analysis of the prediction and various treatment options. It should be appreciated that the health condition classification and information associated with such classification may be provided in a variety of ways, including, for example, output as an audio signal via a speaker or a digital assistant.

[0023] Advantageously, utilizing a wearable analyte monitoring device and a machine learning model that is insensitive to manufacturing variations of the analyte monitoring device's sensor to generate a prediction of a user's health condition classification increases the accuracy rate of the prediction, eliminates many of the unpleasant aspects of the diagnostic tests described above, and does not limit who can be tested. Unlike HbA1c, for example, pregnant women can safely wear the analyte monitoring device for the observation period. Moreover, because the machine learning model is applied to a robust combination of features extracted from analyte measurements collected over multiple days, the inconsistencies associated with traditional tests are reduced, thereby improving the accuracy rate of the prediction compared to traditional tests based on a single blood sample. By accurately predicting health condition classifications and informing users, health care providers, and / or telemedicine services, the described machine learning model enables early detection of health conditions such as diabetes and identifies treatment options that may mitigate potentially harmful health conditions. In so doing, significant damage to the heart, blood vessels, eyes, kidneys, and nerves resulting from diabetes can be largely avoided.

[0024] A technical effect of selecting a robust combination of features extracted from analyte measurements based on a performance metric associated with accurately predicting a health condition classification and a robustness metric associated with insensitivity to manufacturing variations in the analyte sensor obtaining the analyte measurements is that a machine learning model trained to predict a health condition classification using the robust combination of features may have an increased accuracy rate compared to a machine learning model trained using a single analyte feature or a combination of non-robust features.

[0025] In some aspects, techniques described herein relate to methods that include obtaining a plurality of features of an analyte measurement, selecting a feature combination of the plurality of features based on a robustness metric associated with insensitivity to manufacturing variations of the analyte sensor and a performance metric associated with predicting a health condition classification, and training one or more machine learning models for predicting a health condition classification using the feature combination.

[0026] In some aspects, techniques described herein relate to methods where the analyte measurements are historical analyte measurements from a user population associated with outcome data for the user population, and where selecting a combination of features of the plurality of features based on a robustness metric and a performance metric includes generating a model prediction of a health condition classification for each of a plurality of candidate combinations of the plurality of features, and determining a performance metric for each of the plurality of candidate combinations based on the model prediction of the health condition classification for the outcome data of the user population.

[0027] In some aspects, the techniques described herein relate to methods, where the performance metric indicates one or both of a sensitivity for predicting a health condition classification and a specificity for predicting a health condition classification based on a model prediction of the health condition classification on outcome data for each of a plurality of candidate combinations.

[0028] In some aspects, the techniques described herein relate to methods in which the outcome data is indicative of a clinically determined health state classification for each user of a user population.

[0029] In some aspects, techniques described herein relate to methods where selecting a combination of features of a plurality of features based on a robustness metric and a performance metric further includes simulating manufacturing variations of the analyte sensor in the analyte measurements over a plurality of simulation rounds, each introducing a different percentage of the simulated variation into the analyte measurements, and determining a robustness metric for each of the plurality of candidate combinations based on a change in the performance metric for each percentage of the simulated variation.

[0030] In some aspects, the technology described herein relates to methods where simulating manufacturing variations of an analyte sensor in analyte measurements over multiple simulation rounds includes simulating different performance variations and analyte sensor characteristics such that each simulation round introduces a different percentage of the simulated variation into the analyte measurements.

[0031] In some aspects, techniques described herein relate to a method in which selecting a combination of features of a plurality of features based on a robustness metric and a performance metric further includes filtering a plurality of candidate combinations of the plurality of features based on a robustness metric of each of the plurality of candidate combinations relative to a robustness threshold, and selecting as the combination of features the filtered candidate combination having the highest value for the performance metric.

[0032] In some aspects, techniques described herein relate to a method, where the combination of features of the plurality of features includes a first feature and a second feature, and the method further includes determining an individual performance metric and an individual robustness metric for each of the plurality of features using the model prediction of the health condition classification for each of the plurality of features.

[0033] In some aspects, the techniques described herein relate to methods where at least one of the first feature and the second feature is a trend-related feature of the analyte measurement.

[0034] In some aspects, the techniques described herein relate to methods where the second features are variability and stability features of the analyte measurements.

[0035] In some aspects, techniques described herein relate to a method, after training one or more machine learning models to predict a health condition classification using the feature combination, further including obtaining new analyte measurements from an analyte measurement device worn by a user over an observation period, extracting features of the feature combination from the new analyte measurements, inputting the extracted features of the feature combination into the one or more machine learning models, and receiving a health condition classification of the user as an output of the one or more machine learning models.

[0036] In some aspects, the techniques described herein relate to a device comprising one or more processors and a memory having computer-readable instructions stored thereon, the computer-readable instructions being executable by the one or more processors to perform operations including: acquiring analyte data of a user measured by an analyte sensor; extracting at least two features of the analyte data, where the at least two features are included in a multivariate model of analyte features determined to be robust to manufacturing variability of the analyte sensor based on a variability simulation performed on historical analyte data of a user population; inputting a combination of the at least two features into a machine learning model; predicting a health condition classification of the user via the machine learning model; and receiving the health condition classification as an output of the machine learning model.

[0037] In some aspects, the techniques described herein relate to devices in which a user's analyte data is measured by an analyte sensor over an observation period, and the health condition classification is an index that describes the user's status during the observation period with respect to health condition.

[0038] In some aspects, the techniques described herein relate to a device where the health condition is diabetes and the health condition classification is one of a diabetic status, a pre-diabetic status, and a non-diabetic status.

[0039] In some aspects, the techniques described herein relate to a device in which a machine learning model is trained with a training input portion including a combination of at least two features extracted from historical analyte data of a user population, and an expected output portion including a label representing a health condition classification for each user of the user population.

[0040] In some aspects, the techniques described herein relate to a device that includes a wearable analyte monitoring device including a sensor subcutaneously inserted into a user's skin to collect the user's analyte measurements during an observation period; a storage device for maintaining the user's analyte measurements collected during the observation period; and a prediction system for predicting a health condition classification of the user by extracting robust analyte feature combinations from the user's analyte measurements and processing the robust analyte feature combinations with one or more machine learning models.

[0041] In some aspects, techniques described herein relate to a system where one or more machine learning models are generated based on historical analyte measurements and historical outcome data of a user population, the system further including a model manager for acquiring historical analyte measurements and historical outcome data of the user population, the historical analyte measurements being provided by analyte monitoring devices worn by users of the user population, extracting robust analyte feature combinations from the historical analyte measurements, providing the robust analyte feature combinations extracted from the historical analyte measurements to one or more machine learning models, and generating the one or more machine learning models by adjusting weights of the one or more machine learning models based on a comparison of training health condition classifications received from the one or more machine learning models and clinically validated health condition classifications indicated by the historical outcome data.

[0042] In some aspects, the techniques described herein relate to systems in which clinically validated health condition classifications indicated by historical outcome data are associated with one or more diagnostic measures that are independent of historical analyte measurements provided by analyte monitoring devices worn by users of a user population.

[0043] In some aspects, the techniques described herein relate to a system in which one or both of the storage device and the predictive system are implemented, at least in part, in a wearable analyte monitoring device.

[0044] In some aspects, the techniques described herein relate to systems in which the predictive system is implemented in one or more computing devices remote from the wearable analyte monitoring device.

[0045] In the following discussion, an example of an environment in which the techniques described herein can be used is first described. For example, an analyte monitoring device may be used to measure one or more analytes from a user over an observation period, such as according to an example implementation shown in FIG. 1. The analyte monitoring device may measure one or more analytes using a sensor as shown in FIG. 2. According to the techniques described herein, analyte data generated by the analyte monitoring device may be routed to different systems in relation to health state classification as illustrated in FIG. 3. Example implementation details and procedures that may be performed in the discussed environment, as well as other environments, are then described. For example, as illustrated in the implementation of FIG. 4 and the procedure (e.g., method) of FIG. 11, a model manager may perform variability simulations on historical analyte measurements of a user population to identify combinations of analyte features that generate accurate health state predictions even in the presence of simulated variability. Exemplary graphs illustrating performance metrics and variability sensitivity metrics for multiple analyte features alone and in combination are shown in FIG. 5. Once the combination of analyte features is selected, the model manager may train a machine learning model to predict a health condition classification using a combination of analyte features of historical analyte measurements and outcome data of a user population, as illustrated in the implementation of FIG. 6 and the procedure of FIG. 12. Once trained, the machine learning model may be used to predict a health condition classification of a user based on a combination of analyte features extracted from the user's analyte data, such as following the implementation illustrated in FIG. 7 and the procedure of FIG. 13. The health condition classification may be input via the exemplary user interfaces illustrated in FIG. 8 and FIG. 9. Additionally, additional health data that may be used in predicting the health condition classification may be input via the exemplary user interface of FIG. 10. An example of how metrics derived from glucose measurements may be used in a diabetes classification algorithm is discussed with respect to FIG. 14. FIG. 15 illustrates an exemplary system including various components that may be implemented as any type of computing device for implementing and executing the techniques discussed above.The execution of those procedures is not limited to this example environment, and this example environment is not limited to the execution of those procedures.

[0046] Example Environment 1 is an illustration of an environment 100 in an example implementation operable to use the analyte measurements and health state predictions using machine learning described herein. The illustrated environment 100 includes a human 102 who is shown wearing an analyte monitoring device 104. The illustrated environment also includes an observation kit provider 106 and an observation analysis platform 108.

[0047] In the illustrated environment 100, the analyte monitoring device 104 is shown to be provided to the human 102 by an observation kit provider 106, e.g., as part of an observation kit. The analyte monitoring device 104 may be provided as part of an observation kit, e.g., for the purpose of monitoring one or more analytes in the blood and / or interstitial fluid of the human 102 over an observation period lasting multiple days or lasting a different amount of time, e.g., minutes, hours, etc. As used herein, the term "analyte" may refer to a biochemical or chemical that is subjected to analysis in determining a user's status with respect to a health condition. For example, the term "analyte" may refer to a metabolite or other chemical, a protein (e.g., an enzyme), or another measurable indicator of the severity or presence (or absence) of a health condition present in the blood and / or interstitial fluid. For example, the analyte monitoring device 104 may be configured to measure one or more of glucose, lactate, ketones, potassium, insulin, phosphate, bicarbonate, calcium, magnesium, sodium, and blood urea nitrogen. As an example, a human 102 may have his or her glucose monitored to predict whether or not he or she has diabetes (e.g., type 1 diabetes, type 2 diabetes, gestational diabetes mellitus (GDM), cystic fibrosis diabetes, etc.), whether or not he or she is at risk for developing diabetes (e.g., prediabetes), and / or whether or not he or she is predicted to experience side effects associated with diabetes (e.g., comorbidities, dysglycemia, macrosomia requiring a cesarean section (C-section), and neonatal hypoglycemia, to name a few). As another example, an analyte monitoring device 104 may monitor one or more analytes to predict whether a human 102 has an acute condition (e.g., ketonemia, acidosis, or sepsis) or a chronic condition (e.g., liver disease, kidney disease, or sleep apnea).

[0048] In connection with the observation period, instructions may be provided to the human 102 instructing the human 102 to perform one or more activities during the observation period, such as instructing the human 102 to consume a beverage or a particular meal (e.g., the same beverage consumed in connection with an OGTT), to avoid one or more particular foods, to exercise, and to rest, to name a few. In one or more implementations, the instructions may be provided as part of an observation kit, e.g., written instructions. Alternatively or additionally, the observational analysis platform 108 may cause the instructions to be communicated to and output (e.g., for display or audio output) via one or more computing devices associated with the human 102. The observational analysis platform 108 may provide these instructions for output after a predetermined amount of time has passed in the observation period (e.g., 2 days) and / or based on patterns in the analyte measurements obtained. In connection with providing such instructions, the analyte monitoring device 104 automatically monitors the analyte levels of the human 102 after engaging in the prescribed activity, such as by monitoring the amount by which the human's 102 glucose changes after consuming a prescribed meal, performing a prescribed exercise, etc.

[0049] Although discussed throughout as lasting multiple days, in one or more implementations, the duration of the observation period may be variable, such that the observation period may end when enough analyte measurements have been collected to accurately predict the health status classification of the human 102. As an example, in some cases, glucose measurements of the human 102 over just a few hours may be processed to predict with statistical certainty that the human 102 has diabetes. In this case, the duration of the observation period may be hours rather than multiple days. Generally, however, the observation period lasts multiple days in order to obtain data such that features can be extracted to describe, for example, day-to-day variations in analyte levels, and to prevent erroneous predictions that may otherwise be caused by anomalous measurements or observations.

[0050] To this end, the observation kit provider 106 may represent one or more of a variety of entities associated with obtaining a prediction regarding whether the human 102 has a health condition or is predicted to experience an adverse effect of the health condition. For example, the observation kit provider 106 may represent a provider of the analyte monitoring device 104 and, if corresponding to the provider of the analyte monitoring device 104, a platform for monitoring and analyzing measurements obtained from the analyte monitoring device 104, such as the observation analysis platform 108. Alternatively or additionally, the observation kit provider 106 may correspond to a health care provider (e.g., a primary care physician, OB / GYN, endocrinologist), a clinic, a hospital, an insurance provider, a medical testing facility, or a telemedicine service, to name a few. Alternatively or additionally, the observation kit provider 106 may correspond to a pharmacist or pharmacy, which may have a physical brick-and-mortar storefront and / or provide services online. It should be understood that these are a few examples and that the observation kit provider 106 may represent different entities without departing from the spirit or scope of the described technique.

[0051] With this in mind, the human 102 may be provided with the analyte monitoring device 104 in a variety of ways in accordance with the described techniques. For example, the analyte monitoring device 104 may be provided to the human 102 at a clinic, hospital, medical testing facility, or brick-and-mortar pharmacy, e.g., as part of an observation kit. Alternatively, the analyte monitoring device 104 may be mailed to the human 102 from, e.g., an analyte monitoring device 104 provider, a pharmacy, medical testing facility, telemedicine service, etc. The human 102 may obtain the analyte monitoring device 104 over an observation period in other ways in one or more implementations.

[0052] Regardless of how the analyte monitoring device 104 is acquired by the human 102, the analyte monitoring device 104 is configured to monitor one or more predefined analytes (e.g., glucose) of the human 102 during an observation period lasting over a period spanning multiple days. In one or more implementations, the analyte monitoring device 104 is a wearable device such that the device is worn by the human 102 while the device performs various operations. Additionally or alternatively, the analyte monitoring device 104 performs one or more operations before or after being worn by the human 102. For example, the analyte monitoring device 104 may be configured with an analyte sensor that detects one or more signals indicative of an analyte in the human 102 and enables the generation or estimation of an analyte measurement (e.g., an estimation of a glucose value). These analyte measurements (e.g., glucose measurements) may be corresponded to or otherwise packaged for communication as analyte measurements 110, which are an example of sensor data 118, to one or more of the computing devices or observational analysis platforms 108.

[0053] In at least one implementation, the analyte monitoring device 104 is a glucose monitoring system. As an example, the analyte monitoring device 104 may be configured as a continuous glucose monitoring ("CGM") system, such as a wearable CGM system. As used herein, the term "continuous" used in connection with analyte monitoring may refer to the ability of a device to generate measurements substantially continuously, and thus the device may be configured to generate analyte measurements at regular or irregular time intervals (e.g., approximately every hour, approximately every 30 minutes, approximately every 5 minutes, etc.), such as upon establishing a communication coupling with a different device (e.g., when the observational analysis platform 108 establishes a wireless connection with the analyte monitoring device 104 to retrieve one or more of the measurements). However, in other implementations, glucose (or other analyte) monitoring may not be "continuous" and instead provide glucose measurements when requested. For example, the analyte monitoring device 104 may communicate a current glucose measurement to a computing device in response to a request from the computing device. This request may be initiated in a variety of ways, such as in response to a user input to a user interface displayed by the computing device, in response to placement of the computing device within a threshold proximity to the analyte monitoring device 104, in response to the computing device coming into physical contact with the analyte monitoring device 104, in response to a request from an application implemented on the computing device, etc. This functionality, along with further aspects of the configuration of the analyte monitoring device 104, are discussed in more detail in connection with FIG.

[0054] In addition to generating sensor data 118 (including analyte measurements 110), the analyte monitoring device 104 also transmits the generated sensor data 118, for example, to the observational analysis platform 108. The analyte monitoring system 104 may communicate data in real time (e.g., as the data is generated) using analyte or other sensors. Alternatively or additionally, the analyte monitoring device 104 may communicate data to the observational analysis platform 108 at predetermined time intervals. For example, the analyte monitoring device 104 may be configured to communicate the sensor data 118 to the observational analysis platform 108 approximately every 5 minutes. The interval at which the sensor data 118 is communicated by the analyte monitoring device 104 may vary from the above example without departing from the spirit or scope of the described techniques. The sensor data 118 may be communicated by the analyte monitoring device 104 to other computing devices (e.g., in addition to or instead of the observational analysis platform 108 in accordance with the described techniques).

[0055] The analyte monitoring device 104 may be configured in a manner similar to wearable analyte monitoring devices used to treat health conditions, such as to treat diabetes, although in one or more implementations, the analyte monitoring device 104 may be configured differently from devices used for treatment. These different configurations may be deployed to control for confounding factors of the observation period, such that measurements are obtained that accurately reflect the effects of the user's normal day-to-day behavior. This may include, for example, limiting and / or completely preventing the user from testing those measurements during the observation period. By preventing the user from testing analyte measurements 110 over the course of the observation period, the observation configuration further prevents the user from seeing or otherwise observing an analyte measurement event (e.g., a glucose spike) and changing the user's behavior to counter such an event.

[0056] In some cases, the analyte monitoring device 104 may be a dedicated device specifically designed for the purpose of collecting a user's glucose measurements during an observation period spanning multiple days or some other period of time so that a diabetes classification may be generated that may be distinct in one or more ways from a wearable glucose monitoring device worn by a user to treat diabetes. In other cases, the analyte monitoring device 104 may have the same hardware characteristics as a wearable glucose monitoring device used to treat diabetes, but may include software that disables or enables different features, such as software that prevents a user from testing analyte measurements 110 during the observation period. In these cases, features that were disabled during the observation period may be enabled after the observation period has ended, such that the user may have access to previously disabled features, such as the ability to view glucose measurements in substantially real-time.

[0057] The different configurations may also be based on differences between how the analyte measurements 110 are used in connection with an observation period for health condition prediction and how the measurements are used in connection with treatment of the health condition. In an example where the health condition is diabetes and the analyte is glucose, continuous or near-continuous receipt and output of glucose measurements may be used to accompany treatment and inform treatment decisions, such as to help a human or their caregiver decide what to eat, how to administer insulin, whether to contact a health care provider, etc., substantially as those measurements are generated. In those scenarios, knowing the measurements and / or trends in the measurements in a timely manner (e.g., substantially in real time) may be crucial to effectively mitigate potentially serious side effects. In contrast, receipt and substantially real-time output of glucose measurements to the human being observed (or to a caregiver) may not be necessary in connection with diabetes prediction in those scenarios. Instead, the glucose measurements generated for diabetes prediction are processed such that an accurate prediction regarding diabetes may be generated at the end of the observation period or after some other range (e.g., when sufficient measurements have been generated to achieve statistical certainty).

[0058] Based on such differences in how the analyte measurements are used, the analyte monitoring device 104 may have more local storage than a wearable analyte measurement device used for treatment (e.g., diabetes treatment). By way of example and not limitation, 10-15 days of analyte measurement storage may be provided in an observation configuration, whereas 3 hours of analyte measurement storage may be provided in a treatment configuration. The larger storage capacity of the analyte monitoring device 104 may be suitable for storing analyte measurements 110 for the duration of the observation period. In contrast, wearable analyte measurement devices used for treatment may be configured to offload analyte measurements such that, once measurements are suitably offloaded, the measurements are no longer stored locally on those devices. By way of example, a wearable glucose device used for treatment of diabetes may offload glucose measurements by transmitting glucose measurements over a wireless connection to an external computing device, e.g., at predetermined time intervals and / or in response to establishing or re-establishing a connection with the computing device.

[0059] To the extent that the analyte monitoring device 104 may be configured to store the analyte measurements 110 over the entire observation period, in one or more implementations, the analyte monitoring device 104 may be configured without wireless transmission means, e.g., without an antenna for wirelessly transmitting the analyte measurements 110, and without hardware or firmware for generating packets for such wireless transmission. Alternatively, the analyte monitoring device 104 may be configured with hardware that communicates the analyte measurements 110 via a physical wired connection. In such a scenario, the analyte monitoring device 104 may be "plugged in" to retrieve the analyte measurements 110 from the device's storage.

[0060] Thus, the analyte monitoring device 104 may be configured with one or more ports that enable wired transmission of the analyte measurements 110 to an external computing device. Examples of such physical connections may include a micro universal serial bus (USB) connection, a mini USB connection, and a USB-C connection, to name a few. While the analyte monitoring device 104 may be configured to extract the analyte measurements 110 via a wired connection as just discussed above, in different scenarios the analyte monitoring device 104 may alternatively or additionally be configured to offload the analyte measurements 110 via one or more wireless connections. Implementations involving wired and / or wireless communication of the analyte measurements 110 are discussed further below.

[0061] In addition to storage and communication differences, the analyte monitoring device 104 may also include one or more sensors or sensor circuitry that are configured differently than devices designed for the treatment of health conditions. For example, the sensors and circuitry (including, e.g., measurement algorithms) of a wearable glucose monitoring device used to treat diabetes may be optimized for a measurement range that spans 40 mg / dL to 400 mg / dL. This is because treating diabetes often involves determining what steps should be taken to mitigate severe glycemic events, e.g., hypoglycemia and hyperglycemia, that may occur toward both ends of the range. However, measurement fidelity over a wide range may not be required to predict diabetes. Rather, diabetes predictions may be suitably generated in association with a smaller range, such as a range of glucose measurements that spans 120 mg / dL to 240 mg / dL. Thus, the analyte monitoring device 104 may include one or more sensors or sensor circuitry that are optimized to generate measurements in such a smaller range. It should be understood that the differences discussed above are merely examples of how the analyte monitoring device 104 may differ from wearable analyte monitoring devices configured for treatment of health conditions, and that the analyte monitoring device 104 may differ from those devices in different ways without departing from the spirit or scope of the described techniques.

[0062] When the analyte monitoring device 104 generates an analyte measurement 110, the measurement is provided to the observational analysis platform 108. As described above, the analyte measurement 110 may be communicated to the observational analysis platform 108 as sensor data 118 via a wired and / or wireless connection. For example, in scenarios where the observational analysis platform 108 is implemented partially or entirely on the analyte monitoring device 104, the analyte measurement 110 may be transferred from the device's local storage to the device's processing system via a bus. In scenarios where the analyte monitoring device 104 is configured to generate a prediction of a health condition classification by processing the analyte measurement 110, the analyte monitoring device 104 may also be configured to provide the predicted health condition classification as an output, for example, by communicating the health condition classification to an external computing device. In other scenarios, the analyte measurement 110 may be processed by an external computing device configured to predict a health condition classification.

[0063] In one or more implementations, the analyte monitoring device 104 is configured to transmit the analyte measurements 110 to the external device via a wired connection with the external device, e.g., via USB-C or some other physical communication coupling. Here, a connector may be plugged into the analyte monitoring device 104, or the analyte monitoring device 104 may be inserted into an apparatus having a receptacle that interfaces with a corresponding contact on the device. The analyte measurements 110 may then be retrieved from the storage device of the analyte monitoring device 104 via this wired connection and transferred, for example, to the external device via a wired connection. Such a connection may be used in a scenario where the analyte monitoring device 104 is mailed by the human 102 after the observation period to a health care provider, a telemedicine service, a provider of the analyte monitoring device 104, a medical testing facility, or the like. To this end, an observation kit (not shown) may include packaging (e.g., an envelope or box) for mailing the analyte monitoring device 104 to such an entity after observation. Such a connection may also be used in scenarios where the analyte monitoring device 104 is delivered by the human 102 after the observation period to a clinic or hospital (or other facility of a healthcare provider), a pharmacy, or a medical testing facility, etc. Alternatively or additionally, scenarios involving a wired connection may involve the human 102 plugging the analyte monitoring device 104 into an external computing device after the observation period, for example, using a cord provided as part of an observation kit. In these scenarios, the external computing device may communicate the analyte measurements 110 to the observational analysis platform 108 over a network (not shown), such as the Internet.

[0064] Alternatively or additionally, providing the analyte measurements 110 to the observational analysis platform 108 may include the analyte monitoring device 104 transmitting the analyte measurements 110 via one or more wireless connections. For example, the analyte monitoring device 104 may wirelessly transmit the analyte measurements 110 to an external computing device, such as a mobile phone, a tablet device, a laptop, a smart watch, other wearable health tracking device, etc. Thus, the analyte monitoring device 104 may be configured to communicate with the external device using one or more wireless communication protocols or techniques. By way of example, the analyte monitoring device 104 may communicate with the external device using one or more of Bluetooth (e.g., a Bluetooth Low Energy link), near-field communication (NFC), Long Term Evolution (LTE) standards such as 5G, etc. The analyte monitoring device 104 may be configured with a corresponding antenna and other wireless transmission means in scenarios where the analyte measurements 110 are transmitted to an external device for processing. In these scenarios, the analyte measurements 110 may be communicated to the observational analysis platform 108 in a variety of manners, such as at predetermined time intervals (e.g., daily, hourly, or every five minutes), in response to the occurrence of some event (e.g., a storage buffer of the analyte monitoring device 104 filling up), or in response to the end of an observation period, to name a few.

[0065] Thus, regardless of where the observational analysis platform 108 is implemented, the observational analysis platform 108 obtains analyte measurements 110 generated by the analyte monitoring device 104. In one or more implementations, the observational analysis platform 108 may be implemented in whole or in part on the analyte monitoring device 104. Alternatively or additionally, the observational analysis platform 108 may be implemented in whole or in part using one or more computing devices external to the analyte monitoring device 104, such as one or more computing devices associated with the human 102 (e.g., a mobile phone, a tablet device, a laptop, a desktop, or a smartwatch) or one or more computing devices associated with a service provider (e.g., a healthcare provider, a telemedicine service, a service corresponding to the provider of the analyte monitoring device 104, a medical testing facility, etc.). In the latter scenario, the observational analysis platform 108 may be implemented at least in part on one or more server devices.

[0066] In the illustrated environment 100, the observational analytics platform includes a storage device 112. In accordance with the described techniques, the storage device 112 is configured to maintain the analyte measurements 110. The storage device 112 may represent one or more databases and other types of storage devices capable of storing the analyte measurements 110. The storage device 112 may also store a variety of other data, such as demographic information describing the human 102, information about a health care provider, information about an insurance provider, payment information, prescription information, determined health indicators, account information (e.g., usernames and passwords), and the like. As discussed in more detail below, the storage device 112 may also maintain data for other users of the user population.

[0067] In the illustrated environment 100, the observational analytics platform 108 also includes a prediction system 114. The prediction system 114 represents functionality for processing the analyte measurements 110 to generate disease and condition predictions, such as predicting whether the human 102 has diabetes (e.g., type 2 diabetes, GDM, cystic fibrosis diabetes, etc.), is at risk for developing diabetes (e.g., prediabetes), and / or is predicted to experience side effects associated with diabetes (e.g., comorbidities, dysglycemia, macrosomia requiring cesarean section, and neonatal hypoglycemia, to name a few). As discussed in more detail below, the prediction system 114 uses machine learning to predict health condition classifications. The use of machine learning may include, for example, leveraging one or more models generated using machine learning techniques and using historical analyte measurements and historical outcome data of a user population to identify robust features of the analyte measurements to use to predict the health condition classifications.

[0068] The illustrated environment 100 also includes a health condition classification 116 that may be output by the prediction system 114. According to the described techniques, the health condition classification 116 may indicate whether the human is predicted to have the indicated health condition or whether the human is predicted to experience side effects associated with the health condition. The health condition classification 116 may also be used to generate one or more notifications or user interfaces based on the classification, such as a report directed to a health care provider, including the health condition classification (e.g., the human is predicted to have a health condition, such as diabetes), or a notification directed to the human 102 instructing the human 102 to contact his or her health care provider. Examples of user interfaces that may be generated based on the health condition classification 116 are described in more detail in connection with FIGS. 8 and 9. Consider the following discussion of FIG. 2 in the context of measuring analytes, e.g., continuously analyzing one or more analytes of a subject, and obtaining data describing such measurements.

[0069] Figure 2 illustrates in more detail an example implementation 200 of the analyte monitoring device 104 of Figure 1. Accordingly, components previously introduced in Figure 1 are numbered the same and will not be introduced again. In particular, the illustrated example 200 includes a top view and a corresponding side view of the analyte monitoring device 104. It should be understood from the following discussion that the analyte monitoring device 104 may vary in implementation without departing from the spirit or scope of the described techniques.

[0070] In this example 200, the analyte monitoring device 104 is illustrated to include an analyte sensor 202 (e.g., a glucose sensor) and a sensor module 204. Here, the analyte sensor 202 is shown in a side view being subcutaneously inserted into the skin 206 of the human 102, for example, of FIG. 1. The sensor module 204 is approximated as a dashed rectangle in a top view. The analyte monitoring device 104 also includes a transmitter 208 in the illustrated example 200. The use of a dashed rectangle for the sensor module 204 indicates that the sensor module may be contained or otherwise implemented within a housing of the transmitter 208. An antenna and / or other hardware used to enable the transmitter 208 to generate a signal for communicating data, for example, via a wireless connection to the observational analysis platform 108, may also be contained or otherwise implemented within a housing of the transmitter 208. In this example 200, the analyte monitoring device 104 further includes an adhesive pad 210.

[0071] In operation, the analyte sensor 202 and adhesive pad 210 may be assembled to form an applicator assembly configured to be applied to the skin 206 such that the analyte sensor 202 is inserted subcutaneously as depicted. In such a scenario, the transmitter 208 may be attached to the assembly after affixing to the skin 206 via an attachment mechanism (not shown). Alternatively, the transmitter 208 may be incorporated as part of the applicator assembly such that the analyte sensor 202, adhesive pad 210, and transmitter 208 (with sensor module 204) can be affixed to the skin 206 all at once. In one or more implementations, this applicator assembly is affixed to the skin 206 using a separate sensor applicator (not shown). Unlike the finger prick required by conventional blood glucose meters, for example, user-initiated application of the analyte monitoring device 104 with a sensor applicator is substantially painless and does not require the drawing of blood. Additionally, the automated sensor applicator generally allows the human 102 to implant the analyte sensor 202 subcutaneously in the skin 206 without the assistance of a clinician or health care provider.

[0072] The analyte monitoring device 104 may also be removed by peeling the adhesive pad 210 from the skin 206. It should be understood that the analyte monitoring device 104 and its various components are illustrated as one exemplary form factor and that the analyte monitoring device 104 and its components may have different form factors without departing from the spirit or scope of the described techniques.

[0073] In operation, the analyte sensor 202 is communicatively coupled to the sensor module 204 via at least one communication channel, which may be a wireless or wired connection. Communications from the analyte sensor 202 to the sensor module 204 or from the sensor module 204 to the analyte sensor 202 may be implemented actively or passively, and these communications may be continuous (e.g., analog) or discrete (e.g., digital).

[0074] The analyte sensor 202 may be a device, molecule, and / or chemical that changes or causes a change in response to an event that is at least partially independent of the analyte sensor 202. The sensor module 204 is implemented to receive an indication of a change to or caused by the analyte sensor 202. For example, the analyte sensor 202 may include glucose oxidase that reacts with glucose and oxygen to form hydrogen peroxide that is electrochemically detectable by the sensor module 204, which may include electrodes. In this example, the analyte sensor 202 may be configured as or include a glucose sensor configured to detect an analyte in blood or interstitial fluid that is indicative of glucose levels using one or more measurement techniques. In one or more implementations, the analyte sensor 202 may also be configured to detect analytes in blood or interstitial fluid that are indicative of other markers, such as lactate levels, ketones, or ionic potassium, which may improve accuracy in identifying or predicting glucose-based events (e.g., hyperglycemia or hypoglycemia). Additionally or alternatively, the analyte monitoring device 104 may include additional sensors and / or architectures to the analyte sensor 202 for detecting those analytes indicative of other markers.

[0075] In another example, the analyte sensor 202 (or additional, not shown, sensors of the analyte monitoring device 104) can include first and second conductors, and the sensor module 204 can electrically detect a change in electrical potential between the first and second conductors of the analyte sensor 202. In this example, the sensor module 204 and the analyte sensor 202 are configured as a thermocouple, such that the change in electrical potential corresponds to a change in temperature. In some examples, the sensor module 204 and the analyte sensor 202 are configured to detect a single analyte, e.g., glucose. In other examples, the sensor module 204 and the analyte sensor 202 are configured to use multiple sensing modes to detect multiple analytes, e.g., ionic sodium, ionic potassium, carbon dioxide, and glucose. Alternatively or additionally, the analyte monitoring device 104 includes multiple sensors to detect not only one or more analytes (e.g., ionic sodium, ionic potassium, carbon dioxide, glucose, and insulin), but also one or more environmental conditions (e.g., temperature, humidity, movement). Thus, the sensor module 204 and analyte sensor 202 (as well as any additional sensors) may detect the presence of one or more analytes, the absence of one or more analytes, and / or a change in one or more environmental conditions. Further, the sensor module 204 and analyte sensor 202 (as well as any additional sensors) may be configured to output a signal indicative of the amount (e.g., level) of one or more analytes in the blood or interstitial fluid. As described above, the analyte monitoring device 104 may be configured to generate data describing a single analyte (e.g., glucose) or multiple analytes.

[0076] In one or more implementations, the sensor module 204 may include a processor and memory (not shown). The sensor module 204 may utilize the processor to generate the analyte measurement 110 based on communication with the analyte sensor 202 indicating the change discussed above. Based on the above-mentioned communication from the analyte sensor 202, the sensor module 204 is further configured to generate a communicable data package including at least one analyte measurement 110. In this example 200, the sensor data 118 represents these data packages. Additionally or alternatively, the sensor module 204 may configure the analyte data 118 to include additional data, including, by way of example, supplemental sensor information 214. The supplemental sensor information 214 may include a sensor identifier, a sensor status, a temperature corresponding to the analyte measurement 110, measurements of other analytes corresponding to the analyte measurement 110, and the like. It should be appreciated that the supplemental sensor information 214 may include a variety of data supplementing the at least one analyte measurement 110 without departing from the spirit or scope of the described techniques.

[0077] In implementations in which the analyte monitoring device 104 is configured for wireless transmission, the transmitter 208 may transmit the sensor data 118 as a stream of data to a computing device (e.g., a computing device of the observational analysis platform 108 of FIG. 1 ). Alternatively or additionally, the sensor module 204 may buffer the analyte measurements 110 and / or the supplemental sensor information 214 (e.g., in a memory of the sensor module 204 and / or in another physical computer-readable storage medium of the analyte monitoring device 104) and later cause the transmitter 208 to transmit the buffered sensor data 118 at various regular or irregular intervals, such as time intervals (about every second, about every 30 seconds, about every minute, about every 5 minutes, about every hour, etc.), storage intervals (when the buffered analyte measurements 110 and / or supplemental sensor information 214 reach a threshold amount of data or number of measurements), etc. It should be appreciated that in some implementations, the analyte monitoring device 104 may vary in many ways from the examples described above without departing from the spirit or scope of the described techniques.

[0078] With respect to the supplemental sensor information 214, the sensor identifier represents information that uniquely identifies the analyte sensor 202 from other sensors, such as other sensors of other analyte monitoring devices, other sensors previously or subsequently implanted in the skin 206, etc. By uniquely identifying the analyte sensor 202, the sensor identification may also be used to identify other aspects regarding the analyte sensor 202, such as the manufacturing lot of the analyte sensor 202, the packaging details of the analyte sensor 202, the shipping details of the analyte sensor 202, etc. In this manner, various problems detected with sensors manufactured, packaged, and / or shipped in a similar manner to the analyte sensor 202 may be identified and used in a variety of ways, such as, for example, to calibrate the glucose measurements 110, to notify a user of a defective sensor, to notify a manufacturing facility of a machining issue, etc.

[0079] The sensor status of the supplemental sensor information 214 represents the status of the analyte sensor 202 at a given time, e.g., the state of the sensor at the same time that one of the analyte measurements 110 is generated. To this end, the sensor status may include an entry for each of the analyte measurements 110 such that there is a one-to-one relationship between the analyte measurements 110 and the status captured in the supplemental sensor information 214. For example, the sensor status may describe an operational state of the analyte sensor 202. In one or more implementations, the sensor module 204 may identify one of several predefined operational states for a given analyte measurement 110. The identified operational state may be based on communications from the analyte sensor 202 and / or characteristics of those communications.

[0080] As an example, the sensor module 204 may include (e.g., in memory or other storage) a look-up table with a predetermined number of operating conditions and a basis for selecting one condition from another. For example, the predetermined conditions may include a "normal" operating condition, and the basis for selecting the condition may be that the communication from the analyte sensor 202 falls within a threshold indicative of normal operation, e.g., falls within an expected time threshold, falls within an expected signal strength threshold, the environmental temperature is within a temperature threshold suitable for continued operation as expected. The predetermined conditions may also include operating conditions that indicate one or more characteristics of the analyte sensor 202 communication are outside of the range of normal activity and may result in potential errors in the analyte measurements 110.

[0081] For example, these non-normal operating status basis may include receiving a communication from the analyte sensor 202 outside a threshold expected time, detecting an analyte sensor 202 signal strength outside a threshold of expected signal strength, detecting an environmental temperature outside a suitable temperature for continued operation as expected, detecting that the person 102 has rolled over on the analyte monitoring device 104 (e.g., in bed), etc. The sensor status may indicate a variety of aspects related to the analyte sensor 202 and analyte monitoring device 104 without departing from the spirit or scope of the described techniques.

[0082] Having considered an example environment and an example wearable analyte monitoring device, we now turn to a discussion of some example details of techniques for health prediction using analyte measurements and machine learning in digital media environments, according to one or more implementation aspects.

[0083] Health Status Prediction FIG. 3 illustrates an example of an implementation 300 in which health-related data, including analyte measurements, are routed to different systems in relation to predicting health conditions.

[0084] The illustrated exemplary implementation 300 includes components from FIG. 1, including the observational analytics platform 108 and the human 102. Thus, components previously introduced in FIG. 1 are numbered the same and will not be introduced again. As described in more detail below, the implementation 300 may be used to classify the status of the human 102 with respect to a particular health condition. For example, the status may include the absence of a particular health condition, the presence of a particular health condition, and / or the severity of a particular health condition. In some examples, the status may further include symptoms or other side effects of a health condition that the human 102 is predicted to experience, comorbidities that the human 102 is predicted to experience, and the like. As used herein, the term "health condition" may be understood to broadly include diseases, disorders, and non-pathological conditions that may receive medical treatment. The term "disease" may refer to any health condition that impairs the normal functioning of the body. Thus, a disease such as diabetes may be referred to herein as a health condition. Furthermore, a "disease" or "health condition" may include a group of diseases or health conditions that may be subclassified into different types. For example, diabetes can be subclassified as type 1 diabetes, type 2 diabetes, GDM, cystic fibrosis diabetes, and the like.

[0085] The illustrated exemplary implementation 300 also shows a device 302 associated with the human 102 that may provide analyte measurements 110 to the observational analysis platform 108 and / or storage device 112 in connection with health condition prediction. The illustrated device 302 includes an analyte monitoring device 104 worn by the human 102 during an observation period to generate analyte measurements 110, along with additional devices external to the analyte monitoring device 104. Specifically, the illustrated additional external devices include a mobile phone and a smart watch, although various other devices may be configured to provide analyte measurements 110 to the observational analysis platform 108 and / or storage device 112 in one or more implementations. Other examples of the device 302 may include a laptop, a tablet device, a wearable health tracking device, and the like.

[0086] As mentioned above, the analyte measurements 110 may be communicated or otherwise provided to the observational analysis platform 108 and / or storage device 112 via a wired or wireless connection. For example, the analyte monitoring device 104 may provide the analyte measurements 110 to the observational analysis platform 108 and / or storage device 112 via a wired or wireless connection, as discussed above. In a scenario in which one of the additional external devices 302 provides the analyte measurements 110, the analyte measurements 110 may first be provided from the analyte monitoring device 104 to the additional external device, such that the additional external device communicates or otherwise provides the analyte measurements 110 to the observational analysis platform 108 and / or storage device 112.

[0087] In these scenarios, the additional external device 302 may act as an intermediary between the analyte monitoring device 104 and the observational analysis platform 108 and storage device 112, such that the external device 302 is used to route the analyte measurements 110 from the analyte monitoring device 104 to the observational analysis platform 108 and / or storage device 112. Alternatively or in addition, other devices may route the analyte measurements 110 from the analyte monitoring device 104 to the observational analysis platform 108 and / or storage device 112. These other devices may include dedicated devices configured to extract data from the analyte monitoring device 104 and associated with entities involved in health condition prediction, such as healthcare providers, hospitals, pharmacies, telemedicine services, medical laboratories, etc.

[0088] The illustrated exemplary implementation 300 also includes a user population 304. The user population 304 represents a plurality of users corresponding to humans wearing analyte monitoring devices, such as the analyte monitoring device 104. The analyte measurements 110 of these other users will then be provided to the observational analysis platform 108 and / or storage device 112 by their respective monitoring devices and / or by an external computing device. In one or more implementations, the user population 304 includes users selected as part of one or more studies conducted at least in part for the purpose of collecting data (including the analyte measurements 110) such that machine learning can be used to generate one or more models, e.g., using supervised learning, unsupervised learning, reinforcement learning, etc., that may be used to classify a particular health condition.

[0089] Alternatively or additionally, the user population 304 may include users for whom a health status classification has previously been generated based on their own analyte measurements generated during an observation period involving the analyte monitoring device 104 in a manner similar to how health status predictions are generated for the human 102. Data generated prior to a health status prediction for the human 102 and in association with a survey performed to collect data is referred to as "historical" data because it is generated at a time before the analyte measurements 110 for the human 102 are generated. Similarly, data generated prior to a health status prediction for the human 102 and in association with health status predictions for other users is also historical data. In accordance with the described techniques, historical data includes, for example, historical analyte measurements and historical outcome data. These historical data are used in conjunction with machine learning to train or otherwise learn an underlying model for health status prediction and / or classification, as described in more detail in connection with FIG. 5. In contrast to the historical analyte measurements of the user population 304, the analyte measurements 110 for the health status prediction for the human 102 may be new analyte measurements.

[0090] As an example, a study to gather data related to health status prediction may involve participants wearing analyte monitoring devices for a multi-day period to generate analyte measurements 110 for those participants. The period may have the same or different duration as the observation period used to generate the analyte measurements 110 for the humans 102 without departing from the spirit or scope of the described techniques. In addition to collecting analyte measurements 110, such studies may be utilized to obtain other data regarding the participants. The outcome data 306 may correspond to at least some of this other data and describe various aspects regarding the users of the user population 304.

[0091] For example, in the context of a study to predict diabetes, participants may be tested using conventional techniques to generate one or more diagnostic measures, such as HbA1c, FPG, and / or 2Hr-PG, in addition to wearing an analyte (e.g., glucose) monitoring device. The independent diagnostic measures 308 represent data describing the outcomes of one or more such tests for users of the user population 304. For example, the independent diagnostic measures 308 may describe HbA1c, FPG, 2Hr-PG (or OGTT as a combination of FPG and 2Hr-PG), and / or random plasma glucose (RPG) results in relation to users of the user population 304. The independent diagnostic measures 308 may thus represent clinically determined health status classifications of the user population 304. Given this, the analyte measurements 110 of study participants may be associated with the respective participant's independent diagnostic measures 308, for example, by labeling the measurements. As discussed in more detail below, machine learning, through a training process, may learn patterns in analyte measurements 110 that are indicative of particular values ​​of the independent diagnostic measure 308, such as patterns in analyte measurements 110 that indicate a respective individual is likely to have an HbA1c of 10.0. It may be appreciated that where research involves predicting non-diabetic diseases and conditions, the independent diagnostic measure 308 may differ from those described above.

[0092] As illustrated, outcome data 306 also includes observed side effects 310 and clinical diagnoses 312. Observed side effects 310 represent data describing side effects experienced by users of user population 304. By way of example, in the context of predicting diabetes, observed side effects 310 may describe whether a user experienced any of one or more side effects associated with type 2 diabetes. The one or more side effects associated with type 2 diabetes include diabetic retinopathy, cataracts, glaucoma, blindness, severe hyperglycemia or hypoglycemia, heart and vascular disease, neuropathy, erectile dysfunction, renal failure or end stage renal disease, delayed healing, hearing impairment, skin conditions (e.g., bacterial and fungal infections), sleep apnea, and Alzheimer's disease, to name a few.

[0093] Additionally or alternatively, observed side effects 310 may describe whether the user experienced any of one or more side effects associated with GDM, such as the user's baby having excessive birth weight (requiring a Cesarean section delivery), preterm birth (premature birth), respiratory distress syndrome, the user's baby having neonatal hypoglycemia, the baby becoming obese or developing type 2 diabetes later in life, stillbirth, etc.

[0094] Additionally or alternatively, the observed side effects 310 may describe whether the user experienced one or more side effects associated with other types of diabetes, such as effects associated with type 1 diabetes, cystic fibrosis diabetes, pancreatic diabetes, etc. It can be understood that the observed side effects 310 may differ to predict diseases and conditions other than diabetes, with diabetes prediction being provided as an example. With this in mind, the analyte measurements 110 of the study participants may be associated with the observed side effects 310 of each participant, such as by labeling the measurements. As discussed in more detail below, machine learning, through a training process, may learn patterns in the analyte measurements 110 that are indicative of the occurrence and non-occurrence of the observed side effects 310, such as patterns in the analyte measurements 110 that are indicative of the probability that each person will have a baby with an excessive birth weight that requires a Caesarean section.

[0095] Clinical Diagnosis 312 represents data describing whether a user of user population 304 has been diagnosed (or not) with a health condition by a clinician, or whether the user has been provisionally or preliminarily diagnosed with a health condition. By way of example, the diagnosis may be made by a clinician based on one or more of independent diagnostic measures 308 and / or observed side effects 310. Thus, the diagnosis may provide a clinically validated health condition classification. Additionally or alternatively, clinical diagnosis 312 may be configured to represent a labeling based on a diagnostic test that has not been approved for diagnosis by the Food and Drug Administration (FDA) or the clinical community at large. In the diabetes example, the values ​​of clinical diagnosis 312 may indicate that the respective user has been clinically diagnosed with diabetes (or a type of diabetes), clinically diagnosed with prediabetes (or any of different types of prediabetes), provisionally or preliminarily diagnosed with diabetes, does not have diabetes (i.e., has been screened), has been diagnosed with diabetes using a non-approved test, or has been diagnosed with prediabetes using a non-approved test, to name a few. Given this and the independent diagnostic measures 308, for example, the analyte measurements 110 can be correlated with each study participant's independent diagnostic measures 308 and with each participant's clinical diagnosis 312.

[0096] Through training, machine learning may learn patterns in analyte measurements 110 that are indicative of a particular value of the independent diagnostic measure 308 and that are further indicative of a different clinical diagnosis 312. As an illustrative example, machine learning may learn patterns in analyte measurements 110 that indicate a person's HbA1c is likely to be 6.0 (e.g., "estimated A1c") and that a clinician's analysis is likely to result in a diagnosis of prediabetes. Although this example is discussed with respect to a person's HbA1c, it should be understood that a clinical diagnosis of prediabetes or diabetes may be made based on different measurements (e.g., FPG) and / or observations (e.g., weight gain, neuropathy, and sleep apnea) without departing from the spirit or scope of the described technology.

[0097] In one or more implementations, the outcome data 306 may include or be usable as a label. For example, the value of each independent diagnostic measure 308 may be used to label the analyte measurements 110 of each user of the user population 304. Alternatively or additionally, a label indicative of an observed side effect 310 experienced by the respective user may be used to label the analyte measurements 110 of each user. Alternatively or additionally, a label indicative of a clinical diagnosis 312 may be used to label the analyte measurements 110 of each user. For example, the analyte measurements 110 of a user clinically diagnosed with pre-diabetes may be associated with a “pre-diabetes” label, whereas the analyte measurements 110 of a different user clinically diagnosed with diabetes may be associated with a “diabetes” label. Although the exemplary implementation 300 shows separate diagnostic measures 308, observed side effects 310, and clinical diagnoses 312, it should be understood that the outcome data 306 may include data describing different, additional, or fewer aspects of the users of the user population 304 without departing from the spirit or scope of the described techniques.

[0098] As shown in the illustrated exemplary implementation 300, the analyte measurements 110 and outcome data 306 of the users of the user population 304 are communicated or otherwise provided to the observational analytics platform 108 and / or storage device 112. In addition to the analyte measurements 110 and outcome data 306, additional data describing other aspects of the users of the user population 304 may be obtained by the observational analytics platform 108 and / or storage device 112. By way of example, this additional data may include demographic data (e.g., age, gender, ethnicity), medical history data (e.g., height, weight, body mass index (BMI), body fat percentage, presence or absence of various conditions), stress data, nutritional data, exercise data, prescription data, height and weight data, occupational data, and the like. These types of additional data are merely examples, and the additional data may include more, less, or different types of data without departing from the spirit or scope of the techniques described herein. In one or more implementations, the observational analysis platform 108 and / or the storage device 112 may obtain such additional data (or at least some of the additional data) regarding the humans 102 and regarding the users of the user population 304.

[0099] In particular, the illustrated exemplary implementation 300 shows the observational analysis platform 108 and the storage device 112 separately, and also shows a dashed arrow between the storage device 112 and the observational analysis platform 108. Generally speaking, the arrow represents that data maintained in the storage device 112 may be retrieved by the observational analysis platform 108 from the storage device 112. Stated another way, data maintained by the storage device 112 may be provided to the observational analysis platform 108. As discussed above, the storage device 112 may store analyte measurements 110 for the human 102, as well as analyte measurements 110 and outcome data 306 for the user population 304.

[0100] In one or more implementations, the observational analysis platform 108 and the storage device 112 may correspond to the same entity, such as a provider of an analyte monitoring device (e.g., analyte monitoring device 104) and services related to analyte monitoring. In such implementations, the observational analysis platform 108 and the storage device 112 may be implemented in a "cloud" across multiple computing devices (e.g., servers) and storage resources allocated to the entity or otherwise associated (e.g., via subscription or ownership) with the entity. To this end, the analyte measurements 110 of the humans 102, as well as the analyte measurements 110 and outcome data 306 of the user population 304, may be retrieved by the observational analysis platform 108 from the storage device 112, by way of a server associated with the service provider retrieving the data from storage associated with the service provider.

[0101] In other implementations, the observational analysis platform 108 and the storage device 112 may correspond to different entities. As an example, the storage device 112 may correspond to a first entity, such as a computing device (e.g., a mobile phone or tablet device) of the human 102, and the observational analysis platform 108 may correspond to a second entity, such as a provider of analyte monitoring devices and services related to analyte monitoring. In this example, the observational analysis platform 108 may be implemented, at least in part, as an application of the second entity running on the computing device of the human 102. Alternatively or additionally, the observational analysis platform 108 may be implemented using a server device of the second entity. In an application implementation, the application of the second entity may retrieve one or more of the analyte measurements 110 of the human 102, the analyte measurements 110 of the user population 304, or the outcome data of the user population 304 from the storage device 112 implemented locally on the computing device, e.g., via a bus or other local transmission means of the computing device. In a server implementation, a second entity's server may retrieve data from a storage device 112 implemented on a computing device over one or more networks, such as the Internet.

[0102] In another example where the observational analysis platform 108 and the storage device 112 correspond to different entities, the storage device 112 may correspond to a first entity, such as a provider of analyte monitoring devices and services related to analyte monitoring (or limited services related to analyte monitoring). In this latter example, the observational analysis platform 108 may correspond to a second, different entity, such as a service provider, e.g., a data partner of the first entity. In this example, the second entity may be considered a "third party" with respect to the entity corresponding to the storage device 112 (and the analyte monitoring device 104). If it corresponds to a data partner, the observational analysis platform 108 may obtain data from the first entity (i.e., the storage device 112) pursuant to one or more legal agreements between the first entity and the second entity. The provision of data maintained in the storage device 112 to the observational analysis platform 108 may be controlled by an application programming interface (API).

[0103] In this type of scenario, such an API may be considered an "egress" for data, such as analyte measurements 110 and outcome data 306. "Egress" means that the flow of data is generally outward from a first entity to a third party (e.g., a second entity). In the context of data provision, an API may expose one or more "calls" (e.g., a particular format for data requests) to a third party. As an example, an API may expose those calls to a third party after the third party enters into an agreement, e.g., with a business corresponding to the first entity, that allows the third party to retrieve data from the storage device 112 via the API. As part of this agreement, the third party may agree to exchange payments to retrieve data from the first entity. Alternatively or additionally, the third party may agree to exchange data that the third party generates, e.g., via an associated device, to retrieve data from the first entity. The parties who enter into an agreement to retrieve data (e.g., analyte measurements 110) from a first entity via an API may be referred to as "data partners." In operation, the API allows a third party to make a request for data (e.g., analyte measurements 110 and / or outcome data 306) maintained on the storage device 112 in a particular request format, and if the request is made in the particular format, the first entity provides the requested data in a particular response format. The requested data may be provided in the particular response format (e.g., packets) in one or more communications over a network, e.g., the Internet. Examples of second entities that may be considered "third parties" include various service providers, such as service providers that provide one or more health monitoring / tracking services, fitness-related services, telemedicine services, medical laboratory services, and the like.In practice, the storage device 112 and the observational analysis platform 108 may be implemented using a variety of devices and / or resources (e.g., computing, communication, storage, etc.), and the division (or non-division) between entities corresponding to the various devices and / or resources may vary from that described above without departing from the spirit or scope of the technology described herein.

[0104] In any event, the observational analysis platform 108 is configured to obtain the analyte measurements 110 of the humans 102, as well as the analyte measurements 110 and outcome data 306 of the user population 304, and process them according to the described techniques. For example, using the analyte measurements 110 and outcome data 306 of the user population 304, the prediction system 114 is configured to generate one or more machine learning models, such as regression models, neural networks, and / or reinforcement learning agents. Once one or more such models are generated, the prediction system 114 is configured to process the analyte measurements 110 of the humans 102 to predict a health state classification 116 of the humans 102 using the one or more models, as described further herein with respect to FIG.

[0105] In the illustrated exemplary implementation 300, the prediction system 114 is shown to output a notification 314. The notification 314 may be based on or include the health condition classification 116 (not shown in FIG. 3). Consider an example where the health condition classification 116 output by one or more machine learning models of the prediction system 114 is a label indicating that the human 102 is predicted to have diabetes, e.g., a text label such as "1" (where "0" indicates no diabetes), or "diabetes". In this case, simply providing the health condition classification 116 to the human 102 may be undesirable. If such information is not delivered with relevant educational materials or delivered in an appropriate setting and personalized manner, providing such information may affect the human 102 in a variety of negative ways, such as by causing confusion, anger, depression, and the like. Thus, the notification 314 may be based solely on the health condition classification 116, such as by informing the human 102 that the results of the observation period are available and instructing them to schedule an appointment with their associated healthcare provider.

[0106] In contrast, providing the health condition classification 116 to a healthcare provider of the human 102 may not be undesirable. Instead, it may be preferable to provide the health condition classification 116 to the healthcare provider (as opposed to not providing a classification) so that the healthcare provider can appropriately inform the human 102 and develop a treatment plan for the human 102. In such a scenario, the notification 314 may simply correspond to the health condition classification 116. Alternatively, the notification communicated to the healthcare provider (or others) may be configured as a report that includes the health condition classification 116 along with other information, such as a trace of the analyte measurements 110 of the human 102 over an observation period, measures derived from those analyte measurements 110, treatment recommendations (e.g., learned from historical data of the user population 304), etc. Examples of these notifications are discussed in more detail in connection with FIGS. 8 and 9.

[0107] As mentioned above and discussed in further detail below (e.g., with respect to FIG. 6), one or more machine learning models of the prediction system 114 may be trained using various features of analyte measurements 110 from the user population 304 to predict a health condition classification 116 of the human 102. However, it may be desirable to select a feature (or combination of features) that not only has clinical relevance for classifying a health condition, but also is robust (or less sensitive) to manufacturing variations of the analyte monitoring device 104 (e.g., of the analyte sensor 202 of the analyte monitoring device 104 shown in FIG. 2) to increase the accuracy rate of the classification. In contrast, using features of the analyte measurements 110 that are sensitive to manufacturing variations, such as sensor bias, to predict the health condition classification 116 may result in misclassification. For example, the misclassification may include a false positive for the health condition, a false negative for the health condition, etc.

[0108] Accordingly, Figure 4 illustrates an example implementation 400 for selecting features for training and using one or more machine learning models for classifying health conditions. Components previously introduced in Figures 1-3 are numbered the same and will not be introduced again.

[0109] The implementation 400 includes a model manager 402 configured to, among other tasks, evaluate and select features of the analyte measurements 110 that are robust for accurately classifying a health condition of interest, even in the presence of observed variability in the analyte measurements 110 that may be caused by manufacturing variability of the analyte sensor 202 (not shown in FIG. 4 ). The model manager 402 is shown to include multiple modules, including a pre-processing manager 404, a variability simulator 406, a feature builder 408, a predictor 410, and an evaluator 412. A “module” may include hardware and / or software systems that operate to perform one or more functions, such as those described below. For example, a module may include or be included in a computer processor, controller, or another logic-based device that performs operations based on instructions stored on a tangible and non-transitory computer-readable storage medium, such as a computer memory. Alternatively, a module may include a hardwired device that performs operations based on the device's hardwired logic. The various modules illustrated in the accompanying figures, including Figure 4, may represent hardware operating based on software or hardwired instructions, software directing the hardware to perform operations, or a combination thereof. The hardware may include electronic circuitry including one or more logic-based devices, such as a microprocessor, processor, controller, etc., and / or connected to these logic-based devices, as described in more detail with respect to Figure 15. It may be understood that the model manager 402 may include more, fewer, or different modules than those illustrated in Figure 4 without departing from the scope of the present disclosure.

[0110] In the illustrated implementation 400, the model manager 402 is shown to obtain analyte measurements 110 and outcome data 306 of a user population 304, for example from a storage device 112. In general, the pre-processing manager 404 is configured to pre-process the analyte measurements 110 to determine a time-ordered sequence of the analyte measurements 110 according to their respective timestamps. Due to corruption and communication errors, the analyte measurements 110 obtained by the prediction system 114 may not only be out of time order, but one or more measurements may be missing. For example, there may be gaps in the time-ordered sequence where one or more measurements are expected. In these cases, the pre-processing manager 404 may be further configured to interpolate the missing analyte measurements and incorporate these values ​​into the time-ordered sequence. Additionally or alternatively, the pre-processing manager 404 may be configured to filter out portions of the analyte measurements 110 according to a predetermined criterion, such as to remove corrupted or poor signal quality data. Although this functionality is discussed, in one or more implementations, the analyte measurements 110 obtained by the prediction system 114 may already be in chronological order (e.g., one or more time series of analyte measurements 110) such that ordering of the measurements and interpolation of missing measurements is not performed by the pre-processing manager 404.

[0111] In the illustrated implementation 400, the pre-processing manager 404 is shown to output pre-processed data 414 including analyte measurements 110 in a sequential time series format, as discussed above. The pre-processed data 414 is input to a variability simulator 406. The variability simulator 406 is configured to introduce manufacturing-related sensor variability into the pre-processed data 414 by running variability simulations. For example, the variability simulator 406 may run multiple variability simulations over multiple simulation rounds, each with a different percentage of simulated manufacturing-related analyte sensor variability added to the pre-processed data 414 before the data is input to the feature builder 408. As an example, the variability simulator 406 may apply different performance variability and analyte sensor characteristics to the pre-processed data 414 during each of the multiple simulation rounds. In some examples, the variability simulator 406 may simulate bias with a fixed variability (e.g., standard deviation) applied per trace of the pre-processed data 414. As an example, a multiplicative variability (e.g., percentage variability) may be drawn from a normal distribution with a fixed standard deviation and a mean that is swept between a first and a second percentage variability for each simulation round and applied to the preprocessed data 414. The second percentage variability may be greater than the first percentage variability. As a non-limiting example, the first percentage variability may be a negative value (e.g., −8%, or a value between −10% and −5%) and the second percentage variability may be a positive value (e.g., 11%, or a value between 10% and 15%). As another non-limiting example, the fixed standard deviation may be 8.

[0112] The preprocessed data 414, including those with no simulated variability (e.g., 0% variability) and those with a known amount of simulated variability, are input to a feature builder 408. The feature builder 408 may be configured to generate data (e.g., one or more feature vectors) that may be evaluated in connection with predicting a health condition classification. In the illustrated implementation 400, the feature builder 408 is shown to output extracted analyte features 416 of the preprocessed data 414, such as from each user of the user population 304. The feature builder 408 may determine the extracted analyte features 416 by processing the preprocessed data 414 according to one or more predefined algorithms or functions. Each of the different extracted analyte features 416 may correspond, at least in some examples, to a different algorithm or function by which the feature builder 408 processes the preprocessed data 414.

[0113] Here, the extracted analyte features 416 include trend-related features 418, time- and day-related features 420, variability and stability features 422, frequency-related features 424, and value-based features 426. It should be understood that the extracted analyte features 416 may vary from the illustrated combination without departing from the spirit or scope of the described technique. As discussed above, the preprocessed data 414 may be configured as time series data such that each measured analyte level is associated with a time point and is ordered with respect to time. That is, a first analyte measurement taken at an earlier time is placed in the time series data before a second analyte measurement taken at a later time. This time series arrangement of the preprocessed data 414 may enable the feature builder 408 to extract features related to the time trend and stability of the analyte measurements 110, which may provide feature candidates that are less sensitive to differences in the output of the analyte sensor 202 due to manufacturing variations of the analyte sensor 202.

[0114] The trend-related features 418 may include features describing patterns or trends in the time series data. For example, the trend-related features 418 may include features describing relationships between analyte values ​​(e.g., levels) taken at different times (e.g., 5-minute intervals, 10-minute intervals, 15-minute intervals), such as autocorrelation features. The trend-related features 418 may further include rate-of-change features. These rate-of-change features may correspond to the difference in the analyte measurements 110 of a given user of the user population 304 over a unit of time. As an example, the pre-processing manager 404 may determine the difference in the measured amount of the analyte and the difference in time between at least two measurements, such that a change in the amount of the analyte over a unit of time may be determined. It should be appreciated that such a rate of change may be determined using three or more of the analyte measurements 110 in the pre-processed data 414. In some examples, the rate-of-change features may include statistics regarding the rate of change of the analyte measurements 110, such as the standard deviation of the rate of change of the analyte measurements 110.

[0115] The time and day related features 420 may include features describing how analyte dynamics vary from day to day and / or based on time of day. Examples of time and day related features 420 may include, for example, mean analyte levels for a particular day, mean analyte levels for a particular time of day, rate of change of analyte levels for a particular day, rate of change of analyte levels between particular times of day, and corresponding statistics (e.g., standard deviation, coefficient of variation, skewness, kurtosis, etc.). For example, the time and day related features 420 may include day statistical features and time statistical features. As another example, the time and day related features 420 may include differences between mean values ​​of analyte measurements 110 for different days (e.g., mean day differences), differences between mean values ​​of analyte measurements 110 for different times of day (e.g., between awake and asleep hours), differences between standard deviations of analyte measurements 110 for different times of day, etc.

[0116] The variability and stability features 422 may include features indicative of how stable or variable the analyte measurements are. For example, the variability and stability features 422 may include a setpoint frequency feature and a peak feature. The setpoint frequency feature may be, for example, a measure of how often the analyte level is within a predefined range of a setpoint value. The setpoint value may be, for example, a mode of the analyte measurement 110. The peak feature may include statistics describing the analyte dynamics during a peak in the analyte measurement 110 (e.g., where the analyte level reaches a local maximum). For example, the peak feature may be a measure of the peak width and / or height relative to a stable baseline analyte level of the user. The stable baseline analyte level may be, for example, a mode or a mode proxy.

[0117] The frequency related features 424 may include features extracted from the frequency transformed data of the time series preprocessed data 414. The frequency related features 424 may indicate, for example, a dominant frequency of analyte variability. The frequency related features 424 may enable additional information to be extracted from the time domain data, such as different frequencies into which the time domain data may be resolved.

[0118] The value-based features 426 may include, for example, the mean analyte value and / or median analyte value of the analyte measurements 110 for a given user of the user population 304, and corresponding statistical measures (e.g., standard deviation, skewness, kurtosis, coefficient of variation, statistical distribution, etc.). The value-based features 426 may further include interquantile range differences and time-based threshold measures. For example, the value-based features 426 may include an in-range time measure corresponding to the amount of time during the data collection period that the user's analyte measurements 110 are between a first analyte level corresponding to the upper and lower limits of the range, respectively, and a second analyte level that is less than the first analyte level. As another example, the pre-processing manager 404 may determine an out-of-range time measure corresponding to the amount of time over the course of the observation period that the user's analyte measurements 110 are outside such range. Additionally or alternatively, the value-based features 426 may include event occurrence-based features that may indicate the occurrence of a user's analyte measurement 110 increasing above a first analyte level (e.g., a hyperglycemic event) and / or decreasing below a second analyte level (e.g., a hypoglycemic event). Additional examples of value-based features 426 may include values ​​corresponding to a threshold percentile for the analyte (e.g., a statistically significant threshold percentile such as the 94th percentile or higher), a 10-90th percentile range for the analyte, and amplitude-based features (e.g., the mean amplitude of analyte level excursions), to name a few.

[0119] The predictor 410 is configured to provide model predictions 428 for the extracted analyte features 416, alone and in combination. For example, the model predictions 428 can be binary or probability-based outputs of health state predictions for each of the extracted analyte features 416 for each amount of manufacturing variability (or lack thereof, such as when the percent variability is 0%) simulated by the variability simulator 406. The predictor 410 can utilize both univariate feature models that include one of the extracted analyte features 416 and multivariate models (e.g., bivariate models) that combine at least two of the extracted analyte features 416. As a non-limiting example, the predictor 410 can utilize regression models, such as linear or logistic regression models, of the extracted analyte features 416 and various combinations thereof.

[0120] The evaluator 412 is configured to receive the model predictions 428 and categorize the one or more extracted analyte features 416 according to one or more metrics. Here, the evaluator 412 is shown to determine a performance metric 430 and a robustness metric 432. The performance metric 430 may provide an indication of how well each of the one or more extracted analyte features 416 classifies a health condition based on the outcome data 306 for a given user of the user population 304 compared to the model predictions 428, and the robustness metric 432 may provide an indication of how insensitive each of the one or more extracted analyte features 416 is to simulated manufacturing variations, as described in more detail below.

[0121] The performance metric 430 may be a measure of one or both of the true positive rate (e.g., sensitivity) at which the model prediction 428 of the one or more extracted analyte features 416 correctly predicts that an associated user of the user population 304 has a health condition and the true negative rate (e.g., specificity) at which the model prediction 428 of the one or more extracted analyte features 416 correctly predicts that an associated user of the user population 304 does not have a health condition. For example, the higher the performance metric 430, the more sensitive and specific the one or more extracted analyte features 416 are predicted to be for classifying a health condition. As an example, the performance metric 430 may rank the one or more extracted analyte features 416 on a predefined scale, such as a scale of 0 to 1, with 0 indicating that none of the corresponding model predictions 428 (e.g., 0%) are accurate and 1 indicating that all of the corresponding model predictions 428 (e.g., 100%) are accurate.

[0122] The robustness metric 432 may be a measure of how much the performance metric 430 changes for each percent variability added to the preprocessed data 414 by the variability simulator 406 as averaged over iterations for each user of the user population 304. For example, the robustness metric 432 may combine variability sensitivity to true positive rate and true negative rate. The robustness metric 432 may rank the one or more extracted analyte features 416 on a predefined scale, with the highest value indicating no change in the performance metric 430 in response to the simulated variability and the lowest value indicating the greatest change in the performance metric 430 in response to the simulated variability. The higher the robustness metric 432, the less sensitive the corresponding one or more extracted analyte features 416 are predicted to be to manufacturing variability of the analyte sensor. In some examples, in addition to or instead of the robustness metric 432, the evaluator 412 may determine a variability sensitivity metric, which may be the inverse of the robustness metric 432. For example, the higher the variability sensitivity metric, the corresponding one or more extracted analyte features 416 are predicted to be more sensitive (and less robust) to manufacturing variations in the analyte sensor that may affect the analyte measurement 110. Generally, highly robust analyte features (e.g., those having a high robustness metric 432 or a low variability sensitivity metric) correspond to extracted analyte features 416 that exhibit little change in performance metric 430 with various amounts of simulated variability. As a non-limiting example, trend-related features 418 and variability and stability features 422 may produce extracted analyte features 416 that have a relatively high robustness metric 432 (and a relatively low variability sensitivity metric).

[0123] Based on the robustness metric 432 and the performance metric 430 of the various extracted analyte features 416, the evaluator 412 is configured to select and output a robust analyte feature combination 434. The robust analyte feature combination 434 may balance performance and robustness to generate a health condition classification model that accurately predicts a health condition with high sensitivity and specificity that is relatively unaffected by manufacturing variations that may affect analyte sensor output and performance. As described further below with respect to FIG. 5, the robust analyte feature combination 434 may include at least two of the extracted analyte features 416. For example, a model built from at least two extracted analyte features 416 may exhibit a greater combination of robustness and performance than a single extracted analyte feature 416. As an example, a first one of the analyte features of a robust analyte feature combination 434 may have a higher performance metric 430 than a second one of the analyte features in the robust analyte feature combination 434, which may have a higher robustness metric 432 than the first one. In another example, a first one of the analyte features may have a higher performance metric 430 and a higher robustness metric 432 than a second one of the analyte features in the robust analyte feature combination 434.

[0124] As a non-limiting example where diabetes is the health condition and glucose is the analyte, a value-based feature 426 such as mean glucose may have a relatively high performance metric 430 because elevated glucose is a hallmark of diabetes. However, at least some of the value-based features 426 may also be relatively sensitive to manufacturing variations in the analyte sensor 202. For example, increasing the simulated positive bias, where glucose measurements are shifted higher, may decrease the true negative rate of univariate models of at least some of the value-based features 426 due to the model incorrectly predicting that users without diabetes have diabetes due to the elevated glucose levels. Similarly, increasing the simulated negative bias, where glucose measurements are shifted lower, may decrease the true positive rate of univariate models of at least some of the value-based features 426 due to the model incorrectly predicting that users with diabetes do not have diabetes due to the lowered glucose levels. As a result, at least some of the univariate models of the value-based features 426 may have a relatively low robustness metric 432. Thus, in one example, one of the value-based features 426 may be balanced with a feature having a high robustness metric 432, such as one of the trend-related features 418, in a combination of robust analyte features 434.

[0125] Although the above examples describe the robust analyte feature combinations 434 as including features belonging to different categories (or types), it can be appreciated that in other examples, the robust analyte feature combinations 434 can include features from the same category. For example, each of the features in the robust analyte feature combinations 434 can be one of the trend-related features 418. As another example, each of the features in the robust analyte feature combinations 434 can be one of the value-based features 426. Thus, it should be appreciated that the above examples are provided for illustration and not limitation.

[0126] To select the robust analyte feature combination 434, the evaluator 412 may filter out multivariate model predictions 428 that have a robustness metric 432 that is below a predetermined threshold. The evaluator 412 may select the multivariate model prediction 428 with the highest performance metric 430 of the remaining candidates, and the associated extracted analyte features 416 may be output as the robust analyte feature combination 434.

[0127] 5, an example of a set of graphs 500 for categorizing extracted analyte features (e.g., extracted analyte features 416 of FIG. 4) to identify robust analyte feature combinations 434 is shown. The graphs 500 include a first graph 502 of a univariate feature model and a second graph 504 of a bivariate feature model, where the vertical axis of each graph represents a variability sensitivity metric and the horizontal axis of each graph represents a performance metric. The variability sensitivity metric may be, for example, the inverse of the robustness metric 432 of FIG. 4, and the performance metric may be the performance metric 430 of FIG. 4. Each data point represents the variability sensitivity metric and the performance metric of each univariate (first graph 502) or bivariate (second graph 504) model prediction (e.g., model prediction 428 of FIG. 4) for predicting a health condition classification using the associated extracted analyte features. That is, a first graph 502 shows individual performance metrics and individual variability sensitivity metrics for model predictions using each extracted analyte feature individually, while a second graph 504 shows performance metrics and variability sensitivity metrics for model predictions using two of the extracted analyte features in combination.

[0128] Referring initially to the first graph 502, the legend 506 represents the category of each extracted analyte feature according to the fill pattern of the data points. In this example, the high-performing features 508 are indicated by darker dot shading and have a relatively higher performance metric than the other features, e.g., a performance metric of at least 0.97. As described above with respect to FIG. 4, the performance metric may rank each extracted analyte feature on a scale of 0 to 1, with 0 indicating that none (e.g., 0%) of the univariate model predictions for that feature are accurate in predicting the health condition classification, and 1 indicating that all (e.g., 100%) of the univariate model predictions for that feature are accurate in predicting the health condition classification. In the example shown in the first graph 502, the high-performing features 508 also all have a relatively high variability sensitivity metric, indicating that these high-performing features 508 are more sensitive to the effects of manufacturing variability on the analyte measurements in predicting the health condition classification.

[0129] Intermediate features 510 are indicated by intermediate dot shading that is lighter than the dark dot shading of the high-performing features 508. In accordance with the described techniques, intermediate features 510 are those that have both an intermediate performance metric and an intermediate variation sensitivity metric. For example, in the example shown in the first graph 502 of FIG. 5, intermediate features 510 have a performance metric between 0.75 and 0.97 and a variation sensitivity metric between 0.3 and 1.2. Different ranges of performance metric and variation sensitivity may be used to define intermediate features 510 (and other types of features) in variation.

[0130] In this example, highly robust features 512 are indicated by horizontal shading and have a relatively lower variability sensitivity metric than the other features, e.g., in this example, a variability sensitivity metric of less than 0.2. As described above with respect to FIG. 4, highly robust features are identified as those that experience little change in performance when simulated manufacturing related variability is added to the analyte measurements. The highly robust features 512 shown in the first graph 502 have a performance metric in the range of 0.80 to 0.95.

[0131] In this example, poorly performing features 514 are indicated by white fill (e.g., no shading). Here, poorly performing features 514 are defined as features with a performance metric less than 0.65, indicating that these features may not be accurately reliable for predicting health condition classifications using machine learning models. However, in various scenarios, poorly performing features 514 also have high robustness, as indicated by their low variability sensitivity metric.

[0132] Thus, in some examples, such as the example shown in FIG. 5, it may be beneficial to combine at least two extracted analyte features to generate a feature combination with a higher performance metric and lower variability sensitivity metric than any single extracted analyte feature. Referring now to the second graph 504, a legend 516 represents the category of analyte feature combinations shown at each data point according to the fill of the data point. In the example shown in the second graph 504, a high performing feature combination 518 including two high performing features from the first graph 502 is indicated by a dark dot shading. A high and medium performing feature combination 520 including one high performing feature and one medium feature from the first graph 502 is indicated by a vertical line shading. A medium feature combination 522 including two medium features from the first graph 502 is indicated by a medium dot shading. A high robustness and medium feature combination 524 including one high robustness feature and one medium feature of the first graph 502 is indicated by lane shading, a high robustness feature combination 526 including two high robustness features of the first graph 502 is indicated by horizontal line shading, and a low robustness feature combination 528 including two low performing features of the first graph 502 is indicated by white fill (or no fill).

[0133] The robustness threshold 530 shown in the second graph 504 may be used (e.g., by the evaluator 412) to filter out feature combinations of the bivariate model for consideration as robust analyte feature combinations 434. In one or more implementations, the robustness threshold 530 is a non-zero, pre-defined variability sensitivity metric value calibrated to distinguish feature combinations that are robust to analyte sensor manufacturing variability from feature combinations that are not robust to analyte sensor manufacturing variability. Thus, feature combinations having a variability sensitivity metric below the robustness threshold 530 are considered robust. As a non-limiting example, the robustness threshold 530 may be 0.5, as shown in FIG. 5. In other examples, the robustness threshold 530 may be set to another value, such as 0.4.

[0134] In yet other examples, a robustness metric may be used in place of a variability sensitivity metric. In such examples, the robustness threshold 530 may be a non-zero predetermined robustness metric value above which a feature combination is deemed robust to manufacturing variability of the analyte sensor. Thus, when a robustness metric is used in place of a variability sensitivity metric, feature combinations that are below the robustness threshold 530 may be filtered out from consideration for a robust analyte feature combination 434.

[0135] In some examples, such as the example shown in Figure 5, the feature combination with the largest performance metric that is not filtered out by the robustness threshold 530 (e.g., is less than the robustness threshold 530 when the robustness threshold 530 is the variability sensitivity metric value) is selected as the robust analyte feature combination 434. The robust analyte feature combination 434 is the high robustness and medium feature combination 524 in the example shown in the second graph 504, but in other examples, the robust analyte feature combination 434 may be a combination of features of different categories. Thus, it can be understood that the example shown in Figure 5 is for illustration and not for limitation.

[0136] In some implementations, additional aspects may be evaluated (e.g., in addition to the performance metric and robustness threshold 530) in selecting the robust analyte feature combinations 434. The additional aspects may include, but are not limited to, ease of computation of the features in the feature combinations, ability to determine the features in the feature combinations using hardware alone, amount of data needed to compute the features in the feature combinations (e.g., collection over multiple days vs. collection over several hours), etc. Thus, in some implementations, for a selected number of top candidate feature combinations with the highest performance metrics that are not filtered out by the robustness threshold 530, an additional metric (or score) may be determined (e.g., via the evaluator 412) based on these additional aspects. As a non-limiting example, the additional metric may be determined for the candidate feature combinations with the three largest performance metrics. In some implementations, the additional metric may be added to the performance metric of each of the selected number of top candidate feature combinations, and the top candidate feature combinations with the highest sum may be selected as the robust analyte feature combinations 434. In this way, robust analyte feature combinations 434 may include feature combinations that are easier to analyze than the feature combinations with the highest performance metrics.

[0137] 5 illustrates filtering the feature combinations of the second graph 504 using a robustness threshold 530, in other examples, a performance metric threshold may be used instead of or in addition to the robustness threshold 530 to filter the feature combinations. For example, the feature combination with the lowest variability sensitivity metric (or highest robustness metric) greater than the performance metric threshold may be selected as the robust analyte feature combination 434. Additional examples of how robust analyte feature combinations 434 may be selected based on a variability sensitivity metric (or robustness metric) and a performance metric are described below with respect to FIG. 11.

[0138] The robust analyte feature combination 434 determined via the example implementation 400 of Figure 4 and illustrated in the second graph 504 of Figure 5 may be used to train a machine learning model for predicting a health condition classification. Referring now to Figure 6, an example of an implementation 600 of the prediction system 114 in which a machine learning model 602 is trained via a model manager 402 is shown in more detail. Components previously introduced in Figures 1-4 are numbered the same and will not be introduced again.

[0139] In the illustrated implementation 600, the prediction system 114 includes a machine learning model 602 and a model manager 402 that manages the machine learning model 602. According to the described techniques, the machine learning model 602 may represent a single machine learning model or an ensemble of multiple models. The machine learning model 602 may correspond to different types of machine learning models, where the underlying models are trained using various techniques, such as using supervised learning, unsupervised learning, and / or reinforcement learning. By way of example, these models may include regression models (e.g., linear, polynomial, and / or logistic regression models), classifiers, neural networks, and reinforcement learning-based models, to name a few.

[0140] The machine learning models 602 may be configured as or include other types of machine learning models without departing from the spirit or scope of the described techniques. These different machine learning models may each be built or trained (or the models learned differently) using different data and different algorithms due at least in part to different architectures and / or paradigms. Thus, it should be understood that the following discussion of the functionality of the model manager 402 is applicable to a variety of machine learning models. However, for purposes of explanation, the functionality of the model manager 402 in training the machine learning models 602 is generally described in the context of statistical models and neural networks.

[0141] Generally speaking, the model manager 402 is configured to manage machine learning models, including the machine learning model 602. This model management includes, for example, building the machine learning model 602, training the machine learning model 602, updating the model, etc. In particular, the model manager 402 is configured to perform this model management, at least in part, using a wealth of data maintained in the storage device 112. As illustrated, this data includes the analyte measurements 110 and outcome data 306 of the user population 304. Stated another way, the model manager 402 builds the machine learning model 602, trains the machine learning model 602 (or otherwise learns the underlying model), and updates the model using the analyte measurements 110 and outcome data 306 of the user population 304. For example, the model manager 402 may select robust analyte feature combinations 434 based on the analyte measurements 110 and outcome data 306 of the user population 304 and use the robust analyte feature combinations 434 as inputs to train the machine learning model 602 according to the implementation 400 discussed above with respect to FIG. 4. In implementations in which the machine learning model 602 receives data in addition to the robust analyte feature combinations 434 as inputs, such as additional data from the user population 304, the model manager 402 also uses such additional data to build, train, and update the machine learning model 602.

[0142] In one or more implementations, the model manager 402 generates training data for training the machine learning model 602 or otherwise learning the parameters of the model. Generally speaking, the generation of the training data depends on the health condition classification that the machine learning model is designed to output. This training data will be different, for example, if the machine learning model 602 is configured to generate a human diagnostic measure, a side effect predicted to be experienced by a human, or a prediction of a human clinical diagnosis. Regardless of the predicted outcome, generating the training data may include time-ordering the analyte measurements 110 of the user population 304 (if the analyte measurements 110 are not already time-ordered) and extracting analyte features from those time-ordered analyte measurements 110, as described above with respect to FIG. 4. The model manager 402 may leverage the functionality of the preprocessing manager 404 to form the preprocessed data 414 of the time-ordered analyte measurements 110 and the functionality of the feature builder 408 to extract analyte features, for example, in a manner similar to that discussed above in connection with generating the extracted analyte features 416. As an example, once the robust analyte feature combinations 434 are selected, the model manager 402 may use the feature builder 408 to extract the robust analyte feature combinations 434 from the training data. The model manager 402, at least in some examples, may not utilize the variability simulator 406 in training the machine learning model 602.

[0143] Generating the training data also includes associating the traces of analyte measurements 110, or the combination of robust analyte features extracted from the analyte measurements 110, 434, with the outcome data 306 of each user of the user population 304. With this in mind, the analyte traces or extracted analyte features corresponding to a particular user are associated with the outcome data 306 of the particular user. As an example, a particular user may have been clinically diagnosed with diabetes, and their glucose may be above a threshold for an amount of time corresponding to 27% of the observation period. With this in mind, the model manager 402 may form a training instance that includes an input portion with a value indicative of the user's time above the threshold 27% of the time, and an associated output portion with a value indicative of the person having diabetes, e.g., "1" or some other corresponding value.

[0144] In one or more implementations, the model manager 402 may build a statistical model by extracting from the outcome data 306 observed values ​​or labels corresponding to at least one type of outcome, such as values ​​of the clinical diagnosis 312, e.g., “diabetes”, “pre-diabetes”, “non-diabetes”, or values ​​indicative of those labels. Once built, the statistical model is configured to predict the values ​​or labels of the at least one outcome type and output them as the health condition classification 116. The values ​​or labels indicative of the at least one outcome type do not act as inputs to the model. In a scenario where the statistical model is a regression model, for example, the outcome values ​​or labels may correspond to one or more dependent variables. In contrast, each feature of the robust analyte feature combinations 434 extracted from the analyte measurements 110 may act as an input to the model. Thus, in a scenario where the machine learning model 602 is configured as a statistical model, at least two analyte features of the robust analyte feature combinations 434 may correspond to at least two explanatory (or independent) variables.

[0145] Given a set of outcome values ​​or labels from the outcome data 306 and a set of values ​​of the robust analyte feature combinations 434 extracted from the analyte measurements 110, the model manager 402 uses one or more known techniques to "fit" these sets of values ​​to an equation such that, in response to the input of the extracted analyte feature values, the model manager 402 produces outcome values ​​or labels within some tolerance. Examples of such fitting techniques include using least squares, using least absolute deviations regression, minimizing a penalized version of a least squares cost function (e.g., ridge regression or lasso), etc. By "fitting" it is meant that the model manager 402 estimates the model parameters of the equation using one or more techniques and these sets of values ​​from the training data.

[0146] The estimated parameters include, for example, weights that are applied to values ​​of independent variables (e.g., robust analyte feature combinations 434) as they are input to the machine learning model during operation. The model manager 402 incorporates these parameters estimated from fitting the observations of the user population 304 into a formula to generate the machine learning model 602 as a statistical model. During operation, the prediction system 114 inputs the values ​​of the independent variables (e.g., values ​​of robust analyte feature combinations 434) into the statistical model (e.g., as one or more vectors or matrices), which applies the estimated weights to these input values, and then outputs values ​​or labels of one or more dependent variables. This output corresponds to a health condition classification 116.

[0147] In the following discussion, the ability of the model manager 402 to build and train machine learning models is discussed in relation to the configuration of machine learning models 602 that correspond to or include at least one neural network.

[0148] With respect to the training data used, the model manager 402 may generate training data instances that include an input portion and an expected output portion, i.e., ground truth for comparison with the model's output during training, as described above. The input portion of the training data instance may correspond to one or more traces of analyte measurements 110 and / or one or more extracted features in the robust analyte feature combinations 434 for a particular user. The output portion may correspond to one or more values ​​of outcome data 306 for a particular user, such as a value indicative of or confirming a clinical diagnosis of a health condition (e.g., the user's observed HbA1c when the health condition is diabetes). Again, whether a trace is used for training, as well as the features of the robust analyte feature combinations 434 used for training, and which outcome data is used for training, depend on the data that the machine learning model 602 is designed (and trained) to receive as input and the data that it is designed (and trained) to output.

[0149] The model manager 402 uses the training input portions along with the respective expected output portions to train the machine learning model 602. In the training context, the model manager 402 may train the machine learning model 602 by providing instances of data from a set of training input portions to the machine learning model 602. In response, the machine learning model 602 generates a prediction of the disease of the condition classification, such as by predicting a value indicative of a clinical diagnosis. The model manager 402 takes this training prediction from the machine learning model 602 as an output and compares the training prediction to the expected output portion corresponding to the training input portion. For example, if the machine learning model 602 outputs a diabetes classification indicating that the user has diabetes, the prediction is compared to the output data (e.g., classifying the user as having or not having diabetes) to determine if the prediction was correct. Based on this comparison, the model manager adjusts the internal weights of the machine learning model 602 such that the machine learning model can substantially reproduce the expected output portion when the respective training input portion is provided as an input in the future.

[0150] This process of inputting instances of training input portions into the machine learning model 602, receiving training predictions from the machine learning model 602, comparing the training predictions (e.g., using a loss function such as mean squared error) to expected output portions (observed) that correspond to the input instances, and adjusting the internal weights of the machine learning model 602 based on these comparisons may be repeated over hundreds, thousands, or even tens of thousands of iterations, using an instance of the training data for each iteration.

[0151] The model manager 402 may perform such iterations until the machine learning model 602 is able to consistently generate predictions that substantially match the expected output portion. The ability of the machine learning model to consistently generate predictions that substantially match the expected output portion may be referred to as "convergence." With this in mind, it may be said that the model manager 402 trains the machine learning model 602 until the machine learning model 602 "converges" to a solution, e.g., the model's internal weights are suitably adjusted due to training iterations such that the model consistently generates predictions that substantially match the expected output portion.

[0152] As described above, the machine learning model 602, in one or more implementations, may be configured to receive inputs in addition to the analyte measurements 110 and / or the robust analyte feature combinations 434 extracted from those measurements. In such implementations, the model manager 402 may form a training instance that includes the training input portions, the respective expected output portions, and additional input data describing any other aspects of the user population 304 being used to predict the health condition classification, such as demographic information, medical history, exercise, and / or stress. This additional data as well as the training input portions may be processed by the model manager 402 according to one or more known techniques to generate an input vector. This input vector describing the training input portions as well as other aspects may then be provided to the machine learning model 602. In response, the machine learning model 602 may generate a prediction of the health condition classification in a manner similar to that discussed above, such that the prediction is compared to the expected output portions of the training instances, and the model weights may be adjusted based on the comparison.

[0153] It may be appreciated that in some examples, at least a portion of the implementation 400 of Figure 4 and the implementation 600 of Figure 6 may be performed in combination. For example, the variation simulator 406 may introduce manufacturing-related sensor variation when adjusting the machine learning model 602. As an example, the simulated manufacturing-related sensor variation may be applied within cross-validation to a validation fold of a candidate model of the machine learning model 602 before evaluating the candidate model on the validation fold.

[0154] Once the machine learning model 602 is trained, it is used to predict a health condition classification. Consider the following discussion of FIG. 7 in the context of predicting a health condition classification 116 of a human 102 from an analyte measurement 110 of the human 102.

[0155] Figure 7 illustrates in more detail one example of an implementation 700 of the prediction system of Figure 1 in which a health state classification is predicted using machine learning. Accordingly, components previously introduced in Figures 1-6 are numbered the same and will not be introduced again.

[0156] In the illustrated implementation 700, the prediction system 114 is shown to obtain an analyte measurement 110, for example, from a storage device 112 (not shown in FIG. 7), where the analyte measurement 110 corresponds to a human 102. In the implementation 700, the prediction system 114 is shown to include a pre-processing manager 404, a feature builder 408, and a machine learning model 602, which are configured to generate a prediction of a health condition classification 116 based on the analyte measurement 110 of the human 102. Although the prediction system 114 is shown to include these three components, it should be understood that the prediction system 114 may have more, fewer, and / or different components to generate a health condition classification 116 based on the analyte measurement 110 without departing from the spirit or scope of the described techniques.

[0157] In one or more implementations, the analyte measurements 110 are configured as time-ordered data, such that each of the analyte measurements 110 corresponds to a timestamp, as described above with respect to FIG. 4. Thus, the pre-processing manager 404 may be configured to determine a time-ordered sequence of the analyte measurements 110 of the human 102 according to the respective timestamps and generate pre-processed data 414. The pre-processed data 414 may be input to a feature builder 408, which may be configured to extract analyte features from the pre-processed data 414 (e.g., feature vectors), which may be provided as input to the machine learning model 602, and data that may be reported in association with the health condition classification 116 (e.g., included as part of the notification 314 shown in FIG. 3). In particular, the machine learning model 602 may be trained to predict the condition classification 116 using the robust analyte feature combinations 434 selected via the implementation 400 of FIG. 4, as described above with respect to the implementation 600 of FIG. 6. Thus, in the illustrated implementation 700, the feature builder 408 is shown to output the robust analyte feature combinations 434. The feature builder 408 may determine the robust analyte feature combinations 434 by further processing the preprocessed data 414 of the analyte measurements 110 according to one or more predefined algorithms or functions. Each of the different extracted analyte features of the robust analyte feature combinations 434 may correspond to a different algorithm or function by which the feature builder 408 processes the preprocessed data 414 of the analyte measurements 110.

[0158] In addition to the robust analyte feature combinations 434, the feature builder 408 may also, at least in some examples, incorporate features from additional data describing different aspects of the human 102. This additional data may include data for a second analyte different from the analyte measurement 110 (e.g., a lactate measurement where the analyte measurement 110 is a glucose measurement), environmental data (e.g., a human's temperature), previously observed side effect data (e.g., data describing any of a variety of side effects associated with a previously observed health condition classification), demographic data collected or otherwise obtained through a questionnaire (e.g., describing age, gender, ethnicity), medical history data, stress data, nutritional data, exercise data, prescription data, height and weight data, occupational data, etc. In other words, the data provided as input to the machine learning model 602 or ensemble of machine learning models may, in one or more implementations, describe various aspects about the human 102 (e.g., as features of an input feature vector) in addition to the robust analyte feature combinations 434 without departing from the spirit or scope of the described techniques. In such a scenario, the machine learning model 602 is trained using similar historical data of the user population 304 (see FIG. 6).

[0159] Although the illustrated implementation 700 shows the feature builder 408 processing the preprocessed data 414 to generate robust analyte feature combinations 434 and using those features as inputs to the machine learning model 602 (e.g., feature vectors indicative of the extracted features), in one or more implementations, the feature builder 408 may also generate feature vectors that represent (alone or with other features) one or more time series of analyte measurements 110 (e.g., traces). Thus, the input data to the machine learning model 602 may correspond to or otherwise include a vectorized time series of analyte measurements 110, or multiple vectorized time series of analyte measurements 110. In implementations in which the time series of analyte measurements 110 are vectorized, the machine learning model 602 may correspond to, for example, a neural network. In implementations in which the robust analyte feature combinations 434 are vectorized, the machine learning model may correspond, for example, to a multivariate regression model, such as a multivariate linear regression model or a logistic regression model.

[0160] In response to receiving input data from the feature builder 408, the machine learning model 602 is configured to generate and output the health condition classification 116. In particular, the machine learning model 602 may be trained to output the health condition classification 116. By way of example, as detailed above with respect to FIG. 6, the machine learning model 602 may be trained, or underlying representations may be learned, based on one or more training techniques and using historical analyte and outcome data from which a health condition classification may be derived, such as using the analyte measurements 110 and outcome data 306 of the user population 304. In accordance with the described techniques, the machine learning model 602 may represent one or more models, including, for example, a model trained to predict whether a person has a health condition, and in one or more implementations, an additional model for predicting whether a person does not have a health condition (e.g., a diabetes classification that may be used to screen a person as not having diabetes with a degree of certainty). Each of the models in the multi-model configuration may receive differently configured input data describing different aspects, such as a feature vector having features that represent different aspects related to a health condition. It should be understood that in other implementations, a single model may be configured to generate both types of predictions. In one or more implementations, the machine learning models 602 may be configured as an ensemble of models, each of which generates a different prediction for a health condition than the other models.

[0161] The health condition classification 116 may classify the human 102 with respect to one or more outcomes corresponding to outcomes described by the outcome data 306 used to train the machine learning model 602. In an implementation in which the machine learning model 602 is trained or learned using clinical diagnoses 312 of the user population 304, the machine learning model 602 may then classify the analyte measurements 110 of the human 102 into a class corresponding to one of the diagnoses. In an exemplary implementation in which the health condition is diabetes, the machine learning model 602 may classify the analyte measurements 110 of the human 102 as diabetes, pre-diabetes, or non-diabetes. To this end, a health care provider may use the health condition classification 116 to treat the human 102 or develop a treatment plan similar to how a health care provider would do so if the human 102 were diagnosed with diabetes according to conventional techniques, e.g., HbA1c, FPG, and / or 2Hr-PG.

[0162] Similarly, if the machine learning model 602 is trained or learned using the observed side effects 310 of the user population, the machine learning model 602 may output probabilities that the analyte measurements 110 of the human 102 indicate that the human is experiencing various side effects, e.g., a probability from 0 to 1 that the human experiences any of a variety of side effects associated with a health condition. In some implementations, there may be a machine learning model trained or constructed for each effect such that the machine learning model 602 represents an ensemble of models capable of generating predictions regarding whether the human 102 will experience each effect (or the probability that the human will experience each effect).

[0163] In implementations where the machine learning model is trained or learned using the independent diagnostic measures 308 of the user population, the machine learning model 602 may then output a prediction of the value of the particular diagnostic measure. Continuing with the exemplary implementation where the health condition is diabetes, the machine learning model 602 may output a prediction of the HbA1c value, the FPG value, the 2Hr-PG value, or the OGTT value. The health condition classification 116 output depends largely on how the machine learning model 602 was trained, and in particular, the information that the health condition classification 116 represents depends on the training. For example, the condition classification 116 may represent a label indicating whether or not a person has a health condition or whether or not a person is at risk of developing a health condition (e.g., a diabetes label, a pre-diabetes label, or a non-diabetes label), a label indicating whether or not a person has a particular type of health condition (e.g., type 1 diabetes, type 2 diabetes, and GDM), a probability, or a measurement. In addition, different types of machine learning models may be better suited to generate predictions regarding different types of outcomes that may be represented by the health condition classification 116.

[0164] Additionally, the condition classification 116 output by the machine learning model 602 may serve as the basis for a variety of information provided to the human 102 in connection with which the prediction is generated, as well as other information associated with the human, such as the human's 102 healthcare provider, caregiver, telemedicine or health tracking services, etc. In the context of information that may be output based on a prediction, consider the following discussion of Figures 8-9.

[0165] 8 illustrates an example of a user interface implementation 800 displayed to inform a user regarding a health condition prediction generated based on analyte measurements generated during an observation period. The following description refers to the components and features illustrated in FIGS. 1-7 for generating a health condition prediction. However, it can be understood that implementation 800 may use other features and components to inform a user regarding a health condition prediction generation without departing from the scope of the present disclosure.

[0166] The illustrated exemplary implementation 800 includes a computing device 802 displaying a user interface 804. In this exemplary implementation 800, the user interface 804 may correspond to the notification 314 of FIG. 3. This exemplary implementation 800 represents a scenario in which the notification 314 (e.g., output via the user interface 804) is generated based on, but does not include, the health condition classification 116 introduced in FIG. 1. Here, the computing device 802 may be associated with the human 102 for whom analyte measurements are collected during an observation period and in which the health condition classification 116 is generated. Alternatively, the computing device 802 may be associated with another human associated with the human 102, such as a caregiver.

[0167] To this end, the user interface 804 may be displayed to inform the human 102 (or a human associated with the human 102) regarding the health condition classification 116 without revealing the predicted classification. This is because the output of the health condition classification 116 to the human 102 may affect the human 102 in a variety of negative ways, such as by causing confusion, anger, depression, and the like. In this exemplary implementation 800, the user interface 804 includes an overview regarding the processing of the analyte measurements of the human 102. The user interface 804 also includes a recommendation of an actionable action based on the health condition classification (in this case, recommending that the human 102 follow up with his or her healthcare provider). In addition, the user interface 804 includes graphical user interface elements 806 that are selectable to execute the recommended actionable action. Each of the graphical user interface elements 806 may be selectable to schedule a follow-up appointment with the healthcare provider of the human 102, such as an appointment at the healthcare provider's physical location or via telephone or video conference, e.g., associated with telemedicine and / or telehealth services. It should be understood that notifications that are generated based on health condition classifications 116, but do not include classifications, may be configured in a variety of ways without departing from the spirit or scope of the described techniques.

[0168] FIG. 9 illustrates an example of a user interface implementation 900 displayed to report a user's health forecast along with other information generated in association with the health forecast. The following description refers to the components and features illustrated in FIGS. 1-7 for generating a health forecast. However, it can be understood that the implementation 900 can report a health forecast based on data generated and analyzed using other features and components without departing from the scope of the present disclosure. The exemplary implementation 900 includes diabetes as a health condition determined based on glucose measurements obtained from the user. However, it can be understood that the implementation 900 can be adapted to report predictions and associated information for other diseases and conditions without departing from the scope of the present disclosure.

[0169] The illustrated exemplary implementation 900 includes a display device 902 displaying a user interface 904 configured as a report. In this example, the user interface 904 may correspond to the notification 314 of FIG. 3. In contrast to the exemplary implementation 800 shown in FIG. 8, this exemplary implementation 900 represents a scenario in which the notification includes the health condition classification 116 introduced in FIG. 1. In this illustrated exemplary implementation 900, a graphical diagnostic element 906 represents or otherwise indicates the health condition classification 116. Here, the display device 902 may be associated with a health care provider affiliated with the human 102 for whom analyte measurements are collected during an observation period and for whom the health condition classification 116 is generated.

[0170] To this end, the user interface 904 may be displayed to report the health condition classification 116 to the health care provider, and to report additional information that may be related to the classification. In operation, the health care provider may independently analyze the reported additional information and provide a diagnosis different from that indicated by the health condition classification 116. In this exemplary implementation 900, the additional information includes glucose traces 908, 910. The glucose traces 908, 910 represent analyte measurements 110 of the human 102 collected over a two-day observation period. The user interface 904 is also shown with controls that may enable the user to navigate to other analyte measurements 110 collected, such as traces corresponding to days earlier or later in the observation period.

[0171] The user interface 904 also includes a graphical glucose feature element 912 that represents or otherwise indicates one or more of the extracted analyte features 416 determined by the feature builder 408 based on the analyte measurements 110 of the human 102. In addition to the predicted clinical diagnosis, as indicated by the graphical diagnosis element 906, the user interface 904 also includes a predicted side effect element 914 and a probability element 916. The inclusion of the predicted side effect element 914 and the probability element 916 indicates that the machine learning model 602 introduced in FIG. 6 may be configured (e.g., via configuration as an ensemble of models and / or based on architecture and training) to generate predictions of two or more types of diabetes classifications. By way of example, the machine learning model 602 may be configured to predict a clinical diagnosis of the human 102, one or more values ​​of a plurality of clinical diagnostic values ​​associated with diabetes (e.g., HbA1c, FPG, 2Hr-PG, and OGTT), and a probability that the human 102 will experience one or more of a plurality of side effects of diabetes.

[0172] In particular, the predicted side effects element 914 corresponds to side effects that the diabetes classification indicates the human 102 is more likely to experience than not experience, for example based on a probability of experiencing those effects output by the machine learning model being greater than 50%. It should be appreciated that side effects that the machine learning model 602 predicts to have any probability of occurrence may also be output in one or more scenarios with a corresponding probability. The probability element 916 includes a probability that the side effect indicated by element 914 will occur. The probabilities indicated by these probability elements 916 may be output by the machine learning model 602 in one or more implementations. It should be appreciated that the report including the health condition classification 116 may be configured in different ways without departing from the spirit or scope of the described techniques, such as a document suitable for printing.

[0173] 10 illustrates an example of an implementation 1000 of a user interface displayed to collect additional data that may be used as input to a machine learning model for generating a health state prediction. The following description refers to the components and features illustrated in FIGS. 1-7 and described above for generating a health state prediction. However, it may be understood that the implementation 1000 may collect additional data for use by other features and components without departing from the scope of the present disclosure.

[0174] The illustrated exemplary implementation 1000 includes a computing device 1002 displaying a user interface 1004. In this exemplary implementation 1000, the user interface 1004 may be displayed to collect data about the human 102 in addition to the analyte measurements 110 collected during an observation period. These additional data may be provided as input to the machine learning model 602 along with traces of the analyte measurements 110 and / or one or more of the extracted analyte features 416. That is, the additional data may be represented in features of a feature vector that is input to the model. To train the machine learning model 602, these additional data may also be collected from users of the user population 304. Thus, the user interface 1004 may be displayed to users of the user population 304 to collect these additional data from those users. The additional data may include, for example, data describing demographics, medical history, exercise, and / or stress.

[0175] In the illustrated example implementation 1000, the user interface 1004 includes various graphical elements 1006 with which a user may interact (e.g., select or enter values) to provide additional data about the user. However, it should be understood that the included graphical elements 1006 are merely examples, and that a user interface for collecting such additional data may be configured in different ways to include more, fewer, or different elements that enable collection of various additional data without departing from the spirit or scope of the described techniques.

[0176] Having discussed example details of techniques for health state prediction using analyte measurements and machine learning, we now consider some example procedures that illustrate some additional aspects of the techniques.

[0177] Step-by-step example This section describes examples of procedures (e.g., methods) for health condition prediction using analyte measurements and machine learning. Aspects of the procedures may be implemented in hardware, firmware, software, or combinations thereof. The procedures are illustrated as a set of blocks that specify operations to be performed by one or more devices and are not necessarily limited to the order shown for performing the operations by the respective blocks. In some examples, multithreading or parallel processing may be used in performing the described procedures. In at least some implementations, the procedures are performed by a predictive system, such as the predictive system 114 of FIG. 1 utilizing the model manager 402 of FIG. 4. Thus, reference is made to features and components described above with respect to FIGS. 1-6, but it may be understood that the procedures may be performed using similar or different components without departing from the scope of the present disclosure.

[0178] FIG. 11 illustrates a procedure 1100 in an example implementation in which a combination of robust analyte features for predicting a health condition classification is selected based on historical analyte measurement and outcome data for a user population.

[0179] Analyte measurements collected by wearable analyte monitoring devices worn by users of the user population are obtained (block 1102). As an example, the model manager 402 obtains analyte measurements 110 of users of the user population 304. In some examples, the model manager 402 generates analyte data from the analyte measurements 110, such as via a preprocessing manager 404 of the model manager 402. The analyte data may be preprocessed data (e.g., preprocessed data 414 of FIG. 4), which may include a chronological and / or filtered sequence of the analyte measurements 110.

[0180] Outcome data describing one or more aspects of the users of the user population related to a health condition is obtained (block 1104). As an example, the model manager 402 obtains the outcome data 306. In the example discussed above, the outcome data describes exemplary aspects such as one or more of: independent diagnostic measures 308 of the users of the user population 304, observed side effects 310 of the users of the user population 304, and clinical diagnoses 312 of the users of the user population 304.

[0181] Analyte sensor manufacturing-related variation is simulated in the analyte data (block 1106). As an example, the variation simulator 406 of the model manager 402 introduces the analyte sensor manufacturing-related variation into the analyte data generated from the analyte measurements 110. The analyte sensor manufacturing-related variation may refer to differences in the response of different analyte sensors (e.g., the analyte sensor 202 of FIG. 2) to the same analyte level due to, for example, sensor bias, different sensor models, different sensor lots, different sensor manufacturers, etc. For example, the variation simulator 406 may run multiple simulations for each portion of the analyte data (e.g., each trace), and each simulation may introduce a different percentage of the simulated manufacturing-related variation into the analyte data. For example, the variation simulator 406 may simulate the manufacturing-related variation by applying a multiplicative or additive variation to each portion of the analyte data. As an example, the variability is drawn from a normal distribution with a fixed standard deviation and a mean value that is swept within a predetermined range of percent manufacturing-related variability. As a non-limiting example, the standard deviation may be set to 8 and the predetermined range of percent manufacturing-related variability may be -8% to 11%. Thus, a new manufacturing-related variability value may be determined for each portion of the analyte data during each round of the simulation. As a non-limiting example, the simulation may be run for 20 to 40 rounds (e.g., 30 rounds).

[0182] A number of features are extracted from the analyte data (block 1108). As an example, the feature builder 408 of the model manager 402 may receive analyte data from each round of simulation, including analyte data without simulated analyte sensor manufacturing-related variation and analyte data with varying amounts of simulated analyte sensor manufacturing-related variation, and output extracted analyte features 416. Thus, extracted analyte features 416 may be output for analyte data having each percentage of simulated manufacturing-related variation, including 0%.

[0183] The combination of the plurality of features is evaluated for a robustness metric associated with sensitivity to analyte sensor manufacturing-related variability and a performance metric associated with predicting the health condition classification (block 1110). As an example, at least two features of the plurality of features are combined as a multivariate model (e.g., bivariate model) and a model prediction of the health condition classification is generated and compared to the outcome data 306. Alternatively, a single feature of the plurality of features may be used in a univariate model. As described with respect to FIG. 4, the performance metric (e.g., performance metric 430) may rank each candidate combination of the plurality of features according to the accuracy rate of predicting the disease of the condition classification, and the robustness metric (e.g., robustness metric 432) may be used to rank each candidate combination of the plurality of features according to the percentage change in the performance metric per percentage change in the simulated manufacturing-related variability. A higher performance metric may indicate that the candidate combination is more sensitive and specific for predicting disease of the condition, and a higher robustness metric may indicate that the performance of the candidate combination is less affected by analyte sensor manufacturing-related variability. Alternatively, instead of the robustness metric, a variability sensitivity metric that is the inverse of the robustness metric may be used. A higher variability sensitivity metric may indicate that the performance of the candidate combination is more affected by analyte sensor manufacturing-related variability (e.g., more sensitive to manufacturing-related variability).

[0184] A combination of features is selected to be a robust feature combination for predicting a health condition classification via the machine learning model based on a robustness metric and a performance metric (block 1112). As an example, a robustness threshold (e.g., robustness threshold 530 in FIG. 5) may be used to filter the candidate combinations of features such that candidate combinations having robustness metrics below the robustness threshold (or sensitivity metrics above the robustness threshold) are removed from further consideration. In some examples, the filtered candidate combination with the highest performance metric may be selected as the robust feature combination (e.g., robust analyte feature combination 434 in FIG. 4). Thus, the robust feature combination may be the candidate combination with the highest performance metric among combinations having robustness metrics greater than the robustness threshold (or variability sensitivity metrics that are less than the robustness threshold).

[0185] As another example, a performance metric threshold may be used instead of or in addition to a robustness threshold. For example, a performance metric threshold may be used to filter out candidate combinations of features that have a performance metric that is less than the performance metric threshold. The remaining candidate combinations with the highest robustness metric (or lowest variability sensitivity metric) may be selected as the robust feature combination.

[0186] As yet another example, additional aspects may be evaluated in selecting robust feature combinations. As mentioned above with respect to FIG. 5, these additional aspects may include, but are not limited to, ease of computation of features in the candidate feature combinations, ability to determine features in the candidate feature combinations using hardware alone, amount of data required to compute features in the candidate feature combinations (e.g., collection over multiple days vs. collection over several hours), etc. Thus, in some implementations, an additional metric may be determined for a selected number of top candidate feature combinations (e.g., those with top performance metrics that are not filtered out by a robustness threshold or those with top robustness metrics that are not filtered out by a performance metric threshold). In some implementations, the additional metric may be added (e.g., as a weighted or unweighted sum) with the performance metric and / or robustness metric of each of the top candidate feature combinations, and the top candidate feature combinations with the highest sum may be selected as the robust feature combinations.

[0187] As yet another example, a robust feature combination may be selected based on a sum of a robustness metric and a performance metric for each candidate combination to maximize the combination of the robustness metric and the performance metric. For example, the candidate combination with the highest sum of the robustness metric and the performance metric may be selected as the robust feature combination. In some examples, the sum may be a weighted sum to give more or less weight to the robustness metric versus the performance metric.

[0188] In this manner, a combination of robust analyte features may be selected that may produce a machine learning model with consistent performance and high accuracy over the range or expected variation in analyte measurements 110 due to manufacturing variations between sensors and between subjects. By using a combination of robust analyte features, the performance and robustness of the resulting machine learning model may be more effectively balanced than could be achieved using a single analyte feature.

[0189] FIG. 12 illustrates a procedure 1200 in an example implementation in which a machine learning model is trained to predict health condition classifications based on robust analyte features extracted from historical analyte measurement and outcome data of a user population.

[0190] Analyte measurements collected by wearable analyte monitoring devices worn by users of the user population are obtained (block 1202). As an example, the model manager 402 obtains analyte measurements 110 of users of the user population 304. Outcome data describing one or more aspects of the users of the user population related to a health condition is obtained (block 1204). As an example, the model manager 402 obtains outcome data 306. In the example discussed above, the outcome data describes exemplary aspects such as one or more of: independent diagnostic measures 308 of users of the user population 304, observed side effects 310 of users of the user population 304, and clinical diagnoses 312 of users of the user population 304.

[0191] An instance of training data is generated that includes a training input portion and an expected output portion, where the training input portion includes a combination of features of the users' analyte measurements selected based on the robustness metric and the performance metric, and the expected output portion includes one or more values ​​of outcome data corresponding to the users (block 1206). According to principles discussed herein, the combination of features includes at least two extracted features of the analyte measurements of each user (e.g., robust analyte feature combinations 434 introduced with respect to FIG. 4). Selecting the combination of features based on the robustness metric and the performance metric is further described above with respect to step 1100 of FIG. 11. Furthermore, the expected output portion includes one or more values ​​of outcome data corresponding to each user. By way of example, the model manager 402 generates an instance of training data by associating, for each user of the user population 304, a combination of features extracted from the users' analyte measurements obtained in block 1202 with one or more values ​​of the users' outcome data obtained in block 1204. In one or more implementations, the model manager 402 "labels" the feature combinations with one or more labels that represent values ​​of outcome data corresponding to the user. The outcome data may include, for example, clinically validated indicators of health condition classifications.

[0192] Here, blocks 1208-1214 may be repeated until the machine learning model is suitably trained, such as until the machine learning model "converges" to a solution, e.g., the model's internal weights are suitably adjusted due to training iterations such that the model consistently produces predictions that substantially match the expected output portion. Alternatively or additionally, blocks 1208-1214 may be repeated for some instances (e.g., all instances) of the training data.

[0193] The training input portion of the instances of training data is provided as an input to the machine learning model (block 1208). As an example, the model manager 402 provides the training input portion of the instances of training data generated in block 1206 as an input to the machine learning model 602.

[0194] A prediction of the health condition classification is received as an output from the machine learning model (block 1210). According to principles discussed herein, the prediction of the health condition classification corresponds to one or more values ​​of the user's outcome data included in the training instance and the same aspects related to the health condition. As an example, the machine learning model 602 predicts a health condition classification (e.g., a classification of the user into a class of "health condition exists", a class of "at risk of developing a health condition", a class of "health condition does not exist", or a value indicating one of the classes) based on the training input portion provided in block 1208, and the model manager 402 receives the health condition classification as an output of the machine learning model 602.

[0195] The prediction of the health condition classification is compared to the expected output portion of the training data instances (block 1212). By way of example, the model manager 402 compares the health condition classification predicted in block 1210 to the expected output portion of the training instances generated in block 1206 by using a loss function such as, for example, mean squared error (MSE). It should be understood that the model manager 402 may use other loss functions during training to compare the predictions of the machine learning model 602 to the expected output without departing from the spirit or scope of the described techniques.

[0196] The weights of the machine learning model are adjusted based on the comparison (block 1214). By way of example, the model manager 402 may adjust the internal weights of the machine learning model 602 based on the comparison. In one or more implementations, the model manager 402 may optionally utilize one or more hyperparameter optimization techniques during training to tune the hyperparameters of the learning algorithm employed.

[0197] 13 illustrates a procedure 1300 in an example implementation in which a machine learning model predicts a health condition classification based on a user's analyte measurements collected by a wearable analyte monitoring device during an observation period. The machine learning model may be trained, for example, according to procedure 1200.

[0198] A user's analyte measurements are obtained, the analyte measurements being collected by a wearable analyte monitoring device worn by the user during an observation period (block 1302). As an example, the machine learning model 602 obtains analyte measurements 110 collected by an analyte monitoring device 104 worn by the human 102 (e.g., a user) during an observation period. The analyte monitoring device 104 may be provided, for example, as part of an observation kit for purposes of monitoring the analytes of the human 102. Regardless of how the analyte monitoring device 104 is obtained by the human 102, the device is configured to monitor the analyte levels of the human 102 during an observation period that typically lasts over multiple days and periods of time. The analyte monitoring device 104 may be configured with an analyte sensor 202 that may be inserted, for example, subcutaneously into the skin of the human 102 and used to measure an analyte in the blood or interstitial fluid of the human 102.

[0199] Although discussed throughout as being inserted subcutaneously into the skin of the human 102, in one or more implementations, the analyte sensor 202 may not be inserted subcutaneously. In such implementations, the analyte sensor 202 may instead be disposed on the skin or muscle of the human 102. For example, the analyte sensor 202 may be a patch that adheres to the skin of the human 102 for an observation period. The patch may then be peeled off. Alternatively or additionally, the non-invasive analyte sensor may be optically based, for example using photoplethysmography (PPG). The analyte sensor 202 may be configured in a variety of ways to obtain measurements indicative of analyte levels in the human 102 without departing from the spirit or scope of the described techniques.

[0200] A robust feature combination is extracted from the analyte measurements, where the robust feature combination is less sensitive to manufacturing variations of the analyte sensor 202 and is accurate for predicting the health condition classification (block 1304). As described above with respect to FIGS. 4-5 and 11, the robust feature combination may include at least two extracted analyte features from the plurality of extracted analyte features 416. By way of example, a first analyte feature of the robust feature combination may have a higher performance metric for accurately predicting the health condition classification (e.g., performance metric 430 of FIG. 4) and a lower robustness metric for insensitivity to manufacturing variations of the analyte sensor, such as analyte sensor bias (e.g., robustness metric 432 of FIG. 4), and a second analyte feature of the robust feature combination may have a higher robustness metric and a lower performance metric (e.g., than the first analyte feature). As an illustrative example, the first analyte feature may represent an analyte level or degree of analyte elevation in the blood of the human 102, such as one of the value-based features 426 of Figure 4. As another illustrative example, the second analyte feature may represent an analyte pattern or trend, such as one of the trend-related features 418 or the variability and stability features 422 of Figure 4.

[0201] The user's health condition classification is predicted (block 1306) by processing the combination of the extracted robust features using one or more machine learning models. According to principles discussed herein, the one or more machine learning models are generated based on the historical analyte measurements and historical outcome data of the user population, such as following the procedure 1200 described above with respect to FIG. 12. As an example, the machine learning model 602 predicts the health condition classification 116. The machine learning model 602 generates this prediction by processing the combination of the extracted robust features based on the patterns of the robust feature combinations learned during training and the outcome data 306 of the user population 304. As described above, the user population 304 includes users who wear a wearable analyte monitoring device, such as the analyte monitoring device 104. By processing the combination of robust features together, rather than the individual extracted analyte features, performance and manufacturing variability robustness can be balanced for consistent and accurate model performance.

[0202] The health condition classification is output (block 1308). As an example, the machine learning model 602 outputs the health condition classification 116. As discussed throughout, the health condition classification 116 may indicate whether the human is predicted to have the health condition or whether the human is predicted to experience a side effect associated with the health condition. The health condition classification 116 may also be used to generate one or more notifications or user interfaces based on the classification, such as a report directed to a health care provider that includes the health condition classification (e.g., the human is predicted to have the health condition), or a notification directed to the human that instructs the human 102 to contact his or her health care provider.

[0203] Example of implementation: Diagnosing type 2 diabetes using a factory calibrated CGM system Type 2 diabetes (T2D) is a progressive disease that affects over 400 million people worldwide and can lead to long-term vascular complications. Before progressing to T2D, many people first develop prediabetes, which is characterized by mildly elevated glucose levels. For those with prediabetes and T2D, proper diagnosis and management can improve glycemic control and reduce the risk of diabetes-related complications.

[0204] As previously described herein, clinical and regulatory bodies have developed criteria for diagnosing T2D that are typically based on fasting plasma glucose (FPG), 2-hour plasma glucose (2Hr-PG), random plasma glucose (RPG) values, and hemoglobin HbA1c (HbA1c). Both FPG and 2Hr-PG are part of an oral glucose tolerance test (OGTT), but FPG can be tested separately from an OGTT. Each test has several advantages and limitations. FPG tests are easy to perform in the clinic, but rely on an overnight fast before blood is drawn. 2Hr-PG measurements from an OGTT are more sensitive than FPG alone, capturing post-prandial glucose spikes that are indicative of diabetes. Blood glucose levels below 140 mg / dL are considered "normal," whereas readings greater than 200 mg / dL indicate diabetes. Readings between 140 and 199 mg / dL indicate prediabetes. However, this test relies on fasting and meal preparation, is time-consuming with at least two blood draws over a 2-h period, and has relatively low reproducibility (e.g., in the range of 60-80%). Therefore, unless both FPG and 2Hr-PG values ​​are above the diagnostic threshold, a repeat OGTT is recommended to confirm the diagnosis.

[0205] RPG testing can be performed at any time, but is not an accepted diagnostic by itself unless accompanied by overt diabetic symptoms. HbA1c testing does not rely on fasting and is accurate and reproducible. However, HbA1c is not a direct measure of glucose, and HbA1c values ​​can be affected by factors including anemia, hemoglobinopathies, race, and the so-called "glycation gap" which corresponds to the discrepancy between HbA1c and fructosamine measurements. These conditions, along with suboptimal agreement between HbA1c and OGTT results, can lead to misdiagnosis and delayed treatment, which can have adverse health and economic consequences.

[0206] A continuous glucose monitoring system (CGM) system reports glucose levels from blood or interstitial fluid, for example, at predetermined intervals. As a non-limiting example, the predetermined intervals are 5-minute intervals. In various implementations, the CGM system includes an analyte monitoring device 104. Accordingly, additional details regarding the CGM system are described above with respect to FIG. 1 and FIG. 2. An individual sensor session in which the CGM system is worn continuously (e.g., without removal) may last for a predetermined observation period, such as 10 days, and may make glucose concentration data available to the CGM system wearer (e.g., user) to assist with diabetes management. Alternatively or additionally, for sessions performed in "blind mode," sensor data may be recorded but not displayed. Real-time CGM data includes, for example, current glucose concentration estimates and trends that may inform immediate treatment decisions. Additionally, retrospective analysis of CGM data may yield summary statistics that inform longer-term interventions, adjustments, or risk of diabetes-related complications.

[0207] Given the limitations of current standard of care diagnostic testing, according to the described technique, an algorithm uses CGM data from the observation period as a surrogate diagnosis for T2D. In this example, a binary classification diagnostic CGM (dCGM) algorithm is used to generate a classification for T2D. In one or more implementations, HbA1c measurements are paired with tags that classify the measurements as corresponding to a diabetes status of euglycemia, prediabetes, or T2D. Observed CGM-derived metrics may also be paired with observed HbA1c measurements, so that diabetes status can be paired with such CGM-derived metrics in the training data. To train the dGCM algorithm, a wide range of CGM-derived metrics (e.g., features) may be combined with diabetes status, including metrics such as glycemic variability, time in range, risk assessment scores, etc. Further examples of such metrics are represented by extracted analyte features 416 and are described with respect to FIG. 4. In one or more implementations, feature importance is evaluated in k-fold cross-validation, and desired CGM metrics can be down-selected based on discriminatory power, interpretability, and collinearity.

[0208] In one or more implementations, a dCGM algorithm is trained to utilize as input a combination of metrics assessing blood glucose levels (e.g., time above 140 mg / dL) and quantified variability (e.g., interquartile range). To further illustrate the utility of these CGM data-derived metrics, a correlation between blood glucose levels and variability with respect to diabetes status is shown in FIG. 14. FIG. 14 shows a plot 1400 of percent time above 140 mg / dL (vertical axis) and interquartile range (horizontal axis). A legend 1402 indicates the diabetes status associated with each data point according to the fill and shape. In this example, data points from euglycemic users 1404 (e.g., users with HbA1c measurements less than 5.7%) are represented by solid black inverted triangles, data points from prediabetic users 1406 (e.g., users with HbA1c measurements greater than or equal to 5.7% and less than 6.5%) are represented by diagonally shaded circles, and data points from T2D users 1408 (e.g., users with HbA1c measurements of at least 6.5%) are represented by solid white triangles (e.g., no shading).

[0209] As illustrated by plot 1400, plotting the percent time above 140 mg / dL against the interquartile range provides a separation between users with and without T2D. For example, T2D users 1408 are relatively clustered at higher percent time above 140 mg / dL values ​​and higher interquartile range values ​​(e.g., top right of plot 1400), while euglycemic users 1404 and prediabetic users 1406 are relatively clustered at lower percent time above 140 mg / dL values ​​and lower interquartile range values ​​(e.g., bottom left of plot 1400). Because time spent above 140 mg / dL is rare in healthy individuals, this CGM metric can be used as a threshold for diagnosing T2D or assessing risk of developing T2D.

[0210] The wealth of information generated by CGM data measured under real-life conditions enables feature extraction and feature selection for training machine learning based algorithms for screening for diabetes and predicting diabetes risk, as described in detail above with respect to Figures 4-7 and 11-13. While the present embodiment employs a factory calibrated CGM system worn for several days and blinded retrospective data from a single sensor wearing session, other embodiments may use data from CGM sensors with other calibrations, other wearing durations, more than one sensor wearing session, etc. Additionally, while the present embodiment describes a binary classification algorithm for diagnosing T2D, the dCGM algorithm may be adapted to classify type of diabetes, risk of developing diabetes, prediction of developing diabetes related health side effects, etc.

[0211] The percent time above 140 mg / dL and interquartile range represent one example of an interpretable set of CGM metrics that have high discriminatory power for classifying those with and without T2D, although it should be understood that other combinations of different and / or additional metrics based on high density CGM data may be used to resolve differences in glycemic abnormalities between these users in other dCGM algorithms.

[0212] Having described example procedures according to one or more implementations, we now turn to example systems and devices that can be utilized to implement the various techniques described herein.

[0213] System and device embodiments 15 illustrates, generally at 1500, an embodiment of a system including an embodiment of a computing device 1502, which represents one or more computing systems and / or devices that may implement various techniques described herein. This is illustrated through the inclusion of a prediction system 114 at a platform level and at an individual computing device level. The prediction system 114 may be implemented at one level or the other level, or at least partially at both levels. The computing device 1502 may be, for example, a server of a service provider, a device associated with a client (e.g., a client device), an on-chip system, and / or any other suitable computing device or computing system.

[0214] An embodiment of a computing device 1502 as illustrated includes a processing system 1504, one or more computer-readable media 1506, and one or more input / output (I / O) interfaces 1508 communicatively coupled to one another. Although not shown, the computing device 1502 may further include a system bus or other data and command transfer system coupling the various components together. The system bus may include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus utilizing any of a variety of bus architectures. Various other examples, such as control and data lines, are also contemplated.

[0215] The processing system 1504 represents a function for performing one or more operations using hardware. Thus, the processing system 1504 is illustrated as including hardware elements 1510, which may be configured as a processor, a functional block, or the like. This may include implementations in hardware as application specific integrated circuits or other logic devices formed using one or more semiconductors. The hardware elements 1510 are not limited by the materials from which they are formed or the processing mechanisms employed therein. For example, a processor may be comprised of semiconductors and / or transistors (e.g., electronic integrated circuits (ICs)). In such a context, processor-executable instructions may be electronically executable instructions.

[0216] The computer-readable medium 1506 is illustrated as including memory / storage 1512. The memory / storage 1512 represents memory / storage capacity associated with one or more computer-readable media. The memory / storage 1512 may include volatile media (such as random access memory (RAM)) and / or non-volatile media (such as read only memory (ROM), flash memory, optical disks, magnetic disks, etc.). The memory / storage 1512 may include fixed media (e.g., RAM, ROM, fixed hard drives, etc.) as well as removable media (e.g., flash memory, removable hard drives, optical disks, etc.). The computer-readable medium 1506 may be configured in a variety of other ways, as further described below.

[0217] Input / output interface 1508 represents functionality that allows a user to input commands and information into computing device 1502 and / or allow information to be presented to a user and / or other components or devices using various input / output devices. Examples of input devices include keyboards, cursor control devices (e.g., a mouse), microphones, scanners, touch capabilities (e.g., capacitive or other sensors configured to detect physical contact), cameras (e.g., employing visible wavelengths or non-visible wavelengths such as infrared frequencies to recognize movements as non-touch gestures), and the like. Examples of output devices include display devices (e.g., a monitor or projector), speakers, printers, network cards, haptic response devices, and the like. Thus, computing device 1502 can be configured in a variety of ways to support user interaction, as described further below.

[0218] Various techniques may be described herein in the general context of software, hardware elements, or program modules. Generally, such modules include routines, programs, objects, elements, components, data structures, etc. that perform particular tasks or implement particular abstract data types. As used herein, the terms "module," "function," and "component" generally refer to software, firmware, hardware, or combinations thereof. Aspects of the techniques described herein are platform-independent, meaning that the techniques may be implemented on a variety of commercial computing platforms having a variety of processors.

[0219] An implementation of the described modules and techniques may be stored on or transmitted via some form of computer-readable media, such as computer-readable medium 1506. Computer-readable media may include a variety of media that may be accessed by computing device 1502. By way of example, and not limitation, computer-readable media may include "computer-readable storage media" and "computer-readable signal media."

[0220] A "computer-readable storage medium" may refer to a medium and / or device that allows for persistent and / or non-transient storage of information, as opposed to merely a signal transmission, carrier wave, or signal itself. Thus, a computer-readable storage medium refers to a non-signal-bearing medium. A computer-readable storage medium includes hardware, such as volatile and non-volatile, removable and non-removable media, and / or storage devices implemented in a manner or technology suitable for storage of information, such as computer-readable instructions, data structures, program modules, logic elements / circuits, or other data. Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, hard disk, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or other storage device, tangible media, or other articles of manufacture suitable for storing the desired information and that can be accessed by a computer.

[0221] A "computer-readable signal medium" may refer to a signal-bearing medium configured to transmit instructions to the hardware of the computing device 1502, such as via a network. Signal media may typically embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, data signal, or other transport mechanism. Signal media also includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media include wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, and other wireless media.

[0222] As previously described, the hardware elements 1510 and computer-readable media 1506 represent modules, programmable device logic, and / or fixed device logic implemented in hardware form that may be employed in some embodiments to implement at least some aspects of the techniques described herein, such as to execute one or more instructions. The hardware may include integrated circuits or components of on-chip systems, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other implementations in silicon or other hardware. In this context, the hardware may operate as a processing device that executes program tasks defined by instructions and / or logic embodied by the hardware, as well as hardware utilized to store instructions for execution, such as the computer-readable storage media previously described.

[0223] Combinations of the foregoing may be used to implement various techniques described herein. Thus, software, hardware, or executable modules may be implemented as one or more instructions and / or logic embodied on some form of computer-readable storage medium and / or by one or more hardware elements 1510. Computing device 1502 may be configured to implement specific instructions and / or functions corresponding to software and / or hardware modules. Thus, implementation aspects of modules executable as software by computing device 1502 may be achieved at least in part in hardware, for example, through use of computer-readable storage medium 1506 and / or hardware elements 1510 of processing system 1504. The instructions and / or functions may be executable / operable by one or more articles of manufacture (e.g., one or more computing devices 1502 and / or processing system 1504) to implement the techniques, modules, and examples described herein.

[0224] The techniques described herein may be supported by various configurations of computing device 1502 and are not limited to the specific examples of the techniques described herein. This functionality may also be implemented in whole or in part through the use of a distributed system, such as on a "cloud" 1514 via platform 1516, as described below.

[0225] Cloud 1514 includes and / or represents a platform 1516 for resources 1518. Platform 1516 abstracts the underlying functionality of hardware (e.g., servers) and software resources of cloud 1514. Resources 1518 may include applications and / or data available while computer processing is running on a server remote from computing device 1502. Resources 1518 may also include services provided over the Internet and / or over a subscriber network, such as a cellular network or a Wi-Fi network.

[0226] The platform 1516 can abstract resources and functionality for connecting the computing device 1502 with other computing devices. The platform 1516 can also serve to abstract the scaling of resources to provide a level of scale corresponding to the demand faced by the resources 1518 implemented via the platform 1516. Thus, in an embodiment of interconnected devices, implementation aspects of the functionality described herein can be distributed throughout the system 1500. For example, functionality can be implemented partially on the computing device 1502 as well as via the platform 1516 that abstracts the functionality of the cloud 1514.

[0227] In some aspects, the techniques described herein relate to a method in which selecting a feature combination of a plurality of features based on a robustness metric and a performance metric further includes filtering a plurality of candidate combinations of the plurality of features based on a performance metric of each of the plurality of candidate combinations relative to a performance threshold, and selecting as the feature combination the filtered candidate combination having the highest value for the robustness metric.

[0228] In some aspects, techniques described herein relate to a method, where selecting a combination of features from a plurality of features based on a robustness metric and a performance metric further includes selecting a candidate combination of features from a plurality of candidate combinations of features that maximizes a combination of the robustness metric and the performance metric as the combination of features.

[0229] In some aspects, the techniques described herein relate to methods that further include training one or more machine learning models to predict a health condition classification using the outcome data of the user population.

[0230] In some aspects, the techniques described herein relate to methods where an individual performance metric is higher for the first feature than the second feature, and an individual robustness metric is higher for the second feature than the first feature.

[0231] In some aspects, the techniques described herein relate to methods in which the respective performance metric and the respective robustness metric are both higher for the first feature than for the second feature.

[0232] In some aspects, the techniques described herein relate to methods where at least one of the first feature and the second feature is an analyte measurement value-based feature.

[0233] In some aspects, the techniques described herein relate to methods wherein the health condition classification is an index describing the presence of the health condition, the absence of the health condition, a degree of severity of the health condition, a side effect associated with the health condition, and / or a clinical diagnostic value associated with the health condition.

[0234] In some aspects, the techniques described herein relate to a method further including generating a notification based on the health status classification of the user.

[0235] In some aspects, the techniques described herein relate to methods in which the notification includes a report directed to the user's healthcare provider that includes the user's health condition classification.

[0236] In some aspects, the techniques described herein relate to methods in which a notification is output to a user and includes instructions to contact the user's healthcare provider.

[0237] In some aspects, the techniques described herein relate to devices where a user's analyte data is measured by an analyte sensor over an observation period, the observation period spanning multiple days.

[0238] In some aspects, the techniques described herein relate to a device where a user's status with respect to a health condition is at least one of: absence of the health condition, presence of the health condition, severity of the health condition, type of the health condition, and a probability that the user will experience a side effect associated with the health condition.

[0239] In some aspects, techniques described herein relate to a device that includes obtaining an analyte measurement of a user during an observation period via an analyte sensor of an analyte monitoring device worn by the user during the observation period, extracting a first feature and a second feature from the user's analyte measurement, providing a combination of the first feature and the second feature to a machine learning model, and receiving a prediction of a health condition classification from the machine learning model.

[0240] In some aspects, the techniques described herein relate to a method further including generating a notification based on the prediction of the health condition classification received from the machine learning model.

[0241] In some aspects, the techniques described herein relate to a method in which generating a notification includes generating at least one of a first notification output to a user and including instructions that do not include a prediction of a health condition classification, and a second notification output to a healthcare provider of the user and including a prediction of a health condition classification.

[0242] In some aspects, the techniques described herein relate to a method, wherein the first feature comprises an amplitude-based feature, a time-in-range-based feature, a time-out-of-range-based feature, a stability-based feature, or an event occurrence-based feature.

[0243] In some aspects, the techniques described herein relate to methods where the second feature comprises a trend-based feature, a variability-based feature, a frequency-related feature, or an autocorrelation feature.

[0244] In some aspects, the techniques described herein relate to methods where the health condition classification is a diabetes classification and the analyte measurement is a glucose measurement.

[0245] In some aspects, the techniques described herein relate to methods, wherein the diabetes classification is an index describing the user's status during the observation period as having gestational diabetes or not having gestational diabetes.

[0246] In some aspects, the techniques described herein relate to methods where the diabetes classification is an indication describing the user's status during the observation period as having diabetes, not having diabetes, or having pre-diabetes.

[0247] In some aspects, the techniques described herein relate to methods in which the diabetes classification is indicative of one or more side effects of diabetes that the user is predicted to experience.

[0248] In some aspects, the technology described herein relates to methods where the first features are value-based features of the analyte measurement and the second features are pattern-based features of the analyte measurement.

[0249] In some aspects, the techniques described herein relate to methods in which each of the first feature and the second feature is an analyte measurement value-based feature.

[0250] In some aspects, the techniques described herein relate to methods in which each of the first feature and the second feature is a pattern-based feature of the analyte measurement.

[0251] In some aspects, techniques described herein relate to methods in which a combination of a first feature and a second feature is selected based on the performance of the combination of the first feature and the second feature in predicting a clinically determined health state classification of a user population while simulating manufacturing-related analyte sensor variability in the historical analyte measurements of the user population.

[0252] In some aspects, techniques described herein relate to a method that includes obtaining analyte measurements of a user during an observation period via an analyte sensor of an analyte monitoring device worn by the user during the observation period, extracting features from the user's analyte measurements, providing the features to a machine learning model, and receiving a prediction of a health condition classification from the machine learning model.

[0253] In some aspects, the techniques described herein relate to methods that further include selecting features based on a robustness metric associated with insensitivity to manufacturing variations of the analyte sensor and a performance metric associated with predicting the health condition classification.

[0254] In some aspects, the techniques described herein relate to a method further including generating at least one of a first notification including instructions without a prediction of a health condition classification and a second notification including a prediction of a health condition classification.

[0255] conclusion Although the systems and techniques have been described in language specific to structural features and / or methodological acts, it should be understood that the systems and techniques defined in the appended claims are not necessarily limited to the particular features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claimed subject matter. [Explanation of symbols]

[0256] 100 Environment 102 Humans 104 Analyte monitoring devices 106 Observation Kit Provider 108 Observation Analysis Platform 110 Analyte measurement results 112 Storage Devices 114 Prediction Systems 116 Health Status Classification 118 Sensor Data

Claims

1. Obtaining a plurality of features of the analyte measurement; selecting a feature combination of the plurality of features based on a robustness metric associated with insensitivity to manufacturing variations of the analyte sensor and a performance metric associated with predicting a health state classification; training one or more machine learning models using the combination of features to predict the health condition classification; and A method comprising:

2. the analyte measurements are historical analyte measurements from a user population associated with outcome data for the user population, and selecting the feature combination of the plurality of features based on the robustness metric and the performance metric. generating a model prediction of the health condition classification for each of a plurality of candidate combinations of the plurality of features; and determining the performance metric for each of the plurality of candidate combinations based on the model prediction of the health state classification for the outcome data of the user population.

3. 3. The method of claim 2, wherein the performance metric indicates one or both of a sensitivity for predicting the health state classification and a specificity for predicting the health state classification based on the model prediction of the health state classification on the outcome data for each of the plurality of candidate combinations.

4. The method of claim 2 or 3, wherein the outcome data is indicative of a clinically determined health status classification for each user of the user population.

5. selecting the feature combination of the plurality of features based on the robustness metric and the performance metric; simulating the manufacturing variation of the analyte sensor in the analyte measurement over a number of simulation rounds, each of which introduces a different percentage of simulated variation into the analyte measurement; and determining the robustness metric for each of the plurality of candidate combinations based on a change in the performance metric for each of the simulated percentages of variation.

6. 6. The method of claim 5, wherein simulating the manufacturing variation of the analyte sensor in the analyte measurements over the multiple simulation rounds comprises simulating different performance variations and analyte sensor characteristics to introduce a different proportion of the simulated variation into the analyte measurements in each simulation round.

7. selecting the feature combination of the plurality of features based on the robustness metric and the performance metric; filtering the candidate combinations of the features based on the robustness metric of each of the candidate combinations relative to a robustness threshold; The method of claim 2 , further comprising: selecting the filtered candidate combination having the highest value for the performance metric as the feature combination.

8. 8. The method of claim 1, wherein the feature combination of the plurality of features comprises a first feature and a second feature, the method further comprising: determining an individual performance metric and an individual robustness metric for each of the plurality of features using a model prediction of the health state classification for each of the plurality of features.

9. The method of claim 8 , wherein at least one of the first feature and the second feature is a trend-related feature of the analyte measurement.

10. The method of claim 8 , wherein the second feature is a variability and stability feature of the analyte measurement.

11. After training the one or more machine learning models to predict the health condition classification using the combination of features, obtaining new analyte measurements from an analyte measuring device worn by the user over an observation period; extracting features of the combination of features from the new analyte measurements; and inputting the extracted features of the feature combinations into the one or more machine learning models; receiving the health condition classification of the user as an output of the one or more machine learning models; The method of any one of claims 1 to 10, further comprising:

12. one or more processors; and a memory having computer readable instructions stored thereon, the computer readable instructions being operable by the one or more processors to: obtaining analyte data of the user measured by the analyte sensor; extracting at least two features of the analyte data, the at least two features being included in a multivariate model of analyte features determined to be robust to manufacturing variability of the analyte sensor based on variability simulations performed on historical analyte data of a user population; inputting the combination of at least two features into a machine learning model; predicting a health status classification of the user via the machine learning model; and receiving the health state classification as an output of the machine learning model. device.

13. The device of claim 12 , wherein the analyte data of the user is measured by the analyte sensor over an observation period, and the health condition classification is an index describing the user's status during the observation period with respect to health condition.

14. The device of claim 13 , wherein the health condition is diabetes and the health condition classification is one of a diabetic status, a pre-diabetic status, and a non-diabetic status.

15. 15. The device of claim 12, wherein the machine learning model is trained using a training input portion comprising the combination of the at least two features extracted from the historical analyte data of the user population, and an expected output portion comprising a label representative of the health state classification of each user of the user population.