Predictive models for early identification of pregnancy disorders

Abstract

Embodiments include a computing device executing software routines and / or one or more machine learning architectures that provides a clinical prediction model for predicting health complications resulting from pregnancy. The prediction model may identify and extend variables available in electronic health record systems during early pregnancy and may utilize machine learning methods to predict problematic outcomes in late pregnancy or the postpartum period. The prediction model follows several steps to identify likely pregnancy disorders, including identifying data sources and experts for classifiers, collating data with clinical experts, applying statistical and machine learning methods, and assessing model performance and interpretability. The prediction model and associated methods provide unprecedented early detection of pregnancy complications for early intervention and treatment to improve maternal and fetal health outcomes.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 63 / 424,717, filed November 11, 2022, which is incorporated by reference in its entirety.

[0002] This application relates generally to machine learning architectures for healthcare management, including predictive models for early identification of pregnancy disorders, and more particularly to machine learning models and techniques for predicting potential complications in the second and third trimesters of pregnancy. [Background technology]

[0003] Disorders that emerge during pregnancy, such as hypertension, diabetes, and perinatal depression, can increase the risk of adverse maternal and neonatal outcomes. Hypertension during pregnancy complicates 10% of pregnancies and is the leading cause of pregnancy-related death in the United States. Gestational diabetes complicates approximately 4% of pregnancies and is associated with preterm labor, fetal macrosomia, and fetal anomalies; furthermore, 50% of women with gestational diabetes develop type 2 diabetes after delivery. Early intervention, such as monitoring or initiation of medication or therapy, can reduce the risk of adverse outcomes and diagnoses. Therefore, the ability to clinically identify patients at risk for various pregnancy-related disorders early in pregnancy is essential for initiating interventions and improving pregnancy-related health outcomes.

[0004] Predefined clinical criteria, such as the United States Preventive Services Task Force (USPSTF) guideline for low-dose aspirin prophylaxis for individuals at high risk for preeclampsia and the American College of Obstetricians and Gynecologists (ACOG) guideline for early gestational diabetes screening in women with risk factors, currently guide clinical care for detecting pregnancy complications. Both guidelines define "risk" as a simple, rule-based algorithm, meaning that a person is classified as "high risk" if they meet any one feature from a short list and "low risk" if they meet none. This approach is overly simplistic because it does not consider the relative importance of each clinical feature, does not account for the multiplicative effect when multiple conditions are met, does not consider risk as a continuous measure with uncertainty, and does not include additional clinical factors, such as vital signs or laboratory data, that are collected as part of standard prenatal care.

[0005] For example, the incidence of hypertensive disorders of pregnancy (HDP) doubled in the United States between 2008 and 2019. Approximately one-quarter of maternal deaths occurring during hospitalization for delivery had a documented diagnosis code for HDP. Early identification of patients who would benefit from intervention may reduce the incidence of HDP. Current ACOG risk criteria could also be improved by using machine learning. Summary of the Invention

[0006] Current technological approaches and standards related to healthcare data present challenges for implementing machine learning architectures to predict pregnancy disorders. For example, certain industry interoperability standards for generating, storing, hosting, and sharing healthcare data may present challenges for machine learning architectures to access certain types of data necessary for making predictions of pregnancy disorders. Systems and methods are disclosed herein that can address the shortcomings discussed above and may also provide any number of additional or alternative benefits and advantages. Embodiments of the present disclosure relate specifically to a clinical curation process for identifying and expanding variables available in electronic health record systems during early pregnancy and then utilizing machine learning methods to predict problematic outcomes in late pregnancy. In an exemplary embodiment, the process may follow four steps, which may include identifying data sources and experts for classifiers, collating data with clinical experts, applying statistical and machine learning methods, and assessing model performance and interpretability.

[0007] In one embodiment, a computer-implemented method for determining an identification of an adverse pregnancy outcome includes receiving input data for training data, the input data including a plurality of health parameters for each of previous patients who have experienced the adverse pregnancy outcome; selecting a subset of the plurality of health parameters associated with the respective outcome, the selecting based on known outcome associations or outcome associations calculated based on machine learning output; assigning the subset of parameters for each patient associated with the respective outcome to training data and test data; fitting one or more machine learning models using the training data; evaluating performance of the one or more machine learning models using the test data to select a best-performing machine learning model based on a statistical comparison of performance between the one or more machine learning models; and updating, by the computer, a classification threshold of the selected machine learning model as a risk probability cutoff value for a given patient to experience the adverse pregnancy outcome.

[0008] In one embodiment, a computer-implemented method for predicting a patient's adverse pregnancy outcome includes collecting a plurality of health parameters of the patient before or during the first trimester of pregnancy, inputting the plurality of health parameters into a machine learning model, and determining a risk of the patient experiencing an adverse pregnancy outcome during pregnancy from an output of the trained machine learning model, wherein the machine learning model is trained by one or more training steps, the one or more training steps including receiving input data including a respective plurality of health parameters for each previous patient who has experienced an adverse pregnancy outcome, and determining exclusion conditions for excluding any of the respective plurality of health parameters based on determining whether any each previous patient meets the exclusion conditions; selecting a subset of each of the plurality of health parameters associated with each outcome, the selecting being based on known outcome associations or outcome associations calculated based on machine learning outputs; allocating the subset of parameters for each patient associated with each outcome to training data and test data; fitting one or more machine learning models using the training data; evaluating performance of the one or more machine learning models using the test data to select a best-performing machine learning model based on a statistical comparison of performance between the one or more machine learning models; and determining a classification threshold of the selected machine learning model as a risk probability cutoff value for a given patient to develop an adverse pregnancy outcome.

[0009] In one embodiment, a computer-implemented method for predicting a patient's pregnancy disorder outcome includes: collecting, by a computer, a plurality of health parameters of the patient before or during the first trimester; feeding, by a computer, the plurality of health parameters into a trained machine learning model having the ability to predict the risk of the outcome; and determining, by a computer, the patient's risk of developing a pregnancy disorder during pregnancy from the output of the trained machine learning model.

[0010] In one embodiment, a system for predicting a patient's adverse pregnancy outcome includes a computing device operable to execute computer-readable instructions configured to: receive a plurality of health parameters of the patient before or during the first trimester, input the plurality of health parameters into a trained machine learning model having a performance for predicting risk of the outcome as assessed by an area under the curve (AUC) of at least about 0.7, determine from the output of the trained machine learning model the patient's risk of experiencing an adverse pregnancy outcome during pregnancy, and output the risk as a numerical or semantic categorical value.

[0011] In one embodiment, a system for predicting a patient's adverse pregnancy outcome includes a computing device operable to execute computer-readable instructions configured to perform the steps of receiving a plurality of health parameters of the patient before or during the first trimester, inputting the plurality of health parameters into a machine learning model, determining a risk of the patient experiencing an adverse pregnancy outcome during pregnancy from an output of the trained machine learning model, and outputting the risk as a numerical or semantic categorical value, wherein the machine learning model is trained by one or more training steps, the one or more training steps including receiving input data including each of the plurality of health parameters for each previous patient who has experienced an adverse pregnancy outcome, and setting exclusion conditions for excluding any of the each of the plurality of health parameters for each previous patient. the selected plurality of health parameters satisfy an exclusion condition based on a determination of whether the selected plurality of health parameters satisfy an exclusion condition; selecting a subset of the respective plurality of health parameters associated with the respective outcome, the selecting being based on a known outcome association or an outcome association calculated based on a machine learning output; assigning the respective outcome-associated subset of the parameters for each patient to training data and test data; fitting one or more machine learning models using the training data; evaluating performance of the one or more machine learning models using the test data to select a best-performing machine learning model based on a statistical comparison of performance between the one or more machine learning models; and determining a classification threshold of the selected machine learning model as a risk probability cutoff value for the given patient to experience the adverse pregnancy outcome.

[0012] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

[0013] The present disclosure may be better understood by referring to the following drawings, in which components are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the present disclosure, in which reference numbers indicate corresponding parts throughout the different views. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 illustrates components of a system in accordance with an exemplary embodiment. [Figure 2] FIG. 1 illustrates components of a system in accordance with an exemplary embodiment. [Figure 3] FIG. 1 illustrates an exemplary timeline for early prediction and intervention of gestational hypertension. [Figure 4] 1 is a graph showing the performance of an exemplary HDP model on an internal validation cohort from a large academic medical center in the Midwestern United States. [Figure 5] 1 is a graph showing the performance of an exemplary HDP model in an external cohort of primiparous pregnancies, demonstrating that the model outperforms the ACOG checklist in external validation. [Figure 6] 1 is a graph showing HDP model performance across different races / ethnicities. [Figure 7] FIG. 1 illustrates an exemplary timeline for early prediction and intervention of GDM. [Figure 8] 1 is a graph showing the performance of an exemplary model for detecting GDM. [Figure 9] FIG. 1 illustrates an exemplary timeline for early prediction and intervention of excessive gestational weight gain (eGWG). [Figure 10] 1 is a graph showing the performance of an exemplary excessive gestational weight gain model. [Figure 11] FIG. 1 illustrates operational steps of an exemplary process for implementing a predictive machine learning method. [Figure 12] 1 is a graph showing the performance of an exemplary model for first trimester HDP compared to a reduced model. [Figure 13] FIG. 13 is a graph showing the reduced model of FIG. 12 compared to the USPSTF guidelines for aspirin initiation. [Figure 14] 1 is a graph showing the performance of an exemplary early pregnancy gestational diabetes model. DETAILED DESCRIPTION OF THE INVENTION

[0015] Reference will now be made to the exemplary embodiments illustrated in the drawings, and specific language will be used herein to describe the exemplary embodiments. Notwithstanding the foregoing, it will be understood that no limitation of the scope of the invention is thereby intended. Alterations and further modifications of the inventive features disclosed herein, and further applications of the inventive principles disclosed herein, will occur to those skilled in the art and in possession of this disclosure, and are considered within the scope of the invention.

[0016] The embodiments disclosed herein generally provide early detection of pregnancy disorders, often as early as the first trimester. Early detection and intervention are important for providing optimal health outcomes for mothers and children despite the risk of pregnancy disorders. For example, many conventional guidelines provide detection of pregnancy disorders later in pregnancy, or even after birth. Furthermore, conventional guidelines are based on limited insight that cannot incorporate the additive or multiplicative effects of patient health parameters (traits or health conditions, demographics, test results, etc.). The present invention in embodiments provides a method for training a machine learning model for the much-needed early detection of pregnancy disorders, a method and system for early detection of pregnancy disorders, and a method for treatment after early detection of pregnancy disorder risk.

[0017]

[0003] Embodiments described herein include a computing system including hardware and software components that implement a machine learning architecture for predicting health-related complications during pregnancy. The computing device executes software programming for the machine learning architecture, including classifiers trained to predict potential complications in the second and third trimesters from health data routinely collected during the first trimester, such as biometrics, laboratory values, and past health history, which can be used to indicate and predict the risk of potential complications in the second and third trimesters. In some implementations, the computing device may perform a unique clinical curation process to identify and expand variables available in electronic health record (EHR) systems during the first trimester or early stages of pregnancy (e.g., 13 weeks), and then apply the machine learning architecture to predict adverse outcomes in the third trimester.

[0018] The machine learning architecture includes machine learning models or techniques that are trained for early prediction of patient health outcomes. Machine learning models may include, or may include features of, multilayer perceptron (MLP) neural networks, logistic regression (LR), ensemble (histogram gradient boosting, AdaBoost, LR, MLP), histogram-based gradient boosting, adaptive boosting (AdaBoost), histogram-based gradient boosting (bagging), random forests, generalized additive models (GAM), logistic regression + LASSO, and stochastic gradient boosting. These are non-limiting examples of machine learning models, and various models may be implemented as described herein to create a predictive model for early detection of pregnancy disorders.

[0019] The machine learning model of the machine learning architecture may include one or more classifiers trained with training data including various types of patient care data. The training data typically includes various health parameters of patients with previous diagnoses of pregnancy disorders. In one embodiment, data sources are identified through a quantitative or iterative process. In a further embodiment, a clinical expert is involved in the selection of data sources based on several factors, including, but not limited to, the feasibility of data acquisition, the source population, and the inclusion of relevant data types (see, for example, Table 1). If necessary, the clinical expert may guide the selection of appropriate data sources.

[0020] [Table 1] TIFF2025539072000003.tif94170

[0021] In various embodiments, data from selected data sources may be collated by an automated process or a process that includes input from clinical experts. In many cases, pregnancy disorder outcomes (i.e., health outcomes) may need to be defined. For example, health outcomes may already exist as separate fields in a dataset, or may need to be developed based on multiple inputs. When an output needs to be developed based on multiple inputs, various factors may be considered, including, but not limited to, clinical criteria based on a combination of laboratory and screening test results and diagnostic codes. In embodiments, determining whether a previous patient has been diagnosed with a condition may rely on various measurements recorded in health records, in contrast to clinical criteria based on laboratory and screening test results. For example, to determine whether a person has preeclampsia, they must have blood pressure above 140 / 90 and evidence of kidney problems. In the case of development using diagnostic codes, multiple diagnostic codes may be combined to represent a particular health outcome. For example, to determine whether a person has preeclampsia, diagnosis code O14.10 refers to "severe preeclampsia of unspecified time," and diagnosis code O14.05 refers to "mild to moderate preeclampsia in the third trimester." In this case, the expert determines that a person with a code beginning with "O14" has been diagnosed with preeclampsia.

[0022] In various embodiments, health parameters may be selected based on availability, quality, and relevance. See, for example, Table 1 for exemplary health parameters. In embodiments, the model may be flexible to include other types of health parameters as needed by a particular implementation. Parameters may be evaluated for inclusion in the model based on several factors, including, but not limited to, availability in standard electronic health records, evidence that the parameter may be associated with the outcome, and the inclusion of socioeconomic parameters. Furthermore, the relevance of a parameter to an outcome may be computationally discovered, for example, by machine learning training, testing, or discovery in an implementation. Thus, in some embodiments, included parameters may be iteratively reevaluated in the model development process.

[0023] In various embodiments, parameters may be available in selected data sources, but it may be advantageous to determine whether the parameters will be readily available for a majority of patients (approximately 80% or more complete) in other health data systems to which the classifier will be applied. For example, receipt of a complete blood count blood test is part of routine prenatal care, and the results of this blood test should be available in health data systems. In addition to data availability, the quality of the parameters in standard health records may also be considered. "Data quality" is an assessment of whether a parameter is suitable for informing clinical decision-making and involves evaluating the accuracy and standardization of a given parameter. Accuracy means that the parameter's value reflects truth, and standardization means that the parameter value is standardized across clinical settings.

[0024] In various embodiments, evidence (clinical or calculated) may be available that indicates experts may select parameters that may be associated with outcomes based on their clinical knowledge and the consensus of the scientific community. For example, a previous diagnosis of anxiety would be included in a model for gestational diabetes due to the extensive literature detailing the link between anxiety and gestational diabetes. Additionally, socioeconomic parameters may be included. In embodiments, to avoid propagating systematic bias into the model, race and ethnicity, income, and other socioeconomic parameters may be included in model development if they are deemed clinically relevant to the outcome.

[0025] In some embodiments, health records for a particular patient may be incomplete for a variety of reasons. In embodiments, the methods and systems herein may handle missing values ​​in health records to maintain robust models. In some cases, the reason for the missing value may be determined. For example, an individual with a missing screening test may actually be unlikely to have the condition, meaning that the missing value for the parameter is meaningful. Various other missing parameters may be meaningful, while other missing parameters may be missing because the record is incomplete or for reasons that are not meaningful. Meaningfulness may be determined by knowledge or research, or may be determined computationally by iterative machine learning training, as described herein.

[0026] In various embodiments, the machine learning model is trained based on a certain patient population, for example, patients who do not meet the exclusion or inclusion criteria, and is useful for that population. For example, a specific population (based on clinical or sociodemographic factors) to which the classifier is relevant can be determined. For example, only individuals without pre-gestational diabetes can be diagnosed with gestational diabetes. Therefore, individuals with pre-gestational diabetes are excluded from the development and testing of classifiers for gestational diabetes. In other words, pre-gestational diabetes can be, as a non-limiting example, an exclusion criterion for gestational diabetes.

[0027] Various data handling steps may be performed on the input data and health parameter data as needed for a given implementation. For example, selected parameters may be combined for a selected study population at the individual level. In one embodiment, additional pre-processing of parameters may be performed prior to model fitting, including transformation of existing parameters, handling of missing values, and characterization of complex inputs.

[0028] In one embodiment, health parameter input data may be transformed. The transformation may include combining existing parameters into one parameter. For example, height and weight measurements may be combined into a body mass index. In addition, it may be advantageous to categorize continuous parameters based on clinically relevant cutoff values. For example, a continuous measurement of body mass index may be categorized as "underweight," "healthy weight," "overweight," or "obese."

[0029] In various embodiments, missing values ​​may be handled. For example, if the missing value itself is deemed informative, the missing value may be coded as a missing category, thereby allowing the model to learn from the missing value. If the missing value itself is deemed uninformative, the value is imputed using an imputation approach such as K-nearest neighbors. It may be understood that other methods of handling missing variables may be implemented as needed.

[0030] In various embodiments, complex health parameters may be handled. For example, a health parameter may include a complex signal of a dependent variable (health parameter) measured over several independent variables, such as time. In one embodiment, characterization provides a way to convert various forms of data into numerical data for use in machine learning models. For example, in the case of continuous time series data of fetal heart rate, characterization involves extracting a finite set of time- and frequency-related measurements.

[0031] Training a machine learning model generally relies on the use of a training dataset and a test set. In one embodiment, individuals are randomly assigned to a "training set" or a holdout "test set." The training set may be used to construct a classifier, and the test set may be used to determine the performance of the classifier. In some cases, categorical outcome parameters may have an unbalanced distribution. Balancing may be performed on the training dataset to prevent skewing toward the majority. Examples of balancing methods include balancing class weights, undersampling the majority class, or oversampling the minority class. It should be understood that any balancing technique is contemplated to improve the reliability of a machine learning model.

[0032] In one embodiment, various machine learning models are matched to identify the high-performing or best-performing model. Examples of machine learning models include logistic regression, random forest, or any other known model. Logistic regression estimates parameter values ​​(i.e., log-odds ratios) for each parameter included in the model, which can be used to return the probability of a binary outcome (e.g., a diagnosis of gestational diabetes) for each individual based on those variable values. Random forest models use random subsets of training data sampled with replacement to generate a collection of decision trees for predicting outcomes. The overall outcome of the random forest is selected by majority consensus among all trees, allowing the strengths of one tree to compensate for the weaknesses or gaps of other trees. In various embodiments, the method includes specifying hyperparameters. For example, random forest requires specifying the number of decision trees. The values ​​of the hyperparameters are determined by further dividing the training data into K datasets (K is any positive integer), enumerating possible values ​​of the hyperparameters, and then testing model performance at all values ​​of the hyperparameters. Typically, the hyperparameters with the best performance are selected.

[0033] In one embodiment, to improve model interpretability (i.e., to make it easier to explain to clinicians), variable selection methods can be used to create "reduced models" with a subset of variables. For logistic regression models, lasso (least absolute shrinkage and selection) can be employed, which reduces the relative contribution of each variable to zero depending on its importance. For other models, recursive feature elimination (RFE) can be employed. RFE iteratively removes variables by fitting a given machine learning algorithm, ranking features by importance, discarding the least important features, and refitting the model. This process can be repeated until a number of features remains that balances model interpretability (e.g., fewer variables) and model performance.

[0034] Model performance and interpretability are generally assessed during the optimization process. For example, specific performance metrics may be relied upon to assess the performance of a given model and compare the relative performance of various models. In one embodiment, evaluation metrics include a plot of the receiver operating characteristic (ROC) curve and the area under the curve (AUC). ROC curves are threshold-independent, meaning that model performance is displayed without considering specific sensitivity or specificity thresholds, allowing for the selection of desired cutoff values ​​based on clinical objectives. AUC is often used as the primary performance metric for predictive models because it provides a numerical summary of the ROC curve. 95% confidence intervals for each AUC may be calculated using the fast DeLong method. In various embodiments, the "best" model may be selected based on a statistical comparison of AUCs between models using the DeLong test. If a model has an AUC significantly higher (p-value < 0.05) than all other models, this model may be selected for clinical use. However, if there is some correlation between model performance, the model with the highest clinical interpretability may be selected. In various embodiments, the models herein can achieve AUC values ​​of at least about 0.7 or greater.

[0035] In various embodiments, if applicable to a given outcome, the performance of a predictive model may be compared to existing classifiers to illustrate how the model outperforms the current paradigm of clinical care. For example, the USPSTF recommends aspirin at 12 weeks' gestation in patients at high risk for preeclampsia, where "high risk" is defined as any individual with preeclampsia in a previous pregnancy, multiple fetuses, chronic hypertension, kidney disease, diabetes, autoimmune disease, or multiple moderate risk factors. In various embodiments, after the "best" model is determined, a probability cutoff value for classifying an individual as "high risk" for a given outcome is determined. Thus, in some embodiments, the predicted outcome may include a numerical probability or a semantic risk classification, such as "high risk," "moderate risk," or "average risk." In various embodiments, the numerical or semantic risk outcome may be utilized to inform treatment or intervention for the disease for which the patient is at risk.

[0036] The pregnancy disorder outcomes described herein may include various disorders such as gestational diabetes, gestational hypertension, excessive gestational weight gain, small for gestational age (i.e., small-for-gestational-age infants), and perinatal depression, among others. It is understood that each outcome may be defined differently based on different guidelines, and the model training process may take into account the guidelines under which patients in the training data were diagnosed.

[0037] In one embodiment, a computing device may be used to execute computer-readable instructions. Such computer-readable instructions may be programmed to receive patient health parameters, input the patient health parameters into a trained machine learning model, determine a risk for the patient based on the input, and output the risk to the patient. It should be appreciated that the patient health parameters and the risk to the patient may be displayed graphically through a patient portal or similar computerized display. This organized display of data may allow for better patient engagement and understanding.

[0038] Components of an Exemplary System FIG. 1 illustrates components of a system 100 for predicting and caring for pregnancy disorders, according to one embodiment. System 100 includes analysis system 101, care system 110, and patient devices 114a-114n (collectively referred to as one or more patient devices 114). The components of system 100 may communicate with each other via one or more networks 104. Analysis system 101 and care system 110 represent computing network infrastructures 101, 110, including physically and logically associated software and electronic devices, managed or operated by various business entities, including healthcare analysis services and healthcare providers or similar organizations (e.g., hospitals, clinics, medical offices, insurers, research institutions). Analysis system 101 includes an analysis server 102 that executes the software programming of a pregnancy prediction engine 105, a management device 103, and an analysis database 106. Care system 110 includes a provider device 116 and a provider database 118. The system 100 illustrated in FIG. 1 is merely an example. Embodiments may include additional or alternative components, or may omit certain components from those of Figure 1, and still be within the scope of the present disclosure. Embodiments may include or be otherwise implemented with any number of devices capable of performing the various features and tasks described herein.

[0039] Network 104 hosts and manages communications within system 100. Network 104 includes various hardware components (e.g., switches, routers) and software components of one or more public or private networks that interconnect the various components of system 100. Non-limiting examples of such networks 104 may include a local area network (LAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), and the Internet. Communications over network 104 may occur according to various communication protocols, such as Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol, and IEEE communications protocols, and are implemented by components of network 104, patient devices 114, and devices of analysis system 101 and care system 110.

[0040] In some embodiments, the system 100 includes a patient device 114. The patient device 114 may include any electronic computing device including hardware and software components capable of performing the various processes and tasks described herein. The patient uses the patient device 114 to access and interact with care applications, which perform various operations to provide patient output for the analysis server 102, such as the output of the pregnancy prediction engine 105. The patient device 114 may include an electronic device including a processor and / or software data streaming over a TCP / IP network or other computing network channel. In some cases, the care applications are originally installed on and executed by the patient device 114, and the care applications may contact the analysis server 102 to access and interact with certain features and associated data of the care applications. In some cases, the patient device 114 executes a web browser that accesses the features and functionality of the care applications, which are hosted and executed by web server software on the analysis server 102. Non-limiting examples of patient devices 114 may include a patient computer 114a, a virtual private cloud (VPC), or a patient mobile device 114b, among other types of electronic computing devices. The patient computer 114a may include any type of computing device, such as a workstation computer, laptop, tablet, server, etc. The patient mobile device 114b may include any type of mobile electronic computing device, such as a smartphone, tablet, or edge device, among others.

[0041] The analysis system 101 hosts care applications, a pregnancy prediction engine 105, and / or other computing services for collecting healthcare data to develop and run software-based components of the pregnancy prediction engine 105. In some embodiments, the care applications of the analysis server 102 interact with and provide output to a patient device 114. Additionally or alternatively, in some embodiments, the care applications of the analysis server 102 interact with and provide output to a provider device 116.

[0042] The analysis server 102 comprises any computing device including hardware (e.g., processor, non-transitory machine-readable storage memory) and software components (e.g., pregnancy prediction engine 105, web server software) capable of performing the various processes and tasks described herein. While FIG. 1 shows only a single analysis server 102, the analysis server 102 may include any number of computing devices. In some cases, a computing device of the analysis server 102 may perform all or a portion of the processes and benefits of the analysis server 102. The analysis server 102 may include computing devices operating in a distributed or cloud computing configuration and / or a virtual machine configuration. It should also be understood that in some embodiments, the functionality of the analysis server 102 may be performed in part or entirely by a computing device of the care system 110 or a patient device 114.

[0043] The pregnancy prediction engine 105 includes software programming for predicting and detecting one or more types of pregnancy disorders. As further described in FIG. 2 , the pregnancy prediction engine 105 may include software routines and machine learning models of a machine learning architecture that define specific action engines, among other potential actions for adjusting patient care data. The pregnancy prediction engine 105 takes patient care data as input, detects pregnancy instances, extracts features and feature vectors from data available during the pregnancy instances, and applies one or more classifier models to the extracted feature vectors to determine a disorder prediction score indicative of the likelihood of a pregnancy disorder. The pregnancy prediction engine 105 detects a pregnancy disorder when a corresponding classifier trained to detect a given disorder determines that the disorder prediction score meets a detection threshold.

[0044] The pregnancy prediction engine 105 logically operates in several operational phases, including a training phase and a deployment phase (sometimes referred to as “inference time”). During training, the pregnancy prediction engine 105 extracts training features and training vectors from a training dataset to generate various predicted outputs. The pregnancy prediction engine 105 or a user may compare the predicted outputs to labels comprising the expected outputs to determine the level of error. The user or the machine learning functionality of the pregnancy prediction engine 105 may adjust or tune, for example, algorithms, heuristics, data input types, hyperparameters, weights, thresholds, or other aspects of the pregnancy prediction engine 105's functional engine to reduce the level of error. During deployment, the patient device 114 or provider device 116 provides patient data to the pregnancy prediction engine 105. The pregnancy prediction engine 105 extracts patient features and feature vectors from the patient care data to generate various detection outputs.

[0045] The management device 103 may include any computing device including hardware (e.g., processor, non-transitory machine-readable storage memory) and software components capable of performing the various processes and tasks described herein. Non-limiting examples of the management device 103 may include a personal computer (e.g., a workstation computer, a laptop computer), a mobile device, a tablet, etc. The management device 103 includes a user interface that allows a system administrator user to interact with the configuration of the system 100, including the configuration of the pregnancy prediction engine 105. The administrator may enter various configuration inputs that, for example, adjust or tune algorithms, heuristics, data input types, weights, and thresholds, among other aspects of the pregnancy prediction engine's 105 functional engine.

[0046] Analytic database 106 may be hosted on any number of computing devices, including hardware (e.g., processors, non-transitory machine-readable storage memory) and software components capable of performing the various processes and tasks described herein. Analytic database 106 may store patient care data about patients associated with analysis system 101 (e.g., patients treated by clinicians utilizing pregnancy prediction engine 105, patients operating patient devices 114 that execute software utilizing pregnancy prediction engine 105). Analytic database 106 may store configurations of pregnancy prediction engine 105, as received from provider device 116 or management device 103, and / or as automatically tuned by machine learning models and functionality of pregnancy prediction engine 150. In some embodiments, analytical database 106 may further include various instances or versions of pregnancy prediction engine 105 that can be adapted by administrators or clinicians or patients with different care system 110 configurations.

[0047] As mentioned, the care system 110 includes the computing network infrastructure for various clinical entities, such as hospitals, clinics, or research organizations, among others. The hardware and software components of the care system 110 generate and host patient care data and may access and interact with software services hosted by the analysis system 101 (e.g., care software, pregnancy prediction engine 105).

[0048] The provider device 116 includes any computing device including hardware (e.g., processor, non-transitory machine-readable storage memory) and software components capable of performing the various processes and tasks described herein. Non-limiting examples of the provider device 116 may include a personal computer (e.g., a workstation computer, a laptop computer), a mobile device, a tablet, etc. In some cases, the provider device 116 includes a user interface that allows a clinician-user (e.g., a physician, a medical professional, a researcher) to interact with a particular configuration of the pregnancy prediction engine 105 and / or the patient care data in the provider database 118 or the analysis database 106. An administrator may input various configuration inputs, for example, to adjust or tune algorithms, heuristics, data input types, weights, and thresholds, among other aspects of the functional engines of the pregnancy prediction engine 105. The provider device 116 may input a particular configuration, for example, indicating what type of data should be used to identify pregnancy cases or detect particular types of pregnancy disorders. In some cases, provider device 116 may contain patient care data in memory and provide the patient care data as additional patient care data input to analysis system 101. Output generated by analysis server 102 may be transmitted to and presented in a clinician user interface on provider device 116 so that the clinician may review the results with the patient or other process.

[0049] The provider database 118 may be hosted on any number of computing devices, including hardware (e.g., processors, non-transitory machine-readable storage memory) and software components capable of performing the various processes and tasks described herein. In some cases, the provider database 118 may include various types of patient electronic health records (EHRs) according to EHR standards, although embodiments are not so limited. The provider database 118 may include other types of data related to patient care other than official or standardized EHR patient care data. The analysis server 102 may receive or retrieve patient care data from the provider database 118, or the provider device 116 may transmit patient care data from the provider database 118 to the analysis server 102.

[0050] FIG. 2 illustrates data flow among components of a system 200 for predicting and caring for pregnancy disorders, according to one embodiment. The system 200 includes a data source 201, a user device 216, and an analysis server 203. The analysis server 203 includes machine-executable pregnancy prediction software programming, including the software routines of a pregnancy prediction engine 220. The software components of the pregnancy prediction engine 220 may include functionality for specific operational engines that define a machine learning architecture. As shown in FIG. 2, the pregnancy prediction engine 220 includes software programming that defines or performs the functions of, for example, a data ingestion engine 202, a pregnancy detector 204, a feature extractor 206, a label generator 208, and disorder classifiers 210a-210n (collectively referred to as one or more disorder classifiers 210). The software routines of the pregnancy prediction engine 220 are executed by the analysis server 203 of the exemplary system 200, although the software components of the pregnancy prediction engine 220 may be executed by any computing device including a processor capable of performing the operations of the pregnancy prediction engine 220, and by any number of such computing devices.

[0051] The data source 201 may include any electronic computing device or software component that contains or provides various types of data to the pregnancy prediction engine 220 of the analysis server 203. In some cases, the data source 201 includes a database (e.g., analysis database 106, provider database 118) containing an EHR or other type of data stored in non-transitory machine-readable storage memory. In some cases, the data source 201 includes a user device 216 that may store particular types of data in non-transitory storage or may receive user input indicating particular types of data via a user interface of the user device 216. In some cases, the data source 201 is hosted on the analysis server 203 with the pregnancy prediction engine 220. Additionally or alternatively, in some cases, the data source 201 is hosted on a computing device (not shown) separate from the analysis server 203 with the pregnancy prediction engine 220. In such cases, the data source 201 transmits various types of data to the analysis server 203 over one or more networks.

[0052] The user devices 216 (e.g., the management device 103, the patient device 114, and the provider device 116) may include any type of computing device operated by a user to interact with or configure the functionality of the pregnancy prediction engine 220. Users operating the user devices 216 may include, for example, administrators of the system 200, clinicians (e.g., physicians, care workers, researchers), and patients, among others. Users may operate the user devices 216 to enter various configuration inputs including instructions for configuring the operation of the pregnancy prediction engine 220. Users may operate the user devices 216 to provide various types of data input parameters for training, tuning, or deploying the pregnancy prediction engine 220.

[0053] Analysis Server The analysis server 203 (e.g., analysis server 102) may include any computing device including hardware and software components capable of performing the features and functions described herein. The analysis server 203 may include, for example, non-transitory machine-readable storage memory and one or more processors for executing the software components of the pregnancy prediction engine 220. The analysis server 203 may receive various data and configuration inputs from the data sources 201 and the user devices 216. The analysis server 203 may perform the operations of the pregnancy prediction engine 220 using the data input parameters and according to the configuration.

[0054] The data ingestion engine 202 of the pregnancy prediction engine 220 includes software routines programmed to receive or retrieve that type of data from the data sources 201. The data ingestion engine 202 may be programmed to obtain data from the data sources 201 at preconfigured intervals or in response to user commands received as configuration or other user input from a user operating the user device 216. The data ingestion engine 202 may perform various functions to ingestion and prepare data for use by the pregnancy prediction engine 220, such as collating, normalizing, organizing, parsing, and / or validating data input, among other possible functions.

[0055] Specific data inputs or system configurations may be received at the data ingestion engine 202 (or other components of the system 200) from the user device 216. For example, a clinician (e.g., physician, health care professional, researcher) may indicate and configure the data source 201, the type of data input parameter, or which disorder classifier 210 is associated with which type of data parameter. In some cases, the user may verify the type or amount of data input to be acquired from the data source 201 or configure thresholds requiring assessment of the type or amount of data input to be acquired from the data source 201. Non-limiting examples may include verification of, or threshold requirements for, data acquisition feasibility (e.g., data volume, data transfer time frame, or throughput), source population data, and inclusion of relevant data types, among others. In some implementations, the data ingestion engine 202 receives specific correction inputs from the user device 216. The data ingestion engine 202 may identify missing or inaccurate data values ​​and may correct or update the data values ​​according to the correction inputs received from the user device 216.

[0056] For example, a data input may be available at the data source 201, but the user or the data ingestion engine 202 may determine whether the data input parameters will be readily available to a majority (e.g., 80% or more) of patients for a given data source 201 (e.g., a target database for a target health data system database to which the disorder classifier 210 is applied, a past health data system database to which the disorder classifier 210 was previously applied). Additionally or alternatively, the quality of the data input parameters within a standard EHR database record may also be considered, where "data quality" may be an automated or manual assessment of whether a particular type of data input is suitable to inform clinical decision-making and involves assessing the accuracy and standardization of a given parameter, where "accuracy" means that the parameter's value reflects truth and "standardization" means that the parameter value is standardized across clinical sites.

[0057] The data ingestion engine 202 (or other software components of the analytics server 203) may perform data augmentation operations on the data inputs obtained from the data sources 201. The data augmentation operations may generate additional types of data inputs or data points and / or generate synthetic training data for training the machine learning models of the machine learning architecture of the pregnancy prediction engine 220.

[0058] In some cases, the data ingestion engine 202 may convert certain types of data into health parameters. For example, the data ingestion engine 202 may combine or parse certain data input parameters into a combined parameter. As an example, the data ingestion engine 202 may algorithmically combine height and weight measurements to calculate or output a body mass index. In some cases, the data ingestion engine 202 may categorize continuous health parameters based on clinically relevant cutoff values ​​or thresholds. For example, continuous measurements of body mass index may be categorized as “underweight,” “healthy weight,” “overweight,” or “obese” according to the health parameter's corresponding threshold. As mentioned, in some embodiments, the data ingestion engine 202 performs operations to address missing values. In such cases, when the user device 216 inputs, or when the data ingestion engine 202 indicates that there are informative missing values, the missing values ​​are coded into a “missing” category, allowing a machine learning model of the machine learning architecture to learn from the missing values. However, if the user device 216 or the data ingestion engine 202 determines that there are uninformative missing values, the data ingestion engine 202 may perform a function in which the missing values ​​are imputed using an imputation method such as K-nearest neighbors.

[0059] The pregnancy detector 204 of the pregnancy prediction engine 220 includes software routines for detecting, predicting, or otherwise identifying instances in which a patient is pregnant based on an analysis of the patient's care data, such as the patient's EHR data obtained from an EHR database (e.g., provider database 118), among other potential data sources 201.

[0060] In some embodiments, the pregnancy detector 204 includes pre-configured functionality and algorithms for analyzing various types of data within a particular patient's patient care data to identify pregnancy markers. Pregnancy markers include specific patient care data, or patterns of patient care data, that are indicative of pregnancy within a given time period. The pregnancy detector 204's functionality may include applying various heuristics and relative weights to the patient data to identify pregnancy markers. The pregnancy detector 204 identifies or detects instances of pregnancy where the pregnancy marker data meets one or more thresholds. For example, the pregnancy detector 204 may query (e.g., perform a regular expression search) the patient care data to identify matching pregnancy indicators, such as specific metrics (e.g., blood test metrics), diagnostic or treatment codes, and data types (e.g., instances of ultrasound images), among other data types. In some configurations, the pregnancy detector 204 may detect an instance of pregnancy in response to determining that the amount of identified pregnancy indicators meets a threshold amount of matching. Additionally or alternatively, in some configurations, the pregnancy detector 204 may detect an instance of pregnancy in response to determining that an identified pregnancy indicator is associated with a timestamp in the care data that meets one or more timing thresholds.

[0061] As mentioned, an administrator or clinician may operate the user device 216 to enter configuration inputs into the analysis server 203 to configure the operation of the pregnancy prediction engine 220. The configuration inputs may indicate, for example, pregnancy indicators, pregnancy markers, pregnancy detection thresholds, query parameters for identifying pregnancy indicators or pregnancy markers within the patient's care data.

[0062] In some embodiments, the pregnancy detector 204 may include machine learning models and techniques of a machine learning architecture trained to detect instances of pregnancy using patient care data. For example, the pregnancy detector 204 may extract a feature vector including pregnancy indicator data and apply a pregnancy detector classifier (not shown) that calculates a pregnancy likelihood score. The pregnancy detector 204 detects an instance of pregnancy if the pregnancy detector classifier determines that the pregnancy likelihood score meets a pregnancy detection threshold score.

[0063] The feature extractor 206 of the pregnancy prediction engine 220 includes software routines for identifying and extracting pregnancy care features or feature vectors. When the pregnancy detector 204 detects a pregnancy case, the pregnancy detector 204 applies the feature extractor 206 to the care data to extract features from the patient's care records associated with the patient's pregnancy case. For example, the pregnancy detector 204 may select a collection of one or more care data records with timestamps within a duration threshold of the detected pregnancy case. The pregnancy detector 204 may then extract features and feature vectors for the given pregnancy case. The feature extractor 206 may store the features in storage memory of the analytic server 203 or a database (e.g., the analytic database 106) of the system 200.

[0064] Feature extractor 206 may perform characterization operations that convert various forms of data input (from data sources 201) into numerical data for use in the machine learning models of the machine learning architecture of pregnancy prediction engine 220. For example, for continuous time series data of fetal heart rate, the characterization operation involves extracting a finite set of time- and frequency-related measurements.

[0065] In some embodiments, to improve model interpretability (i.e., to facilitate explanation to clinicians), the user device 216 provides user interface options for selecting and configuring data input parameters for the data ingestion engine 202 and features extracted by the feature extractor 206. In some cases, variable selection techniques in the user device 216 or pregnancy prediction engine 220 may create a "reduced model" with a subset of feature variables. In some implementations, the disorder classifier 210 performs logistic regression to train on the extracted features. In such implementations, the disorder classifier 210 may implement a lasso method, which reduces the relative contribution of each feature variable to zero according to a preconfigured importance or weighted value. In some implementations, the disorder classifier 210 may implement other types of machine learning models, such as recursive feature elimination (RFE). In such an embodiment, the RFE training operation may iteratively remove feature variables by, for example, fitting a given machine learning algorithm of the disorder classifier 210 on the training features of the training data, ranking the features by weight importance, discarding one or more of the least important features (having relatively the lowest ranking weights), and re-fitting the model of the disorder classifier 210. The pregnancy prediction engine 220 may repeat the RFE training process until the remaining features balance the interpretability of the machine learning model (e.g., fewer variables) with the performance of the machine learning model.

[0066] In some embodiments, the pregnancy prediction engine includes a label generator 208 that includes software routines for generating or associating extracted screening features and feature vectors with training labels, where the training labels indicate expected outcomes corresponding to the screening features. The expected outcomes indicate the "ground truth" for a given feature vector extracted from specific data values ​​associated with a patient's pregnancy cases. In some circumstances, for example, the pregnancy detector 204 may misclassify (e.g., false positive, false negative) the outcome label for a portion of a patient's data record. The label generator 208 generates training labels for the feature vectors that indicate the ground truth of whether the feature vector extracted from a portion of the patient's data record is actually a pregnancy disorder and / or an instance of a particular disorder.

[0067] The label generator 208 may manually or automatically generate training labels indicating ground truth for the screening feature vectors extracted from the patient data input. In some cases, a user (e.g., an administrator, a clinician) may operate the user device 216 to enter configuration input indicating expected outcomes for the training labels, which may be provided to the label generator 208 for manual association with the feature vectors by the pregnancy prediction engine 220. In some cases, the label generator 208 may be pre-configured to algorithmically determine whether ground truth for the screening feature vectors has been assigned to a type of data input parameter according to pre-configured heuristics and weights, in which case the label generator 208 may execute a separate algorithm or threshold from the pregnancy detector 204 to determine the ground truth of whether the patient's record indicates a pregnancy instance and / or pregnancy disorder.

[0068] The training labels may be used to tune the algorithms of the pregnancy detector 204 or the feature extractor 206. As an example, a user may compare the generated training output of the pregnancy detector 204 with the training labels representing the ground truth to determine whether the pregnancy detector 204 properly detected pregnancy cases in the patient's care data. The user may adjust the algorithm's features, heuristics, types of data input parameters, weights, or thresholds to improve the accuracy of the pregnancy detector 204. As another example, a user may compare the generated training output of the feature extractor 206 or the disorder classifier 210 with the training labels representing the ground truth to determine whether the feature extractor 206 extracted optimal screening features or whether the disorder classifier 210 properly detected a given pregnancy disorder. The user may adjust the feature extractor 206's algorithm's features, heuristics, types of data input parameters, weights, or thresholds to improve the accuracy of the screening features extracted from the patient's data records. The user device 216 or pregnancy prediction engine 220 may calculate or determine which features have the greatest impact, remove features with relatively low impact, and adjust the functionality of the pregnancy detector 204, feature extractor 206, or disorder classifier 210. In some cases, for example, the pregnancy prediction engine 220 may implement a feature reduction method (e.g., LASSO) to adjust the pregnancy detector 204, feature extractor 206, or disorder classifier 210.

[0069] 2, the label generator 208 is coupled to the feature extractor 206 to generate features or feature vectors or to associate them with training labels. Embodiments need not be so limited. Additionally or alternatively, in some embodiments, the label generator 208 is coupled to one or more data sources 201 or the data ingestion engine 202 such that the label generator 208 generates labels or associates labels with care data obtained from the data source 201 when the data ingestion engine 202 receives care data from the data source 201.

[0070] In some implementations, the disorder classifier 210 may implement a machine learning model or technique using a difference or loss function to train or tune the hyperparameters or weights of the disorder classifier 210. The pregnancy prediction engine 220 may determine the difference or loss (e.g., level of error) based on comparing the predicted outcomes of the compared disorder classifiers 210 with training labels that indicate the predicted outcomes.

[0071] A disorder classifier 210 of the pregnancy prediction engine 220 for detecting pregnancy disorders in patient care data. The machine learning architecture of the pregnancy prediction engine 220 includes any number of disorder classifiers 210 for classifying specific pregnancy disorders in the patient care data, with each disorder classifier 210 programmed and trained to detect whether the patient care data is indicative of a particular type of pregnancy disorder. In some implementations, the disorder classifiers 210 include the functionality of a binary classifier. Additionally or alternatively, in some implementations, one or more of the disorder classifiers 210 include the functionality of a multi-class classifier. For example, the first disorder classifier 210a includes a hypertension detector trained to detect and classify whether the patient care data is indicative of hypertension. As another example, the second disorder classifier 210b includes a GDM detector trained to detect and classify whether the patient care data is indicative of an instance of GDM. Embodiments may include additional or alternative examples of disorder classifiers 210 implemented in the pregnancy prediction engine 220, as described herein. During operation, the pregnancy prediction engine 220 applies the disorder classifier 210 to the screening feature vector to output a detection outcome. The disorder classifier 210 generates a disorder likelihood score that indicates the likelihood or probability that a disorder will be indicated in the patient care data. The disorder classifier 210 detects a disorder in response to determining that the disorder likelihood score meets the detection threshold score.

[0072] As mentioned, the disorder classifier 210 may execute a machine learning model or technique to train or tune the hyperparameters or weights of the disorder classifier 210 using a difference or loss function. The pregnancy prediction engine 220 may determine the difference or loss (e.g., level of error) based on comparing the predicted outcomes of the compared disorder classifiers 210 with training labels indicative of the expected outcomes.

[0073] In some embodiments, the pregnancy prediction engine 220 may employ a data augmentation operation to train the disorder classifier 210 based on one or more temporal features included in or associated with the feature vector used to train the disorder classifier 210. In many situations, certain types of data are not always available to the pregnancy prediction engine 220 at deployment inference time. Due to real-world delays, there may be a time lag between the time a test is administered or a sample is taken and the time the measurement or metric is included in the patient care data. The disorder classifier 210 is trained according to a distribution curve that plots the output generated for each of the training input vectors. Real-world delays in the data may skew the results of the disorder classifier 210. At inference time, the presence or absence of certain types of data inputs used to extract a patient's feature vector at a given time point in the patient's pregnancy may cause the disorder classifier 210 to predict a skewed outcome associated with the disorder classifier's 210 distribution and potentially an inaccurate outcome relative to the patient's real-world condition. To address these issues, the data augmentation process associates timestamps with the training data or feature vectors at training time for robustness. The pregnancy prediction engine 220 may include or associate one or more timestamps (e.g., when a data input value was generated as a measurement, when a data input value was available to the pregnancy prediction engine 220 in the patient care data). These timestamps may be associated with specific data input values ​​in the training data and / or may be associated with the training feature vector as a whole prior to providing such feature vector to the disorder classifier 210 at training time. [Example]

[0074] Illustrative Embodiments Example 1: Prediction of hypertension of pregnancy (HDP) The incidence of HDP doubled in the United States between 2008 and 2019. Alarmingly, 24% of maternal deaths occurring during hospitalization for delivery had a documented diagnosis code for HDP. Early identification of patients who would benefit from HDP intervention may reduce the incidence of HDP by enabling physicians to initiate lifestyle-based interventions or prescribe preventive or interventional treatments. Additionally, current ACOG risk criteria could be improved by using machine learning.

[0075] Figure 3 shows an exemplary timeline for early prediction and intervention of HDP. As shown in the timeline, early prediction may involve information collected during the first trimester. Non-limiting examples of health parameters that may be collected and useful in the prediction method include family history of hypertension, chronic hypertension, diabetes, autoimmune disease, kidney disease, aspirin use, reproductive history, time since last birth, history of stillbirth, history of HDP, use of IVF, age, race, ethnicity, height, weight, MCV, hematocrit, platelets, hemoglobin, and blood pressure. As also shown in the timeline in Figure 3, a diagnosis of HDP may not be made until 20 weeks of pregnancy under current guidelines, or until two weeks after delivery. This creates a significant gap in the time in which a patient may develop adverse HDP. Using early detection models, early interventions, such as blood pressure monitoring or aspirin (or other medication) intervention, may be prescribed. In this way, the harmful effects of HDP on the patient and child may be reduced or mitigated.

[0076] As shown in Figure 4, the machine learning model was trained and validated with internal data. The ROC curve in Figure 4 shows an AUC of 0.78, indicating good model performance in predicting HDP in the internal validation cohort. Furthermore, as shown in Figure 5, the model performed well in an external cohort of primiparous pregnancies, even outperforming the ACOG checklist (USPSTF aspirin initiation guidelines). Based on this surprising result, it is demonstrated that the system and method herein can provide earlier (i.e., first trimester) and more reliable disease prediction than existing methods. The study population included 18,456 pregnancies from a large academic medical center in the Midwestern United States, and the model type used in this example was a logistic regression model.

[0077] The HDP model was also tested for performance differences across different races / ethnicities, as shown in Figure 6. No differences in model performance by race / ethnicity were observed, with AUC values ​​ranging from 0.77 to 0.93, indicating excellent performance across the categories examined.

[0078] Example 2: Prediction of Gestational Diabetes Mellitus (GDM) Between 2016 and 2020, the incidence of GDM increased by 30%. Pregnancies complicated by GDM are at increased risk of preterm birth, macrosomia, and large-for-gestational-age babies. Additionally, 10% of women with GDM develop type 2 diabetes shortly after giving birth.

[0079] FIG. 7 shows an exemplary timeline for early prediction and intervention of GDM. As shown in the timeline, early prediction may involve information collected during the first trimester. Non-limiting examples of health parameters that may be collected and useful in the prediction method include family history of hypertension, chronic hypertension, diabetes, exercise status, parity, time since last birth, history of GDM, age, race, ethnicity, height, weight, MCV, hematocrit, platelets, hemoglobin, and blood pressure. As also shown in the timeline in FIG. 7, a diagnosis of GDM may not be made until 24 weeks of gestation under current guidelines, and may not occur until 28 weeks of gestation. This creates a significant gap in the time during which a patient may develop adverse GDM symptoms. Using an early detection model, early interventions, such as exercise intervention, nutritional counseling, and timely receipt of blood glucose testing, may be prescribed. In this way, the harmful effects of GDM on the patient and child may be reduced or mitigated.

[0080] The machine learning model was trained and validated on internal data, as shown in Figure 8. The ROC curve in Figure 8 shows an AUC of 0.73, indicating good model performance in predicting GDM in the internal validation cohort. The study population included 23,202 pregnancies from a large academic medical center in the Midwestern United States. Patients with pregestational diabetes were excluded from the model. The model type used in this example was a histogram-based bagged tree ensemble.

[0081] Example 3: Prediction of Excessive Gestational Weight Gain (eGWG) Nearly 50% of US pregnancies exceed gestational weight gain (GWG) goals, which can lead to significantly increased risk of HDP, GDM, and long-term metabolic outcomes. Identifying individuals at highest risk for eGWG in early pregnancy provides an opportunity for counseling and behavioral interventions to reduce weight gain.

[0082] FIG. 9 shows an exemplary timeline for early prediction and intervention of eGWG. As shown in the timeline, early prediction may involve information collected during the first trimester. By way of non-limiting example, health parameters that may be collected and useful in the prediction method may include OB history, chronic hypertension, pre-gestational diabetes, age, race, height, weight, pre-pregnancy weight, etc. As also shown in the timeline of FIG. 9, a diagnosis of eGWG may not be made until 37 weeks of gestation under current guidelines, or even 40 weeks of gestation. This creates a significant gap in the time during which a patient may develop adverse eGWG. Using early detection models, early interventions, such as exercise interventions and nutritional counseling, may be prescribed. In this way, the adverse effects of eGWG on the patient and child may be reduced or mitigated.

[0083] The machine learning model was trained and validated on internal data, as shown in Figure 10. The ROC curve in Figure 10 shows an AUC of 0.82, indicating good model performance in predicting eGWG in the internal validation cohort. The study population included 10,867 pregnancies from a large academic medical center in the Midwestern United States. Patients with preterm delivery or missing weight measurements were excluded from the model. The model type used in this example was logistic regression.

[0084] Example 4: Comparison of model performance As described herein, models for gestational hypertension, gestational diabetes, and excessive gestational weight gain have been developed and shown to effectively predict these pregnancy disorders. A comparison of these developed models is shown below in Table 2.

[0085] [Table 2]

[0086] * The values ​​given in Table 2 for sensitivity and specificity correspond to the maximum value of sensitivity plus specificity across all likelihood cutoff values.

[0087] Example 5: Optimized health parameter sets for outcome prediction As described herein, robust predictive model development involves data selection and training. The present invention involves discovering groupings of input health parameters that provide high-performance predictive models for predicting pregnancy disorders. FIG. 11 generally illustrates the operational steps of a predictive machine learning method. For example, these groupings of health parameters may advantageously provide a model with an AUC of at least about 0.7, and often higher. In embodiments herein, the model may be limited to one of the groupings of input health parameters (i.e., the input health parameters may be composed of a grouping). In embodiments herein, the model may include, or alternatively may be composed of, about 80% of the parameters within a particular grouping of input health parameters. An exemplary advantageous embodiment including a grouping set of input health parameters is shown in Table 3 below.

[0088] [Table 3] TIFF2025539072000006.tif175169 TIFF2025539072000007.tif219169 TIFF2025539072000008.tif148169 TIFF2025539072000009.tif160169 TIFF2025539072000010.tif170169 TIFF2025539072000011.tif154169 TIFF2025539072000012.tif154169 TIFF2025539072000013.tif176169 TIFF2025539072000014.tif192169 TIFF2025539072000015.tif170169

[0089] Example 6: Reliable prediction of HDP in early pregnancy A machine learning model was trained for effective prediction of HDP in first trimester, as shown in Figure 12. The full model included key health parameters BMI, pre-pregnancy weight, SBP, DBP, hypertension, asthma, migraine, UTI, anemia, mental health, gonorrhea, chlamydia, yeast infection, bacterial vaginosis, diabetes, hemoglobin, hematocrit, MCV, platelets, blood type, urine culture, fetal nuchal edema, smoking, age, and insurance.

[0090] In this study, the research data utilized came from a publicly available dataset of nulliparous pregnancies. The primary diagnosis was any diagnosis of hypertension of pregnancy (HDP) between 20 weeks of gestation and 2 weeks after delivery, and outcomes were readily available in the dataset. HDP included diagnoses of gestational hypertension, preeclampsia with or without severe features, aggravated preeclampsia with or without severe features, and eclampsia.

[0091] In this study, 207 predictor health parameters were included in constructing the full model; the model did not include information on maternal race and ethnicity in training the predictive model. However, maternal race and ethnicity were used to assess model performance by race and ethnicity in stratified analyses. Additionally, missingness was determined to be informative for HDP outcomes and was included in the analysis. The final sample size was 9,124. Any predictors in either set with missing values ​​were coded "-10000" specifically for individuals whose values ​​were unknown, allowing subsequent models to be trained even in the absence of a predictor.

[0092] To train the model, 80% of patients were randomly assigned to the training set and 20% to the holdout training set. Balanced class weights were used to account for imbalances in HDP diagnosis, ensuring that misclassification in the minority class (HDP diagnosis) was penalized similarly to the majority class (non-HDP diagnosis). For the outcome of HDP diagnosis, a random forest model was developed using the full set of predictors / parameters to distinguish between individuals with and without HDP. To find optimal hyperparameters for the model, such as the number of decision trees to include, a grid search with three-fold cross-validation was used with sensitivity as the scoring metric. This resulted in a maximum tree depth of 50, a minimum number of samples per leaf node of 1, and a minimum number of samples required for splitting, as well as two internal nodes. To improve model interpretability, a reduced model with 30 predictors was created using recursive feature elimination.

[0093] The full model performed well, with an AUC of 0.73 (95% CI: 0.70, 0.75). However, as explained, the full model produces output variables that can be difficult for medical professionals to understand. In many cases, understanding of these variables can be improved by reducing the model to limit the number of output variables as described herein. As shown in Figure 12, reducing the model had a limited effect on performance, as assessed by an AUC value of 0.71 (95% CI: 0.68, 0.74), indicating that the model is robust enough to undergo procedural reduction while retaining acceptable performance.

[0094] Example 7: HDP model outperforms USPSTF guidelines In pregnant patients at risk for cardiovascular problems, USPSTF guidelines provide information on when physicians should initiate aspirin management. Figure 13 shows the reduced HDP model of Example 6 compared to the aspirin management guidelines. When set at the same specificity level as the USPSTF guidelines (53%), the reduced HDP model performed with a sensitivity of 80%. Compared to the USPSTF guidelines, this represents an additional 15 HDP patients for every 100 HDP patients who were successfully initiated on aspirin. This result demonstrates the failure of the traditional guidelines: 15% of patients who could benefit from aspirin management are not treated, potentially suffering from preventable health outcomes. Additionally, when set at the same level of sensitivity, the reduced model had a specificity of 0.65. This represents an additional 12 non-HDP patients for every 100 non-HDP patients for whom aspirin was not initiated using the current model, compared to the traditional guidelines. Thus, the machine learning models and related methods described herein provide unprecedented predictions of pregnancy disorder outcomes to direct disease interventions and improve patient outcomes.

[0095] Example 8: GDM models consistently predict disease risk in early pregnancy Figure 14 shows a comparison of five machine learning models for predicting GDM in early pregnancy with high confidence. The primary outcome is a diagnosis of gestational diabetes based on meeting one of the following criteria:

[0096] Glycemic control test (GCT) > 200 mg / dL, or GCT > 130 mg / dL and two or more of the following for a 3-hour glucose tolerance test (GTT): fasting >= 95 mg / dL, time 1 >= 180 mg / dL, time 2 >= 155 mg / dL, time 3 >= 140 mg / dL, or one or more of the following for a 2-hour GTT: fasting >= 92 mg / dL, time 1 >= 180 mg / dL, time 2 >= 153 mg / dL. Table 4 below provides information about the models used.

[0097] In the five models used, important parameters included the ACOG guideline variables (Models 1–5): BMI, first-degree relatives with diabetes, hypertension, PCOS, cardiovascular disease, and race and ethnicity, as well as additional variables in Models 2–5: age, parity, smoking, insurance type, asthma, migraine, blood type, hemoglobin, MCV, hematocrit, platelet PAPP-A, fetal nuchal edema, anxiety, depression, and urine culture. Models 2 and 4 excluded race and ethnicity. Based on the obtained AUC values, each model performed well for GDM prediction, but including clinical testing and past diagnostic history (i.e., additional variables) generally improved model performance. Furthermore, including race and ethnicity in the model significantly improved performance. Overall, these results demonstrate the robustness of the described models for predicting GDM in early pregnancy.

[0098] The various illustrative logic blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the invention.

[0099] Computer software-implemented embodiments may be implemented in software, firmware, middleware, microcode, hardware description languages, or any combination thereof. A code segment or machine-executable instruction may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, attributes, or memory contents. Information, arguments, attributes, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, or network transmission.

[0100] The actual software code or specialized control hardware used to implement these systems and methods is not a limitation of the invention. Thus, although the operation and behavior of the systems and methods have been described without reference to specific software code, it will be understood that software and control hardware can be designed to implement the systems and methods based on the description herein.

[0101] When implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. The steps of a method or algorithm disclosed herein may be embodied in a processor-executable software module, which may reside on a computer-readable or processor-readable storage medium. Non-transitory computer-readable or processor-readable media includes both computer storage media and tangible storage media that facilitate transferring a computer program from one place to another. Non-transitory processor-readable storage media may be any available medium that can be accessed by a computer. By way of example and not limitation, such non-transitory processor-readable media may include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, or any other tangible storage medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer or processor. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-Ray discs, where disks typically replicate data magnetically and discs replicate data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of code and / or instructions on a non-transitory processor-readable medium and / or computer-readable medium, which may be incorporated into a computer program product.

[0102] The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the spirit and scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the appended claims and the principles and novel features disclosed herein.

[0103] While various aspects and embodiments have been disclosed, other aspects and embodiments are contemplated. The various aspects and embodiments disclosed are for purposes of illustration and not limitation, with the true scope and spirit being indicated by the following claims.

Claims

1. To determine the discrimination of adverse pregnancy outcomes and receiving input data for training data, the input data including a plurality of respective health parameters for each previous patient who experienced the adverse pregnancy outcome; selecting a subset of the respective plurality of health parameters associated with a respective outcome, the selecting being based on known outcome associations or outcome associations calculated based on machine learning output; assigning a subset of patient parameters associated with each outcome to the training data and the test data; fitting one or more machine learning models using the training data; evaluating the performance of the one or more machine learning models using the test data to select a best-performing machine learning model based on a statistical comparison of performance between the one or more machine learning models; updating, by the computer, the classification threshold of the selected machine learning model as a risk probability cutoff value for a given patient to develop the adverse pregnancy outcome; 20. A computer-implemented method comprising:

2. 10. The method of claim 1, further comprising determining exclusion criteria for excluding any of the respective plurality of health parameters based on determining whether any respective previous patient meets the exclusion criteria.

3. 10. The method of claim 1, further comprising determining the classification threshold of the selected machine learning model as a risk probability cutoff value for a given patient to develop the adverse pregnancy outcome.

4. 2. The method of claim 1, further comprising: transforming each relevant subset having some parameters of the respective plurality of health parameters into a respective transformed parameter having a fewer number of parameters; and replacing the respective subset with the respective transformed parameter.

5. The method of claim 1 , further comprising identifying one or more missing parameters in one or more of the respective plurality of health parameters.

6. 6. The method of claim 5, further comprising identifying whether the missing parameters are informative for disease outcome.

7. The method of claim 6 , further comprising, for missing values ​​that are not identified as informative, imputing a representative value to replace the missing value.

8. The method of claim 7 , wherein the imputation is based on a k-nearest neighbor method.

9. The method of claim 1 , wherein complex input data having multiple values ​​of each dependent parameter for each independent parameter is transformed into a number of finite parameters.

10. 10. The method of claim 1, further comprising determining whether a test set produces an unbalanced distribution of outcome variables and balancing the training set by undersampling a majority input parameter or oversampling a minority input parameter.

11. The method of claim 1 , further comprising outputting variables from the selected machine learning model, the output variables being associated with one or more statistically relevant input parameters.

12. The method of claim 11 , further comprising reducing the machine learning model to produce a reduced number of output variables associated with statistically relevant input parameters.

13. 1. A computer-implemented method for predicting a patient's pregnancy outcome, comprising: collecting a plurality of health parameters of the patient before or during the first trimester of pregnancy; inputting the plurality of health parameters into a machine learning model; determining a risk of the patient developing the pregnancy disorder during the pregnancy from an output of the trained machine learning model; Including, The machine learning model includes training it through one or more training steps, the one or more training steps comprising: receiving input data, the input data including a plurality of respective health parameters for each prior patient who has experienced the adverse pregnancy outcome; determining exclusion criteria for excluding any of the respective plurality of health parameters based on a determination of whether any respective previous patient meets the exclusion criteria; selecting a subset of the respective plurality of health parameters associated with a respective outcome, the selecting being based on known outcome associations or outcome associations calculated based on machine learning output; assigning a subset of patient parameters associated with each outcome to the training data and the test data; fitting one or more machine learning models using the training data; evaluating the performance of the one or more machine learning models using the test data to select a best-performing machine learning model based on a statistical comparison of performance between the one or more machine learning models; determining a classification threshold of the selected machine learning model as a risk probability cutoff value for a given patient to develop the adverse pregnancy outcome; 20. A computer-implemented method comprising:

14. The method of claim 13 , wherein the parameters are iteratively reassessed in the development of the model.

15. 1. A computer-implemented method for predicting a patient's pregnancy outcome, comprising: collecting, by a computer, a plurality of health parameters of the patient before or during the first trimester; acquiring, by the computer, the plurality of health parameters into a trained machine learning model having the ability to predict the risk of the outcome; determining, by the computer, from the output of the trained machine learning model, a risk of the patient developing the pregnancy disorder during the pregnancy; 20. A computer-implemented method comprising:

16. The method of claim 15 , wherein the computer is configured to iteratively adjust the parameters of the model.

17. 1. A system for predicting a patient's pregnancy outcome, comprising: a computing device operable to execute computer readable instructions, said computer readable instructions comprising: receiving a plurality of health parameters of the patient before or during an early pregnancy; inputting the plurality of health parameters into a trained machine learning model having a performance to predict risk of the outcome as assessed by an area under the curve (AUC) of at least about 0.7; determining, from the output of the trained machine learning model, the patient's risk of developing the pregnancy disorder during the pregnancy; outputting the risk as a numerical or semantic classification value; A system configured to run

18. The system of claim 17 , wherein the parameters are iteratively reassessed in determining the model.

19. 1. A system for predicting a patient's pregnancy outcome, comprising: a computing device operable to execute computer readable instructions, said computer readable instructions comprising: receiving a plurality of health parameters of the patient before or during an early pregnancy; inputting the plurality of health parameters into a machine learning model; determining, from the output of the trained machine learning model, the patient's risk of developing the pregnancy disorder during the pregnancy; outputting the risk as a numerical or semantic classification value; configured to run The machine learning model includes training it through one or more training steps, the one or more training steps comprising: receiving input data, the input data including a plurality of respective health parameters for each prior patient who has experienced the adverse pregnancy outcome; determining exclusion criteria for excluding any of the respective plurality of health parameters based on a determination of whether any respective previous patient meets the exclusion criteria; selecting a subset of the respective plurality of health parameters associated with a respective outcome, the selecting being based on known outcome associations or outcome associations calculated based on machine learning output; assigning a subset of patient parameters associated with each outcome to the training data and the test data; fitting one or more machine learning models using the training data; evaluating the performance of the one or more machine learning models using the test data to select a best-performing machine learning model based on a statistical comparison of performance between the one or more machine learning models; determining a classification threshold of the selected machine learning model as a risk probability cutoff value for a given patient to develop the adverse pregnancy outcome; Including, the system.

20. The system of claim 19 , wherein the parameters are iteratively reassessed in the determination of the model.

Images