Systems and methods for assessing neonatal health risk and uses thereof

Abstract

Embodiments herein describe systems and methods for assessing neonatal health risk and uses thereof. Many embodiments utilize one or more machine learning models to determine neonatal disorders based on an electronic health record (EHR) of an individual. Additional embodiments treat, remedy, ameliorate, or ameliorate the disorder.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 268,689, entitled “Systems and Methods to Assess Neonatal Health Risk and Uses Thereof,” filed on February 28, 2022, by Aghaeepour et al., the disclosure of which is hereby incorporated by reference in its entirety.

[0003] STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0004] This invention was made with government support under Contracts HL139844 and GM138353 awarded by the National Institutes of Health. The government has certain rights in this invention. Field of the Invention

[0005] The present invention relates to neonatal health; more particularly, the present invention relates to systems and methods that incorporate machine learning to determine neonatal risk, particularly preterm birth.

[0006] background

[0007] Preterm birth is the leading cause of death in children under 5 years of age. Although gestational age and birth weight and other anthropometric measures provide clinicians with a rough estimate of the risk of neonatal morbidity and mortality, these data are increasingly recognized as poor surrogates. For example, although gestational age is often considered a surrogate for biological immaturity, this variable alone performs poorly as a predictor of risk. For example, many infants born before 28 weeks of gestation will develop at least one of the sequelae of prematurity, including intraventricular hemorrhage (IVH), respiratory distress syndrome (RDS), necrotizing enterocolitis (NEC), sepsis (early or late), retinopathy of prematurity (ROP), BPD, and periventricular leukomalacia (PVL). 6 Some preterm neonates will develop more than 1–2 of these conditions; few infants are free of these conditions.

[0008] Accurate risk prediction and prognosis are essential in perinatal and neonatal medicine. Validated clinical prediction calculators have estimated risk trajectories for common outcomes associated with preterm birth, including death, neurodevelopmental impairment, bronchopulmonary dysplasia (BPD), and others. Prognostic estimates help clinicians and families select appropriate interventions to achieve the outcome(s) they value or most desire. Historically, antenatal and postnatal risk calculators have incorporated a small set of clinical risk factors assessed at a single time point to provide families and providers with a rough estimate of their fetal or neonatal risk. To date, most clinical prediction calculators have limited predictive power and clinical utility due to the small number of parameters considered and the single time point utilized.

[0009] Understanding which premature newborns are more likely to develop acquired complications of prematurity based on their potential individual risk levels is a critical task in line with the requirements of precision medicine in the 21st century. SUMMARY OF THE INVENTION

[0011] This summary is intended to provide some embodiments and is not intended to limit the scope of the present invention in any way. For example, any feature included in the embodiments of this summary is not required by the claims unless the claims clearly record the feature. Various features and steps described elsewhere in this disclosure may be included in the embodiments summarized here, and the features and steps described here and elsewhere may be combined in a variety of ways.

[0012] In some aspects, the techniques described herein relate to a machine learning model comprising a multi-task neural network, wherein the neural network comprises an encoder, a hidden state, and a decoder, wherein the encoder reads an input, wherein the hidden state represents an internal learned representation of the entire input, and wherein the decoder interprets the internal learned representation and reconstructs the input.

[0013] In some aspects, the techniques described herein relate to a machine learning model in which the input comprises an individual's electronic health record (EHR).

[0014] In some aspects, the technology described herein relates to a machine learning model, wherein the machine learning model is trained using EHRs of a plurality of individuals and a plurality of newborns, wherein each of the plurality of individuals has given birth to at least one of the plurality of newborns.

[0015] In some aspects, the technology described herein relates to a method for assessing neonatal risk, the method comprising obtaining or having obtained an electronic health record (EHR) of an individual and determining at least one neonatal disorder for the individual's child based on metabolites in the EHR using a machine learning model, the machine learning model comprising a deep learning neural network having at least one bottleneck layer.

[0016] In some aspects, the technology described herein relates to a method in which the machine learning model determines respiratory support strategies (including ventilator settings) to reduce adverse outcomes.

[0017] In some aspects, the technology described herein relates to a method wherein the respiratory support strategy includes ventilator settings.

[0018] In some aspects, the technology described herein relates to a method wherein the model identifies medications prescribed to a mother that may affect neonatal morbidity.

[0019] In some aspects, the technology described herein relates to a method, wherein the at least one neonatal disorder is selected from bronchopulmonary dysplasia (BPD), intraventricular hemorrhage (IVH), necrotizing enterocolitis (NEC), retinopathy of prematurity (ROP), bronchopulmonary dysplasia (BPD), intraventricular hemorrhage (IVH), necrotizing enterocolitis (NEC), retinopathy of prematurity (ROP), pulmonary hypertension, pulmonary hemorrhage, jaundice, periventricular leukomalacia (PVL), respiratory distress syndrome (RDS), early-onset sepsis, late-onset sepsis, patent ductus arteriosus (PDA), cerebral palsy, and neurodevelopmental impairment (NDI).

[0020] In some aspects, the technology described herein relates to a method wherein the EHRs are from multiple institutions.

[0021] In some aspects, the technology described herein relates to a method further comprising treating at least one neonatal disorder in the child.

[0022] In some aspects, the technology described herein relates to a method for providing intravenous nutrition to a premature infant, the method comprising obtaining or having obtained an electronic health record (EHR) of an individual, wherein the EHR includes detailed information about the individual's health, and the individual is a premature infant, and selecting a nutrient bag containing a mixture of nutrients to supplement the health of the individual.

[0023] In some aspects, the technology described herein relates to a method wherein the machine learning model comprises a multi-task neural network, wherein the neural network comprises an encoder, a hidden state, and a decoder, wherein the encoder reads an input, wherein the hidden state represents an internal learned representation of the entire input, and wherein the decoder interprets the internal learned representation and reconstructs the input.

[0024] In some aspects, the technology described herein relates to a method, wherein the nutrient bag is one bag in a group of nutrient bags, wherein each bag in the group of nutrient bags contains a composition of nutrients generated by bottleneck layer clustering.

[0025] In some aspects, the technology described herein relates to a method wherein the nutrient bag improves wound healing.

[0026] In some aspects, the technology described herein relates to a method wherein the nutritional bag improves neurocognitive development.

[0027] In some aspects, the technology described herein relates to a method wherein the nutrient bag improves respiratory health.

[0028] In some aspects, the technology described herein relates to a method wherein the nutrient bag improves gastrointestinal health.

[0029] In some aspects, the technology described herein relates to a method wherein the nutrient bag improves ocular health.

[0030] In some aspects, the technology described herein relates to a method for nutritional support, the method comprising obtaining health information about an individual and providing dietary recommendations to the individual.

[0031] In some aspects, the technology described herein relates to a method wherein providing dietary advice includes providing food advice.

[0032] In some aspects, the technology described herein relates to a method wherein the food suggestions include at least one baby food suggestion.

[0033] In some aspects, the technology described herein relates to a method in which providing dietary advice includes interacting with a database of food and nutritional information.

[0034] In some aspects, the technology described herein relates to a method for preparing an intravenous nutritional supplement solution, the method comprising developing a set of nutritional prescriptions for intravenous supplementation using a machine learning model, and producing a nutritional bag including a prescription from the prescription set.

[0035] In some aspects, the technology described herein relates to a method wherein the machine learning model comprises a multi-task neural network, wherein the neural network comprises an encoder, a hidden state, and a decoder, wherein the encoder reads an input, wherein the hidden state represents an internal learned representation of the entire input, and wherein the decoder interprets the internal learned representation and reconstructs the input.

[0036] In some aspects, the technology described herein relates to a method wherein producing a nutritional bag includes producing a nutritional bag for each prescription in the formulary.

[0037] In some aspects, the technology described herein relates to a method wherein the nutritional formulary includes at least 5 prescriptions.

[0038] In some aspects, the technology described herein relates to a method wherein the nutritional formulary includes 15 prescriptions.

[0039] In some aspects, the technology described herein relates to a method wherein the nutritional bag is a sterile IV bag.

[0040] Other features and advantages of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The specification and claims will be more fully understood with reference to the following drawings and data graphs, which are provided as exemplary embodiments of the invention and should not be construed as a complete recitation of the scope of the invention.

[0043] Figure 1 An exemplary LSTM-based autoencoder according to various embodiments is shown that is capable of objectively determining a subgroup of an AI model with enhanced performance. Specifically, subgroup discovery performed on a feature latent space obtained using an LSTM autoencoder determines a subgroup of newborns in which the AI ​​model has a high precision-recall ratio. Figure 1 An exemplary architecture of an exemplary LSTM-based autoencoder is provided for extracting a low-dimensional encoded representation of an input sequence containing a mother's EHR history. Subgroup discovery is iteratively performed at each level by dividing the data set into many overlapping subgroups defined by the variables of the obtained latent space. The search path for a single subgroup is performed two levels downward. At the end of the program, the subgroups are scored and sorted for further analysis based on predefined scoring criteria (i.e., AUPRC). The classification accuracy in the subgroups determined by subgroup discovery and in the complete data set is measured by AUC, AUPRC, and AUPRC, compared to a random classifier.

[0044] Figure 2 An exemplary correlation graph of EHR codes in maternal medical history and measurements is shown, where each node represents a code or measurement value, according to various embodiments. The size of the node is proportional to the metric described in the method for evaluating feature importance, and is averaged over all results; the correlation of the nodes connected by the edge is in the top 1% of all correlations.

[0045] Figure 3 Exemplary AUCs of AI models for different neonatal outcomes in preterm newborns (born at <37 weeks of gestation) and term newborns (born at ≥37 weeks of gestation) according to various embodiments are shown. Since none of the term newborns had PVL or anemia, these outcomes were excluded from the chart.

[0046] Figures 4A-4DAn exemplary overview of an AI pipeline for predicting neonatal outcomes according to various embodiments is shown. Figure 4A An example of a hypothetical patient timeline is provided that includes multiple visits before and after labor / birth; at each visit, any combination of conditions, observations, medications, and procedures can be recorded. Figure 4B An exemplary architecture of a multi-input multi-task deep learning model is provided: code sequences from maternal / neonatal medical history are input to a bidirectional LSTM layer with 128 units after code embedding, while maternal / neonatal sociodemographic information, maternal measurements, and (when specified) gestational age and birth weight are input to a dense layer of 4 units. The outputs of these two networks are then concatenated and input to a dense single-layer neural network with 64 units, followed by a set of dense layers, one for each outcome, each set consisting of two dense layers and a single unit output. Figure 4C An exemplary bidirectional LSTM layer is provided for learning bidirectional long-term dependencies between codes within a sequence: Each code in the sequence is input to both the forward and backward LSTM layers, and the outputs of both layers are further concatenated. While processing, the hidden state from the previous code layer in the sequence is passed to the next code layer in the sequence; the hidden state acts as the memory of the neural network, preserving information about previous data that the network has seen before. Figure 4D An exemplary structure of a single LSTM layer for the t-th code in a sequence is provided: ct is the cell state that carries the relevant information during the entire sequence processing, ht is the hidden state of the t-th code in the sequence that is passed to the next code layer in the sequence, and xt is the input to the layer processing the t-th code in the sequence, i.e., the vector corresponding to the embedding of the t-th code in the sequence. Each line carries a complete vector, the circles represent point-by-point operations, and the boxes represent the learned neural network layers with the indicated activation functions. Lines merge to indicate connections, and lines fork to indicate that the content is copied and the copies are sent to different locations. Basically, the LSTM layer learns which information must be discarded from the cell state and which new information must be stored in the cell state; finally, the output is calculated based on the cell state and the processed input.

[0047] Figures 5A-5D The exemplary data provided shows the code ratio in various situations. Specifically, Figure 5A The proportion of codes for each feature set among the total number of unique codes found in the medical histories of 32,354 newborns and 27,519 mothers is shown; Figure 5B The proportion of codes for each feature set among all codes found in the medical history is shown; Figure 5C The average percentage drop in AUC across 24 outcomes for AI models that do not include a specific feature set is shown; Figure 5DThe average percentage reduction in AUPRC across 24 outcomes for AI models that do not include specific feature sets is shown.

[0048] Figure 6 Exemplary data shown show the proportion of maternal medical history present up to delivery for each concept code in Stanford (x-axis) and UCSF (y-axis) pregnancies; the axes are on a logit scale; solid black lines indicate perfect agreement (i.e., the percentages are the same in UCSF and Stanford); dashed black lines indicate when the percentage in one dataset is between half and twice the percentage in the other dataset; concept codes outside these dashed lines were excluded from subsequent analyses.

[0049] Figure 7A-7B The exemplary data shown show a multi-task analysis of EHR data between 2014 and 2020, resulting in longitudinal and comprehensive prediction models for neonatal morbidity before and after birth. The prediction AUC at birth was >0.7 for 22 / 24 outcomes, >0.8 for 17 / 24, and >0.9 for 10 / 24. Neonatal outcomes with AUC>0.9 at birth include BPD, ROP, anemia of prematurity, death, IVH, heart failure, PVL, pulmonary hemorrhage, NEC, and lung correction. At 1 week after birth, the AUC was >0.9 for 21 / 24, and included outcomes with long latencies before diagnosis, such as BPD, ROP, and NEC. Real-time predictions of neonatal morbidity can be difficult to determine, especially for less common outcomes such as IVH, NEC, sepsis, and death. The heat map is colored according to the multi-task modeling output, which comes from 1) a specific time period and 2) contains relevant clinical inputs (drugs, measurements, conditions, observations, and procedures), which produces: ( Fig. 7A ) Fold increase / decrease in AUPRC of the AI ​​model compared to the random classifier at different time points, from 5 months before parturition / birth (-5M) to 2 months after parturition / birth (+2M), where 0 represents parturition / birth, or ( Figure 7B) AUC of individual results at different time points. All results before birth cover maternal codes of 5 months, 4 months, 3 months, 2 months, 1 month, 2 weeks or 1 week before birth. All results at birth cover all maternal inputs until and including delivery. All results after birth cover maternal and neonatal inputs until a specific time point after birth (e.g., 1 week, 2 weeks, 1 month or 2 months). The connection between maternal (n=27,519) and neonatal (n=32,354) clinical data allows us to establish a generalizable longitudinal model that provides risk prediction for all neonatal results. The neonatal results on the y-axis are arranged based on cluster analysis. A comprehensive longitudinal calculator combining maternal, fetal and neonatal risk factors has the potential to transform clinical care by standardizing risk assessment, early identification of high-risk populations that may benefit from additional treatment and / or participation in clinical trials, and timely intervention before the development of results.

[0050] Figure 8 The exemplary data shown show four correlation plots for the 24 neonatal outcomes considered: the size of the nodes is proportional to the prevalence in the study dataset; nodes are connected if the correlation is greater than 0.5, and the thickness and color of the edges are proportional to the strength of the correlation, with darker green and thicker lines showing stronger correlations; preterm outcomes including RDS, IVH, sepsis, NEC, ROP, and anemia of prematurity are shown as highly correlated.

[0051] Fig. 9 An exemplary hypothetical prediction timeline for a newborn with BPD is shown; the prediction scores of the AI ​​model at different time points are based on various risk factors obtained from EHR records in the mother's and newborn's history. Additional data is incorporated into the model throughout pregnancy, at birth, and in the postnatal period, and the prediction model is iteratively improved. Patients with higher BPD prediction scores (i.e., 0.2) have a greater tendency to develop BPD (twice as likely) than patients with lower BPD prediction scores (i.e., 0.1), and can therefore be directly compared. The BPD prediction score should not be interpreted as an individual probability of later development of BPD.

[0052] Figures 10A-10B Exemplary data showing the AUC of an AI model for predicting 24 neonatal outcomes at different time points, from 5 months before delivery / birth (-5M) to 2 months after delivery / birth (+2M); the vertical dashed line indicates delivery / birth; the shaded area indicates the 95% confidence interval of the AUC; the horizontal dashed line indicates the AUC of a random classifier (i.e., 0.5).

[0053] Figures 10C-10DExemplary data showing the AUPRC of an AI model used to predict 24 neonatal outcomes at different time points, from 5 months before delivery / birth (-5M) to 2 months after delivery / birth (+2M); the vertical dashed line indicates delivery / birth; the shaded area indicates the 95% confidence interval of the AUPRC; the horizontal dashed line indicates the AUPRC of a random classifier, which is equivalent to the prevalence of the outcome in the dataset.

[0054] Fig.11 An example of neonatal outcome prediction scores for an individual dichorionic patient born at Lucile Packard Children's Hospital is shown, who was born on April 13, 2020 at 24 weeks of gestational age, 2 days after PPROM, chorioamnionitis, and spontaneous PTL. Risk predictions are calculated based on maternal and neonatal codes, which are arranged in chronological order to specific diagnoses and include specific diagnoses, but not exceeding the date of individual diagnoses (when this occurs). The patient ultimately had EHR diagnoses of RDS, IVH (grade I bilateral), BPD, sepsis, PDA, anemia of prematurity, ROP, and hyperbilirubinemia. The highest individual prediction scores at birth were ROP, anemia of prematurity, RDS, and hyperbilirubinemia, all of which were diagnosed as what the patient ultimately suffered from. The lowest prediction scores at birth were NEC, pulmonary hypertension, CP, PVL, and death. Although the infant had a high risk of these diagnoses, the patient was still alive and never developed any of these results except transient pulmonary hypertension. This model allows us to predict individual outcomes based on population-level data. We thank the patient's parents for allowing us to create and publish this individual's risk prediction score.

[0055] Figures 12A-12B An exemplary time-shifted experimental data set is shown. AUC of the AI ​​model for predicting 24 neonatal outcomes at delivery / birth in the entire data set (tested using 5-fold cross validation, black line), in newborns born in 2019 only (n=5,852, light green line), and in newborns born in 2020 only (n=4,397, dark green line).

[0056] Figures 12C-12D An exemplary time-shifted experimental dataset is shown. AUPRC for the AI ​​model used to predict 24 neonatal outcomes at delivery / birth in the entire dataset (tested using 5-fold cross validation, black line), in newborns born only in 2019 (n=5,852, light green line), and in newborns born only in 2020 (n=4,397, dark green line).

[0057] Fig.13AShown are exemplary AUPRCs for a simplified model used to predict five selected outcomes in the training dataset (Stanford) and in the external validation data (UCSF).

[0058] Fig. 13B Shown are exemplary AUCs for a simplified model predicting five selected outcomes in the training dataset (Stanford) and in the external validation data (UCSF).

[0059] Figures 14A-14B Exemplary AUCs of the AI ​​model at delivery / birth (solid black line) and APGAR at 1 minute (dashed orange line) are shown; the dashed grey line represents the AUC of a random classifier. The APGAR score at 1 minute consists of 5 discrete subjective scores (0-2 for each score), which consist of: 1) appearance 2) heart rate 3) grimace 4) activity, and 5) respiratory effort. The AI ​​model had similar AUCs to the APGAR model for outcomes such as RDS, death, sepsis, pulmonary hemorrhage, and other CNS outcomes, including hypoxic ischemic encephalopathy (mild, moderate, and severe). It is worth noting that the APGAR score is a set of subjective physical examination results obtained shortly after birth and does not have sufficient predictive power for any known neonatal outcome.

[0060] Figures 14C-14D Exemplary AUPRCs for the AI ​​model at labor / birth (solid black line) and APGAR at 1 minute (dashed orange line) are shown.

[0061] Fig.15A Exemplary odds ratios between condition concept codes (rows) and neonatal outcomes (columns) are shown; the 50 condition concept codes with the highest average odds ratios between outcomes are shown; colors indicate the strength and direction of the association, with red indicating a positive association (i.e., the presence of the concept code in the mother's EHR history is associated with an increased risk of the neonatal outcome) and blue indicating a negative association (i.e., the presence of the concept code in the mother's EHR history is associated with a decreased risk of the neonatal outcome).

[0062] Fig. 15B Exemplary odds ratios between observed concept codes (rows) and neonatal outcomes (columns) are shown; the 50 observed concept codes with the highest average odds ratios between outcomes are shown; colors indicate the strength and direction of the association, with red indicating a positive association (i.e., the presence of the concept code in the mother's EHR history is associated with an increased risk of the neonatal outcome) and blue indicating a negative association (i.e., the presence of the concept code in the mother's EHR history is associated with a decreased risk of the neonatal outcome).

[0063] Fig. 15CExemplary odds ratios between medication concept codes (rows) and neonatal outcomes (columns) are shown; the 50 medication codes with the highest average odds ratios between outcomes are shown; colors indicate the strength and direction of the association, with red indicating a positive association (i.e., the presence of the concept code in the mother's EHR history is associated with an increased risk of the neonatal outcome) and blue indicating a negative association (i.e., the presence of the concept code in the mother's EHR history is associated with a reduced risk of the neonatal outcome). It is noteworthy that there is an inherent indication bias for medications commonly used in the setting of preterm birth, such as magnesium sulfate, indomethacin, and 17-α-hydroxyprogesterone. In addition, medications used to prevent hypertensive disorders of pregnancy (nifedipine, aspirin) or medications for various forms of diabetes (insulin, glibenclamide, glucagon) all show the expected positive association.

[0064] Fig.15D Exemplary odds ratios between procedure concept codes (rows) and neonatal outcomes (columns) are shown; the 50 procedure concept codes with the highest average odds ratios between outcomes are shown; colors indicate the strength and direction of the association, with red indicating a positive association (i.e., the presence of the concept code in the mother's EHR history is associated with an increased risk of the neonatal outcome) and blue indicating a negative association (i.e., the presence of the concept code in the mother's EHR history is associated with a decreased risk of the neonatal outcome).

[0065] Fig.15E Exemplary odds ratios between the last measurement recorded one week before delivery / birth (rows) and neonatal outcomes (columns) are shown; the 50 measurements with the highest average odds ratios between outcomes are shown; colors indicate the strength and direction of the association, with red indicating a positive association (i.e., values ​​above the median are associated with an increased risk of neonatal outcomes) and blue indicating a negative association (i.e., values ​​below the median are associated with a reduced risk of neonatal outcomes). Notably, the protective effect of higher albumin levels may reflect high-quality nutrition associated with many neonatal outcomes.

[0066] Fig.16 An exemplary correlation network is shown for the top 20 conditions, medications, observations, procedures, and measurements that are most strongly associated with all 24 neonatal outcomes; the 20 conditions, medications, observations, procedures, and measurements were ranked using a metric derived from the odds ratios described in the Methods, and the top 20 with the highest mean neonatal outcome were selected from each group. A tSNE plot of the resulting features was constructed; nodes represent conditions, observations, procedures, medications, and measurements; edges connect nodes with correlations exceeding 0.8 (red edges indicate negative correlations, green edges indicate positive correlations; correlations were assessed using the tetrachoric, biserial, or Pearson correlation coefficients (when appropriate); the size of the node is proportional to the mean odds ratios for the 24 outcomes, with the larger the node, the stronger the average association between the outcomes.

[0067] Figures 17A-17B Shown are exemplary AUCs for a multi-task AI model that simultaneously predicts 24 neonatal outcomes (dark blue), and for separate single-task models that each predict one individual outcome (light blue); the dashed grey line indicates the AUC of a random classifier.

[0068] Figures 17C-17D An exemplary AUPRC is shown for a multi-task AI model that simultaneously predicts 24 neonatal outcomes (dark blue), as well as for separate single-task models (light blue) that each predict one individual outcome.

[0069] Figures 18A-18F Exemplary data of potential pathological mechanisms of NEC that were exploited by a multi-task approach to improve NEC prediction are shown. Specifically, Fig.18A A correlation network is shown for the top 20 conditions, medications, observations, procedures, and measurements with the strongest associations for all 24 neonatal outcomes, including NEC; conditions, medications, observations, procedures, and measurements were ranked using a metric derived from the odds ratios described in Methods, and the top 20 with the highest neonatal outcome means were selected from each group. A tSNE plot of the resulting features was constructed; nodes represent conditions, observations, procedures, medications, and measurements; edges connect nodes with correlations exceeding 0.8; associations were assessed using the tetrachoric, biserial, or Pearson correlation coefficients (when appropriate); the size of the node is proportional to the odds ratio for NEC. The larger the node, the stronger the association with the outcome, regardless of direction, i.e., positive or negative association. Fig.18B Exemplary AUCs for NEC predicted by a single-task model (black dashed line), a dual-output multi-task model that simultaneously predicts NEC and polycythemia (green line), and a dual-output multi-task model that simultaneously predicts NEC and anemia of prematurity (blue line) are shown. Fig.18C An exemplary comparison of maternal hemoglobin levels in infants diagnosed with NEC compared to infants not diagnosed with NEC is shown. There were statistically significant differences in maternal hemoglobin levels at 5 months, 4 months, and 1 month prior to delivery. Fig.18D An exemplary comparison of neonatal hemoglobin levels at birth, 1 month, 2 months, 3 months, and 4 months of age is shown for neonates diagnosed with NEC compared to neonates not diagnosed with NEC. Infants with NEC have lower hemoglobin concentrations at birth compared to infants without NEC. Fig.18E Exemplary data showing maternal hemoglobin levels at delivery relative to NEC prediction scores for newborns diagnosed with NEC and newborns never diagnosed with NEC are shown. Fig.18FExemplary data showing neonatal hemoglobin levels at birth relative to NEC prediction scores for neonates diagnosed with NEC and neonates never diagnosed with NEC are shown.

[0070] Figures 19A-19B The exemplary data presented show that the AI ​​model is able to differentiate based on IVH grade and provide insights into the underlying pathological mechanisms of IVH. Fig.19A An exemplary correlation network of the top 20 conditions, medications, observations, procedures, and measurements with the strongest associations with all 24 neonatal outcomes is provided; conditions, medications, observations, procedures, and measurements were ranked using a metric derived from the odds ratios described in the methods, and the top 20 with the highest neonatal outcome means were selected in each group. A tSNE plot of outcome features was constructed; nodes represent conditions, observations, procedures, medications, and measurements; edges connect nodes with correlations exceeding 0.8; associations were assessed using the quartic method, biserial method, or Pearson correlation coefficient (when appropriate); the size of the node was proportional to the odds ratio of IVH, with the larger the node, the stronger the association with the outcome, regardless of direction, i.e., positive or negative association. Fig.19B An exemplary IVH prediction score of the AI ​​model at delivery for neonates stratified by IVH grade is provided. The predictive ability of the model gradually increases with the increase of the clinical severity of IVH, i.e., grade IV> grade III> grade II> grade I. Patients with higher IVH prediction scores (i.e., 0.2) have a greater tendency to suffer from IVH (2 times the probability) compared with patients with lower IVH prediction scores (i.e., 0.1), and can therefore be directly compared. The IVH prediction score should not be interpreted as an individual probability of IVH development. The IVH grade reported in the ultrasound report of neonates with unclear grade categories differs from their ICD codes, making it difficult to classify them according to the Papile grading system.

[0071] Detailed Description

[0072] Turning now to the accompanying drawings, systems and methods for assessing neonatal health risks and uses thereof are provided. Many embodiments provide methods including machine learning models that improve risk prediction by integrating continuous and abundant newborn and maternal information contained in electronic health records (EHRs) collected before and after birth. Other embodiments describe methods for predicting the possibility of one or more disorders that pre-term infants are susceptible to. Certain embodiments describe suggestions for treating, repairing, improving or alleviating one or more disorders predicted. Other embodiments allow the possibility of a pre-term infant suffering from any disorder to be risk graded for the infant.

[0073] Over the past decade, hospital systems have increasingly implemented EHR systems to capture and store clinical data in real time. Longitudinal data collection and serialization of clinical information for patients with acute and chronic health conditions, inpatient hospitalizations, and outpatient care has revolutionized clinical medicine. EHR has allowed large amounts of data to be formally communicated between providers, simplified billing, and to some extent simplified research workflows. However, EHR clinical data are very complex and difficult to query. They are also heterogeneous and lack standardization. Recent computational advances have helped alleviate such limitations through data linkage and the use of large amounts of demographic, diagnostic, drug, and clinical data. In addition, these data are typically available at a fraction of the time and cost of prospective cohort studies or clinical trials, and include thousands or tens of thousands of additional patients.

[0074] From an analytical point of view, EHR data presents challenges that cannot be addressed by traditional computing solutions. These challenges include combining longitudinal information with time dependencies and modeling thousands of potential predictors arising from the complexity and granularity of the data. Recent developments in artificial intelligence (AI) methods allow many of these challenges to be addressed, thereby fully leveraging the breadth of EHR data. AI models (such as artificial neural networks) can process large amounts of structured and unstructured data with a large number of input variables. In particular, recurrent neural networks (such as long short-term memory (LSTM) models) are designed to exploit time dependencies and do not require a priori specification of which potential predictor variables should be considered. In addition, multi-task learning allows us to predict multiple outcomes simultaneously. By exploiting the potential commonalities between outcomes, the knowledge learned when predicting one outcome can be shared when predicting other outcomes, thereby improving predictive power compared to models developed to predict each outcome independently.

[0075] Many of the embodiments herein utilize multi-task learning to simultaneously predict the risk of the most important adverse neonatal outcomes using longitudinal EHR data spanning a period starting shortly after conception and ending several months after birth.

[0076] Machine Learning Models

[0077] Artificial neural networks (NNs) are a family of computing systems based on a collection of connected units or nodes that receive signals (input data or signals returned by previous units), process them and then transmit them to subsequent units. Units are aggregated into layers, and each layer can perform different transformations on its input. Signals are transmitted from the first layer (input layer) to the last layer (output layer containing the predicted object). Many implementations use NNs because they are able to process large amounts of data (to learn and model complex nonlinear relationships that can be generalized to unseen data) and because NNs do not require strict assumptions about the distribution of input variables and their associations. In the presence of multiple outcomes, multi-task learning allows multiple outcomes to be predicted simultaneously by exploiting representations shared between related outcomes. In addition, recurrent NNs (RNNs) use their internal states (e.g., memories) to obtain information from previous inputs to affect current inputs and outputs. Unlike traditional NNs where inputs and outputs are independent of each other, the outputs of recurrent NNs depend on previous elements in the sequence. Long short-term memory (LSTM) is a special type of RNN designed to solve long-term dependency problems, for example, when the previous state that affects the current prediction did not occur in the recent past, but in the more distant past.

[0078] In many embodiments, LSTM RNNs are characterized by a "cell" in a hidden layer of the NN that has three gates (e.g., an input gate, an output gate, and a forget gate). These gates control the flow of information, allowing the LSTM layer to remember information for longer periods of time. In a conventional (unidirectional) LSTM NN, input flows in one direction, usually forward, i.e., from the past to the future. In a bidirectional LSTM NN, input flows in both directions to retain future and past information.

[0079] Many embodiments use one or more multi-input multi-task deep neural networks to predict neonatal outcomes, determine nutritional needs and / or any other use described herein. Some embodiments utilize one or more multi-input multi-task deep neural networks to determine and / or discover subgroups within a population. In many embodiments, the one or more multi-input multi-task deep neural networks include an autoencoder. An autoencoder is a self-supervised learning model that can learn a compressed, low-dimensional representation of input data. An autoencoder typically consists of an encoder and a decoder: the encoder reads an input sequence; the hidden state or output of the encoder represents an internal learning representation of the entire input sequence, which is then provided as input to a decoder model that interprets the internal learning representation and reconstructs the input sequence. In addition, subgroup discovery is a data mining technique that is used to determine a description of a subset of data that exhibits an interesting distribution relative to a pre-specified target. For example: given a data set X and a search space S identified by a set of descriptors (i.e., variables), subgroup discovery finds and sorts subgroups with high or low target concepts in X.

[0080] Go to Figure 1 In one exemplary embodiment, the input to the autoencoder consists of a series of concept codes. These codes can be input into one or more appropriate layers of the encoder. In various embodiments, the layers are LSTM layers, convolutional layers, and / or any appropriate layer time. In the illustrated embodiment, the layers include a 256-unit bidirectional LSTM layer followed by a 128-unit bidirectional LSTM layer. The output of the second layer is an encoded 128-dimensional latent space of the input data, which can be used to determine subgroups using subgroup discovery. A bridge layer can be used to connect the encoder and decoder. As further shown in this embodiment, the bridge layer is a repeated vector layer, but any appropriate layer can be used in an embodiment. The decoder can use any number of layers and / or layer types to reconstruct the input sequence. In some embodiments, such as Figure 1 As shown, the decoder can consist of two bidirectional LSTM layers that mirror the two layers of the encoder—for example, a 128-unit bidirectional LSTM layer followed by a 256-unit bidirectional LSTM layer.

[0081] Training machine learning models

[0082] Many embodiments train models based on inputs from the EHR, including, but not limited to, conditions, observations, medications, procedures, and measurements recorded under the mother's patient identification number. Exemplary measurements may include test results indicating one or more of an individual's genetics, blood test results, enzymes, metabolites, fatty acids, and / or any other measurable components. In various embodiments, the records include: 1) conditions: the presence of a disease or medical condition, 2) observations: observed clinical sequelae obtained as part of a medical history, 3) medications: the use of any prescription and over-the-counter medications, vaccines, and macromolecular biotherapies, 4) procedures: records of activities or processes ordered or performed by a healthcare provider for the purpose of diagnosis or treatment, and 5) measurements: structured values ​​obtained by systematic and standardized examinations or tests (such as laboratory tests, vital signs, quantitative findings in pathology reports, etc.) of patients or patient samples. Organize conditions, observations, medications, and procedures by patient and time, and exclude records corresponding to conditions used to determine neonatal outcomes to avoid potential leakage of information about outcomes into the input data. The resulting sequence of chronologically arranged records throughout the predicted time point (e.g., delivery, one week before delivery, two weeks before delivery, etc.) forms one of the personalized inputs to the model's newborn. In additional embodiments, the most common measurements (e.g., available in ≥10% of mothers) are extracted to form additional personalized inputs to the newborn along with maternal demographics (age and race at delivery) and (when specified) the sex, gestational age at delivery, and birth weight of the newborn. Table 1 provides a list of measurements that can be used in various embodiments.

[0083] In addition, many embodiments also include a complete medical history for each newborn, including all conditions, observations, medications, and procedures - these events are extracted and organized in time. For each newborn, the resulting record sequence is combined with the corresponding mother's record sequence (until delivery / birth) to form input data for the model at the prediction point after delivery. Certain embodiments may further use inputs derived from medications, nutritional supplements, diet, and / or any other relevant aspects that may play a role in health and / or development.

[0084] In order to predict a series of neonatal outcomes and to fully interrogate common neonatal pathologies by a multi-task approach, a list of neonatal outcomes can be obtained as the presence or absence of any record associated with each of these outcomes at any time in the available neonatal medical history when extracting data. When quantity permits, some obstacles affecting the same organ system can be grouped to form a single, non-universal outcome (e.g., other CNS obstacles). Table 2 provides a code list for determining the presence / absence of each outcome in various embodiments.

[0085] Many embodiments extract data from clinical records and calculate risk scores. To accomplish these tasks, in many embodiments, gestational age and birth weight at delivery are extracted from clinical records in the EHR of the newborn. Regular expressions for "gestational age" and "birth weight" can be used to systematically search for free text in clinical records. Text related to these mentioned information (e.g., after or before) can be extracted and converted into days of gestational age and grams of birth weight. When there are multiple clinical records for the same newborn and the values ​​are inconsistent, the most frequently occurring value can be retained, or if two or more values ​​occur with the same frequency, the average of all different values ​​can be retained.

[0086] Several neonatal risk scores have been developed to quantify the risk of neonatal death and / or severe outcomes. Most of these scoring systems are derived from pre-term newborns and are directed to a single outcome, such as death. Many embodiments described herein predict a wider range of neonatal outcomes for all newborns, including death, regardless of gestational age at delivery.

[0087] In the sequence processing deep learning algorithm used by various embodiments, each element of the sequence (i.e., code) can be represented as a real-valued vector that encodes the meaning of the element, so that elements that are closer in the vector space are expected to have similar meanings. These vectors that encode the meaning of each potential element that can be found in the sequence are called embeddings. Given the large number of unique codes, this scheme can be superior to one-hot encoding, in which each code k will be represented by a K-dimensional vector of 0, except for the kth element which will be 1.

[0088] In various embodiments, an input such as a code is embedded to reduce the code to a lower dimensional space. For example, the code can be reduced to a 128-dimensional space. As a non-limiting example, a sequence of n codes is converted to a 128xn matrix and input into a bidirectional long short-term memory (LSTM) recurrent NN with 128 units. Recurrent NN is a class of NNs that use sequence data or time series data. Certain embodiments train global vector (GloVe) embeddings for all codes present in the medical history of a mother or a newborn to reduce the code to a 128-dimensional space. The GloVe model can be trained on the non-zero entries of a global code-code co-occurrence matrix, which lists the frequency of co-occurrence of codes with each other in the EHR medical history of a patient. The main intuition behind the GloVe model is that the ratio of code-code co-occurrence probability has the potential to encode a certain form of meaning. The obtained code embedding can be projected to two dimensions and divided by group (i.e., condition, observation, drug, and procedure) using tSNE to achieve visualization purposes. Similarly, a two-dimensional tSNE graph of the measurement results can be obtained. Figure 2An exemplary tSNE plot is shown, in which the 20,172 codes present in the exemplary data are visualized. Figure 2 The size of the nodes is proportional to the metric for assessing feature importance described in Methods and averaged over all results; the relevance of the nodes connected by the edge is in the top 1% of all relevances.

[0089] The resulting encoded 128-dimensional space can be divided into a training data set and a test data set in a ratio of 60%-40%. Subgroup discovery can be applied to a training data set containing an encoded 128-dimensional latent space, and the classification metric is evaluated in a test data set. Each dimension in the latent space can be discretized into groups (e.g., 2 groups, 3 groups, 4 groups, 5 groups, etc.) using appropriate quantiles to form a search space. The target concept can be AUPRC, so that the determined subgroup is the subgroup with the highest AUPRC. AUPRC can be obtained based on the actual presence / absence of a given neonatal outcome and the predicted score output by the AI ​​model at delivery / birth. It should be noted that the foregoing embodiments are exemplary only, and some models and / or training methods may include one or more layers; each layer may have more or fewer units; and / or the layers, data extraction and / or training methods may be optimized for computer performance or specific uses.

[0090] Neonatal Outcome Prediction

[0091] Neonates born after 37 weeks of gestation have traditionally been considered a relatively low-risk group for adverse neonatal outcomes, with lower rates of neonatal morbidity and mortality compared with preterm neonates. Nevertheless, term neonates, especially those with cardiac, neurologic, or genetic disorders, are at increased risk for prolonged NICU hospitalization secondary to overlapping pathology with preterm infants.

[0092] Certain embodiments can be used to predict neonatal outcomes. Such outcomes are at different time points in pregnancy and / or postpartum, such as any time from about 5 months antenatal to about 2 months postpartum. In such embodiments, a machine learning model (such as described herein) can be trained with:

[0093] (i) a code sequence from the medical history of the mother and the newborn up to the predicted time point,

[0094] (ii) maternal / neonatal sociodemographic information, maternal measurements closest to the prediction time, and (when specified) gestational age and birth weight.

[0095] In some specific embodiments, input (i) includes all maternal EHR records up to the prediction or delivery time point (whichever occurs first), plus the newborn EHR record up to the prediction time point (only for models predicting delivery / post-birth outcomes). The measurements in input (ii) are updated to select the results closest to the prediction or delivery time point (all within a 30-day time window) (whichever occurs first), while gestational age and birth weight are added only to models obtained at or after delivery. For example, for a model trained using data available at delivery / birth, input (i) includes all maternal EHR records up to delivery (i.e., date of birth of the newborn) and does not include records from the medical history of the newborn, and input (ii) includes the measurements closest to delivery (within a 30-day time window), gestational age at delivery, birth weight, plus maternal / newborn sociodemographic information. On the other hand, the model obtained one week after delivery is based on a combination of maternal medical history up to delivery and the HER of the newborn up to one week after birth [input (i)]; and maternal / newborn sociodemographic information, gestational age at delivery, birth weight, and the measurements closest to delivery form input (ii).

[0096] In various embodiments, input (i) is fed into a bidirectional long short-term memory (LSTM) recurrent neural network with 128 units after code embedding, while input (ii) is processed by a dense single-layer neural network with 4 units. The outputs of these two networks are then concatenated and fed into a dense single-layer neural network with 64 units, followed by a set of dense layers, one for each result, consisting of two dense layers and a single unit output.

[0097] Many embodiments were able to predict outcomes for both preterm and term births. However, all 24 morbidity and mortality outcomes were more prevalent in preterm neonates (n=3,639, 11.5%) than in term neonates (n=27,998, 88.5%) (see Tables 1 and 3). For example, in one embodiment, the prevalence of IVH in preterm and term neonates was 5.1% and 0.2%, respectively, while the prevalence of NEC was 1.8% and 0.04%, respectively, and Figure 3The AUC for each outcome in preterm and term neonates is shown because the AUC is independent of the prevalence of the outcome. For most outcomes, the AUC in term neonates was similar to that seen in preterm neonates; however, a decrease in the AUC was observed for RDS (0.661 in term neonates vs. 0.833 in preterm neonates), ROP (0.665 vs. 0.918), sepsis (0.707 vs. 0.814), hyperbilirubinemia (0.606 vs. 0.728), candidiasis (0.626 vs. 0.711), cardiac instability (0.683 vs. 0.773), and neonatal gastroesophageal reflux (0.659 vs. 0.734).

[0098] By such predictions, many embodiments can provide solutions to treat or ameliorate certain conditions, such as providing light therapy (including intensity and / or timing settings), breathing strategies (e.g., gas type, pressure and / or other ventilator settings), and / or specific medical treatments, supplements, medications, or strategies that can be used prenatally (e.g., during the mother's pregnancy) or in the neonatal period to prevent, ameliorate and / or alleviate neonatal morbidity or conditions.

[0099] Nutritional supplementation for newborns

[0100] Some embodiments can be used to generate the formula and / or preparation of the intravenous (IV) nutritional supplement bag of newborn.A major problem is the ability of producing the nutritional supplement bag of premature infant.Many such babies cannot absorb nutrients because the digestive system is not yet fully formed.At present, the mode of IV supplementation is taken based on the assessment of laboratory results.This process involves the temporary preparation of customized bags for each individual newborn, and is susceptible to supply chain problems, pollution, human error and / or preparation and / or mixing for the impact of any other problems involved in the components of IV supplementation.

[0101] In order to solve this problem, many embodiments relate to a nutritional bag for IV supplementation. Such a nutritional bag can be developed by formula and / or preparation designed using an automatic encoder (as described herein). Such a model can be trained using a neonatal EHR (as described herein), including the nutritional supplement preparation included in such an EHR.

[0102] Using the model trained as described, many embodiments provide formulas and / or formulations of standardized nutritional bags. Such embodiments can output any number of formulas and / or formulations, such as 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, 25, etc. In various embodiments, the efficacy obtained from additional formulas and / or formulations may be reduced (e.g., 15 unique formulas / formulations may be 97% effective for supplementation, while 20 unique formulas / formulations may only provide 97.5% efficacy, but also require additional storage and / or production lines).

[0103] Nutrition bags as described herein can be formulated into sterile IV bags. Such IV bags can be prepared so that they can be reconstituted with diluents (e.g., sterile water, whether locally derived or independently derived) or prepared in liquid form, fully formulated for use. In addition, some formulations can be used to prevent and / or improve specific conditions and / or disorders in newborns, such as neurocognitive development, respiratory health, gastrointestinal health, eye health and / or any other developmental conditions or problems.

[0104] While the above describes methods for determining IV-based nutritional supplementation, further embodiments can expand upon this foundation to provide oral health supplementation, such as through infant formula (e.g., ) or baby food. Some embodiments provide suggestions and / or formulas for newborn nutrition, such as which specific foods to provide (e.g., peas, carrots, beets, bananas, apples, etc.) and how much to provide (e.g., 1 can, 1 / 2 can, etc.) so that the child obtains proper nutrition.

[0105] Using the Model

[0106] Embodiment can use the EHR obtained from multiple institutions (for example, clinics, hospitals, children's hospitals, etc.) to predict neonatal outcomes.For example, many embodiments can obtain the EHR of individuals (women, women, girls, people with uterus and / or any other fertile individuals) who give birth to children.Such EHR can be obtained for individuals who give birth to children at any time before or during pregnancy, such as for the purpose of planning children, consulting or other planning on behalf of individuals who give birth to children.In some embodiments, such EHR is used by medical practitioners (such as obstetricians, gynecologists, neonatologists and / or any other medical professionals).In such embodiments, the results can be used to prepare medical treatments (for example, surgery, antibiotics, etc.), nutrition plans and / or any other behaviors that benefit children (preterm or full term) who may be susceptible to or prone to poor health conditions.

[0107] The EHR may be obtained from a public source or a proprietary data source (such as a database). These data sources may be local (e.g., a hard drive) or remote (e.g., a server accessed via network communications). Data sources may include hospital records, health system records, records from obstetricians, records from gynecologists, and / or electronic records from any other medical or health source. Some EHRs may be compiled from multiple sources, such as when an individual receives care from multiple locations and / or medical professionals.

[0108] Exemplary embodiments

[0109] Although the following examples provide details regarding certain embodiments of the invention, it should be understood that these are merely exemplary in nature and are not intended to limit the scope of the invention.

[0110] Example 1: Longitudinal risk prediction of maternal and child health using artificial intelligence and electronic health records

[0111] method:

[0112] Data Source

[0113] This was a cohort study based on routinely collected EHRs at Stanford Hospital and Clinics and Lucile Packard Children's Hospital (Stanford, CA, USA). Interconnection of EHRs from both hospitals allowed for a unique combination of a range of maternal and neonatal data. All EHRs from inpatient and outpatient data were mapped to the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) version 5.3.1.16,17. Data included patient demographics, provider orders, diagnoses, procedures, medications, laboratory tests, and clinical information collected during all inpatient and outpatient visits. This study was approved by the Stanford University Institutional Review Board (#39225).

[0114] Delivery queue

[0115] A cohort of 193,546 women was first identified, with women aged between 14 and 45 years and with at least one pregnancy-related record between 2014 and September 2020. Pregnancy-related records consisted of any record containing one of the codes identified by Matcho et al. (cited earlier) to identify pregnancy events, broadly including live birth, stillbirth, miscarriage (spontaneous and induced), delivery, pregnancy test, and ectopic pregnancy. Of the 193,546 women identified, 27,521 were associated with 32,356 newborns born in one of the two hospitals between April 2014 and October 2020. For the remaining pregnancies, no delivery outcome was found because it was only a record of pregnancy testing without an actual pregnancy, delivery outside of the two hospitals, or the pregnancy was terminated. Of the 32,356 newborns identified, 2 were further excluded because they had less than 30 days of observation available after birth or because there was no record of the corresponding mother before delivery. The final dataset consisted of 32,354 births from 27,519 mothers, including 3639 preterm infants and 28,715 full-term infants. Of the 32,354 pregnancies, there were 644 twin pregnancies and 60 triplet pregnancies. Of the 27,519 mothers, 4,449 gave birth to multiple infants at different time points; 4,087 gave birth to two newborns, 340 gave birth to three newborns, 20 gave birth to four newborns, and 2 gave birth to a total of five newborns.

[0116] Maternal medical history and feature extraction

[0117] For each newborn, the entire maternal medical history available in the EHR until delivery is extracted. This consists of all conditions, observations, medications, procedures, and measurements recorded under the mother's patient identification number. Different types of records are: 1) condition: the presence of a disease or medical condition, 2) observations: observed clinical sequelae obtained as part of the medical history, 3) medications: the use of any prescription and over-the-counter medications, vaccines, and macromolecular biological therapies, 4) procedures: records of activities or processes ordered or performed by a healthcare provider for diagnostic or therapeutic purposes, and 5) measurements: structured values ​​obtained by systematic and standardized examinations or tests (such as laboratory tests, vital signs, quantitative findings in pathology reports, etc.) performed on patients or patient samples. Organize conditions, observations, medications, and procedures by patient and time, and exclude records corresponding to conditions used to determine neonatal outcomes to avoid potential leakage of information about outcomes into the input data. The entire chronological sequence of records obtained, until the predicted time point (e.g., delivery, one week before delivery, two weeks before delivery, etc.), forms one of the neonatal personalized inputs to the model. In addition, the most common measurements available in ≥10% of mothers were extracted to form additional neonatal personalized inputs along with maternal demographics (age at delivery and race) and (when specified) neonatal sex, gestational age at delivery, and birth weight. Table 1 reports the complete list of measurements used. For each measurement, the result closest to the predicted time point (e.g., delivery) was extracted, which was within 15 days before or after the predicted point ( Figures 4A-4D ).

[0118] Neonatal medical history and outcomes

[0119] Similarly, the entire medical history of all conditions, observations, medications, and procedures of the newborn is extracted and organized in time. For each newborn, the resulting record sequence is combined with the corresponding mother's record sequence (until delivery / birth) to form the model input data at the prediction point after delivery.

[0120] In addition, because of interest in understanding whether AI can accurately predict a wide range of neonatal outcomes, and in order to comprehensively interrogate common neonatal pathologies through a multitasking approach, a list of 24 neonatal outcomes was obtained as the presence or absence of any record related to each of these outcomes at any time in the neonatal medical history available at the time of data extraction (i.e., January 2021, allowing for at least three months of follow-up). For the outcome of death, we only considered deaths within two months of birth. These outcomes were selected from those included in the “neonatal disorder” code (SNOMED code ‘22925008’), with a sufficient number of cases (i.e., n ≥ 100) to allow meaningful analysis, and transient disorders such as tachypnea, vomiting, electrolyte disturbances, etc. were excluded. When numbers allowed, certain disorders affecting the same organ system were grouped to form a single, non-generic outcome (e.g., other CNS disorders). Table 2 reports the list of codes used to determine the presence / absence of each outcome.

[0121] Data extraction from clinical records and calculation of neonatal risk scores

[0122] Gestational age at delivery and birth weight were extracted from clinical records in the neonates' EHRs. Free text in clinical records was systematically searched using regular expressions for "gestational age" and "birth weight". Text following either of these mentions was extracted and converted to days for gestational age and grams for birth weight. When multiple clinical records were available for the same neonate and the values ​​were inconsistent, the most frequently occurring value was retained, or if two or more values ​​occurred with the same frequency, the average of all different values ​​was retained.

[0123] Several neonatal risk scores have been developed to quantify the risk of neonatal death and / or severe outcomes. Most of these scoring systems are derived from pre-term newborns and are directed to a single outcome, such as death. The method is more comprehensively intended to predict a wider range of neonatal outcomes for all newborns, including death, regardless of gestational age at delivery. The classification performance of the proposed model is compared with that of two neonatal risk calculators: the APGAR score and the National Institute of Child Health and Human Development (NICHD)-Neonatal Research Network (NRN) death risk score. The APGAR score is routinely used in pediatrics and obstetrics to quickly evaluate the physical condition of all newborns after birth. Regular expressions such as "APGAR score:" or "APGAR total" are systematically searched in clinical records, and the text following any one of these regular expressions is extracted, and the mention information of "1min:", "1 minute:", "one min:" or "one minute:" is further searched. The APGAR score for one minute after delivery is then obtained by extracting the number following any one of these regular expressions. Given that the APGAR score is a subjective measure of physical examination findings obtained shortly after birth and that our proposed model is more comprehensive, comparisons between models must recognize their significant differences and objectives.

[0124] Information for calculating the NICHD-NRN risk of death score was also obtained. In addition to gestational age at delivery, birth weight, and sex of the neonate, multiple births and antenatal steroid use were derived from extracted conditions, observations, medications, and procedures recorded in the maternal EHR history. Specifically, codes contained within the ‘multiple birth’ code (SNOMED code ‘45384004’) were used to identify multiple births, and codes associated with ‘betamethasone’ and ‘dexamethasone’ (RxNorm codes ‘1514’ and ‘3264’) within the 2 weeks before delivery were used to infer antenatal steroid use. The NICHD-NRN risk of death score is frequently used in clinical practice in calculators commonly used to assess the risk of survival and neurodevelopmental impairment in preterm neonates. The score provides a risk estimate for neonates born between 22 and 25 completed weeks of gestation with a birth weight between 401 g and 1,000 g. The coefficient associated with the highest gestational age category (i.e., 25 weeks) was applied to preterm neonates born after 25 completed weeks (and before 37 weeks) to expand the calculation of the score to all preterm neonates in the study population. Similarly, when the gestational age was less than 22 weeks, the coefficient for 22 weeks was applied. Again, given that our model includes a wide range of gestational ages and predictions for 24 queried neonatal outcomes, comparisons with the NICHD model must be interpreted with caution.

[0125] Code Embedding

[0126] The medical histories of the mothers and newborns contained a total of 20,172 unique codes, of which 44.6% were condition codes and 43.0% were procedure codes. Of the 11,182,582 records, 29.0% were condition records, 29.0% were procedure records, 25.6% were observation records, and 16.4% were medication codes ( Figures 5A-5D ). In sequence processing deep learning algorithms, each element of the sequence (i.e., code) can be represented as a real-valued vector that encodes the meaning of the element such that elements that are close in vector space are expected to have similar meanings. These vectors that encode the meaning of each potential element that can be found in the sequence are called embeddings. Given the large number of unique codes (K=20,172), this approach outperforms one-hot encoding, where each code k would be represented by a K-dimensional vector of 0s, except for the kth element which would be 1.

[0127] A global vector (GloVe) embedding was trained for all 20,172 codes present in the medical history of the mother or newborn to reduce the 20,172 codes to a 128-dimensional space. 22 The GloVe model was trained on the non-zero entries of a global code-code co-occurrence matrix, which lists the frequency with which codes co-occur with each other in the patient's EHR medical history. The main intuition behind the GloVe model is that the ratio of code-code co-occurrence probabilities has the potential to encode some form of meaning. The embeddings of the 20,172 codes obtained were projected into two dimensions and partitioned by group (i.e., condition, observation, medication, and procedure) using tSNE for visualization purposes. Similarly, a two-dimensional tSNE plot of the measurements was obtained ( Figure 2 ).

[0128] Multi-input multi-task deep learning model

[0129] Multiple multi-input multi-task deep neural networks were trained to simultaneously predict 24 neonatal outcomes at different time points from 5 months before delivery to 2 months after delivery. For each of these models, the input to the model is (i) a code sequence from the medical history of the mother and the newborn until the prediction time point, (ii) maternal / newborn socio-demographic information, maternal measurements closest to the prediction time, and (when specified) gestational age and birth weight. Specifically, input (i) includes all maternal EHR records up to the prediction or delivery time point (whichever occurs first), plus the newborn EHR records up to the prediction time point (only for models predicting delivery / post-birth outcomes). The measurements in input (ii) are updated to select the results closest to the prediction or delivery time point (all within a 30-day time window) (whichever occurs first), while gestational age and birth weight are added only to models obtained at or after delivery. For example, for a model trained using data available at delivery / birth, input (i) includes all maternal EHR records up to delivery (i.e., date of birth of the newborn) and does not include records from the newborn's medical history, and input (ii) includes measurements closest to delivery (within a 30-day window), gestational age at delivery, birth weight plus maternal / newborn sociodemographic information. On the other hand, a model obtained one week after delivery is based on a combination of maternal medical history up to delivery and newborn HER up to one week after birth [input (i)]; and maternal / newborn sociodemographic information, gestational age at delivery, birth weight, and measurements closest to delivery form input (ii).

[0130] Input (i) is fed into a bidirectional long short-term memory (LSTM) recurrent neural network with 128 units after the code is embedded, while input (ii) is processed by a dense single-layer neural network with 4 units. The outputs of these two networks are then concatenated and fed into a dense single-layer neural network with 64 units, followed by a set of dense layers, one for each result, which consists of two dense layers and a single unit output (see the supplementary material for further details).

[0131] To avoid overfitting the data, a five-fold cross-validation was performed. First, the newborns were randomly divided into five parts. Subsequently, the model was trained five times: each time the model was trained as follows: inputs from newborns from four of the five parts were used as training / validation data, while the remaining parts were used as test data so that the predictions for each newborn came from a model that was not trained using data related to the newborn. The classification performance of the model was evaluated using the cross-validation area under the precision-recall curve (AUPRC) and under the receiver operating characteristic curve (AUC). The reference value of the AUC, that is, the AUC achieved by a random classifier, is always 0.5, regardless of the prevalence of the outcome; on the other hand, the reference value of the AUPRC corresponds to the prevalence of the outcome and therefore varies with the outcome. For visualization purposes, we also report the fold increase / decrease in the AUPRC obtained by the AI ​​model compared to the AUPRC of the random classifier; Table 4 reports the prevalence of each outcome.

[0132] When measurements, gestational age, and birth weight were missing, they were imputed using the corresponding means from the available data. All analyses were performed using R v3.6.3 and multi-input multi-task deep neural networks were implemented using Keras via the R package ‘keras’. AI models were trained using a batch size of 512, Adam optimization, binary cross entropy loss, and early stopping (training was stopped after 10 consecutive epochs with no improvement in validation loss) or stopping after 100 epochs.

[0133] Robustness to dataset shift and external validation

[0134] EHR data can be affected by changes in patient populations, clinical and administrative workflows, and updates to coding systems. These changes can result in temporal dataset drift, which can impact the deployment of AI models and lead to degradation of predictive performance over time. To test for any potential dataset drift, an experiment was conducted in which an AI model at delivery / birth was trained using newborns born between 2014 and the end of 2018 and tested on newborns born in 2019 and 2020. The AUC and AUPRC from the original model were then compared to the AUC and AUPRC obtained for newborns born in 2019 and 2020.

[0135] In addition, a simplified model was trained for five selected outcomes (RDS, NEC, IVH, PDA, and anemia of prematurity) and performance was validated in an external EHR from UCSF. Maternal-infant linked EHRs for 12,258 newborns, including conditions, medications, procedures, and measurements, were available in the UCSF EHR database. The model first identified 1,808 different OMOP CDM concept codes that were present in the maternal medical history up to delivery for at least 0.2% of pregnancies identified in the Stanford delivery cohort. These were mapped to the relevant coding systems used by the UCSF EHR: ICD 9 and 10 for conditions, RxNorm for medications, CPT4 for procedures, and LOINC for measurements. The mapping was completed as shown in the OMOP CDM concept relationship table. For each code, the proportion of maternal medical history with these codes present up to delivery was compared in Stanford and UCSF pregnancies ( Figure 6 To avoid bias due to code mapping from different coding systems, subsequent analyses were restricted to the 850 concept codes that were similarly proportioned in both datasets (i.e., when the proportion of concept codes present in maternal medical histories at UCSF was at least half of the proportion at Stanford and less than twice the proportion at Stanford).

[0136] Binary variables were generated in both the Stanford and UCSF data indicating the presence / absence of these 850 selected concept codes in the mother's medical history up to delivery. For each selected outcome, the concept codes were ranked based on their association with the outcome in the Stanford data (assessed using OR), and a logistic model with the top 10 codes plus gestational age was trained using the Stanford data. These models were then tested in the USCF data, and AUC and AUPRC were calculated. These simplified models were used to validate the generalizability and transferability of more complex multi-input, multi-task models to external healthcare settings. External validation of the full model was not possible due to differences in the coding systems used.

[0137] Subgroup Discovery

[0138] Subgroup discovery was used to identify subgroups of newborns where the AI ​​model at delivery / birth showed the highest predictive ability in terms of AUPRC. Subgroup discovery can be used for heterogeneous study populations, such as the one used in this dataset. First, an LSTM autoencoder was used to obtain a 128-dimensional latent space ( Fig. 7A). An autoencoder is a self-supervised learning model that can learn a compressed, lower-dimensional representation of the input data. An autoencoder typically consists of an encoder and a decoder: the encoder model reads the input sequence; the hidden state or output of the model represents the internal learned representation of the entire input sequence, which is then provided as input to the decoder model, which interprets it and reconstructs the input sequence. The input to the autoencoder consists of a sequence of concept codes, i.e., input (i), after the code is embedded. These are input to a bidirectional LSTM layer of 256 units, followed by a bidirectional LSTM layer of 128 units. The output of this layer is an encoded 128-dimensional latent space of the input data, which is used to determine subgroups using subgroup discovery. The repeated vector layer serves as a bridge between the encoder and decoder modules, and the decoder consists of two bidirectional LSTM layers that mirror the two layers of the encoder. The bridge layer can represent the bottleneck layer in the autoencoder. In some embodiments, certain details can be extracted from the bottleneck layer, such as for subgroup discovery.

[0139] Subgroup discovery is a data mining technique that determines descriptions of subsets of data that show interesting distributions relative to a pre-specified target. Given a dataset X and a search space S identified by a set of descriptors (i.e., variables), subgroup discovery finds and ranks subgroups of X that are high or low in the target concept. The resulting encoded 128-dimensional space is split into a training dataset and a test dataset in a 60%-40% ratio. Subgroup discovery is applied to the training dataset containing the encoded 128-dimensional latent space ( Figure 7B ), and the classification metric is evaluated in the test dataset. Each dimension in the latent space is discretized into 2, 3, 4, and 5 groups using appropriate quantiles to form the search space. The target concept is AUPRC, so that the determined subgroup is the one with the highest AUPRC. The AUPRC is obtained based on the actual presence / absence of a given neonatal outcome and the predicted score output by the AI ​​model at delivery / birth. A beam search with a depth of 2 is used, that is, the subgroups are determined by a combination of no more than two descriptors (e.g., dimensions of the latent space). The AUPRC within each subgroup in the training data is calculated to rank the subgroups26; the ranked subgroups are then gradually merged until they cover at least 30% of the test dataset, and the classification metric is calculated in the resulting subclusters.

[0140] Correlation between input features and outcomes

[0141] To investigate which information drives prediction of neonatal outcomes, the importance of each EHR code was assessed and grouped into groups of conditions, medications, observations, and procedures. Maternal medical history and measurements up to one week before delivery were considered to identify features that lead to the development of outcomes beyond those immediately before and during delivery.

[0142] First, a code set deletion experiment was performed. For each set of conditions, medications, observations, and procedures, all codes belonging to that set were removed from the input (i) of the AI ​​model derived 1 week before delivery (one set at a time). For each set, the AI ​​model was then retrained using the modified input (i), which excluded all codes from that set (but included codes from other sets), and the AUC and AUPRC of the 5-fold cross-validation were calculated. In addition, to evaluate the significance of the measurements, the AUC and AUPRC were calculated for an AI model that was not trained with input (ii), and therefore only included input (i), which had codes for all sets up to 1 week before delivery. For each set (condition, medication, observation, procedure, and measurement), the percentage decrease in AUPRC and AUC due to the removal of that set compared to the AUPRC and AUC of the AI ​​model including all sets was calculated.

[0143] In addition, the importance of each EHR code in predicting each neonatal outcome was explored. A total of 13,668 unique codes were found in the maternal medical history until 1 week before delivery; 7,082 of them appeared in less than 5 maternal medical histories and were therefore excluded from this analysis. For each of the 6,586 unique codes found in at least five maternal medical histories, a binary variable was created indicating the presence / absence of the code in the maternal medical history until 1 week before delivery. Then, the odds ratio for each of these 6,586 binary variables and each of the 24 neonatal outcomes was calculated, as well as the respective p-values ​​to assess their significance. Similarly, the odds ratio was calculated for each of the measurements considered (using the results closest to 1 week before delivery) and each of the 24 outcomes. Each measurement was dichotomized by the corresponding median, and the odds ratio and corresponding p-value were calculated using logistic regression.

[0144] In order to balance the strength of association (indicated by the odds ratio) and statistical significance, and to distinguish between positive and negative associations, a new metric was obtained as follows. The odds ratio (or the negative of its reciprocal, i.e. -1 / odds ratio, for odds ratio <1) was multiplied by 1 minus the corresponding p-value. The upper limit of the obtained metric was set to 10 (or -10) to reduce the influence of outliers. The range of the obtained metric is -10 (there is a very strong negative association between the code and the result, which means that the presence of the code in the mother's medical history or the measurement result above the median reduces the risk of the result) to +10 (a very strong positive association, indicating that the presence of the code in the mother's medical history or the measurement result above the median increases the risk of the result).

[0145] Evaluating the benefits of a multi-tasking approach

[0146] The predictive performance of the multi-input multi-task model at delivery / birth (as described above) was compared with the predictive performance of 24 separate multi-input single-task models, each of which was trained to predict one of the 24 outcomes of interest. The multi-task and single-task models had the same inputs, which contained information available at delivery / birth, namely (i) code sequences from the mother's medical history up to delivery, (ii) maternal / newborn socio-demographic information, maternal measurements at delivery, gestational age and birth weight. The single-task model had the same architecture as the multi-task model, with a bidirectional LSTM layer for input (i) and a dense layer for input (ii), concatenated and then input into the dense layer. In the multi-task model, this last layer was followed by a set of dense layers for each outcome, while in the single-task model, this was followed by only a set of dense layers, which was responsible for predicting that specific outcome. Five-fold cross-validation ROC and precision-recall curves were obtained, as well as the respective areas under the curve (AUC and AUPRC) to compare the predictive performance of the single-task model for each of the 24 outcomes compared to the multi-task model.

[0147] In addition, a separate experiment was conducted to show the benefits of the multi-task approach, particularly in terms of correlated outcomes. When predicting NEC, a large difference was noted between the performance of the single-task and multi-task models. Therefore, two additional multi-task models were trained with the goal of monitoring changes in the ability to predict NEC. One multi-task model was trained to predict NEC and the outcome most closely correlated with NEC, i.e., anemia of prematurity. Similarly, one multi-task model was trained to predict NEC and polycythemia vera, i.e., the outcome least correlated with NEC. Both multi-task models had the same structure described for the main multi-task model, with all 24 outcomes, with two separate sets of final dense layers, one for NEC and the other for anemia of prematurity and polycythemia vera, respectively.

[0148] result:

[0149] Maternal / neonatal characteristics and neonatal outcomes

[0150] A total of 32,354 live births from 27,519 unique women occurring from 2014 to 2020 were included in the study. Table 4 reports maternal and neonatal sociodemographic characteristics and the prevalence of each of the 24 neonatal outcomes, with frequencies ranging from 0.07% (PVL) to 46.4% (hyperbilirubinemia). Table 3 includes a comparison of the prevalence of outcomes for term and preterm neonates. The codes extracted from the EHR by category have been listed by percentage, with conditions and procedures each accounting for more than 40% of the overall feature set ( Figures 5A-5D ). The overall prevalence of neonatal outcomes at our center is comparable to the national average.

[0151] To investigate the associations between the 24 neonatal outcomes, a correlation network was constructed that showed four correlations greater than 0.5 between pairs of outcomes ( Figure 8 ), and based on maternal factors extracted from EHR ( Figure 2 ). Several correlations were observed between various neonatal comorbidities, with sepsis, pulmonary hemorrhage, and atelectasis each showing correlations greater than 0.5 with 14 other outcomes. In contrast, candidiasis, polycythemia, and meconium aspiration syndrome (MAS) did not correlate more than 0.5 with any of the other outcomes. Fig. 9 is a hypothetical BPD prediction model that includes Figure 2 In summary, the input data ( Figure 2 ) shows strong internal correlation ( Figure 8 ), which demonstrated the use of multi-task learning and formed the basis for hypothesis testing based on known clinical risks ( Fig. 9 ).

[0152] AI model predicts neonatal comorbidities before, during and after birth

[0153] Fig. 7A The AUC and AUPRC (equivalent to the prevalence of the outcome compared to a random classifier) ​​of the AI ​​model at different prediction stages from 5 months before delivery to 2 months after delivery are reported. Prediction at the time of delivery achieved AUCs ranging from 0.64 (MAS) to 0.99 (BPD and anemia of prematurity), with AUCs exceeding 0.9 for 10 of the 24 neonatal outcomes considered (IVH, NEC, ROP, BPD, PVL, pulmonary hemorrhage, death, atelectasis, heart failure, and anemia of prematurity), and AUCs between 0.8 and 0.9 for another 7 outcomes (RDS, PDA, sepsis, CP, pulmonary hypertension, cardiac instability, and seizures). Compared to a random classifier, AUPRCs were 62.7 times higher for PVL, 57.9 times higher for BPD, 41.4 times higher for death, and 39.4 times higher for NEC (absolute numbers reported in Figures 10A-10D The calculator developed includes detailed longitudinal outcome data so that clinicians can better quantify the risk to the fetus or infant.

[0154] Importantly, the AI ​​model showed good predictive performance before birth: in the week before delivery, the AUC for death and ROP was above 0.9, and the AUC for IVH, NEC, BPD, PDA, PVL, pulmonary hemorrhage, CP, pulmonary HTN, atelectasis, heart failure, and anemia of prematurity was between 0.8 and 0.9. Similarly, for 12 outcomes, the AUPRC in the week before delivery / birth was at least 10 times that of the random classifier, in particular, 30.6 times for BPD, 25.1 times for atelectasis, and 24.8 times and 24.4 times for ROP and PVL, respectively. Fig.11 The same AI prediction model for an individual patient born at 24 weeks and 2 days of gestational age was presented, which contained the patient's unique maternal, neonatal and infant time series data to develop predictions for various outcomes related to preterm birth. The patient was selected to serve as an individual test of the model's ability to predict neonatal outcomes. The patient had an EHR diagnosis of RDS, IVH (grade I bilateral), BPD, sepsis, PDA, anemia of prematurity, ROP and hyperbilirubinemia. ROP, anemia of prematurity, RDS, hyperbilirubinemia and sepsis had the highest individual prediction scores at birth, all of which were diagnosed as what the patient ultimately suffered from. IVH, NEC, pulmonary hypertension, CP, PVL and death had the lowest prediction scores at birth. In summary, the data show that our AI model can predict individual outcomes at both the population and individual levels.

[0155] Predictive ability was consistent in term and preterm neonates

[0156] Neonates born after 37 weeks of gestation have traditionally been considered a relatively low-risk group for adverse neonatal outcomes, with lower rates of neonatal morbidity and mortality compared with preterm neonates. Nevertheless, term neonates, especially those with cardiac, neurological, or genetic disorders, are at increased risk for prolonged NICU hospitalization secondary to overlapping disease pathology with preterm infants. Therefore, we sought to evaluate the ability of our AI model to predict neonatal outcomes in term neonates.

[0157] All 24 morbidity and mortality outcomes were more prevalent in preterm neonates (n = 3,639, 11.5%) than in term neonates (n = 27,998, 88.5% - Tables 4 and 3). For example, the prevalence of IVH in preterm and term neonates was 5.1% and 0.2%, respectively, while the prevalence of NEC was 1.8% and 0.04%, respectively. Figure 3The AUC for each outcome in preterm and term neonates is depicted because the AUC is independent of the prevalence of the outcome. For most outcomes, the AUC for term neonates was similar to that seen in preterm neonates; however, a decrease in the AUC was observed for RDS (0.661 in term neonates vs. 0.833 in preterm neonates), ROP (0.665 vs. 0.918), sepsis (0.707 vs. 0.814), hyperbilirubinemia (0.606 vs. 0.728), candidiasis (0.626 vs. 0.711), cardiac instability (0.683 vs. 0.773), and neonatal gastroesophageal reflux (0.659 vs. 0.734).

[0158] AI models are robust to time series dataset shifts and are expected to translate to other healthcare scenarios

[0159] The AI ​​model was robust to potential temporal dataset shifts; the performance of the AI ​​model at delivery / birth, trained on newborns born between 2014 and the end of 2018 (n=22,101) and tested on newborns born in 2019 (n=5,852) and 2020 (n=4,397), was similar to that of the original model ( Figures 12A-12D ). Table 5 reports the AUC, AUPRC, and AUPRC compared to a random classifier. Performance in 2019 and 2020 was consistent with that seen for the original model; for example, the AUPRC for predicting NEC compared to a random classifier was 39.4 for the original model, and 44.0 and 45.1 in 2019 and 2020, respectively. For IVH, this increased from 20.4 for the original model to 26.2 in 2019 and 35.6 in 2020.

[0160] The simplified models built for validation in the external dataset were tested on 12,258 neonates obtained from the UCSF EHR and are described in Table 6. Details of the simplified models trained using the Stanford data are listed in Table 7, and the results are visualized in Figures 13A-13BFor all five outcomes, the AUCs of the models were similar between the two datasets (IVH: 0.903 in Stanford vs. 0.925 in UCSF; NEC: 0.942 vs. 0.923; anemia of prematurity: 0.988 vs. 0.944; RDS: 0.805 vs. 0.793; PDA: 0.849 vs. 0.866). AUPRCs were comparable for IVH (0.188 in Stanford vs. 0.230 in UCSF), PDA (0.316 vs. 0.225), and RDS (0.504 vs. 0.388); however, AUPRCs decreased in the test data for NEC (0.195 vs. 0.032) and anemia of prematurity (0.668 vs. 0.275).

[0161] Subgroup discovery algorithm identifies subsets of newborns for which the AI ​​model improves its predictive ability

[0162] Using a 128-dimensional latent space of maternal EHR sequences, subgroup discovery yielded a subset of newborns representing at least 30% of the entire study population, where the AI ​​model at the time of labor / birth achieved higher levels of precision and recall ( Figure 1 and Table 8). For most neonatal outcomes, the identified subgroups achieved higher AUPRCs, especially compared to the random classifier. Most importantly, the predictive ability of the AI ​​model improved in the subgroups identified for NEC (from 0.096 in the full dataset to 0.516 in the subgroup), ROP (from 0.690 to 0.860), BPD (from 0.487 to 0.641), PDA (from 0.394 to 0.466), hyperbilirubinemia (from 0.627 to 0.753), and anemia of prematurity (from 0.717 to 0.963). For most outcomes, subgroup findings identified a subset of newborns with a lower prevalence of the outcome of interest compared to the full dataset. Since the baseline AUPRC (i.e., the AUPRC of the random classifier) ​​is equivalent to the prevalence of the outcome, it is important to compare the improvement compared to the random classifier. Among the identified subgroups, the AI ​​model's AUPRC improved particularly compared to the random classifier for NEC (from 39.8 in the full dataset to 588.8 in the subgroup), anemia of prematurity (from 30.9 to 301.3), candidiasis (from 3.2 to 16.1), heart failure (from 16.7 to 64.3), atelectasis (from 29.4 to 103.2), and ROP (from 40.3 to 125.2). The subgroup discovery algorithm ultimately enhanced the model's predictive power, especially for infrequent outcomes such as NEC.

[0163] AI model outperforms currently used risk scores

[0164] As shown in Table 9 and Figures 14A-14B As shown in , the AI ​​model at delivery / birth is much better than the Apgar score at 1 minute in terms of AUC and AUPRC. For 22 of the 24 results, the AUPRC and AUC of the AI ​​model were significantly higher than those of the APGAR score (all p values ​​<0.001), including RDS, IVH, NEC, ROP, BPD, PDA, sepsis, pulmonary hemorrhage, CP, pulmonary HTN, hyperbilirubinemia and death. When evaluated in preterm newborns, the AI ​​model was also significantly better in AUPRC and AUC compared with the NICHD risk score (Table 10). Except for polycythemia and other CNS disorders, the AI ​​model showed significant improvement compared with the NICHD score for all results. It is worth noting that the AI ​​model is designed to measure many additional results in addition to the results measured by the NICHD-NRN or APGAR scoring model. Therefore, the comparison must be interpreted with caution.

[0165] Using EHR data to explore the underlying pathological processes of neonatal conditions

[0166] For each of the five identifiable categories of conditions, medications, observations, procedures, and outcomes, the Figures 15A-15D ) which shows the odds ratios for the 50 codes (rows) with the highest average odds ratios across all 24 outcomes (columns) and for each of the 24 outcomes (columns). All associations between concept codes and neonatal outcomes can also be interactively queried, visualized, and downloaded via the following links:

[0167] It is worth noting that supplementation Fig.15D is a heat map of odds ratios between maternal laboratory measurements in the week before delivery and 24 neonatal outcomes. Higher laboratory values ​​indicate positive or negative odds ratios. Notable laboratory measurements that showed protective associations with neonatal outcomes included serum albumin, serum proteins, platelets, basophils, lymphocytes, and eosinophils. The data suggest that there is an interaction between the maternal immune system in the week before delivery and the relative health of the fetus that continues into the neonatal period and beyond.

[0168] Fig.16The correlation network in Figure 2 shows the codes with the highest average odds ratios for all 24 outcomes and the interactions between codes. The codes that were strongly associated with the neonatal outcomes were maternal outcomes, including postpartum sepsis, PROM (premature rupture of membranes), preterm premature rupture of membranes (PPROM) with unknown onset of labor, PPROM with onset of labor more than 24 hours after PROM, opioid dependence in remission, fetomaternal transfusion syndrome, various congenital heart diseases, renal failure, and / or dialysis dependence. In addition, some codes appeared to be novel risk factors for the outcomes, such as methicillin-susceptible S. aureus carrier status, renal failure, blood cell indices (such as hematocrit), chemotherapy exposure, and certain medications (including phosphodiesterase inhibitors and opioids).

[0169] Simultaneous modeling of neonatal morbidity improves predictive power

[0170] exist Figures 17A-17D The results of multi-task experiments relative to single-task experiments are reported in and show significant improvements in AUPRC and AUC for the multi-task model compared to the single-task model. In terms of AUC, the largest improvements were observed for PVL (0.515 for the single-task model vs. 0.934 for the multi-task model), NEC (0.629 vs. 0.957), heart failure (0.618 vs. 0.940), and pulmonary hemorrhage (0.687 vs. 0.969).

[0171] Given that several neonatal outcomes are particularly difficult for clinicians to predict, we explored NEC and related outcomes of interest in detail. Figures 18A-18F The relationship between maternal anemia, neonatal anemia, anemia of prematurity, and NEC is depicted in . The AUPRC and AUC for NEC using the single-task model were 0.007 and 0.629, respectively, while the AUPRC and AUC obtained by the multi-task model were 0.095 and 0.957, respectively. The four-item correlation between NEC and polycythemia was 0.13, while the four-item correlation between NEC and anemia of prematurity was 0.75. The dual-output multi-task model that simultaneously predicted NEC and polycythemia achieved an AUPRC of 0.010 and an AUC of 0.636 for NEC, while the multi-task model that predicted NEC and anemia of prematurity achieved an AUPRC of 0.056 and an AUC of 0.897 ( Fig.18A ).

[0172] AI model differentiates newborns with IVH based on IVH grade

[0173] Because IVH can also be difficult for clinicians to predict, we analyzed the IVH prediction scores output by the AI ​​model for neonates with IVH at the time of labor / birth. Neonates with IVH were grouped based on IVH grade using records in the neonates’ EHR (grade was ambiguous when there was no record associated with IVH grade). Importantly, the AI ​​model was able to differentiate neonates based on IVH grade; neonates with lower actual IVH grades had lower IVH prediction scores on average compared to neonates with higher IVH grades, while the mean IVH prediction score increased as IVH grade increased ( Figures 19A-19B ). Given that IVH typically occurs within the first 96 hours after birth, the IVH prediction score includes only maternal and neonatal inputs that occurred at and before birth to avoid reverse contamination of the algorithm. Fig.19B The IVH prediction score depicted in demonstrates a dose-dependent relationship between the model input and the severity of IVH.

[0174] Discussion: Using data from >27,000 mothers associated with >32,000 newborns collected at a single center between 2014 and 2020, it has been demonstrated that it is possible to predict neonatal outcomes for a variety of maternal conditions extracted specifically from the EHR. Prediction of neonatal morbidity remains a critical issue in the NICU. Almost all neonatal morbidities are classified based on clinical signs or findings of disease rather than by the use of biomarkers that represent disease pathophysiology. Although prematurity is an important risk factor for the development of a given neonatal morbidity, gestational age alone is not sensitive and specific for reliably predicting neonatal illness. Often the diagnosis occurs too late, resulting in severe morbidity or mortality. Therefore, rapid and early diagnosis of neonatal illness is necessary to prevent severe morbidity and mortality. AI methods that can make predictions from large, heterogeneous datasets have the potential to transform neonatal care by standardizing risk assessments across diverse patient populations. To our knowledge, this is the first study to employ these methods to interrogate the large amounts of clinical data collected in maternal and neonatal EHRs to make clinically meaningful predictions of neonatal outcomes.

[0175] This work differs from previous risk prediction approaches utilizing EHRs in several important ways. First, to our knowledge, this is one of the largest sample sizes of maternal and neonatal records extracted from EHRs utilizing discrete clinical data (including 5 categories of codes). Second, using advanced machine learning methods, we have discovered novel associations between maternal conditions and neonatal outcomes that are clinically plausible. Third, our findings confirm that fetal exposure to maternal health conditions (anemia, certain medication exposures, social determinants of health) appears to increase neonatal susceptibility to conditions such as NEC, BPD, IVH, PDA, and CP. Fourth, we have developed the first longitudinal clinical risk calculator ( Figure 7A-7B and Fig.11 ), thereby incorporating real-time clinical data to predict neonatal outcomes beginning before birth and extending chronologically to 2 months of age. This calculator has the potential to transform clinical care in a number of different ways, including: 1) minimizing interindividual variability in management between providers; 2) providing personalized care based on standardized risk assessment tools; 3) understanding longitudinal group-level risk that applies to individuals; 4) assisting in targeting individual patients who are best suited for recruitment into translational and clinical trials based on longitudinal risk for a given disease; and 5) allowing clinicians to make more informed real-time assessments of their patients and take timely interventions or treatments.

[0176] Most previous clinical risk prediction models use algorithms based on a set of known risk factors, which are captured at a single time point using data from large cohort studies. In adults, examples include the CHA2DS2-VASc score for atherosclerotic cardiovascular disease calculator, thromboembolic risk in atrial fibrillation, and the end-stage liver disease model (insert citation) for predicting the survival of patients with various forms of liver failure. In neonatology, frequently used calculators include the NICHD-NRN calculator, BPD outcome estimator, outcome trajectory estimator, infant clinical risk index (CRIB), and neonatal acute physiology score (SNAP) (insert citation). All of these predict survival and other morbidity rates associated with preterm birth and / or critical illness. But these calculators rely on information collected shortly before or after birth, so it is difficult to rely on them longitudinally. Given that many critically ill newborns require long-term hospitalization and the incubation period of most common diseases varies, this can cause problems. For example, preterm birth usually requires 3-6 months NICU hospitalization. 27 Therefore, the need for a risk prediction calculator that incorporates longitudinal data is urgent, as risk in this population is dynamic, with an ever-increasing set of additional variables that continuously accumulate and interact. 28 The best-known example of a time-based clinical risk prediction tool for pediatric patients is the hour-specific nomogram for the Hyperbilirubinemia Risk Assessment Tool. 29 This hour-specific nomogram and calculator allows clinicians to determine whether serum bilirubin levels meet criteria for phototherapy and / or double volume exchange therapy. This support tool is widely applicable and widely used and has been acclaimed for its ease of use, all desirable features that have led to widespread adoption. However, the tool has only been used to manage hyperbilirubinemia. Our EHR-based longitudinal clinical risk prediction tool ( Figure 7A-7B and Fig.11 ) has incorporated many of the advantageous elements of other calculators, including the ability to predict the risk of multiple morbidities in parallel and to combine both maternal and newborn data.

[0177] The longitudinal nature of our risk assessment tool, combined with the comprehensive nature of the extracted clinical data, allowed us to discover new maternal factors and conditions associated with neonatal risk ( Figures 18A-18F and Fig.19A Indeed, we have found that infant risk of NEC is highly correlated with conditions caused by maternal chronic disease, including maternal anemia, social determinants of health (homelessness, incarceration), and certain prenatal maternal medications, including indomethacin and sildenafil ( Figures 18A-18F The risk of IVH is associated with maternal factors including opioid exposure, renal failure, and methicillin-susceptible S. aureus carrier status ( Fig.19A). Further research is needed to validate these findings. Nonetheless, we believe these are important findings to date, as the risk associated with preterm birth has traditionally been thought to stem from factors such as gestational age, birth weight, and sex (static categories related to anthropometrics) rather than dynamic maternal conditions that may influence the underlying biology reflected by the maternal-fetal interface and subsequently affect the fetus.

[0178] Furthermore, these findings provide nuance to the concept of clustering of acquired diseases in preterm infants. Preterm infants commonly experience co-occurrence of morbidities, including BPD, NEC, ROP, cerebral palsy (CP), and sepsis. Fig. 9 Overlapping patterns of disease were demonstrated, as evidenced by the large number of interconnections between each of the 24 different neonatal outcomes. RDS, anemia, BPD, sepsis, and NEC were all highly correlated with each other as outcomes of preterm birth. These outcomes could be predicted overall based on the clinical trajectory of the mother's pregnancy, but could also be determined individually, i.e., for IVH ( Figures 19A-19B ). Indeed, our model confirmed that, based on the Papile grading system, IVH grade can be predicted at birth, and that the accuracy increases with increasing severity of IVH. This suggests that the multi-task approach is able to classify outcomes in a similar manner to that demonstrated in clinical epidemiological studies. 30

[0179] NEC is a relatively rare disease, even in neonates born before 28 weeks of gestation (incidence 4-10%), thus making it difficult to characterize and prospectively study. The current study is one of the largest studies of NEC risk to combine maternal and neonatal factors in a unified prediction. We found that anemia and / or anemia of prematurity were highly associated with NEC in a multitasking approach ( Fig. 9 , 18D and 18F). This observation further confirms the association from smaller studies. Severe anemia (ie, hemoglobin <8) is thought to be one of the first events in the series of intestinal hypoxia-ischemia. These findings suggest that hemoglobin levels in the mother or the newborn may also be associated with the development of NEC, as lower hemoglobin levels in the mother soon after conception (-9M) were associated with later NEC in the newborn ( Fig.18C In addition, higher hemoglobin levels at birth and greater variability in hemoglobin levels during the first two months of life have also been associated with the development of NEC ( Fig.18F). Impaired placental-fetal transfusion may occur with partial cord occlusion or early cord clamping, which may be the first alarming step in a chain of events that leads to anemia, transient hypovolemic shock, and ischemic stress, which subsequently lead to later NEC. Various studies have demonstrated an association between the severity of anemia and NEC, with some even suggesting a dose-dependent relationship between the two entities. In one of the largest studies to date, which included 598 VLBW infants, 44 of whom had at least stage II NEC, a hazard ratio of 5.99, p = 0.001, was observed for those with severe anemia (defined as hemoglobin <8 g / dL). 39 Alternatively, in the recent Transfusion in Premature Infants (TOP) trial, a prospective study that randomized 1824 preterm neonates of gestational age <29 weeks to higher versus lower hemoglobin thresholds, no difference in NEC rates was observed for neonates with higher versus lower transfusion thresholds, although this was not the primary outcome of the study.

[0180] Although the exact biological mechanisms of NEC have not been definitively determined, neonatal anemia can lead to impaired oxygen delivery, which may result in compromised mesenteric vasculature. The underlying gastrointestinal hypoperfusion is thought to perturb the local microbiome, increase local production of proinflammatory mediators, and exacerbate damage to the immature intestinal barrier. Figures 18A-18F The findings in suggest that: 1) maternal anemia may be a novel risk factor for NEC; and 2) there is a relationship between the degree of anemia in the mother and / or neonate at the time of delivery and the subsequent risk of NEC. Ultimately, further prospective studies are needed to validate this relationship, assess whether transfusions at higher hemoglobin thresholds are protective, and determine whether there is a role for hormone replacement therapy (erythropoietin, darbepoietin alfa) in selected populations at risk for complications secondary to severe anemia. In addition, it should be recognized that given the relative rarity of the diagnosis of NEC, the number of cases requiring screening and / or treatment when evaluating a mother for anemia may be quite high.

[0181] limitation

[0182] Several limitations must be considered when evaluating this study. First, we recognize that ICD codes captured from the EHR do not always fully mimic or replicate clinical findings in patients. This is particularly true for categories such as diagnosis, where clinical variability and interpretation can generate subjectivity. In addition, although the overall prevalence of neonatal illness at our single institutional site is close to the national prevalence in the United States, it is recognized that these results may not be fully generalizable to all institutions. However, such a model is able to predict neonatal outcomes independently of the institution. In addition, clinical risk prediction models should recommend specific decisions that clinicians can take, as studies have shown that the recommendations may influence provider behavior. This study has been designed and intended as a first step in longitudinal risk prediction. Only with additional validation can our prediction model be reasonably used to recommend specific interventions or treatments.

[0183] Conclusions: The machine learning approach employed in this article has allowed the development of predictive models for neonatal outcomes and will potentially serve as an important resource for independent examination by clinicians and researchers. New associations between various maternal and neonatal characteristics and specific neonatal outcomes have been observed. The first longitudinal clinical risk prediction tool for various neonatal outcomes has been developed. A deeper understanding of the impact on the fetal environment and how it may contribute to the risk of neonatal morbidity has been gained.

[0184] Principle of Equivalence

[0185] Several embodiments have been described, and those skilled in the art will recognize that various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the present invention. In addition, in order to avoid unnecessarily obscuring the present invention, many well-known methods and elements have not been described. Therefore, the above description should not be construed as limiting the scope of the present invention.

[0186] Those skilled in the art will appreciate that the foregoing embodiments and the description of the various preferred embodiments of the present invention are only illustrations of the present invention as a whole, and that changes in the components or steps of the present invention may be made within the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments described herein, but is defined by the scope of the appended claims.

[0187] Table 1: List of maternal vital signs and laboratory measurements

[0188]

[0189]

[0190]

[0191]

[0192] Table 2: List of Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) concept IDs used to define each of the 24 neonatal outcomes considered

[0193]

[0194]

[0195]

[0196]

[0197]

[0198]

[0199]

[0200]

[0201]

[0202]

[0203]

[0204]

[0205]

[0206]

[0207] Table 3. Prevalence, AUPRC, AUPRC compared with a random classifier, and AUC of AI models for predicting 24 neonatal outcomes, by preterm and term (gestational age at delivery <37 or ≥37 weeks)

[0208]

[0209]

[0210] Table 4: Summary statistics of maternal / neonatal characteristics and neonatal outcomes. Note: Gestational age at delivery was missing for 713 neonates and neonatal birth weight was missing for 463 neonates. RDS: respiratory distress syndrome, IVH: intraventricular hemorrhage, NEC: necrotizing enterocolitis, ROP: retinopathy of prematurity, BPD: bronchopulmonary dysplasia, PDA: patent ductus arteriosus, PVL: periventricular leukomalacia, CP: cerebral palsy, HTN: hypertension, MAS: meconium aspiration syndrome, CNS: central nervous system

[0211]

[0212]

[0213]

[0214]

[0215] Table 6: Summary statistics of maternal / neonatal characteristics in the external validation data from UCSF

[0216]

[0217] Table 7: Logistic regression models using the Stanford data to predict RDS, NEC, IVH, PDA, and anemia of prematurity based on the first 10 codes for each outcome plus gestational age

[0218]

[0219]

[0220]

[0221]

[0222]

[0223]

[0224]

[0225]

[0226]

Claims

1. A machine learning model comprising: A multi-task neural network, wherein the neural network comprises an encoder, hidden states, and a decoder, wherein the encoder reads an input, wherein the hidden states represent an internal learned representation of the entire input, and wherein the decoder interprets the internal learned representation and reconstructs the input.

2. The machine learning model of claim 1, wherein the input comprises an individual's electronic health record (EHR).

3. The machine learning model of claim 1, wherein the machine learning model is trained using EHRs of a plurality of individuals and a plurality of newborns, wherein each of the plurality of individuals has given birth to at least one of the plurality of newborns.

4. A method for assessing neonatal risk, comprising: Access or has access to an individual’s electronic health record (EHR); and At least one neonatal disorder in the individual's child is determined based on the metabolites in the EHR using a machine learning model comprising a deep learning neural network having at least one bottleneck layer.

5. The method of claim 4, wherein the machine learning model determines respiratory support strategies (including ventilator settings) to reduce adverse outcomes.

6. The method of claim 5, wherein the respiratory support strategy comprises ventilator settings.

7. The method of claim 4, wherein the model identifies medications prescribed to the mother that may affect neonatal morbidity.

8. The method of claim 4, wherein the at least one neonatal disorder is selected from the group consisting of bronchopulmonary dysplasia (BPD), intraventricular hemorrhage (IVH), necrotizing enterocolitis (NEC), retinopathy of prematurity (ROP), bronchopulmonary dysplasia (BPD), intraventricular hemorrhage (IVH), necrotizing enterocolitis (NEC), retinopathy of prematurity (ROP), pulmonary hypertension, pulmonary hemorrhage, jaundice, periventricular leukomalacia (PVL), respiratory distress syndrome (RDS), early-onset sepsis, late-onset sepsis, patent ductus arteriosus (PDA), cerebral palsy, and neurodevelopmental impairment (NDI).

9. The method of claim 4, wherein the EHRs are from multiple institutions.

10. The method of claim 4, further comprising treating the child for at least one neonatal disorder.

11. A method for providing intravenous nutrition to a premature infant, comprising: obtaining or having obtained an electronic health record (EHR) of an individual, wherein the EHR includes detailed information about the health of the individual, and the individual is a premature infant; and Based on the recommendations of the machine learning model, a nutrient bag containing a mixture of nutrients is selected to supplement the health of the individual.

12. The method of claim 11, wherein the machine learning model comprises a multi-task neural network, wherein the neural network comprises an encoder, a hidden state, and a decoder, wherein the encoder reads an input, wherein the hidden state represents an internal learned representation of the input, and wherein the decoder interprets the internal learned representation and reconstructs the input.

13. The method of claim 12, wherein the nutrient bag is one bag in a group of nutrient bags, wherein each bag in the group of nutrient bags contains a combination of nutrients generated by bottleneck layer clustering.

14. The method of claim 13, wherein the nutrient bag improves wound healing.

15. The method of claim 13, wherein the nutritional bag improves neurocognitive development.

16. The method of claim 13, wherein the nutrient bag improves respiratory health.

17. The method of claim 13, wherein the nutritional bag improves gastrointestinal health.

18. The method of claim 13, wherein the nutrient bag improves ocular health.

19. A method of nutritional support, comprising: obtain health information about an individual; and Dietary advice is provided to the individual.

20. The method of claim 19, wherein providing dietary advice comprises providing food advice.

21. The method of claim 20, wherein the food recommendations include at least one baby food recommendation.

22. The method of claim 19, wherein providing dietary advice comprises interacting with a database of food and nutritional information.

23. A method for preparing an intravenous nutritional supplement solution, comprising: Using machine learning models to develop a set of nutritional prescriptions for intravenous supplementation; Nutrient bags are produced containing a formula from the formulary.

24. The method of claim 23, wherein the machine learning model comprises a multi-task neural network, wherein the neural network comprises an encoder, a hidden state, and a decoder, wherein the encoder reads an input, wherein the hidden state represents an internal learned representation of the input, and wherein the decoder interprets the internal learned representation and reconstructs the input.

25. The method of claim 23, wherein producing a nutritional bag comprises producing a nutritional bag for each prescription in the formulary.

26. The method of claim 23, wherein the nutritional formulary comprises at least 5 prescriptions.

27. The method of claim 23, wherein the nutritional formulary comprises 15 prescriptions.

28. The method of claim 23, wherein the nutritional bag is a sterile IV bag.

Images