Method and system for discretizing sepsis risk scores for clinically actionable insights

Abstract

One or more example embodiments relates to a method for training a data model for predicting risk of sepsis. The method comprising creating a first data set indicative of sepsis positive patients, and a second data set, a third data set, and a fourth data set indicative of sepsis negative patients from a patient database. The method further comprises creating a training data and a testing data from the first, second, third and fourth data sets. The method further comprises training and testing the data model based on the training and testing data such that the trained and tested data model may be used for predicting risk of sepsis in patients, providing clinically actionable recommendations based on category of determined risk, and identifying a sub-population of patients likely to benefit from antimicrobial therapy.

Description

METHOD AND SYSTEM FOR DISCRETIZING SEPSIS RISK SCORES FOR CLINICALLY ACTIONABLE INSIGHTSFIELD

[0001] The present disclosure generally relates to predicting sepsis risk, and specifically relates to providing actionable insights based on discretizing sepsis risk scores generated by an ML model.RELATED ART

[0002] The subject matter discussed in the background section should not be assumed to be prior art merely as a result of its mention in the background section. Similarly, a problem mentioned in the background section or associated with the subject matter of the background section should not be assumed to have been previously recognized in the prior art. The subject matter in the background section merely represents different approaches, which in and of themselves may also correspond to implementations of the claimed technology.

[0003] Sepsis occurs when the immune system has a dangerous reaction to an infection. It causes extensive inflammation throughout the body that can lead to tissue damage, organ failure and even death. Many different kinds of infections can trigger sepsis, which is a medical emergency. The quicker the treatment is provided, the better the outcome will be. In recent years, various machine learning algorithms have been proposed for early sepsis prediction to mitigate its impact on mortality, morbidity, and healthcare expenses. Patients suffering from sepsis face a terribly high risk of death. The majority, approximately 80%, of hospital-treated sepsis arises in the community and presents to emergency departments. In intensive care, early detection of the sepsis risk is essential to control the disease, because the treatment of sepsis is highly timesensitive. Early detection of such community-acquired infections during emergency department triage is important for initiating lab analysis, antibiotic administration, and other sepsis treatment protocols early. Despite significant work done in this domain, the impact of sepsis prediction algorithms on patient outcomes in terms of clinically actionable recommendations and antimicrobial stewardship is limited.

[0004] While some existing ML algorithms for prediction of sepsis have been found to be superior to the traditional screening tools in predicting sepsis in emergency rooms, the binary categorization of patients into sepsis positive and sepsis negative performed by such ML models lack clinical interpretation or actionable recommendations. Most of the existing ML algorithms for prediction of sepsis are trained on one or more of continuous vital signs data and laboratory parameters that may not be promptly available in emergency rooms. Further, existing methods for early detection of sepsis in the emergency room involving rule-based sepsis scoring systems such as Systemic Inflammatory Response Syndrome (SIRS) criteria and quick Sequential Organ Failure Assessment score (qSOFA) have low sensitivity and / or low specificity.

[0005] Previously, SIRS criteria had been used to define sepsis. If the SIRS criteria are negative, it is very unlikely the person has sepsis. Alternatively, if SIRS criteria is positive, there is just a moderate probability that the person has sepsis. According to SIRS, there were different levels of sepsis: sepsis, severe sepsis, and septic shock. In recent times, SIRS criteria has been replaced by systemic inflammatory response syndrome (SIRS) with the sequential organ failure assessment (SOFA score) and the abbreviated version (qSOFA). The three criteria for the qSOFA score include a respiratory rate greater than or equal to 22 breaths per minute, systolic blood pressure 100 mmHg or less and altered mental status. Sepsis is suspected when 2 of the qSOFA criteria are met.

[0006] Early diagnosis is necessary to properly manage sepsis, as the initiation of rapid therapy is key to reducing deaths from severe sepsis. By providing timely interventions, these early warning scores can help with early warning programs or specific prehospital treatment with high sensitivity. However, these criteria are poor in specificity. For example, the physiological indicators of viral influenza can often cause false alarms. Further, qSOFA and SOFA criteria may lead to delayed diagnosis of serious infection, leading to delayed treatment. Although SIRS criteria can be too sensitive and not specific enough in identifying sepsis, SOFA also has its limitations and is not intended to replace the SIRS definition. qSOFA has also been found to be poorly sensitive though decently specific for the risk of death with SIRS possibly better for screening. Further still, these criteria do not show a significant change in clinical outcomes.

[0007] Hence there is a need for an accurate method for early prediction of sepsis in emergency rooms that exhibits high sensitivity and specificity, since early prediction or detection in emergency department triage helps in initiating sepsis treatment protocols early.SUMMARY

[0008] One or more example embodiments provides a method for training and developing a data model for early prediction of sepsis.

[0009] One or more example embodiments accurately predicts risk of sepsis in an emergency room setting.

[0010] One or more example embodiments identifies a sub-population within patients of a medical institution who require early intervention with respect to sepsis.

[0011] One or more example embodiments predicts a mortality risk of patients based on predicted risk of sepsis.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Having thus described the embodiments of the disclosure in general terms, reference now will be made to the accompanying drawings, which are not necessarily drawn to scale, and wherein:

[0013] Fig. 1 illustrates a network diagram of a system for developing the data model, in accordance with an embodiment.

[0014] Fig. 2 illustrates a timeline within which data for training a data model is to be collected from a patient admitted in a hospital, in accordance with an embodiment.

[0015] Fig.3 illustrates a split-up of patients used for a training data set and a testing data set of the data model, in accordance with an embodiment.

[0016] Fig. 4 illustrates an AUROC and AUPRC plot for a balanced test set and a prevalence test set, in accordance with an embodiment.

[0017] Fig. 5 illustrates exemplary recommendations generated by a data model corresponding to different categories of sepsis risk, in accordance with an embodiment.

[0018] Figs. 6(a) and 6(b) illustrate outputs of the system, in accordance with an embodiment.DETAILED DESCRIPTION

[0019] The description set forth below in connection with the appended drawings is intended as a description of various embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced. Each embodiment described is provided merely as an example or illustration of the present invention, and should not necessarily be construed as preferred or advantageous over other embodiments. The description includes specific details for the purpose of providing a thorough understanding of the present invention.

[0020] Some embodiments of the present disclosure now will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all, embodiments of the disclosure are shown. Indeed, embodiments of the disclosure may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like numbers refer to like elements throughout.

[0021] As used herein, the tenu “comprising” means including but not limited to and should be interpreted in the manner it is typically used in the patent context. Use of broader terms such as comprises, includes, and having should be understood to provide support for narrower terms such as consisting of, consisting essentially of, and comprised substantially of.

[0022] Sepsis is a life-threating complication of an infection where the body responds improperly to an infection. Sepsis occurs when chemicals released in the bloodstream to fight an infection triggers inflammation throughout the body, causing a cascade of changes that damage multiple organ systems. The damage to the organ systems lead them to fail, and sometimes even results in death. Early detection of sepsis is essential for effective treatment so as to mitigate its impact on mortality and morbidity of patients. Since majority of hospital-treated sepsis cases presents to emergency departments of hospitals, early identification of infections during emergency department triage is important for early detection and treatment of sepsis.

[0023] Existing rule-based methods for early detection of sepsis include SIRS (Systemic Inflammatory Response Syndrome) criteria and qSOFA (quick Sequential Organ Failure Assessment score) diagnose sepsis based on identifying presence of certain criteria. For instance, SIRS is the presence of two or more of the following: abnormal body temperature, heart rate, respiratory rate, or blood gas, and white blood cell count. Further, three criteria for the qSOFA score include a respiratory rate greater than or equal to 22 breaths per minute, systolic blood pressure 100 mmHg or less and altered mental status. Sepsis is suspected when 2 of the qSOFA criteria are met. In addition to the inaccuracies of sepsis detection by the above-mentioned rule-based methods, existing ML algorithms used for predicting sepsis lack clinical interpretation and actionable recommendations.

[0024] The proposed method and system overcome such limitations of existing methods of early detection or prediction of sepsis. The proposed system and method involves developing a data model for predicting risk of sepsis. The proposed system and method also involves discretizing the predictions made by the data model for providing actionable insights regarding treatment protocols to be performed for patients based on their predicted sepsis risk.

[0025] Fig. 1 illustrates a network diagram of a system for developing the data model, in accordance with an embodiment of the present invention. The system 102 may be a server (such as a cloud server or a local server) or a processing device with processing circuitry used for developing the data model. The system 102 is connected to a plurality of data sources 104, where the plurality of data sources relates to a set of patients used for training and testing the data model. In certain embodiments, the plurality of data sources 104 may be associated with patients admittedin emergency departments of a hospital or a group of hospitals. The plurality of data sources 104 include relational databases of the patients and medical instruments or devices monitoring vitals of the patients. The relational databases of the patients may include medical data relating to the patients and medical history of the patients including results of prior lab tests performed for the patients. The vitals of the patient monitored by the medical instruments or devices include temperature, blood pressure, heart rate etc.

[0026] Such data relating to the patients obtained from the plurality of data sources 104 may be retrieved by the system 102 for developing the data model. Firstly, the method for developing the data model performed by the system 102 involves dividing the patients into sepsis positive patients and sepsis negative patients. The patients were tagged as sepsis positive according to sepsis 2 definition, wherein the patients are tagged as having severe sepsis if they satisfied the SIRS criteria, exhibited symptoms of organ dysfunction, and displayed signs of suspicion of infection. To ensure the inclusion of only community acquired infections in the sepsis positive dataset, SIRS criteria, organ dysfunction criteria, and either of blood culture or antibiotic administration should be met between ER arrival and 48-hours post hospital admission, as illustrated in Fig. 2. Further, patients with ICD (International Classification of Diseases) codes belonging to Hospital Acquired Infections are excluded.

[0027] Referring back to Fig. 1, all patients who did not meet the severe sepsis criteria for tagging sepsis positive patients, are considered for identifying the sepsis negative patients. From such a group of patients, patients having ICD codes related to sepsis are removed. ICD is a medical classification list by the World Health Organization containing codes for diseases, signs and symptoms, abnormal findings, complaints, social circumstances, and external causes of injury or diseases. Within the group obtained after removing patients having ICD codes related to sepsis, only patient who did not received antimicrobials during their stay in hospital are retained so as to remove noise. The group of patients thus obtained are identified as sepsis negative patients.

[0028] The sepsis negative patients are further sub-divided into 3 categories to cover the diversity of negatives within the negative cohort and ensure that training and testing data of the data model includes equal proportions of complex cases. The first category of sepsis negative patients includes patients who are SIRS positive and had organ failure within 48 hours of hospitaladmission post-ED discharge. The second category of sepsis negative patients include patients who are SIRS negative, have organ failure within 48 hours of hospital admission post-ED discharge, and whose symptoms mimic those of sepsis. The third category of sepsis negative patients includes patients who are SIRS negative, have organ failure within 48 hours of hospital admission post-ED discharge, and whose symptoms do not mimic sepsis.

[0029] After generating a set of sepsis positive patients and a set of sepsis negative patients, the system 102 extracts training features from databases of the plurality of data sources 104, where training features may include CBC, CMP, vital signs, and demographic parameters such as age and gender. In certain embodiments, additional features derived from parameters that are calculated and used in routine clinical practice may also be extracted by the system 102. After extraction of the training features and the additional features, a missingness analysis is performed by the system 102 to remove parameters that were missing in more than 30% of the total patients. Further, the system also performed a Pearson correlation analysis on the remaining parameters for eliminating parameters that are highly correlated with other parameters, in order to remove redundant parameters and obtain final parameters for developing the data model.

[0030] The sepsis positive patients and sepsis negative patients, along with their corresponding final parameters are used for building the data model and validating the data model. From the sepsis positive patients and sepsis negative patients, a training set of patients and a testing set of patients is generated by the system 102. In an embodiment of the present invention, a fixed number of patients is decided as the size of the training set by the system 102. Sepsis positive patients of the training set and sepsis negative patients of the training set are selected in the ratio 1:2. Further, in order to ensure diversity of sepsis negative patients of the training set, 50% of the sepsis negative patients of the training set is constituted by patients identified as satisfying SIRS criteria but non-septic. Further, 25% of the sepsis negative patients of the training set is constituted by patients tagged with Sepsis mimicking disease ICD Codes I ICD Code prefixes and identified as failing SIRS criteria, and 25% of the sepsis negative patients of the training set is constituted by patients identified as failing SIRS criteria and not tagged with sepsis mimicking disease ICD Codes / ICD Code prefixes.

[0031] The training set generated is thereafter used by the system 102 for training the data model. In one embodiment, the data model may be a voting ensemble classifier, comprising one or more gradient boosting frameworks and one or more Extra Trees classifiers. In an exemplary embodiment, the one or more gradient boosting frameworks may include one or more Light GBMs (Light Gradient Boost Machines) and one or more XGBoosts (extreme Gradient Boosting). In an exemplary embodiment, the one or more extra trees classifiers are trained using the scikit-learn package integrated with python3. The trained ML model is tested using a balanced test set generated by the system 102 comprising equal number of sepsis positive patients and sepsis negative patients. 50% of the sepsis negative patients of the training set are constituted by patients satisfying the SIRS criteria but non-septic. Further, 25% of the sepsis negative patients of the testing set is constituted by patients tagged with Sepsis mimicking disease ICD Codes I ICD Code prefixes and identified as failing SIRS criteria, and 25% of the sepsis negative patients of the testing set is constituted by patients identified as failing SIRS criteria and not tagged with sepsis mimicking disease ICD Codes I ICD Code prefixes. In certain embodiments of the present invention, the testing set also consists of patients tagged with with ICD codes associated with diseases that exhibit similar symptoms as sepsis.

[0032] The data model trained and tested by the system 102 based on the training set and the testing set is capable of differentiating between patients satisfying SIRS criteria who are sepsis positive and patients satisfying SIRS criteria who are sepsis negative, which is one of the difficulties currently faced by clinicians. The data model is also capable of correctly tagging patients with ICD codes associated with diseases that exhibit similar symptoms as sepsis as sepsis negative. The development of a robust negative set, as disclosed above, enhances the ability of the data model to differentiate between sepsis-positive cases and various non-sepsis cases. The data model determines the risk of sepsis as a sepsis risk score. The predictions made by the data model with respect to the testing data is thereafter discretized by the system 102 into low, medium, and high-risk categories. In certain embodiments of the present invention, thresholds for discretization is determined by the system 102 through Bayesian Optimization using Gaussian Processes. ti, a,a) were searched for minimizing the following objective function as provided in Equation 1, where trand at corresponds to lower threshold for medium risk category and window size forthe medium risk category respectively. The upper threshold for the medium risk category is obtained by summation of and a).minimize f(l - a) (Nmedl"m (tl’t2)) - a ( -rP(tl,tz)- + - ™1?Jk »totn / ) VTPCtj.tz) + FPCh.tz) rjvcti.tz) + FJVCtLtz) / ). Equation 1

[0033] With respect to Equation 1, t2is the upper threshold for the medium category, TP(ti, t2) is the number of True Positives given (t1(t2) thresholds, TN tltt2) is the number of True Negatives given (t1(t2) thresholds, FP tt, t2~) is the number of False Positives given (ti, t2) thresholds,t2) is the number of False Negatives given (t1;t2) thresholds, ^medium (ti.is the number of samples in the medium category givent2) thresholds, and N totai is the total number of samples, a and (1 — a) are the weights to manage the inherent tradeoff between the performance and the samples in the medium category.

[0034] A larger window size a) results in a wider medium range, increasing the number of undetermined samples but boosting confidence in predicting the low and high categories. Conversely, a smaller at reduces the number of undetermined samples, but this may come at the cost of the overall performance. The parameter a manages this trade-off: a higher a emphasizes better performance, though it also increases the number of samples in the medium category. To find the optimal balance, both a and (1 — a) must be tuned to maximize their respective objectives.

[0035] The optimal values of parameters ti, ®, and a for output discretization is determined by the system 102, and based on these parameters, the thresholds of the minimum risk category are determined by the system 102 for discretizing the predictions made by the data model. In certain embodiments, the parameters ti, co, and a may be determined during the training stage of the data model, and may be fine-tuned based on different training cohorts. The discretization of the predictions leads to improved performance of the data model, and clinically actionable recommendations may be provided by the system 102 based on the risk of sepsis predicted by the data model for a patient for whom the trained and tested data model is implemented. In an embodiment of the present invention, if a patient is predicted to have low risk of sepsis, it isrecommended to defer antimicrobials and explore other diagnosis, thereby identifying a subpopulation for whom antimicrobial use can be cautiously avoided. This strategy supports effective antimicrobial stewardship, helping to prevent unnecessary antimicrobial exposure and reduces the risk of antimicrobial resistance development. The patients predicted with high risk of sepsis can be confidently triaged, and the recommendation is to promptly initiate antimicrobial administration. This enables the identification of subpopulation that will benefit most from immediate intervention. For the medium risk category, further investigations with sepsis specific tests (CRP, PCT, IL-6) and close monitoring are recommended.

[0036] In certain embodiments of the present invention, if the data model assigns a medium or high risk for sepsis to the patient, supplementary information is retrieved by the system 102 from the plurality of data sources 104 regarding comorbidities and the patient's history of hospital admission. Such information is then forwarded to an additional model, which considers the sepsis risk score provided by the data model along with the presence of comorbidities, in order to estimate the 28-day mortality risk.

[0037] In an exemplary embodiment of the present invention, the system 102 is used to develop the data model for determining sepsis risk score based on data relating to patients admitted in a hospital or medical care center. The data relating to the patients of the hospital is retrieved from the plurality of data sources 104 which include databases maintained by the hospital storing medical data and history of its patients. The plurality of data sources 104 may also include medical instruments used for monitoring the vitals of the patients, and servers used for accessing results of lab tests performed for the patients of the hospital. The system 102 identifies patients in the emergency department of the hospital at risk of sepsis based on routinely orders test, lab results such as CBC and CMP, and vital parameters such as temperature and blood pressure retrieved from the plurality of data sources.

[0038] The system 102 consolidates routine measurements such as lab results and vitals obtained just prior to transfer out of the emergency department for developing the data model. The system 102 selects a set of sepsis positive patients and a set of sepsis negative patients from the set of patients in emergency department of the hospital. The set of sepsis positive patients include patients identified as having severe sepsis. The patients having severe sepsis are identified by thesystem based on patients for whom SIRS criteria, organ dysfunction criteria, and either of blood culture or antibiotic administration should be met between ER arrival and 48-hours post hospital admission. The system 102 also removes patients tagged with ICD codes of hospital acquired infections from the sepsis positive dataset.

[0039] The set of sepsis negative patients are identified by the system 102 from the patients of the emergency department who do not meet the severe sepsis criteria. The sepsis negative dataset is determined by the system 102 after removing any patient with ICD codes related to sepsis and removing patients who received antibiotics during the hospital stay. A total of 43 features including CBC, CMP and vital signs is extracted from the plurality of data sources by the system for training the data model. Additional handcrafted features are also used for training the data model, where the handcrafted features are derived from the parameters that are calculated and used in the routine clinical practice and shown in literature to be informative for identifying sepsis infection.

[0040] Further, the missingness analysis is performed by the system 102 for removing parameters that were missing in more than 30% of the total patients, and the Pearson correlation analysis is performed on the remaining parameters, resulting in elimination of parameters that are highly correlated with other parameters. Based on such operations performed by the system 102, 15 features are selected for training from the initial set of 43 features.

[0041] A total of 1657 sepsis positive and more than 5,000 sepsis negative patients are used by the system 102 for developing and validating the data model, along with the selected 15 parameters, based on the feature importance. 15% of the total positives are held out by the system 102 for model testing, leaving 1345 sepsis positive patients for training the data model. As the sampled dataset is hugely imbalanced, 2690 sepsis negative patients are selected by the system 102 to have a positive to negative ratio of 1:2. From the selected 2690 sepsis negative patients, 1345 patients are identified as satisfying SIRS criteria but non-septic, and 673 patients are tagged with Sepsis mimicking disease ICD Codes / ICD Code prefixes and identified as failing SIRS criteria. The remainder of 672 patients are identified as failing SIRS criteria and not tagged with sepsis mimicking disease ICD Codes I ICD Code prefixes. The final training set thus obtained by the system 102 comprises 1345 sepsis positive patients and 2690 diverse negative patients. Fig. 3illustrates a split-up of patients used for a training data set and a testing data set of the data model, in accordance with an embodiment of the present invention. The data model used in the exemplary embodiment is a voting ensemble classifier, comprising of one or more Light GBM, one or more XGBoost, and one or more Extra Trees classifiers were trained using the scikit-learn package integrated with python3. For example, the voting ensemble classifier comprises 2-6 Light GBM, 1-4 XGBoost, and 3-7 extra trees classifiers.

[0042] The trained ML model exhibits a specificity, sensitivity, PPV and NPV of 86.86%, 80.13%, 85.91% and 81.38% respectively. The test set consist of a total of 612 patients with “balanced” number of sepsis positive patients (312) and sepsis negative patients (312) that were held out from the training set, i.e., patients that the model has encountered before. Further, 50% (156) of the sepsis negative patients of the testing set are comprised patients satisfying the SIRS criteria. More than 70% of patients are tagged as negative by the data model, signifying that the data model has learnt to differentiate between patients satisfying SIRS criteria and are sepsis positive and patients satisfying SIRS criteria and are sepsis negative, which is one of the difficulties faced by clinicians. The test set also consists of 29% of patients (90) tagged with ICD codes associated with diseases that exhibit similar symptoms as sepsis, and it is seen that more than 95% of them were tagged by the data model as negative, indicating learnability of the data model. Additionally, the data model is also tested on a set where the positives and negatives were obtained from a held out set in the ratio of disease prevalence at 3% of positives (256) and 97% (8277) of negatives. The data model exhibits a specificity, sensitivity, PPV and NPV of 98.26%, 80.07%, 58.74% and 99.37% respectively. Fig. 4 illustrates an AUROC and AUPRC plot for a balanced test set and a prevalence test set, in accordance with an embodiment of the present invention. Fig.4 shows the AUROC and AUPRC plots for both Balanced and Prevalence test sets, with 95% confidence interval, computed on 50 different overlapping test sets extracted from the held out set that the data model has not encountered.

[0043] The sepsis risk predictions obtained as an output of testing of the data model is discretized into low, medium, and high-risk categories for converting the predictions into clinically interpretable formats. The thresholds for discretization were determined through BayesianOptimization using Gaussian Processes. t1, α, ω were the searched for minimizing the objective function represented by equation 1, as follows:N medium (Xl» ^2) minimize ] (1 — a)N total TP(tlttz) TN (tt, t2)TP(ti, t2) + FP(tr, t2) TN(tr, t2) + FN(tt, t2)... Equation 1

[0044] In the present exemplary embodiments, minimizing of the objective function is subjected to the following constraints:• 0 ≤ t1≤ 1• 0.2 ≤ α ≤ 0.8• 0.15 ≤ ω ≤ 0.25Q 2•t2> ti

[0045] The optimal ti, a>, and a for output discretization are found to be 0.412, 0.161, 0.28 respectively. Based on these parameters, the thresholds are determined to be 0.41 and 0.57 for discretizing the predictions. The discretized performance of the data model improved to specificity, sensitivity, PPV, and NPV of 91.33%, 85.98%, 90.43%, and 87.24% respectively. Clinically actionable recommendations are provided based on these outputs, and the clinically actionable recommendations are illustrated in Fig. 5. If a patient is predicted to have low risk of sepsis, it is recommended to defer antimicrobials and explore other diagnosis. The patients predicted with high risk of sepsis can be confidently triaged, and the recommendation is to promptly initiate antimicrobial administration. For the medium risk category, further investigations with sepsis specific tests (CRP, PCT, IL-6) and close monitoring are recommended. Table 1 illustrates the performance of the data model after discretizing of sepsis risk scores.Test set (Balanced) Test set (Prevalence) hreshold at jcretized Performance (12. hreshold at icretized Performance (2.< in Med) in Med)Fl Scon sisiliiii iiiiiiAccurac 83.49% 88.05% 97.71% 98.57%Specifier jisliiiii liiliiiiliM iillill 98.90% Sensitivii 80.13% 85.98% 80.07% 85.91%Olllf I 99.63%PPV 85.91% 89.02% 58.74% 67.28%

[0046] Figs. 6(a) and 6(b) illustrate outputs of the system, in accordance with an embodiment of the present invention. The outputs generated by the system 102 may include the sepsis risk score determined by the data model, the category of risk, the clinical recommendations provided corresponding to the category of risk, and a 30-day mortality risk determined by the additional data model.

[0047] The figures of the disclosure are provided to illustrate some examples of the invention described. The figures are not to limit the scope of the depicted embodiments or the appended claims. Aspects of the disclosure are described herein with reference to the invention to example embodiments for illustration. It should be understood that specific details, relationships, and method are set forth to provide a full understanding of the example embodiments. One of ordinary skill in the art recognize the example embodiments can be practiced without one or more specific details and / or with other methods.

[0048] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any disclosures or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular disclosures. Certain features that are described herein in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub combination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a sub combination or variation of a sub combination.

[0049] Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

[0050] It is to be understood that the disclosure is not to be limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation, unless described otherwise.

[0051] It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, components, regions, layers, and / or sections, these elements, components, regions, layers, and / or sections, should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, the term "and / or," includes any and all combinations of one or more of the associated listed items. The phrase "at least one of" has the same meaning as "and / or".

[0052] Spatially relative terms, such as “beneath,” “below,” “lower,” “under,” “above,” “upper,” and the like, may be used herein for ease of description to describe one element or feature’s relationship to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as “below,” “beneath,” or “under,” other elements or features would then be oriented “above” the other elements or features. Thus,the example terms “below” and “under” may encompass both an orientation of above and below. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly. In addition, when an element is referred to as being “between” two elements, the element may be the only element between the two elements, or one or more other intervening elements may be present.

[0053] Spatial and functional relationships between elements (for example, between modules) are described using various terms, including “on," "connected,” “engaged,” “interfaced,” and “coupled.” Unless explicitly described as being “direct,” when a relationship between first and second elements is described in the disclosure, that relationship encompasses a direct relationship where no other intervening elements are present between the first and second elements, and also an indirect relationship where one or more intervening elements are present (either spatially or functionally) between the first and second elements. In contrast, when an element is referred to as being "directly” on, connected, engaged, interfaced, or coupled to another element, there are no intervening elements present. Other words used to describe the relationship between elements should be interpreted in a like fashion (e.g., "between," versus "directly between," "adjacent," versus "directly adjacent," etc.).

[0054] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms "a," "an," and "the," are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the terms “and / or” and “at least one of’ include any and all combinations of one or more of the associated listed items. It will be further understood that the terms "comprises," "comprising," "includes," and / or "including," when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items. Expressions such as “at least one of,” when preceding a list of elements, modify the entire list of elements and do not modify the individual elements of the list. Also, the term “example” is intended to refer to an example or illustration.

[0055] It should also be noted that in some alternative implementations, the functions / acts noted may occur out of the order noted in the figures. For example, two figures shown in succession may in fact be executed substantially concurrently or may sometimes be executed in the reverse order, depending upon the functionality / acts involved.

[0056] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which example embodiments belong. It will be further understood that terms, e.g., those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0057] It is noted that some example embodiments may be described with reference to acts and symbolic representations of operations (e.g., in the form of flow charts, flow diagrams, data flow diagrams, structure diagrams, block diagrams, etc.) that may be implemented in conjunction with units and / or devices discussed above. Although discussed in a particular manner, a function or operation specified in a specific block may be performed differently from the flow specified in a flowchart, flow diagram, etc. For example, functions or operations illustrated as being performed serially in two consecutive blocks may actually be performed simultaneously, or in some cases be performed in reverse order. Although the flowcharts describe the operations as sequential processes, many of the operations may be performed in parallel, concurrently or simultaneously. In addition, the order of operations may be re-arranged. The processes may be terminated when their operations are completed, but may also have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0058] Specific structural and functional details disclosed herein are merely representative for purposes of describing example embodiments. The present invention may, however, be embodied in many alternate forms and should not be constmed as limited to only the embodiments set forth herein.

[0059] In addition, or alternative, to that discussed above, units and / or devices according to one or more example embodiments may be implemented using hardware, software, and / or acombination thereof. For example, hardware devices may be implemented using processing circuitry such as, but not limited to, a processor, Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a System-on-Chip (SoC), a programmable logic unit, a microprocessor, or any other device capable of responding to and executing instructions in a defined manner. Portions of the example embodiments and corresponding detailed description may be presented in terms of software, or algorithms and symbolic representations of operation on data bits within a computer memory. These descriptions and representations are the ones by which those of ordinary skill in the art effectively convey the substance of their work to others of ordinary skill in the art. An algorithm, as the term is used here, and as it is used generally, is conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of optical, electrical, or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0060] It should be borne in mind that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise, or as is apparent from the discussion, terms such as “processing” or “computing” or “calculating” or “determining” of “displaying” or the like, refer to the action and processes of a computer system, or similar electronic computing device / hardware, that manipulates and transforms data represented as physical, electronic quantities within the computer system’s registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

[0061] In this application, including the definitions below, the term ‘module’ or the term ‘controller’ may be replaced with the term ‘circuit.’ The term ‘module’ may refer to, be part of, or include processor hardware (shared, dedicated, or group) that executes code and memory hardware (shared, dedicated, or group) that stores code executed by the processor hardware.

[0062] The module may include one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces that are connected to a local area network (LAN), the Internet, a wide area network (WAN), or combinations thereof. The functionality of any given module of the present disclosure may be distributed among multiple modules that are connected via interface circuits. For example, multiple modules may allow load balancing. In a further example, a server (also known as remote, or cloud) module may accomplish some functionality on behalf of a client module.

[0063] Software may include a computer program, program code, instructions, or some combination thereof, for independently or collectively instructing or configuring a hardware device to operate as desired. The computer program and / or program code may include program or computer-readable instructions, software components, software modules, data files, data structures, and / or the like, capable of being implemented by one or more hardware devices, such as one or more of the hardware devices mentioned above. Examples of program code include both machine code produced by a compiler and higher level program code that is executed using an interpreter.

[0064] For example, when a hardware device is a computer processing device (e.g., a processor, Central Processing Unit (CPU), a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a microprocessor, etc.), the computer processing device may be configured to carry out program code by performing arithmetical, logical, and input / output operations, according to the program code. Once the program code is loaded into a computer processing device, the computer processing device may be programmed to perform the program code, thereby transforming the computer processing device into a special purpose computer processing device. In a more specific example, when the program code is loaded into a processor, the processor becomes programmed to perform the program code and operations corresponding thereto, thereby transforming the processor into a special purpose processor.

[0065] Software and / or data may be embodied permanently or temporarily in any type of machine, component, physical or virtual equipment, or computer storage medium or device, capable of providing instructions or data to, or being interpreted by, a hardware device. The software also may be distributed over network coupled computer systems so that the software isstored and executed in a distributed fashion. In particular, for example, software and data may be stored by one or more computer readable recording mediums, including the tangible or non-transitory computer-readable storage media discussed herein.

[0066] Even further, any of the disclosed methods may be embodied in the form of a program or software. The program or software may be stored on a non-transitory computer readable medium and is adapted to perform any one of the aforementioned methods when run on a computer device (a device including a processor). Thus, the non-transitory, tangible computer readable medium, is adapted to store information and is adapted to interact with a data processing facility or computer device to execute the program of any of the above mentioned embodiments and / or to perform the method of any of the above mentioned embodiments.

[0067] Example embodiments may be described with reference to acts and symbolic representations of operations (e.g., in the form of flow charts, flow diagrams, data flow diagrams, structure diagrams, block diagrams, etc.) that may be implemented in conjunction with units and / or devices discussed in more detail below. Although discussed in a particular manner, a function or operation specified in a specific block may be performed differently from the flow specified in a flowchart, flow diagram, etc. For example, functions or operations illustrated as being performed serially in two consecutive blocks may actually be performed simultaneously, or in some cases be performed in reverse order.

[0068] According to one or more example embodiments, computer processing devices may be described as including various functional units that perform various operations and / or functions to increase the clarity of the description. However, computer processing devices are not intended to be limited to these functional units. For example, in one or more example embodiments, the various operations and / or functions of the functional units may be performed by other ones of the functional units. Further, the computer processing devices may perform the operations and / or functions of the various functional units without sub-dividing the operations and / or functions of the computer processing units into these various functional units.

[0069] Units and / or devices according to one or more example embodiments may also include one or more storage devices. The one or more storage devices may be tangible or non-transitorycomputer-readable storage media, such as random access memory (RAM), read only memory (ROM), a permanent mass storage device (such as a disk drive), solid state (e.g., NAND flash) device, and / or any other like data storage mechanism capable of storing and recording data. The one or more storage devices may be configured to store computer programs, program code, instructions, or some combination thereof, for one or more operating systems and / or for implementing the example embodiments described herein. The computer programs, program code, instructions, or some combination thereof, may also be loaded from a separate computer readable storage medium into the one or more storage devices and / or one or more computer processing devices using a drive mechanism. Such separate computer readable storage medium may include a Universal Serial Bus (USB) flash drive, a memory stick, a Blu-ray / DVD / CD-ROM drive, a memory card, and / or other like computer readable storage media. The computer programs, program code, instructions, or some combination thereof, may be loaded into the one or more storage devices and / or the one or more computer processing devices from a remote data storage device via a network interface, rather than via a local computer readable storage medium. Additionally, the computer programs, program code, instructions, or some combination thereof, may be loaded into the one or more storage devices and / or the one or more processors from a remote computing system that is configured to transfer and / or distribute the computer programs, program code, instructions, or some combination thereof, over a network. The remote computing system may transfer and / or distribute the computer programs, program code, instructions, or some combination thereof, via a wired interface, an air interface, and / or any other like medium.

[0070] The one or more hardware devices, the one or more storage devices, and / or the computer programs, program code, instructions, or some combination thereof, may be specially designed and constructed for the purposes of the example embodiments, or they may be known devices that are altered and / or modified for the purposes of example embodiments.

[0071] A hardware device, such as a computer processing device, may run an operating system (OS) and one or more software applications that run on the OS. The computer processing device also may access, store, manipulate, process, and create data in response to execution of the software. For simplicity, one or more example embodiments may be exemplified as a computer processing device or processor; however, one skilled in the art will appreciate that a hardware device may include multiple processing elements or processors and multiple types of processingelements or processors. For example, a hardware device may include multiple processors or a processor and a controller. In addition, other processing configurations are possible, such as parallel processors.

[0072] The computer programs include processor-executable instructions that are stored on at least one non-transitory computer-readable medium (memory). The computer programs may also include or rely on stored data. The computer programs may encompass a basic input / output system (BIOS) that interacts with hardware of the special purpose computer, device drivers that interact with particular devices of the special purpose computer, one or more operating systems, user applications, background services, background applications, etc. As such, the one or more processors may be configured to execute the processor executable instructions.

[0073] The computer programs may include: (i) descriptive text to be parsed, such as HTML (hypertext markup language) or XML (extensible markup language), (ii) assembly code, (iii) object code generated from source code by a compiler, (iv) source code for execution by an interpreter, (v) source code for compilation and execution by a just-in-time compiler, etc. As examples only, source code may be written using syntax from languages including C, C++, C#, Objective-C, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, Javascript®, HTML5, Ada, ASP (active server pages), PHP, Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, and Python®.

[0074] Further, at least one example embodiment relates to the non-transitory computer-readable storage medium including electronically readable control information (processor executable instructions) stored thereon, configured in such that when the storage medium is used in a controller of a device, at least one embodiment of the method may be carried out.

[0075] The computer readable medium or storage medium may be a built-in medium installed inside a computer device main body or a removable medium arranged so that it can be separated from the computer device main body. The term computer-readable medium, as used herein, does not encompass transitory electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); the term computer-readable medium is therefore considered tangible and non-transitory. Non-limiting examples of the non-transitory computer-readable mediuminclude, but are not limited to, rewriteable non-volatile memory devices (including, for example flash memory devices, erasable programmable read-only memory devices, or a mask read-only memory devices); volatile memory devices (including, for example static random access memory devices or a dynamic random access memory devices); magnetic storage media (including, for example an analog or digital magnetic tape or a hard disk drive); and optical storage media (including, for example a CD, a DVD, or a Blu-ray Disc). Examples of the media with a built-in rewriteable non-volatile memory, include but are not limited to memory cards; and media with a built-in ROM, including but not limited to ROM cassettes; etc. Furthermore, various information regarding stored images, for example, property information, may be stored in any other form, or it may be provided in other ways.

[0076] The term code, as used above, may include software, firmware, and / or microcode, and may refer to programs, routines, functions, classes, data structures, and / or objects. Shared processor hardware encompasses a single microprocessor that executes some or all code from multiple modules. Group processor hardware encompasses a microprocessor that, in combination with additional microprocessors, executes some or all code from one or more modules. References to multiple microprocessors encompass multiple microprocessors on discrete dies, multiple microprocessors on a single die, multiple cores of a single microprocessor, multiple threads of a single microprocessor, or a combination of the above.

[0077] Shared memory hardware encompasses a single memory device that stores some or all code from multiple modules. Group memory hardware encompasses a memory device that, in combination with other memory devices, stores some or all code from one or more modules.

[0078] The term memory hardware is a subset of the term computer-readable medium. The term computer-readable medium, as used herein, does not encompass transitory electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); the term computer-readable medium is therefore considered tangible and non-transitory. Non-limiting examples of the non-transitory computer-readable medium include, but are not limited to, rewriteable non-volatile memory devices (including, for example flash memory devices, erasable programmable read-only memory devices, or a mask read-only memory devices); volatile memory devices (including, for example static random access memory devices or a dynamic random accessmemory devices); magnetic storage media (including, for example an analog or digital magnetic tape or a hard disk drive); and optical storage media (including, for example a CD, a DVD, or a Blu-ray Disc). Examples of the media with a built-in rewriteable non-volatile memory, include but are not limited to memory cards; and media with a built-in ROM, including but not limited to ROM cassettes; etc. Furthermore, various information regarding stored images, for example, property information, may be stored in any other form, or it may be provided in other ways.

[0079] The apparatuses and methods described in this application may be partially or fully implemented by a special purpose computer created by configuring a general purpose computer to execute one or more particular functions embodied in computer programs. The functional blocks and flowchart elements described above serve as software specifications, which can be translated into the computer programs by the routine work of a skilled technician or programmer.

[0080] Although described with reference to specific examples and drawings, modifications, additions and substitutions of example embodiments may be variously made according to the description by those of ordinary skill in the art. For example, the described techniques may be performed in an order different with that of the methods described, and / or components such as the described system, architecture, devices, circuit, and the like, may be connected or combined to be different from the above-described methods, or results may be appropriately achieved by other components or equivalents.

Claims

We claim:

1. A method for training a data model for predicting risk of sepsis, the method comprising: creating, from a patient database, a first data set indicative of sepsis positive patients, wherein the first data set includes patients having SIRS criteria, organ dysfunction criteria, and suspected infection criteria, and wherein the first data set excludes patients having ICD codes relating to hospital acquired infections;creating, from the patient database, a second data set indicative of sepsis negative patients, wherein the second data set includes patients with no antimicrobial administration, having organ dysfunction within 48 hours of hospital admission post-ED (Emergency Department) discharge, and having SIRS criteria, and wherein the second data set excludes patients having sepsis ICD codes;creating, from the patient database, a third data set indicative of sepsis negative patients, wherein the third data set includes patients with no antimicrobial administration, having organ dysfunction within 48 hours of hospital admission post- ED discharge, failing SIRS criteria, and having at least one ICD code of sepsis mimicking diseases, and wherein the third data set excludes patients having sepsis ICD codes;creating, from the patient database, a fourth data set indicative of sepsis negative patients, wherein the fourth data set includes patients with no antimicrobial administration, have organ dysfunction within 48 hours of hospital admission post-ED discharge, and failing SIRS criteria, and wherein the fourth data set excludes patients having ICD codes of sepsis mimicking diseases and sepsis ICD codes;creating a training data selected from the first, second, third, and fourth data sets; training a data model based on the training data selected from the first data set, the second data set, the third data set, and the fourth data set in a ratio of 2:2: 1: 1;creating testing data containing data of patients from the first, second, third and fourth data set held out from the training data, wherein half of the testing data comprises data of patients from the first data set and remaining half of the testing data comprises data of patients from the second, third, and fourth data sets; andtraining and testing the data model based on the training data and testing data respectively, wherein the trained and tested data model is used for predicting risk of sepsis in patients;wherein the predictions of the data model corresponding to the testing data are discretized into different categories of risk;wherein the data model provides clinically actionable recommendations for a patient based on the category of risk determined for the patient; andwherein the data model identifies sub-populations of patients likely to benefit from antimicrobial therapy.

2. The method as claimed in claim 1, wherein the data model is a voting ensemble classifier.

3. The method as claimed in claim 1, wherein the second data set, third data set, and fourth data set constitute sepsis negative patients and the first data set constitutes sepsis positive patients.

4. The method as claimed in claim 1, wherein the patient database includes databases maintained by hospitals or healthcare centres for storing data relating to patients.

5. The method as claimed in claim 1, wherein the data set stored in the patient database includes data relating to routinely ordered tests, lab results, vital parameters, demographic parameters, information regarding comorbidities of patients, and medication information of patients.

6. The method as claimed in claim 1, wherein the predictions of the data model are discretized into different categories based on thresholds determined through Bayesian optimization using gaussian processes.

7. The method as claimed in claim 1, wherein the risk of sepsis predicted by the data model is indicated via a sepsis score.

8. The method as claimed in claim 1, wherein suspected infection criteria relates to suspicion of infection within 48 hours of hospital admission post-ED (Emergency Department) discharge.

9. The method as claimed in claim 1, wherein organ dysfunction criteria relates to exhibition of symptoms of organ dysfunction.

10. The method as claimed in claim 7, wherein the sepsis risk score of a patient and supplementary information regarding comorbidities of the patient is used by a second data model for estimating a 28 -day mortality risk of the patient.

11. A system for training a data model for predicting risk of sepsis, the system comprising: processing circuitry configured to cause the system to,create, from a patient database, a first data set indicative of sepsis positive patients, wherein the first data set includes patients having SIRS criteria, organ dysfunction criteria, and suspected infection criteria, and wherein the first data set excludes patients having ICD codes relating to hospital acquired infections;create, from the patient database, a second data set indicative of sepsis negative patients, wherein the second data set includes patients with no antimicrobial administration, having organ dysfunction within 48 hours of hospital admission post-ED (Emergency Department) discharge, and having SIRS criteria, and wherein the second data set excludes patients having sepsis ICD codes;create, from the patient database, a third data set indicative of sepsis negative patients, wherein the third data set includes patients with no antimicrobial administration, having organ dysfunction within 48 hours of hospital admission post-ED discharge, failing SIRS criteria, and having at least one ICD code of sepsis mimicking diseases, and wherein the third data set excludes patients having sepsis ICD codes;create, from the patient database, a fourth data set indicative of sepsis negative patients, wherein the fourth data set includes patients with no antimicrobial administration, have organ dysfunction within 48 hours of hospital admission post-ED discharge, and failing SIRS criteria, and wherein the fourth data set excludes patients having ICD codes of sepsis mimicking diseases and sepsis ICD codes;create a training data selected from the first, second, third, and fourth data sets;train a data model based on the training data selected from the first data set, the second data set, the third data set, and the fourth data set in a ratio of 2:2: 1: 1;create testing data containing data of patients from the first, second, third and fourth data set held out from the training data, wherein half of the testing data comprises data of patients fromthe first data set and remaining half of the testing data comprises data of patients from the second, third, and fourth data sets; andtrain and test the data model based on the training data and testing data respectively, wherein the trained and tested data model is used for predicting risk of sepsis in patients; wherein the predictions of the data model corresponding to the testing data are discretized into different categories of risk;wherein the data model provides clinically actionable recommendations for a patient based on the category of risk determined for the patient; andwherein the data model identifies sub-populations of patients likely to benefit from antimicrobial therapy.

Images