Method, device, and computer program for predicting whether to perform important intervention on basis of machine learning

Abstract

The present invention relates to a method for predicting whether to perform an important intervention on the basis of machine learning, the method including the steps of: receiving an input of vital sign data of one or more patients, demographic data, arrival path data, and data on whether to perform an important intervention after arriving at an emergency room as training data for training a prediction model; training the prediction model by using the training data so that the prediction model outputs whether to perform an important intervention in response to receiving the input of vital sign data, demographic data, and arrival path data; and inputting vital sign data, demographic data, and patient arrival data of an emergency patient to the trained prediction model to output the degree to which performing an important intervention is necessary, the necessity thereof, and the number of necessary important interventions.

Description

Method, device, and computer program for predicting whether to implement a machine learning-based critical intervention

[0001] This invention was made with the support of the Ministry of Science and ICT under the project identification number 1711182190 and project number 2022R1A2C3004595. The research management specialized organization of the project is the National Research Foundation of Korea, the research project name is "Basic Research Project in Science and Engineering", the research project title is "Research on Development, Application, and Diffusion of AI Decision Support Platform for Improving Patient Safety in Emergency Medicine", the main organization is Sungkyunkwan University, and the research period is from March 1, 2022 to February 28, 2027.

[0002] Rapid recognition of the need for intervention and subsequent follow-up are essential for providing appropriate care, especially to critically ill patients in the emergency department (ED). To facilitate this, triage systems capable of rapidly identifying critically ill patients, such as the Quick Sequential Organ Failure Assessment (qSOFA), and the Modified Early Warning Score (MEWS), are currently being implemented, which can be calculated immediately upon patient arrival at the ED. Furthermore, with the advent of artificial intelligence (AI), several formal studies are developing AI-based clinical decision support systems (CDSS).

[0003] In this regard, Korean Patent No. 10-2450646 discloses a system and method for diagnosing emergency room patients using a big data-based deep learning model. This prior art document features training an AI deep learning model using questionnaire results, treatment-related big data, patient medical history, and vital signs, and using the trained AI deep learning model to diagnose emergency patients.

[0004] As in the above-mentioned prior literature, there are many technologies disclosed for diagnosing emergency room patients using AI. However, there is no technology disclosed that can predict whether to perform important interventions based only on data that can be collected when a patient arrives at the emergency room, without time-consuming tests (e.g., x-ray, ultrasound, MRI, CT, etc.).

[0005] Therefore, there is a need for a machine learning-based method, device, and computer program that can quickly determine whether to implement critical interventions upon arrival of a critically ill patient to the emergency room.

[0006] The purpose of the present invention is to provide a method, device and computer program for predicting whether to perform a critical intervention based on machine learning, which can predict whether to perform a critical intervention using only data that can be acquired during the process of being transferred to an emergency room.

[0007] In order to achieve the above object, the present invention is characterized by comprising a step of inputting vital sign data, demographic data, arrival route data, and data on whether to implement a critical intervention after arrival at an emergency room as training data for training a prediction model; a step of training a prediction model using the training data so that the prediction model outputs whether to implement a critical intervention in response to inputting the vital sign data, demographic data, and arrival route data; and a step of inputting the vital sign data, demographic data, and patient arrival data of an emergency patient into the trained prediction model to output the degree of necessity for implementing a critical intervention, whether it is necessary, and the number of necessary critical interventions to be implemented.

[0008] Preferably, the input receiving step may input only vital sign data, demographic data, and arrival route data collected before the patient arrives at the emergency room as learning data.

[0009] Preferably, the input receiving step may receive at least one of body temperature (BT), heart rate (HR), respiration rate (RR), systolic blood pressure (SBP), and diastolic blood pressure (DBP) as the vital sign data.

[0010] Preferably, the input receiving step may receive at least one of age and gender as the demographic data.

[0011] Preferably, the input receiving step may receive at least one of the means of transportation used to arrive at the emergency room and the reason for visiting the emergency room as the arrival route data.

[0012] Preferably, the input receiving step may receive at least one of the following as input: whether to perform an important intervention, whether to insert an arterial line, whether to perform a massive blood transfusion, whether to insert an A-line, whether to use an inotrope, whether to use a vascular compression device, whether to provide labor support, whether to use high-flow nasal cannula therapy (HFNC), and whether to perform tracheal intubation.

[0013] Preferably, the input receiving step may only receive data on whether or not a critical intervention was implemented within 24 hours after admission to the emergency room.

[0014] Preferably, the step of training the prediction model may use an Extreme Gradient Boost Model as the prediction model.

[0015] In addition, the present invention is a device for predicting whether to perform a critical intervention based on machine learning, comprising: a processor including one or more cores; and a memory; wherein the processor receives, as learning data, one or more patient's vital sign data, demographic data, arrival route data, and data on whether to perform a critical intervention after arrival at an emergency room, for learning a prediction model, and trains the prediction model using the learning data so that the prediction model outputs whether to perform the critical intervention in response to receiving the vital sign data, demographic data, and arrival route data, and receives, as the arrival route data, at least one of a means of transportation used to arrive at an emergency room and a reason for visiting the emergency room.

[0016] In addition, the present invention is a computer program including instructions stored in a computer-readable storage medium that cause a computer to perform the following operations, wherein the operations include: an operation of receiving, as learning data, one or more patient's vital sign data, demographic data, arrival route data, and data on whether to perform important intervention after arrival at an emergency room; and an operation of training the prediction model using the learning data, such that the prediction model outputs whether to perform the important intervention in response to receiving the vital sign data, demographic data, and arrival route data; and another feature of the inputting operation is that only vital sign data, demographic data, and arrival route data collected before the patient arrives at the emergency room are input as learning data.

[0017] The present invention has the advantage of being able to predict whether a critically ill patient admitted to an emergency room needs to undergo a critical intervention using only data that can be obtained prior to arrival at the emergency room without the need for time-consuming examination results.

[0018] The present invention has the advantage of being able to provide information that helps doctors determine whether to perform important interventions on critically ill patients admitted to an emergency room, thereby enabling rapid emergency treatment to be provided to critically ill patients.

[0019] Figure 1 illustrates a flowchart of a method for predicting whether to implement a machine learning-based important intervention according to an embodiment of the present invention.

[0020] Figure 2 shows the degree of correlation between learning data and each intervention trial according to an embodiment of the present invention.

[0021] Figure 3 shows the results of a survey conducted on users (medical staff, etc.) to evaluate the user experience according to an embodiment of the present invention.

[0022] Figure 4 shows the distribution of each intervention time before and after application of the present invention in a positive group according to an embodiment of the present invention.

[0023] FIG. 5 shows a configuration diagram of a device for predicting whether to perform a machine learning-based important intervention according to an embodiment of the present invention.

[0024] Figure 6 illustrates a schematic diagram of a computing environment according to an embodiment of the present invention.

[0025] In a method for predicting whether to implement a machine learning-based important intervention,

[0026] A step of inputting vital sign data, demographic data, arrival route data, and data on whether important interventions were implemented after arrival at an emergency room as training data for training a prediction model;

[0027] A step of training the prediction model using the above learning data so that the prediction model outputs whether to implement a critical intervention in response to inputting vital sign data, demographic data, and arrival route data; and

[0028] A step of inputting vital sign data, demographic data, and patient arrival data of an emergency patient into a learned prediction model to output whether or not a critical intervention is implemented;

[0029] A method comprising:

[0030] Hereinafter, the present invention will be described in detail with reference to the contents described in the attached drawings. However, the present invention is not limited or restricted by the exemplary embodiments. The same reference numerals in each drawing indicate components that perform substantially the same functions.

[0031] The purpose and effects of the present invention can be naturally understood or made clearer by the following description, and the purpose and effects of the present invention are not limited solely by the following description. Furthermore, in describing the present invention, if a detailed description of known technologies related to the present invention is deemed to unnecessarily obscure the gist of the present invention, such detailed description will be omitted.

[0032] The terminology used herein is merely used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the description of the invention, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0033] While terms like "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0034] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0035] When interpreting components, even if there is no explicit description, it is interpreted as including the margin of error. When describing temporal relationships, for example, when temporal continuity is described with phrases such as "after," "following," "next to," or "before," this also includes cases where the relationship is not continuous, unless "immediately" or "directly" is used.

[0036] Hereinafter, the technical configuration of the present invention will be described in detail with reference to the attached drawings.

[0037] Figure 1 illustrates a flowchart of a machine learning-based method for predicting whether a critical intervention will be implemented according to an embodiment of the present invention. Referring to Figure 1, the machine learning-based method for predicting whether a critical intervention will be implemented may include an input receiving step (S100), a prediction model training step (S300), and an outputting step (S500).

[0038] A machine learning-based method for predicting whether critical interventions should be administered can be used to predict whether critical interventions should be administered to critically ill patients admitted to the emergency room, using only data available before arrival, without the need for time-consuming tests. This machine learning-based method can provide physicians with valuable information to help them determine whether critical interventions should be administered to critically ill patients admitted to the emergency room, thereby enabling rapid emergency treatment.

[0039] The method for predicting whether to implement a critical intervention based on machine learning according to the present invention can be used as one of clinical decision support systems (CDSS) for identifying patients requiring intervention in the classification stage.

[0040] The input receiving step (S100) may receive one or more patients' vital signs, demographic data, arrival route data, and data on whether important interventions were implemented after arrival at the emergency room as training data for training a predictive model. The input receiving step (S100) may obtain the training data from an Electronic Medical Record (EMR).

[0041] Figure 2 illustrates the correlation between learning data and each intervention trial according to an embodiment of the present invention. The following description will be provided with reference to Figure 2.

[0042] In addition to training data, the input receiving step (S100) can receive data to predict whether important interventions will be implemented for actual emergency patients. For example, the input receiving step (S100) can receive time information, including the patient's arrival time in the emergency room, admission time, and discharge time. In addition, the input receiving step (S100) can receive the Korean Triage and Acuity Scale (KTAS) score, quick sequential organ failure assessment (qSOFA) score, and final disposition information for each patient. The EMR can receive and update information about emergency patients in real time. The input receiving step (S100) can continuously update information by receiving information about emergency patients from the EMR every 10 minutes.

[0043] The input receiving step (S100) can input only vital sign data, demographic data, and arrival route data collected before the patient arrives at the emergency room as learning data.

[0044] For critically ill patients transported to the emergency room (ER), the speed with which necessary interventions are implemented after arrival can significantly impact the patient's treatment and survival. If tests such as CT or MRI are performed after arrival to determine whether critical interventions should be implemented, this can delay the time needed to implement critical interventions. Furthermore, if the critical intervention is determined solely based on the medical staff's knowledge and experience, the intervention may be inappropriate. Therefore, the present invention constructs training data solely from data collected before the patient's arrival at the ER, enabling the prediction model to quickly and accurately predict whether critical interventions should be implemented after arrival.

[0045] The input receiving step (S100) may receive at least one of body temperature (BT), heart rate (HR), respiration rate (RR), systolic blood pressure (SBP), and diastolic blood pressure (DBP) as vital sign data.

[0046] Vital sign data can be easily measured when an emergency patient is transported to the emergency room via ambulance or other means. Furthermore, even if a patient visits the emergency room on their own, the data can be measured within minutes within the emergency room. Furthermore, as shown in Figure 2, vital sign data such as SBP, DBP, PR, and RR are correlated with the implementation of each intervention. In particular, RR is highly correlated with the use of oxygen therapy and HFNC. Therefore, in one embodiment of the present invention, vital sign data can be easily and quickly measured by training a predictive model or inputting data into a predictive model for prediction, and can be used to determine whether each intervention is implemented.

[0047] The input step (S100) may receive demographic data, at least one of age and gender. Demographic data pertains to the personal information of the emergency patient and can be easily obtained through an ID card or the patient's medical records. Furthermore, as can be seen in Figure 2, demographic data, including age and gender, are related to whether or not each intervention is performed. In particular, age is highly correlated with A-line intubation, oxygen therapy, and MTP. Therefore, in one embodiment of the present invention, demographic data can be easily and quickly acquired as data to be input into a predictive model for training or prediction, and can be used to train a predictive model, and demographic data that is highly related to whether or not each intervention is performed.

[0048] The input receiving step (S100) is arrival route data, and at least one of the means of transportation used to arrive at the emergency room and the reason for visiting the emergency room can be input.

[0049] The means of transportation used to arrive at the emergency room (ER) refers to the actual means of transportation used by emergency patients to reach the ER, including ambulance, private car, and public transportation. This data can be easily obtained by asking patients or from the person transporting them. Furthermore, while relatively mild patients may use public transportation or a private car, severely ill patients often arrive by ambulance. Therefore, the means of transportation used to arrive at the ER is related to the implementation of each intervention, making it a crucial piece of information that can influence the implementation of each intervention. Referring to Figure 2, arrival route data, including whether patients visited in person or were transported by ambulance, are related to the implementation of each intervention.

[0050] Reasons for emergency room visits include self-initiated visits, transfers from the same hospital to an outpatient setting, transfers by ambulance following a 119 or other emergency call, and transfers from other hospitals due to inadequate treatment. Reasons for emergency room visits can be easily obtained by asking the patient, their guardian, or a related healthcare professional. Since transfers from other hospitals are often for critically ill patients, the reason for emergency room visit is relevant to the implementation of various interventions, providing valuable information that can influence the decision to implement key interventions.

[0051] In one embodiment of the present invention, arrival path data that can be easily and quickly acquired as data to be trained as a prediction model or input into a prediction model for prediction, and that is highly relevant to whether or not each intervention is implemented, can be used.

[0052] Table 1 below shows the relationship between the input training data and the patient's severity. Referring to Table 1, we can see that whether a patient visited in person or was transported by ambulance is related to the patient's severity.

[0053] VariablesnUnivariate AnalysisMultivariate AnalysisOR95% CIP-vlaueOR95% CIP-vlaueAGE96,4831.031.03 - 1.03<0.0011.011.00 - 1.01<0.001SEXFemale49,3321(ref)1(ref)Male47,1511.731.61 - 1.85<0.0011.371.26 - 1.49<0.001KTAS346,2781(ref)1(ref)198541.7336.40 - 47.85<0.0014.173.19 - 5.46<0.001278845.485.06 - 5.93<0.0013.282.98 - 3.61<0.0014347530.270.24 - 0.31<0.0010.470.41 - 0.54<0.001565830.090.06 - 0.14<0.0010.180.12 - 0.27<0.001AVPUA93,6521(ref)1(ref)V1,4115.544.76 - 6.44<0.0011.130.92 -1.380.231P91410.218.74 - 11.93<0.0011.080.87 - 1.340.485U506113.2991.48 - 140.31<0.0012.831.78 - 4.50<0.001Ambulance19,6306.426.00 - 6.87<0.0011.871.71 - 2.04<0.001Direct76,3720.440.41 - 0.48<0.0010.580.53 - 0.63<0.001SBP96,4830.980.98 - 0.98<0.0011.001.00 - 1.000.066DBP96,4830.970.96 - 0.97<0.0011.000.99 - 1.000.111PR96,4831.021.02 - 1.02<0.0011.001.00 - 1.000.093RR96,4831.071.07 - 1.07<0.0011.000.99 - 1.010.593TEMP96,4831.051.01 - 1.090.0220.920.88 - 0.97<0.001SPO296,4830.900.90 - 0.91<0.0011.000.99 - 1.010.731Number of Procedure087,4231(ref)1(ref)17,04913.0912.07 - 14.20<0.0014.594.13 - 5.10<0.00121,38152.1146.29 - 58.67<0.00117.3414.90 - 20.18<0.0013519121.07100.18 - 146.30<0.00145.2635.86 - 57.13<0.0014103314.26186.29 - 530.15<0.001217.46113.43 - 416.92<0.00158434.8553.47 - 3536.74<0.001142.2215.54 - 1301.60<0.001.

[0054] The input receiving step (S100) may receive at least one of the following: whether to perform a critical intervention; whether to insert an arterial line; whether to perform a massive blood transfusion; whether to insert an A-line; whether to use an inotrope; whether to use a vascular compression device; whether to provide labor support; whether to use high-flow nasal cannula therapy (HFNC); and whether to perform tracheal intubation. The input receiving step (S100) may receive details of the intervention, such as whether to perform a critical intervention and the timing thereof, as temporal and categorical variables. This is only an example and is not limited to the critical interventions mentioned above; input may be received regarding whether to perform all possible critical interventions.

[0055] Data on the major interventions mentioned above, excluding the presence or absence of massive blood transfusion, are readily available as they are integrated into the electronic medical record system of the secondary hospital.

[0056] The input step (S100) only collects data on whether critical interventions were implemented within 24 hours of emergency room admission. Data for patients under 18 years of age, trauma patients, cardiac arrest patients, and patients who died upon arrival at the hospital can be excluded from the input step (S100).

[0057] Predicting whether critical interventions are needed after 24 hours of emergency room admission is limited by data collected prior to arrival, and accuracy is not guaranteed. Furthermore, since the purpose of the present invention is to rapidly determine whether critical interventions are needed for emergency patients, this also runs counter to the purpose of the present invention. Therefore, embodiments of the present invention can only receive data on whether critical interventions are needed within 24 hours of emergency room admission.

[0058] The step of training the prediction model (S300) can train the prediction model using training data so that the prediction model outputs whether or not to implement a critical intervention in response to inputting vital sign data, demographic data, and arrival route data.

[0059] The step of training the prediction model (S300) can use the Extreme Gradient Boost Model as the prediction model.

[0060] The output step (S500) inputs the vital sign data, demographic data, and patient arrival data of an emergency patient into the learned prediction model to output the degree of necessity for performing a critical intervention, whether it is necessary, and the number of critical interventions required.

[0061] The need for a critical intervention can be expressed as a probability. For example, it could be output as: 80% need for arterial line insertion, 90% need for massive blood transfusion, 40% need for A-line insertion. These need levels can be used by users (medical staff) to determine whether to perform a critical intervention.

[0062] The necessity of implementing a critical intervention can be output as either "necessary" or "unnecessary." The necessity of implementing a critical intervention can be determined by applying a threshold value to the aforementioned probability of necessity. If the probability is higher than the threshold value, the output is "necessary." If the probability is lower than the threshold value, the output is "unnecessary."

[0063] The number of required critical interventions can refer to the number of interventions required, as indicated in the "Need for Critical Interventions" output. The number of required critical interventions can be output along with specific information about the intervention, including the number of required interventions, the name of the intervention, the timing of the intervention, and the intervention method.

[0064] Specifically, the output step (S500) can predict the degree of necessity, necessity, and number of necessary important interventions, such as whether to insert an arterial line, whether to perform massive blood transfusion, whether to insert an A-line, whether to use an inotrope, whether to use a vascular compression device, whether to provide labor support, whether to use high-flow nasal cannula therapy (HFNC), and whether to perform tracheal intubation.

[0065] The output step (S500) can transmit predicted information (the degree of need for critical interventions, whether they are necessary, and the number of critical interventions required) to the EMS. The output step (S500) ultimately enables integrated management of all information in the EMS, thereby increasing medical staff's accessibility to the output information.

[0066] The output step (S500) can provide information to the user (medical staff) to determine whether to perform an important intervention. The output step (S500) can change or accelerate user behavior by providing the user with information to determine whether to perform an important intervention. As a result of examining the changes in decision-making and decision-making speed regarding whether to perform an important intervention after applying the present invention, the number of interventions performed increased in the positive result group after providing the user (medical staff) with the predicted results of the present invention, whereas it decreased in the negative result group. This indicates that the predicted results of the present invention influenced the judgment of the user (medical staff). The action times for four interventions, excluding intubation, inotropes, and vascular compression devices, became faster after applying the present invention (see Table 5 and Figure 4).

[0067] Below, the results of statistical analysis for analyzing the effectiveness of a method for predicting whether to implement a machine learning-based important intervention according to an embodiment of the present invention are described.

[0068] Statistical analysis used descriptive statistics to compare demographic characteristics and time between pre- and post-intervention groups for each intervention. Categorical variables were expressed as counts and percentages relative to the total, and continuous variables were expressed as means and standard deviations. For comparisons between intervention time points, medians and interquartile ranges (IQRs) were used because most data were skewed and not normally distributed. P values ​​were calculated using the Wilcoxon test at the 5% significance level. For medians adjusted for other covariates, quartile regression (e.g., 50th percentile, median regression) was applied, which estimates conditional quartiles and may be more robust to outliers and abnormal data. Estimated medians were compared after adjusting for sex, age, vital signs, and AVPU scale scores.

[0069] Figure 3 presents the results of a survey conducted with users (e.g., medical professionals) to evaluate the user experience according to an embodiment of the present invention. Referring to Figure 3, the survey covered three sections: implementation, content, and system. The implementation section contained three subsections, the content section contained three subsections, and the system section contained two subsections. Each subsection contained multiple questionnaires, and a thorough survey was conducted using open-ended, free-text questions.

[0070] As shown in Figure 3, a total of 18 physicians participated in the survey, with a median of 40.5 months of emergency room experience. Most participants responded positively to the application of the invention and the reliability of its content, but expressed a negative attitude toward the usability sub-item of the system section.

[0071] Statistical analysis was conducted on a total of 37,632 patients. Before the application of the present invention, there were 24,742 emergency room visits. After excluding patients with trauma under the age of 18, patients admitted for the purpose of arrest, and patients who eventually died (DOA), 18,252 patients were ultimately included in the group before the application of the present invention. After the application of the present invention, there were 28,742 emergency room visits. After excluding patients with trauma under the age of 18, patients admitted for the purpose of arrest, and patients who eventually died (DOA), 19,380 patients were ultimately included in the group after the application of the present invention. Table 2 below shows the demographic characteristics of the group before and after the application of the present invention. Here, KTAS refers to the Korea Triage Acuity Scale, SBP refers to systolic blood pressure, and DBP refers to diastolic blood pressure. Additionally, p-values ​​were calculated using the t-test for continuous variables and the chi-square test for categorical variables.

[0072] Pre (n=18,252)Post (n=19,380)PvalueAge (yr)58.0±17.157.9±17.3.75Sex<.01Men9,116 (49.9)9,944 (51.3)Women9,136 (50.1)9,436 (48.7)KTAS<.01140 (0.2)104 (0.5)21,319 (7.2)1,563 (8.1)39,086 (49.8)10,175 (52.5)46,968 (38.2)6,631 (34.2)5839 (4.6)907 (4.7)Method of transportation.05Ambulance2,779 (15.2)2,815 (14.5)Others15,473 (84.8)16,565 (85.5)Route of arrival.02Direct visit13,666 (74.9)14,297 (73.8)Others4,586 (25.1)5,083 (26.2)Vital signsBody temperature (ºC)36.7±0.836.7±0.8.39Heart rate (bpm)90.8±19.991.4±20.1<.01SBP (mmHg)133.3±36.9131.1±38.3<.01DBP (mmHg)80.7±22.379.7±23.7<.01Respiratory rate (breaths / min)18.4±3.618.6±3.6<.01

[0073] Table 3 below shows the AUROC (Area Under the Receiver Operating characteristic Curve), sensitivity, and specificity area after applying the present invention.

[0074] AUROC (95% CI)SensitivitySpecificityA-line0.879 (0.869 - 0.889)0.8550.76Oxygen Therapy0.906 (0.899 - 0.914)0.8520.806HFNC0.95 (0.941 - 0.958)0.9540.835Intubation0.928 (0.905 - 0.951)0.9080.824Inotropes and vasopressors0.88 (0.864 - 0.896)0.8030.805

[0075] Referring to Table 3, the AUROC was 0.879 for A-line insertion, 0.906 for oxygen therapy, 0.95 for HFNC, 0.928 for intubation, and 0.88 for inotropes and vascular compression devices. In all interventions, the present invention exhibited an AUROC of 0.85 or higher.

[0076] Table 4 below presents the results comparing the number of actual interventions before and after the application of the present invention. Here, % represents the proportion of each group in the total population, before and after the application of the present invention, and + indicates that the procedure is recommended to be performed with reference to the predicted results of the present invention.

[0077] preCrIs(%)postCrIs(%)PvalueCris+7884.3178784.530<.01Cris-2301.2601470.760Cris+14397.88417889.226<.01Cris-5092.7892841.468Cris +1410.7733031.563<.01Cris-280.153110.057Cris+810.4441140.588.04Cris-200.110120.062Cris+3071.6823942.033.04Cris-1050.575950.491

[0078] Referring to Table 4, after application of the present invention, the number of interventions increased in the positive group, while the number of interventions decreased in the negative group.

[0079] Table 5 below shows a comparison of the time required for each intervention in minutes before and after the application of the present invention, when the present invention proposes to perform each intervention, i.e., when the present invention is positive (+). Here, ED refers to the Emergency Department, HFNC refers to High-Flow Nasal Cannula, and N refers to the number of interventions performed. In addition, quartile regression analysis was performed to estimate the median after adjusting for age, sex, SBP, DBP, HR, RR, BT, and level of consciousness, which was measured by the alertness, response to verbal stimuli, response to pain stimuli, and no response (AVPU) scales. Each measurement was recorded in minutes (median), and the p-value was calculated using the Wilcoxon test.

[0080] PreCrIs Median Minute [IQR]PostCrIs MedianMinute [IQR]P-valueED admission to A-lineN=78872.3[32.0-222.0]N=87859.2 [27.3-180.0]<.01ED admission to Oxygen TherapyN=14396.3[0.0-139.5]N=17880.0 [0.0-90.9]<.01ED admission to HFNCN=141309.0[69.3-642.5]N=303116.4[38.7-361.6]<.01ED admission to IntubationN=81114.9[58.8-425.9]N=114113.9[19.5-344.2].07ED admission to Inotropes and vasopressorsN=307138.3[42.7-348.6]N=394130.8[40.8-341.3].96After adjustment of Age, Sex, vital signs, and AVPU scaleED admission to A-line80.5[57.2 - 79.4]68.3[68.5 - 92.6]<.01ED admission to Oxygen Therapy13.2[10.7 - 21.2]7.3[89 - 16.5]<.01ED admission to HFNC347.0[237.0-457.0]178.0[101.0-255.0]<.01ED admission to Intubation168.0[104.4-231.0]141.0 [78.1-203].22ED admission to Inotropes and vasopressors155.0[129.0-181.0]161.0[139.0-184.0).57

[0081] As described above, a positive group was selected to observe the effect of the present invention by comparing when the present invention provided results with when it did not. In addition, a quartile regression analysis was performed to estimate the median after adjusting for age, sex, SBP, DBP, HR, RR, BT, and level of consciousness. This was measured using the AVPU scale, which may be another factor affecting the intervention time and the difference between the two groups. All procedures (interventions) were performed within a short period of time after emergency room admission. The time was significantly shortened when arterial line (A-line) therapy, oxygen therapy, and radiofrequency ablation (HFNC) were performed.

[0082] Figure 4 illustrates the distribution of intervention times before and after application of the present invention in a positive group according to an embodiment of the present invention. Referring to Figure 4, it can be seen that, overall, all procedures (interventions) were performed within a shorter period of time after emergency room admission compared to before application of the present invention (green indicates the group before application of the present invention, and red indicates the group after application of the present invention).

[0083] Statistical analysis compared the time required in minutes before and after each intervention, regardless of the results of the present invention, and showed that the median was significantly reduced when A-line administration, HFNC, and intubation were performed.

[0084] Statistical analysis indicates that the group following the application of the present invention had a shorter intervention time overall. The statistical analysis also indicates that the intervention time for the negative group increased in most cases. This suggests that the prediction results of the present invention may influence the judgment of users (medical professionals), as negative outcomes may delay intervention.

[0085] Furthermore, a survey and comparison with the KTAS or qSOFA revealed that the expected number of interventions needed correlated with the patient's severity. Because the present invention can be calculated during patient triage in the early stages of the emergency department, it can also be utilized as an early severity prediction score that can be used in conjunction with the KTAS or qSOFA.

[0086] FIG. 5 is a block diagram of a device (100) for predicting whether to perform a machine learning-based critical intervention according to an embodiment of the present invention. Referring to FIG. 5, the configuration of the device (100) for predicting whether to perform a machine learning-based critical intervention illustrated is merely a simplified example. In one embodiment of the present invention, the device (100) for predicting whether to perform a machine learning-based critical intervention may include other components for performing a computing environment of the device (100), and only some of the disclosed components may constitute the device (100).

[0087] A device (100) for predicting whether to perform a machine learning-based important intervention may include a processor (110) including one or more cores, a memory (120), and a network (130).

[0088] The processor (110) may be configured with one or more cores, and may include a processor for data analysis and deep learning, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of a computing device. The processor (110) may read a computer program stored in the memory (120) and perform data processing for machine learning according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor (110) may perform operations for learning a neural network. The processor (110) may perform calculations for learning a neural network, such as processing input data for learning in deep learning (DL), extracting features from input data, calculating errors, and updating weights of a neural network using backpropagation. At least one of the CPU, GPGPU, and TPU of the processor (110) may process learning of a network function. For example, a CPU and a GPGPU can jointly process network function learning and data classification using network functions. Furthermore, in one embodiment of the present disclosure, processors of multiple computing devices can be jointly used to process network function learning and data classification using network functions. Furthermore, a computer program executed on a computing device according to one embodiment of the present disclosure may be a CPU, GPGPU, or TPU executable program.

[0089] The processor (110) may receive one or more patients' vital signs data, demographic data, arrival route data, and data on whether or not important interventions were implemented after arrival at the emergency room as training data for learning a predictive model. The processor (110) may perform the aforementioned input receiving step (S100).

[0090] The processor (110) can train a prediction model using learning data so that the prediction model outputs whether to implement the above-described critical intervention in response to inputting vital sign data, demographic data, and arrival route data. The processor (110) can receive, as arrival route data, at least one of the means of transportation used to arrive at the emergency room and the reason for visiting the emergency room. The processor (110) can perform the step (S300) of training the aforementioned prediction model.

[0091] The processor (110) can input vital sign data, demographic data, and patient arrival data of an emergency patient into a learned predictive model to output the degree of need for critical intervention, whether it is needed, and the number of critical interventions needed. The processor (110) can perform the outputting step (S500) described above.

[0092] The memory (120) can store any form of information generated or determined by the processor (110) and any form of information received by the network (130).

[0093] The memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. The computing device (100) may also operate in relation to web storage that performs the storage function of the memory (120) on the internet. The description of the above-described memory is merely an example, and the present disclosure is not limited thereto.

[0094] The network (130) may use any known wired or wireless communication system. The network (130) may receive endoscope images and the like from related devices or systems.

[0095] The network (130) can transmit and receive information, user interfaces, etc. processed by the processor (110) through communication with other terminals. For example, the network (130) can provide a user interface generated by the processor (100) to a client (e.g., a user terminal). In addition, the network (130) can receive external input from a user authorized as a client and transmit it to the processor (110). At this time, the processor (110) can process operations such as outputting, modifying, changing, and adding information provided through the user interface based on the external input of the user received from the network (130).

[0096] Meanwhile, a device (100) for predicting whether a machine learning-based important intervention will be implemented may include a server as a computing system that transmits and receives information through communication with a client. In this case, the client may be any type of terminal capable of accessing the server.

[0097] In an additional embodiment, a device (100) for predicting whether to perform a machine learning-based critical intervention may include any type of terminal that receives data resources generated from any server and performs additional information processing.

[0098] Another embodiment of the present invention, a computer program for predicting whether a machine learning-based critical intervention will be implemented, may include an input receiving operation, an operation for training a prediction model, and an output operation. The computer program for predicting whether a machine learning-based critical intervention will be implemented may be stored on a computer-readable storage medium and include instructions for causing a computer to perform the following operations.

[0099] The input receiving operation may receive as training data one or more patients' vital signs, demographic data, arrival route data, and data on whether important interventions were implemented after arrival at the emergency room for the training of a predictive model. The input receiving operation may only receive as training data vital signs, demographic data, and arrival route data collected before the patient arrives at the emergency room. The input receiving operation refers to the operation performed in the aforementioned input receiving step (S100).

[0100] The operation of training a predictive model can train the predictive model using training data so that the predictive model outputs whether or not to implement the above-mentioned important intervention in response to inputting vital sign data, demographic data, and arrival route data. The operation of training a predictive model refers to the operation performed in the aforementioned step of training a predictive model (S300).

[0101] The output operation may input vital sign data, demographic data, and patient arrival data of an emergency patient into a learned predictive model to output the degree of need for critical intervention, whether it is needed, and the number of critical interventions required. The output operation refers to the operation performed in the aforementioned output step (S500).

[0102] Figure 6 illustrates a schematic diagram of a computing environment according to an embodiment of the present invention.

[0103] Although the present disclosure has been described above as being generally implemented by a computing device, those skilled in the art will appreciate that the present disclosure may be implemented in combination with computer-executable instructions and / or other program modules that may be executed on one or more computers and / or as a combination of hardware and software.

[0104] Generally, program modules include routines, programs, components, data structures, and the like that perform particular tasks or implement particular abstract data types. Furthermore, those skilled in the art will appreciate that the methods of the present disclosure can be implemented with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which may be operatively connected to one or more associated devices.

[0105] The described embodiments of the present disclosure can also be practiced in distributed computing environments, where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0106] Computers typically include a variety of computer-readable media. Computer-readable media can be any media that can be accessed by a computer, and includes both volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media. By way of example, and not limitation, computer-readable media can include computer-readable storage media and computer-readable transmission media. Computer-readable storage media includes both volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be accessed by a computer and used to store the desired information.

[0107] Computer-readable transmission media typically includes any information delivery media that embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism. The term modulated data signal means a signal that has one or more of its characteristics set or changed so as to encode information in the signal. By way of example, and not limitation, computer-readable transmission media includes wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, or other wireless media. Combinations of any of the above are also intended to be included within the scope of computer-readable transmission media.

[0108] An exemplary environment for implementing various aspects of the present disclosure is illustrated, including a computer (1000), which includes a processing unit (1020), a system memory (1030), and a system bus (1010). The system bus (1010) connects system components, including but not limited to the system memory (1030), to the processing unit (1020). The processing unit (1020) may be any of a variety of commercially available processors. Dual processors and other multiprocessor architectures may also be utilized as the processing unit (1020).

[0109] The system bus (1010) may be any of several types of bus structures that may be additionally interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. The system memory (1030) includes read-only memory (ROM) (1034) and random access memory (RAM) (1032). A basic input / output system (BIOS) is stored in non-volatile memory (1034), such as ROM, EPROM, or EEPROM, and includes basic routines that help transfer information between components within the computer (1000), such as during start-up. The RAM (1032) may also include high-speed RAM, such as static RAM, for caching data.

[0110] The computer (1000) also includes an internal hard disk drive (HDD) (1050) (e.g., EIDE, SATA) - which may also be configured for external use within a suitable chassis (not shown), a magnetic floppy disk drive (FDD) (1060) (e.g., for reading from or writing to removable diskettes), and an optical disk drive (1070) (e.g., for reading from or writing to CD-ROM disks or other high-capacity optical media such as DVDs). The hard disk drive (1050), the magnetic disk drive (1060), and the optical disk drive (1070) may be connected to the system bus (1010) by a hard disk drive interface, a magnetic disk drive interface, and an optical drive interface, respectively. Interfaces for implementing external drives include at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

[0111] These drives and their associated computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, and the like. In the case of the computer (1000), the drives and media correspond to storing any data in a suitable digital format. While the description of computer-readable media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art will appreciate that other types of computer-readable media, such as zip drives, magnetic cassettes, flash memory cards, cartridges, and the like, may also be used in the exemplary operating environment, and that any such media may contain computer-executable instructions for performing the methods of the present disclosure.

[0112] A number of program modules, including an operating system (1092), one or more application programs (1094), other program modules (1096), and a database (1098), may be stored in the drive and RAM (1032). All or portions of the operating system, applications, modules, and / or data may also be cached in RAM (1032). It will be appreciated that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.

[0113] A user may enter commands and information into the computer (1000) via one or more wired / wireless input devices (1042), such as a keyboard and a pointing device such as a mouse. Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, a touch screen, and the like. These and other input devices are often connected to the processing unit (1020) via an input / output interface (1040) that is connected to the system bus (1010), but may be connected by other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, and the like.

[0114] A monitor or other type of display device is also connected to the system bus (1010) via an interface such as a video adapter. In addition to the monitor, the computer typically includes other peripheral output devices (not shown) such as speakers, a printer, and so on.

[0115] The computer (1000) may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) (1082), via wired and / or wireless communications. The remote computer(s) (1082) may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other conventional network node, and may generally include many or all of the components described for the computer (1000). The logical connections include wired / wireless connections to a local area network (LAN) and / or a larger network, such as a wide area network (WAN). Such LAN and WAN networking environments are common in offices and companies and facilitate enterprise-wide computer networks, such as intranets, all of which may be connected to a worldwide computer network, such as the Internet.

[0116] When used in a LAN networking environment, the computer (1000) is connected to a local network (not shown) via a wired and / or wireless communication network interface or adapter (not shown). The adapter (not shown) may facilitate wired or wireless communication to the LAN (not shown), which may also include a wireless access point installed therein for communicating with the wireless adapter (not shown). When used in a WAN networking environment, the computer (1000) may include a modem (not shown), be connected to a communication computing device on the WAN (not shown), or have other means for establishing communications over the WAN (not shown), such as via the Internet. The modem (not shown), which may be internal or external and wired or wireless, is connected to the system bus (1010) via a serial port interface (not shown). In a networked environment, program modules described for the computer (1000), or portions thereof, may be stored in a remote memory / storage device (not shown). It will be appreciated that the network connections shown are exemplary and that other means of establishing a communications link between computers may be used.

[0117] The computer (1000) operates to communicate with any wireless device or object that is arranged and operates via wireless communication, such as a printer, a scanner, a desktop and / or portable computer, a portable data assistant (PDA), a communication satellite, any equipment or location associated with a radio-detectable tag, and a telephone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure, as in a conventional network, or simply an ad hoc communication between at least two devices.

[0118] Wi-Fi (Wireless Fidelity) enables connections to the Internet and other devices without wires. Wi-Fi is a wireless technology that allows devices, such as computers, to send and receive data anywhere within the coverage area of ​​a base station, both indoors and outdoors, similar to cell phones. Wi-Fi networks use wireless technologies called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in the unlicensed 2.4 and 5 GHz radio bands, at data rates of, for example, 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual-band).

[0119] Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips referenced in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0120] Those skilled in the art will appreciate that the various illustrative logical blocks, modules, processors, means, circuits, and model steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, various forms of programs or design code (referred to herein, for convenience, as software), or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0121] The various embodiments presented herein can be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term article of manufacture includes a computer program, carrier, or media accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Furthermore, various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0122] It should be understood that the specific order or hierarchy of steps in the presented processes is merely an example of exemplary approaches. It should be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but are not intended to be limited to the specific order or hierarchy presented.

[0123] The description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments disclosed herein, but is to be construed in the broadest scope consistent with the principles and novel features disclosed herein.

[0124] The embodiments of the present invention described above are not implemented solely through devices and methods. They may also be implemented through programs that implement functions corresponding to the configurations of the embodiments of the present invention, or through recording media containing such programs. Such recording media may be executed not only on servers but also on user terminals.

[0125] Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims also fall within the scope of the present invention.

[0126] The purpose of the present invention is to provide a method, device and computer program for predicting whether to perform a critical intervention based on machine learning, which can predict whether to perform a critical intervention using only data that can be acquired during the process of being transferred to an emergency room.

Claims

1. In a method for predicting whether to implement a machine learning-based important intervention, A step of inputting vital sign data, demographic data, arrival route data, and data on whether important interventions were implemented after arrival at an emergency room as training data for training a prediction model; A step of training the prediction model using the above learning data so that the prediction model outputs whether to implement a critical intervention in response to inputting vital sign data, demographic data, and arrival route data; and A step of inputting vital sign data, demographic data, and patient arrival data of an emergency patient into a learned prediction model to output whether or not a critical intervention is implemented; A method comprising:

2. In paragraph 1, The above outputting step is: A method wherein the above prediction model outputs the degree of necessity for implementing a critical intervention, whether it is necessary, and the number of necessary critical interventions.

3. In paragraph 1, The above input receiving step is, A method in which only vital sign data, demographic data, and arrival route data collected before a patient arrives at an emergency room are input as learning data.

4. In paragraph 1, The above input receiving step is, A method for receiving at least one of body temperature (BT), heart rate (HR), respiration rate (RR), systolic blood pressure (SBP), and diastolic blood pressure (DBP) as the vital sign data.

5. In paragraph 1, The above input receiving step is, A method for receiving at least one of age and gender as the above demographic data.

6. In paragraph 1, The above input receiving step is, A method for receiving at least one of the means of transportation used to arrive at the emergency room and the reason for visiting the emergency room as the above arrival route data.

7. In paragraph 1, The above input receiving step is, A method of receiving input on whether at least one of the following is performed: whether an arterial line is inserted, whether massive blood transfusion is performed, whether an A-line is inserted, whether an inotrop is used, whether a vasopressor is used, whether labor supply is provided, whether high-flow nasal cannula therapy (HFNC) is used, and whether tracheal intubation is performed.

8. In paragraph 1, The above input receiving step is, A method that only receives data on whether important interventions were implemented within 24 hours of emergency room admission.

9. In paragraph 1, The step of training the above prediction model is: A method using an Extreme Gradient Boost Model as the above prediction model.

10. As a device that predicts whether or not to implement important interventions based on machine learning, a processor comprising one or more cores; and memory; Including, The above processor, To train a prediction model, one or more patients' vital signs data, demographic data, arrival route data, and data on whether important interventions were implemented after arrival at the emergency room are input as training data, and Using the above learning data, the prediction model is trained so that the prediction model outputs whether to implement the above important intervention in response to inputting vital sign data, demographic data, and arrival route data. A device that receives at least one of the means of transportation used to arrive at the emergency room and the reason for visiting the emergency room as the above arrival route data.

11. A computer program stored in a computer-readable storage medium and including commands that cause a computer to perform the following operations, wherein the operations are: An action of inputting vital sign data, demographic data, arrival route data, and data on whether important interventions were implemented after arrival at the emergency room as training data for training a prediction model; and An operation of training the prediction model using the above learning data so that the prediction model outputs whether to implement the important intervention in response to inputting vital sign data, demographic data, and arrival route data; The above input receiving action is, A computer program stored in a computer-readable storage medium, which inputs only vital sign data, demographic data, and arrival route data collected before a patient arrives at an emergency room as learning data.

Images