A system and method for using machine learning to activate a trauma team.

Abstract

A method for allocating medical resources to a subject for the treatment of traumatic injury, comprising: (a) receiving a personal information profile via a user interface of an application running on one or more computer processors; (b) assigning a computable value for any of several clinical conditions in the information profile via a machine learning model; (c) determining a total computable value for the subject's clinical conditions in real time based on an assessment via one or more computer processors via the machine learning model; (d) receiving a trauma onset notification determined by the relationship between the total computable value and a trauma onset level defined by the machine learning model; and (e) notifying a healthcare professional providing medical care to the subject via the user interface about the predicted allocation of medical resources to the subject.

Description

Technical Field

[0001] This application claims priority to U.S. Provisional Application No. 63 / 486,512, filed Feb. 23, 2023, which is incorporated herein by reference.

[0002] This invention was made with government support under TR001999 awarded by the National Institutes of Health. The government has certain rights in this invention.

[0003] The present invention generally relates to the fields of machine learning and e - healthcare as well as analysis and use in trauma treatment.

Background Art

[0004] Traumatic injuries accounted for 74.3% of the deaths of children aged 1 - 18 years in 2020. Although the development of a multi - tiered trauma system has improved survival rates, reduced morbidity, and decreased resource utilization, childhood morbidity and mortality due to traumatic injuries are increasing. These facts emphasize the importance of optimizing an inclusive trauma system to match the needs of injured patients with appropriate hospital resources and services.

[0005] The Optimal Resources document of the American College of Surgeons (ACS) Committee on Trauma (COT) defines the essential minimum criteria for trauma team activation (TTA) based on mechanism of injury, physiological and anatomical criteria. However, due to the diversity of injury patterns and the human physiological responses to those injuries, designing TTA criteria with perfect sensitivity and specificity for hospital triage of trauma patients is extremely difficult, if not impossible. Currently, there are no electronic tools available to assist in making important clinical triage decisions when an injured patient arrives at a trauma center.

Summary of the Invention

[0006] Therefore, there is a need for novel computer-implemented systems and methods to provide effective medical care in the field of trauma treatment. [Means for solving the problem]

[0007] One aspect of the present invention is a method for training a machine learning model to predict the allocation of medical resources necessary to treat patients with traumatic injuries, comprising: (a) establishing a ground truth, the ground truth corresponding to a standardized trauma-inducing level for a standardized patient having a standardized set of clinical conditions; and (b) assigning a first trauma-inducing level to individual patients identified by a plurality of clinical conditions selected from the standardized set of clinical conditions, wherein the machine learning model is a plurality of relative calculable values.(a) Assigning a first trauma trigger level by an algorithm including value, where each relative computable value is assigned to one or more clinical states within a standardized set of clinical states; (c) Determining, by human medical review, which of the set of clinical states each individual patient presented; (d) Determining, by human medical review, a second trauma trigger level that matches the set of clinical states presented by each individual patient; (e) Inputting the second trauma trigger level determined by human medical review based on the set of clinical states presented by each individual patient into a machine learning model; (f) Comparing the second trauma trigger level determined by human medical review to the first trauma trigger level assigned by the machine learning model based on the set of clinical states presented by each individual patient, for each individual patient in whom multiple clinical states selected from the standardized set of clinical states have been identified; (g) Based on the comparison, determining that the first trauma trigger level assigned by the machine learning model is relevant to the clinical environment A method comprising: (h) determining whether an individual patient corresponds to undertriage, overtriage, or an appropriate level of triage; (i) determining whether the second level of trauma-induced trauma corresponds to undertriage, overtriage, or an appropriate level of triage for an individual patient in a clinical setting, based on the set of clinical conditions presented by the individual patient, based on the comparison; (j) adjusting one or more relative computable values ​​used by the machine learning model in assigning the first level of trauma-induced trauma, depending on the determination of whether an individual patient received undertriage, overtriage, or an appropriate level of triage for an individual patient in a clinical setting; and (k) repeating steps (b) to (j) for multiple patients, each identified with multiple clinical conditions selected from a set of clinical conditions, until the machine learning model reaches an appropriate level of triage within the desired parameters.

[0008] Another aspect of the present invention is a method for training a machine learning model to predict the allocation of medical resources necessary to treat patients with traumatic injuries, comprising: (a) establishing a ground truth, the ground truth corresponding to standardized trauma trigger levels for standardized patients having a standardized set of clinical conditions; (b) assigning mechanical trauma trigger levels to individual patients identified by a plurality of clinical conditions selected from the standardized set of clinical conditions, the machine learning model assigning a first trauma trigger level by an algorithm comprising a plurality of relative computable values, each relative computable value being assigned to one or more clinical conditions in the standardized set of clinical conditions; and (c) The method comprises: (d) comparing the mechanical trauma trigger level with ground truth; (e) determining, based on the comparison, whether the mechanical trauma trigger level corresponds to undertriage, overtriage, or an appropriate level of triage for an individual patient; (f) if the mechanical trauma trigger level corresponds to undertriage or overtriage in step (d), adjusting one or more relative computable values ​​used by the machine learning model when assigning the mechanical trauma trigger level; and (g) repeating steps (b) to (e) for multiple patients, each identified with multiple clinical conditions selected from a set of clinical conditions, until the machine learning model reaches an appropriate level of triage within the desired parameters.

[0009] Another aspect of the present invention is a method for allocating medical resources to a subject for the treatment of traumatic injury, comprising: (a) receiving a personal information profile via a user interface of an application running on one or more computer processors, wherein the personal information profile includes inputting personal information relating to a plurality of types of information selected from types of information relating to comorbidities, engineered features, pre-hospital interventions, injury mechanisms, and numerical variables; (b) storing the subject's personal information via one or more computer processors in a database accessible by the application and accessible by the subject via the user interface of the application; and (c) calculating a computable value relating to any of the plurality of types of information in the personal information stored in the database, and one or more computer processors. A method comprising: (d) assigning a computable value point matrix stored in a memory device accessible by a computer processor, wherein the computable value point matrix is ​​generated by a machine learning module using a machine learning model trained by the method of the present application and is continuously updated; (d) determining in real time via one or more computer processors, based on an evaluation via one or more computer processors, a total computable value of the type of information in the subject's personal information, wherein a predictive allocation of medical resources is generated if the total computable value of the type of information in the subject's personal information exceeds or falls below a pre-selected computable value; and (e) notifying a medical professional providing medical care to the subject of the predictive allocation of medical resources to the subject via the user interface.

[0010] Another aspect of the present invention is a system for allocating medical resources to a subject for the treatment of the subject's traumatic injury, comprising one or more computer processors and one or more tangible computer-readable media accessible by the one or more computer processors, wherein the one or more tangible computer-readable media, when executed by the one or more processors, receives a personal information profile via a user interface of an application running on the one or more computer processors, the personal information profile including input of personal information relating to a plurality of types of information selected from types of information including comorbidities, engineered features, pre-hospital interventions, injury mechanisms and numerical variables, and (b) stores the personal information of the subject in a database accessible by the application and accessible by the subject via the user interface of the application, and (c) one (d) assigning a computable value for any of the multiple types of information in the personal information stored in the database and a computable value point matrix stored in a memory device accessible by one or more computer processors, wherein the computable value point matrix is ​​generated and continuously updated by a machine learning module using a machine learning model trained by the method described herein; (d) determining in real time, based on an evaluation via one or more computer processors, the total computable value of the types of information in the personal information of the subject, wherein a predictive allocation of medical resources is generated and determined if the total computable value of the types of information in the personal information of the subject exceeds or falls below a pre-selected computable value; and (e) notifying a healthcare professional providing medical care to the subject of the predictive allocation of medical resources to the subject via the user interface.It is a system that includes instructions to execute.

[0011] One aspect of the present invention, when executed by a computer processor, involves the processor receiving (a) a personal information profile via a user interface of an application running on one or more computer processors, wherein the personal information profile includes inputting personal information relating to a plurality of types of information selected from types of information including comorbidities, engineered features, pre-hospital interventions, injury mechanisms, and numerical variables; (b) storing the subject's personal information via one or more computer processors in a database accessible by the application and accessible by the subject via the application's user interface; and (c) a computable value relating to any of the plurality of types of information in the personal information stored in the database, accessible by one or more computer processors. A tangible, non-temporary, computer-readable storage medium that includes instructions to perform the following: (d) assign a computable value point matrix stored in a memory device, the computable value point matrix being generated and continuously updated by a machine learning module using a machine learning model trained by the method described herein; (b) determine, via one or more computer processors, in real time based on an evaluation via one or more computer processors, a total computable value of the type of information in the subject's personal information, to generate a predictive allocation of medical resources if the total computable value of the type of information in the subject's personal information exceeds or falls below a pre-selected computable value; and (e) notify a medical professional providing medical care to the subject of the predictive allocation of medical resources to the subject via the user interface. [Brief explanation of the drawing]

[0012] [Figure 1-1] Panel A is a diagram showing an overall overview of the steps involved in training a machine learning model for the method and system of the present invention. [Figure 1-2] Panel B is a diagram that provides an overview focused on model training and evaluation of machine learning models. [Figure 2] This diagram shows how the feature weights relate to the algorithm's output. [Figure 3-1] Panel A shows the importance of features when Cribari is ground truth. [Figure 3-2] Panel B shows the importance of features when NFTI is ground truth. Panel C shows the importance of features when Cribari+NFTI is ground truth. [Figure 4] This figure shows the percentage of patients undertriaged by each ground truth. [Figure 5] This figure shows various types of algorithms that may be used in the methods and systems described herein. [Figure 6] Panel A shows the undertriage rate and the number of patients in the dataset by mechanism of injury. The bars represent the undertriage rate (left axis), and the lines on the plot show the total number of patients in the dataset with each mechanism of injury. Panel B shows the undertriage rate by ED staff at different patient arrival times (bars, left y-axis), along with the hospitalization rate of patients at those times (lines, right y-axis). [Figure 7] This figure shows the Cribari Ground Truth trauma activation level for each patient's time of admission. [Figure 8] This figure shows the under and overtriage rates for patients of various ages. Ages are normalized to 0-1. [Figure 9] This figure compares the undertriage rates of patients above and below the median age of 47.57 years in the dataset. [Figure 10-1]Panel A shows the distribution of the triggered ground truth variables across all data. [Figure 10-2] Panel B shows the distribution of triggered ground truth variables among various variables. [Figure 11] This figure shows an example output of the LIME explainability method. [Figure 12] This figure shows the receiver operating characteristic (ROC) curves for each benchmarked machine learning model. The ROC curve shows the discriminative ability of a binary classifier by plotting the true positive rate against the false positive rate at various thresholds. A larger area under the curve (AUC) indicates higher separability between predictions. The random classifier achieved an AUC of 0.5, which is shown by the black dashed line. The best performance was achieved by the support vector machine (red line), followed by logistic regression (blue line), random forest (purple), and ED stuff (yellow). [Figure 13] This diagram shows the importance of features in the support vector machine model. Engineered features were constructed from existing institutional triage criteria, while other features were constructed from injury mechanism, comorbidities, and pre-hospital interventions. [Figure 14-1] This figure shows the distribution of complete and partial trauma team activations determined by trained emergency department (ED) staff. (A) ED staff triggered complete activations in 230 patients (16.8%) and partial activations in 1,136 patients (83.2%). [Figure 14-2](B) With the exception of gunshot wounds (GSW), the activation of the full trauma team for all injury mechanisms ranged from 9.1% to 18.5%. The distribution of gunshot wounds was different, with 50.4% triggering the activation of the full trauma team. Abbreviations: ED: Emergency Department, GSW: Gunshot Wound, MVC: Motor Vehicle Collision. [Figure 14-3] (Continuation of (B). [Figure 14-4] (Continuation of (B). [Figure 14-4] (Continuation of (B). [Figure 15-1] (A) It is a diagram showing the proportion of patients predicted to require full activation by each ground truth. The union (∪) of the two sets Cribari + NFTI is shown outside the colored circle, and the intersection (∩) of Cribari + NFTI is shown. [Figure 15-2] (B) It is a diagram showing the proportion of patients who received full activation and the proportion of patients predicted to require full activation. It shows the proportion of patients who actually received full activation (by ED staff) and the overlapping proportion by the patients predicted by Cribari and NFTI. There were only 96 patients who completely matched.

Mode for Carrying Out the Invention

[0013] Specific aspects and exemplary embodiments of the present invention are referred to in detail and are shown in the accompanying structures and figures. The aspects of the present invention are described in conjunction with exemplary embodiments including methods, materials and examples, and such descriptions are not limiting, and the scope of the present invention shall include all equivalents, alternatives, and modifications that are generally known or incorporated herein. Unless otherwise defined, all technical terms and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention pertains. Those skilled in the art will recognize many techniques and materials similar or equivalent to those described herein that can be used in practicing the aspects and embodiments of the present invention. The described aspects and embodiments of the present invention are not limited to the described methods and materials.

[0014] As used herein and in the appended claims, the singular forms "a," "an," and "the" refer to multiple subjects unless the content explicitly indicates otherwise.

[0015] A range may be expressed herein as “about” one particular value and / or “about” another particular value. When such a range is expressed, other embodiments include one particular value and / or other particular values. Similarly, when values ​​are expressed as approximations, it will be understood that by using the preceding “about,” a particular value forms other embodiments. Furthermore, it will be understood that the endpoints of each range are important, both in relation to the other endpoint and independently of the other endpoint. Also, it will be understood that when a number of values ​​are disclosed herein, each value is disclosed herein not only as the value itself but also “about” that particular value. For example, when the value “10” is disclosed, “about 10” is also disclosed. Also, as will be appropriately understood by those skilled in the art, when a value is disclosed, it will be understood that “less than or equal to that value,” “greater than or equal to that value,” and possible ranges between values ​​are also disclosed. For example, when the value “10” is disclosed, “less than or equal to 10” and “greater than or equal to 10” are also disclosed.

[0016] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as those generally understood by those skilled in the art to which the present invention pertains. Furthermore, terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meanings in the context of the relevant art and this disclosure, and should not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0017] It will be understood that in describing this invention, many techniques and steps are disclosed. Each of these has its own individual advantages, and each can be used in combination with one or more, or possibly all, of the other disclosed techniques. Therefore, for clarity, this description refrains from unnecessarily repeating all possible combinations of the individual steps. Nevertheless, this specification and the claims should be read with the understanding that such combinations are entirely within the scope of the invention and the claims.

[0018] The following description includes numerous specific details for illustrative purposes to provide a complete understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without these specific details.

[0019] I. Definition As used herein, the term “computer” means a machine, apparatus, or device capable of accepting and executing logical operations from software code. The terms “application,” “software,” “software code,” or “computer software” mean any set of instructions that can be operated to cause a computer to perform an action. Software code may be operated by a “rule engine” or processor. Thus, the methods and systems of the present invention may be executed by a computer or computing device having a processor, based on instructions received by computer applications and software.

[0020] As used herein, the term “electronic device” refers to a type of computer that includes circuitry and is configured to generally perform functions such as recording audio, photographs, and video; displaying or playing audio, photographs, and video; storing, retrieving, or manipulating electronic data; providing telecommunications and network connectivity; or any other similar functions. Non-limiting examples of electronic devices include personal computers (PCs), workstations, laptops, tablet PCs including iPads, mobile phones including Apple iOS phones, Android OS phones, Microsoft OS phones, Blackberry phones, digital music players, or electronic devices capable of running computer software and displaying information to a user, memory cards, other memory storage devices, digital cameras, external battery packs, external charging devices, etc. Certain types of electronic devices that are portable and can be easily carried by a person from one place to another may be called “portable electronic devices” or “portable devices.” Some non-limiting examples of portable devices include mobile phones, smartphones, tablet computers, laptop computers, wearable computers such as Apple Watch, other smartwatches, Fitbit, other wearable fitness trackers, Google Glass, etc.

[0021] As used herein, the term “client device” refers to a type of computer that includes circuitry and is configured to generally perform functions such as recording audio, photographs, and video; displaying or playing audio, photographs, and video; storing, retrieving, or manipulating electronic data; providing telecommunications and network connectivity; or any other similar functions. Non-exclusive examples of client devices include personal computers (PCs), workstations, laptops, tablet PCs including iPads, mobile phones including Apple iOS phones, Android OS phones, Microsoft OS phones, Blackberry phones, Apple iPads, Anata digital pens, digital music players, or electronic devices capable of running computer software and displaying information to a user, memory cards, other memory storage devices, digital cameras, external battery packs, and external charging devices. Certain types of electronic devices that are portable and can be easily carried by a person from one place to another may be called “portable electronic devices” or “portable devices.” Some non-exclusive examples of portable devices include mobile phones, smartphones, tablet computers, laptop computers, tablets, digital pens, wearable computers such as Apple Watches, other smartwatches, Fitbits, other wearable fitness trackers, and Google Glass.

[0022] As used herein, the term “computer-readable medium” refers to any medium involved in providing instructions to a processor for execution. Computer-readable medium can take many forms, including but not limited to non-volatile, volatile, and transmission media. Non-volatile media include, for example, optical disks, magnetic disks, and magneto-optical disks, such as hard disks or removable media drives. Volatile media include dynamic memory, such as main memory. Transmission media include coaxial cables, copper wires, and optical fibers, such as the wires that make up a bus. Transmission media can also take the form of acoustic or optical waves, such as those generated during radio and infrared data communications.

[0023] As used herein, the terms “data network” or “network” mean an infrastructure that enables two or more computers, such as client devices, to be connected by wire or wireless means to send and receive data. Non-limiting examples of a data network may include the Internet or a wireless network (i.e., a “wireless network”), which may include Wi-Fi and cellular networks. For example, a network may include a local area network (LAN), a wide area network (WAN) (e.g., the Internet), a mobile relay network, a metropolitan area network (MAN), an ad hoc network, a telephone network (e.g., a public switched telephone network (PSTN)), a cellular network, a Zigby network, or a Voice over IP (VoIP) network.

[0024] As used herein, the term “database” generally means a digital collection of data or information. The present invention uses novel methods and processes for storing, linking, and modifying information such as digital images and videos, as well as user profile information. For the purposes of this disclosure, a database may be stored on a remote server and accessed by a client device over the Internet (i.e., the database is in the cloud), or, in some embodiments, the database may be stored on a client device or the remote computer itself (i.e., local storage). As used herein, “datastore” may include or consist of a database (i.e., information and data from the database may be recorded on a medium on the datastore).

[0025] As used herein, the term “comorbidity” refers to the simultaneous presence of two or more diseases or medical conditions in a patient. In certain embodiments, comorbidities include: aggressive chemotherapy, treatment restrictions by advance directive, alcoholism, Alzheimer's disease, anemia, angina pectoris, anticoagulant therapy, ascites within 30 days, asthma, attention deficit disorder / attention deficit hyperactivity disorder, hemorrhagic disorders, cancer, cardiopulmonary resuscitation (CPR), cerebrovascular disease (CVA), chronic obstructive pulmonary disease (COPD), chronic renal failure, cirrhosis, presence of synchronous or metastatic metastases, congestive heart failure, congenital anomalies, coronary artery disease, coumadin therapy, current smoking, current chemotherapy for cancer, chronic demyelinating disease, chronic drug abuse, Crohn's disease, CVA / hemiplegia (residual stroke), dementia, diabetes, dialysis (excluding transplant patients), disseminated cancer, documented history of cirrhosis, documented history of lung disease with ongoing aggressive treatment, drug use disorder, DVT, esophageal varices, and This may include one or more of the following: insulin-dependent health conditions, heart disease, hemophilia, history of angina pectoris within the past month, history of cardiac surgery, history of myocardial infarction, history of peripheral vascular disease (PVD), history of mental illness, HIV / AIDS, hypertension, insulin dependence, inflammatory bowel disease, irritable bowel disease, major mental illness, psychiatric / personality disorder, multiple sclerosis, myocardial infarction (MI), non-insulin-dependent diabetes mellitus, obesity, pancreatitis, Parkinson's disease, peripheral artery disease (PAD), peripheral vascular disease (PVD), pregnancy, prehospital cardiac arrest with resuscitation by a healthcare provider, prehospital CPR, premature birth, mental illness, pulmonary embolism (PE), renal failure, rheumatoid arthritis, regular steroid use, seizures, spinal cord injury, steroid use, drug abuse disorder, systemic lupus erythematosus, transplantation, ulcerative colitis, currently being treated, and vascular diseases.

[0026] As used herein, the term “pre-hospital intervention” refers to any medical treatment or intervention performed before or during transport to a hospital (at the time of the trauma victim). In certain embodiments, prehospital interventions may include one or more of the following: airway, airway management, alternative airway devices, assisted ventilation with BVM, bleeding control, chest compressions, combitube, CPR initiated by crew, CPR in progress, continued CPR, endotracheal tube, electrocardiogram, ECG monitoring, bleeding management, hemostatic bandage, IO, IO fluid ≤ 500cc, IO fluid 500-2000cc, IO fluid ≥ 2000cc, IO fluid volume unknown, IO fluid trial, IV fluid, IV fluid < 500ml, IV fluid 500-2000ml, IV fluid > 2000ml, IV fluid volume unknown, IV fluid trial, King airway, limb immobilization, medication, nasal ETT, nasal trumpet, decompression by puncture for tension pneumothorax, needle thoracotomy, oral airway, oxygen, oxygen via cannula, oxygen via mask, pelvic binder, pressure, saline solution, spine, spinal immobilization, suction, tourniquet.

[0027] As used herein, the term “mechanism of injury” refers to how damage (trauma) to the skin, muscles, organs, and bones occurred. In certain embodiments, the mechanism of injury may include one or more of the following: abuse, aircraft, plane crash, all-terrain vehicle, bite-animal, human, assault, bicycle collision, boat accident, broken glass, burns, chainsaw, child abuse, crush injury, dirt bike, diving, drowning, electrical injury, explosion, fall, fall-tree stand, fall from snowboard, farm / heavy machinery accident, fight / brawl, fireworks, gunshot wound, handsaw and table saw, hanging, occupational accident, animal injury, jet ski, jump, motorcycle collision, motocross accident, car-pedestrian collision, car collision, motorcycle accident, open wound / separation, pedestrian hit-and-run, playground, electric lawnmower, riding lawnmower, roller skate / blade, scooter, shooting BB gun, skateboard, ski, sled or tubing, snowmobile collision, sports-occurring during sports activities, puncture wound, impact by object-non-automotive related, impact in sports, surgery, trampoline, and unknown.

[0028] As used herein, the term “computable value” refers to an assignable value that can be expressed as a quantity, number, or numerical value. In certain embodiments, the computable value is a weight score assigned to a particular factor in an algorithm used by the systems and methods described herein.

[0029] As used herein, the term “institutional triage criteria” refers to criteria for determining whether a patient is eligible for full trauma triage. In certain embodiments, institutional triage criteria include: (Level I) Adults with consistently confirmed blood pressure less than 90 mmHg; age-specific hypotension in children; gunshot wounds to the neck, trunk, groin, buttocks, or joints; respiratory distress or pre-arrival intubation; mechanism resulting from trauma GCS < 8; transport of patients receiving or having received blood transfusions; ongoing post-traumatic CPR or history of CPR; discretion of the EM physician, triage nurse, or communications nurse; (Level II) Gunshot wounds to the head; gunshot wounds to the arm / leg proximal to the elbow / knee; stab wounds to the head, neck, trunk, groin, or joints; active bleeding requiring a tourniquet or wound packing; suspected spinal cord injury with new motor or sensory loss; proximal trauma to the wrist or ankle Sexual amputation; suspected proximal fractures in two or more locations (including suspected pelvic fractures and open fractures); discretion of the EM physician, triage nurse, or communications nurse; (Level III) severe head injury (cranial fracture or ICH) with trauma to the trunk or limbs, otherwise not meeting the level criteria; multiple rib fractures and rib cage instability, otherwise not meeting the level criteria; traumatic pelvic fracture, otherwise not meeting the level criteria; pregnancy (over 20 weeks) with abdominal trauma; severe injury with pre-existing comorbidities or extremely advanced age; hospitalization for treatment of acute injury associated with known or suspected child physical abuse (non-accidental trauma / NAT); discretion of the ED physician; may include single system injury due to severe mechanism. Facility triage criteria include measures related to serious traumatic injury, including: 1) a history of partial or complete ejection from a vehicle or rollover; 2) a fall of more than 10 feet (all ages); 3) death in the same occupant compartment; 4) a pedestrian / cyclist being thrown, run over, or subjected to a significant impact; 5) a history of a high-speed collision with significant vehicle intrusion; 6) the need for rescue for an trapped patient; 7) a rider being separated from a transport vehicle (motorcycle, ATV, horse, etc.) by a significant impact; and 8) injuries from an explosion or blast.

[0030] II. How to train a machine learning model to predict the allocation of medical resources needed to treat patients with traumatic injuries. One aspect of the present invention is a method for training a machine learning model to predict the allocation of medical resources necessary to treat patients with traumatic injuries, comprising: (a) establishing a ground truth, the ground truth corresponding to a standardized trauma trigger level for a standardized patient having a standardized set of clinical conditions; (b) assigning a first trauma trigger level to individual patients identified by a plurality of clinical conditions selected from the set of clinical conditions, the machine learning model assigning the first trauma trigger level by an algorithm including a plurality of relative computable values, each relative computable value being assigned to one or more clinical conditions in the standardized set of clinical conditions; (c) determining, by human medical review, which of the set of clinical conditions each individual patient presented; (d) determining, by human medical review, a second trauma trigger level matching the set of clinical conditions presented by the individual patient; and (e) determining, by human medical review, based on the set of clinical conditions presented by the individual patient. (i) Input the determined second trauma trigger level into a machine learning model; (f) Compare the second trauma trigger level determined by a human medical review based on a set of clinical conditions presented by each patient with the first trauma trigger level assigned by the machine learning model for each patient in which multiple clinical conditions selected from the set of clinical conditions have been identified; (g) Based on the comparison, determine whether the first trauma trigger level assigned by the machine learning model corresponds to undertriage, overtriage, or an appropriate level of triage for each patient in the clinical setting; (h) Compare the second trauma trigger level determined by a human medical review based on a set of clinical conditions presented by each patient with ground truth; (i) Based on the comparison, determine whether the second trauma trigger level corresponds to undertriage, overtriage, or an appropriate level of triage for each patient in the clinical setting; (j) If each patient is undertriage, overtriage,The present invention relates to a method comprising: (k) adjusting one or more relative computable values ​​used by a machine learning model when assigning a first trauma-inducing level, depending on the determination of which appropriate level of triage was received; (k) repeating steps (b) to (j) with multiple training patients, each identified with multiple clinical states selected from a set of clinical states, until the machine learning model reaches the desired parameter of step (l); and (l) assigning a third trauma-inducing level corresponding to an appropriate level of triage within the desired parameter to multiple test patients, each identified with multiple clinical states selected from a standardized set of clinical states.

[0031] In a particular embodiment, the desired parameter includes machine accuracy, which comprises: (1) assigning a third trauma trigger level to multiple test patients, each identified by a machine learning model, a set of multiple clinical conditions selected from a standardized set of clinical conditions; (2) comparing the third trauma trigger level for each test patient with ground truth and determining, based on the comparison, whether the third trauma trigger level assigned by the machine learning model corresponds to undertriage, overtriage, or an appropriate level of triage for each test patient; and (3) the following formula, i.e., Machine accuracy = (Number of test patients with an appropriate level of triage ÷ Total number of test patients) × 100% This is determined by calculating the machine accuracy for test patients based on the above.

[0032] In certain embodiments, the machine accuracy is 70%, 75%, 80%, 85%, 90%, 95%, or 98% or higher.

[0033] In some embodiments, the desired parameters include the undertriage rate and / or the overtriage rate. The undertriage rate and the overtriage rate are calculated using the following formulas. Undertriage rate = (Number of undertriaged test patients ÷ Total number of test patients) × 100% Overtriage rate = (Number of overtriaged test patients ÷ Total number of test patients) × 100% If the trauma trigger level assigned to a patient is lower than the trauma trigger level assigned to the same patient based on ground truth, it is considered undertriage. If the trauma trigger level assigned to a patient is higher than the trauma trigger level assigned to the same patient based on ground truth, it is considered overtriage.

[0034] In certain embodiments, the undertriage rate is 1%, 2%, 5%, 10%, 15%, 20%, 25%, or 30% or less.

[0035] The emergency room makes decisions regarding the allocation of medical resources based on the inevitably limited information obtained from paramedics en route to the hospital before the patient arrives, or on the information obtained upon arrival at the emergency room itself. Patient classification is critical, as misclassification carries the risk of leading to undertriage, which is detrimental to the patient, or overtriage, which results in the misuse of limited hospital resources. As described herein, determining the ground truth for these classifications is difficult, but it is necessary to establish a supervised learning model with a reliable baseline of proper triage to evaluate instances of undertriage and overtriage. This requires clinical knowledge. Assigning a Trauma Team Activation (TTA) level to suit a specific patient with a traumatic injury can be conceptualized as a simple classification task that assigns a specific TTA level to each patient using a given set of criteria along with available patient data. By activating the trauma system in a timely manner, injured patients can be met with the trauma team upon arrival, so the trauma system is activated as far as possible before patient arrival.

[0036] This invention discloses a method for determining the ideal ground truth for training a machine learning model to accurately predict the TTA level in patients with traumatic injuries (Panels A and B in Figure 1). In certain embodiments, combining the Cribari method with the Need for Trauma Intervention (NFTI) method improves model performance compared to either method alone. Actual trauma team activation levels were compared to the recommended level classifications for each ground truth (Cribari, NFTI, or Cribari+NFTI). Where the classifications did not match, demographics, pre-injury characteristics, and injury mechanism were compared.

[0037] There are several parameters that can be used to determine model performance, and these include, but are not limited to, the following:

[0038] TP=true positive

[0039] FP=false positive

[0040] TN=true negative

[0041] FN=false negative

[0042] Precision rate = Positive predictive value: TP / (TP+FP)

[0043] Recall rate = negative predictive value: TN / (TN+FN)

[0044] F1: Harmonic mean of precision and recall = 2 * (precision * recall) / (precision + recall)

[0045] Accuracy rate = Number of correctly classified (triaged) patients / Total number of tested patients = (TP + TN) / (TP + FP + TN + FN)

[0046] The present invention identifies ground truth available in predictive modeling, and specifically, in certain embodiments, uses hyperparameter tuning (clinical selection of particularly important parameters) to bias the model away from undertriage (to avoid harm to patients), although this may result in instances of overtriage.

[0047] Machine learning, a subtype of artificial intelligence, can be used to optimize classification predictions based on features provided in large datasets. Other advantages of machine learning include its ability to handle complex data and its ability to work with nonlinear and missing data. However, supervised machine learning approaches require that the data used to train the model be provided with correct answers, or "ground truth." The model can learn patterns based on the data and ground truth, and then make predictions or classifications based on those patterns.

[0048] Choosing the ground truth for a TTA level is not easy, as there is no single gold standard method for determining whether a TTA level has been accurately assigned.

[0049] The Cribari matrix method is the most common and is based on the definition of major trauma in the ACS COT Optimal Resources document (patients with an Injury Severity Score (ISS) > 15). In clinical practice, the Cribari classification method is only retrospective, and in reality, it sums up various injuries identified after the patient arrives at the hospital. The Cribari matrix assesses head or neck, face, chest, abdominal / pelvic organs, limbs / pelvic girdle, and external injuries according to a trauma severity score (1-75) calculated from the highest abbreviated injury scale (AIS) code in each of the three most severe areas (full TTA for scores 16-75).

[0050] In various embodiments, alternative scoring systems may be used, including the Need for Trauma Intervention (NFTI), which is more strongly associated with post-traumatic outcomes than the ISS. The NFTI approach uses six criteria, and full trauma team activation is initiated if two or more criteria are met. The NFTI assesses: (1) receiving packed red blood cells within four hours of arrival at the emergency department (ED); (2) transfer from the emergency department (ED) to the operating room within 90 minutes of arrival; (3) transfer from the ED to the radiotherapy room; (4) transfer from the ED to the intensive care unit (ICU) with a length of stay (LOS) of three days or more; (5) mechanical ventilation without procedural anesthesia within three days of arrival; and (6) death within 60 hours of arrival.

[0051] Clinically, the combination of Cribari and NFTI yields the best performance, but this is not necessarily true when machine learning is used, as described herein. Therefore, weighted parameters are used within the model to augment the machine learning model based on clinical knowledge. Figure 2 illustrates how the feature weights relate to the output from the algorithm.

[0052] Another aspect of the present invention relates to a method for training a machine learning model to predict the allocation of medical resources necessary to treat patients with traumatic injuries, comprising: (a) establishing ground truth, where ground truth corresponds to standardized trauma trigger levels for standardized patients having a standardized set of clinical conditions; (b) assigning mechanical trauma trigger levels to individual patients identified by a plurality of clinical conditions selected from the standardized set of clinical conditions, wherein the machine learning model assigns a first trauma trigger level by an algorithm comprising a plurality of relative computable values, each relative computable value being assigned to one or more clinical conditions in the standardized set of clinical conditions; and (c) The method comprises the steps of: (d) comparing a mechanical trauma trigger level with ground truth; (e) determining, based on the comparison, whether the mechanical trauma trigger level corresponds to undertriage, overtriage, or an appropriate level of triage for an individual patient; (f) if, in step (d), the mechanical trauma trigger level corresponds to undertriage or overtriage, adjusting one or more relative computable values ​​used by the machine learning model when assigning the mechanical trauma trigger level; and (g) repeating steps (b) to (e) for a plurality of patients, each identified with a plurality of clinical conditions selected from a set of clinical conditions, until the machine learning model reaches an appropriate level of triage within the desired parameters. In some embodiments, ground truth is based on ISS, NFTI, or a combination of ISS and NFTI. In some embodiments, individual patients have a previous trauma trigger level assigned by a human healthcare provider. In some embodiments, the desired parameters include machine accuracy.

[0053] The methods described herein can be used for both adult and pediatric patients. However, models developed for pediatric treatment must be developed based on a pediatric population, as child patients cannot be considered equivalent to adult patients. Different hyperparameter tuning may be used for pediatric populations.

[0054] A particular feature of the method of the present invention is that the method relies on raw data obtained pre-hospital or in the emergency department of a hospital, and uses this raw data for feature selection when classifying patients for an appropriate level of triage.

[0055] Another particular feature of the method of the present invention is to shift the logic within the model used (e.g., a logistic regression model) by using hyperparameter tuning of various features selected for the purpose of classifying patients for triage.

[0056] The model can be designed to self-improve through methods such as deep learning. By using deep learning, a certain level of internalized quality assurance checks can be introduced into the algorithm for allocating medical resources.

[0057] The artificial intelligence module may include, or function as, artificial intelligence logic stored in memory, which may be executable by the processors of one or more servers and / or client devices. In some embodiments, the artificial intelligence module may function as, or include, a machine learning / deep learning / artificial intelligence platform that queries the system's medical information or data and learns about the medical behavior and trends of one or more patients.

[0058] In further embodiments, the artificial intelligence module may function to provide and recommend solutions to patients and healthcare providers, such as treatments that are cost-effective and can effectively treat a patient's illness.

[0059] In further embodiments, the artificial intelligence module may function to generate collective data and other informatics, such as anonymized general patient population data for healthcare organizations and pharmaceutical companies, using information on one or more patients stored in one or more data stores and / or blockchain databases.

[0060] III. Methods for allocating medical resources to individuals for the treatment of traumatic injuries. Another aspect of the present invention relates to a method for allocating medical resources to a subject for the treatment of traumatic injury. The method includes (a) receiving a personal information profile via a user interface of an application running on one or more computer processors, wherein the personal information profile includes inputting personal information relating to a plurality of types of information selected from types of information including comorbidities, engineered features, pre-hospital interventions, injury mechanisms, and numerical variables; (b) storing the subject's personal information via one or more computer processors in a database accessible by the application and accessible by the subject via the user interface of the application; (c) receiving a data visualization dashboard via the user interface of the application running on one or more computer processors, wherein the dashboard displays predictions of medical resource requests corresponding to the personal information stored in the database; and (d) receiving a data visualization dashboard via a user interface of an application running on one or more computer processors (a) a step of assigning a computable value for any of several types of information in the personal information stored in the database via a sasser and a computable value point matrix stored in a memory device accessible by one or more computer processors, wherein the computable value point matrix is ​​generated and continuously updated by a machine learning module; (e) a step of determining in real time via one or more computer processors the total computable value of the types of information in the personal information of the subject, based on an evaluation via one or more computer processors, wherein a predictive allocation of medical resources is generated if the total computable value of the types of information in the personal information of the subject exceeds or falls below a pre-selected computable value; and (f) a step of receiving a goal setting module via a user interface of an application running on one or more computer processors, wherein the goal setting module(g) a receiving step including input from a healthcare professional regarding a calculable value to be assigned to a type of information in the personal information that is the focus of clinical care based on the personal information stored in the database; and (g) a step of notifying a healthcare professional who provides medical care to the subject via the user interface about the predicted allocation of medical resources to the subject.

[0061] Another aspect of the present invention relates to a method for allocating medical resources to a subject for the treatment of traumatic injury. The method includes (a) receiving a personal information profile via a user interface of an application running on one or more computer processors, wherein the personal information profile includes inputting personal information relating to a plurality of types of information selected from types of information including comorbidities, engineered features, pre-hospital interventions, injury mechanisms, and numerical variables; (b) storing the subject's personal information via one or more computer processors in a database accessible by the application and accessible by the subject via the user interface of the application; and (c) calculating a computable value relating to any of the plurality of types of information in the personal information stored in the database and one or more computer processors. (d) assigning a computable value point matrix stored in a memory device accessible by a computer processor, wherein the computable value point matrix is ​​generated and continuously updated by a machine learning module; (d) determining in real time via one or more computer processors, based on an evaluation via one or more computer processors, a total computable value of the type of information in the subject's personal information, wherein a predictive allocation of medical resources is generated if the total computable value of the type of information in the subject's personal information exceeds or falls below a pre-selected computable value; and (e) notifying a healthcare professional providing medical care to the subject via the user interface about the predictive allocation of medical resources to the subject.

[0062] The allocation of medical resources is based on raw data about patient characteristics identified by pre-hospital staff (e.g., paramedics or other first responders) or hospital staff (e.g., emergency room nurses). The resource allocation models described herein do not necessarily take the sum of weight values ​​assigned to features (although this may be done in certain situations); rather, the models consider the relative values ​​of the assigned weights. Weight values ​​are evaluated in real time within the models, creating a weighted feature map, which is not necessarily used as part of the algorithm for allocating medical resources. Features evaluated include, but are not limited to, comorbidities, mechanisms of injury, and pre-hospital interventions (see, for example, Table 4 and panels A, B, and C in Figure 3).

[0063] Raw patient data can be obtained pre-hospital, in-hospital, or both, but pre-operational variable manipulation requires specialized clinical knowledge. To minimize undertriage when applying the model to allocate medical resources in real time, post-operational hyperparameter tuning is required to bias the model's results away from undertriage. In certain embodiments, various algorithms may be used to construct a supervised learning classifier (Figure 4). In certain embodiments, various classifiers (e.g., logistic regression, decision trees, random forests, support vector machines, K-nearest neighbors, naive Bayes) may be used in the methods and systems herein. In preferred embodiments, logistic regression and random forests are used.

[0064] In a particular embodiment, a method for allocating medical resources to a subject, who is a patient brought to the emergency room, uses a set of standardized questions (e.g., 10 questions) that emergency responders and / or hospital staff answer about the subject. Based on these questions, a model developed through the training method described herein allocates medical resources with a bias designed to avoid the risk of undertriage. The model is trained to be able to allocate medical resources for an appropriate level of triage within a high confidence interval if it can answer four or more questions correctly.

[0065] A graphical user interface (GUI) is provided to the user (e.g., emergency room staff). In certain embodiments, the graphical user interface has fewer than 20 variables into which the user inputs information to obtain a predictive assessment of the medical resources needed. The graphical user interface can operate via dropdown boxes and other standard options known to be provided by the GUI.

[0066] IV. A system for allocating medical resources to individuals for the treatment of their traumatic injuries. Another aspect of the present invention relates to a system for allocating medical resources to a subject for the treatment of traumatic injury. The system comprises one or more computer processors and one or more tangible computer-readable media accessible by the one or more computer processors, wherein the one or more tangible computer-readable media, when executed by the one or more processors, receives a personal information profile via a user interface of an application running on the one or more computer processors, the personal information profile including input of personal information relating to a plurality of types of information selected from types of information including comorbidities, engineered features, pre-hospital interventions, injury mechanisms, and numerical variables, and (b) stores the personal information of the subject in a database accessible by the application and accessible by the subject via the user interface of the application. (c) receiving a data visualization dashboard via the user interface of the application running on one or more computer processors, the dashboard displaying and receiving predictions of medical resource requests corresponding to the personal information stored in the database; (d) assigning a computable value for any of several types of information in the personal information stored in the database and a computable value point matrix stored in a memory device accessible by one or more computer processors, the computable value point matrix being generated and continuously updated by a machine learning module; and (e) determining, via one or more computer processors, the total computable value of the types of information in the personal information of the subject in real time based on evaluations.The system includes instructions to: (f) determine that a predictive allocation of medical resources is generated if the total computable value of the types of information in the subject's personal information exceeds or falls below a pre-selected computable value; (g) receive a goal-setting module via the user interface of an application running on one or more computer processors, the goal-setting module including input from a healthcare professional regarding computable values ​​to be assigned to the types of information in the personal information that are the focus of clinical care based on the personal information stored in the database; and (g) notify a healthcare professional providing medical care to the subject via the user interface about the predictive allocation of medical resources to the subject.

[0067] Another aspect of the present invention relates to a system for allocating medical resources to a subject for the treatment of the subject's traumatic injury. The system comprises one or more computer processors and one or more tangible computer-readable media accessible by one or more computer processors, wherein one or more tangible computer-readable media, when executed by one or more processors, receive from one or more processors: (a) receiving a personal information profile via a user interface of an application running on one or more computer processors, the personal information profile including input of personal information relating to a plurality of types of information selected from types of information including comorbidities, engineered features, pre-hospital interventions, injury mechanisms, and numerical variables; (b) storing the personal information of the subject in a database accessible by the application and accessible by the subject via the user interface of the application; and (c) one or more computer processors The instructions include: (d) assigning a computable value for any of several types of information in the personal information stored in the database to a computable value point matrix stored in a memory device accessible by one or more computer processors, wherein the computable value point matrix is ​​generated and continuously updated by a machine learning module; (e) determining in real time, based on an evaluation via one or more computer processors, the total computable value of the types of information in the personal information of the subject, wherein a predictive allocation of medical resources is generated if the total computable value of the types of information in the personal information of the subject exceeds or falls below a pre-selected computable value; and (f) notifying a medical professional providing medical care to the subject of the predictive allocation of medical resources to the subject via the user interface.

[0068] It will be understood that some exemplary embodiments described herein may include one or more general-purpose or dedicated processors (or “processing devices”), such as microprocessors, digital signal processors, customized processors, and field-programmable gate arrays (FPGAs), and specific built-in programming instructions (including both software and firmware) that control one or more processors to perform some, almost all, or all of the functions of the method and / or system described herein, together with certain non-processor circuits. Alternatively, some or all of the functions may be performed by a state machine without built-in programming instructions, or by one or more application-specific integrated circuits (ASICs) in which each function or some combination of certain functions is performed as custom logic. Naturally, combinations of these two approaches may be used. Also, some exemplary embodiments may be implemented as computer-readable storage media storing computer-readable code for programming computers, servers, appliances, devices, etc., each of which may include a processor for performing the methods described herein and claimed herein. Examples of such computer-readable storage media include, but are not limited to, hard disks, optical storage devices, magnetic storage devices, ROM (read-only memory), PROM (programmable read-only memory), EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), and flash memory.

[0069] The subject matter and functional operating embodiments described herein can be implemented in digital electronic circuits, or in computer software, firmware, or hardware including the structures disclosed herein and their structural equivalents, or in one or more combinations thereof. The subject matter embodiments described herein can be implemented as one or more computer program products, i.e., as one or more modules of computer program instructions coded on a tangible program carrier for execution by a data processing device or for controlling the operation of a data processing device. The tangible program carrier can be a propagating signal or a computer-readable medium. A propagating signal is an artificially generated signal, such as a machine-generated electrical, optical, or electromagnetic signal, that encodes information and transmits it to a suitable receiving device for execution by a computer. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition that generates a machine-readable propagating signal, or one or more combinations thereof.

[0070] Processors suitable for executing computer programs include, for example, both general-purpose and dedicated microprocessors, and any one or more processors in any type of digital computer. Generally, a processor receives instructions and data from read-only memory or random-access memory, or both. Essential elements of a computer are a processor for executing instructions, and one or more memory devices for storing instructions and data. Generally, a computer also includes one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, solid-state drives, or optical disks, or is operablely coupled to them for receiving or transferring data, or both. However, a computer does not have to have such devices.

[0071] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. Processors and memory can be complemented or integrated with dedicated logic circuits.

[0072] To provide user interaction, embodiments of the subject matter described herein can be implemented on a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and a pointing device such as a keyboard and mouse or trackball to which the user can provide input to the computer. Other types of devices can also be used to provide user interaction; for example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user can be received in any form, including acoustic input, voice input, or tactile input.

[0073] Embodiments of the subject matter described herein can be implemented in a computing system including a backend component, for example, a data server; or a computing system including a middleware component, for example, an application server; or a computing system including a frontend component having a graphical user interface or web browser on which a user can interact with an implementation of the subject matter described herein, for example, a client computer; or any combination of one or more such backend, middleware, or frontend components. The components of the system can be interconnected by digital data communication in any form or medium, such as a communication network. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), such as the Internet.

[0074] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via a communication network or cloud. The client-server relationship arises from computer programs running on each computer that have a client-server relationship with one another.

[0075] Furthermore, many embodiments are described in terms of a series of actions performed by elements of a computing device, etc. It will be recognized that the various actions described herein can be performed by a particular circuit (e.g., an application-specific integrated circuit (ASIC)), by program instructions being executed by one or more processors, or a combination of both. Furthermore, these series of actions described herein can be considered to be fully embodied in any form of computer-readable storage medium in which a set of corresponding computer instructions causing the associated processor to perform the functions described herein at runtime is stored. Thus, various aspects of the present invention may be embodied in several different forms, all of which are considered to be within the scope of the claimed subject matter. Furthermore, for each of the embodiments described herein, any corresponding form of such embodiment may be described herein as "logic configured to perform the described actions," etc.

[0076] A computer system may also include main memory, such as random access memory (RAM) or other dynamic storage devices coupled to the bus (e.g., dynamic RAM (DRAM), static RAM (SRAM), and synchronous DRAM (SDRAM)), for storing information and instructions executed by the processor. Furthermore, main memory may be used to store temporary variables or other intermediate information during the execution of instructions by the processor. The computer system may further include read-only memory (ROM) or other static storage devices coupled to the bus (e.g., programmable ROM (PROM), erasable PROM (EPROM), and electrically erasable PROM (EEPROM)), for storing static information and instructions for the processor.

[0077] A computer system may also include a bus-coupled disk controller to control one or more storage devices for storing information and instructions, such as magnetic hard disks and removable media drives (e.g., floppy disk drives, read-only compact disk drives, read / write compact disk drives, compact disk jukeboxes, tape drives, and removable magneto-optical drives). Storage devices may be added to the computer system using an appropriate device interface (e.g., Small Computer System Interface (SCSI), Integrated Device Electronics (IDE), Extended IDE (E-IDE), Direct Memory Access (DMA), or Ultra DMA).

[0078] Computer systems may also include special-purpose logic devices (e.g., application-specific integrated circuits (ASICs)) or configurable logic devices (e.g., simple-programmable logic devices (SPLDs), complex-programmable logic devices (CPLDs), and field-programmable gate arrays (FPGAs)).

[0079] The computer system may also include a bus-coupled display controller to control a display, such as a cathode ray tube (CRT), liquid crystal display (LCD), or any other type of display, for displaying information to the computer user. The computer system may also include input devices, such as a keyboard and a pointing device, for interacting with the computer user and providing information to the processor. Furthermore, a touchscreen may be used in conjunction with the display. The pointing device may be, for example, a mouse, trackball, or pointing stick for transmitting directional information and command selections to the processor and controlling cursor movement on the display. Additionally, a printer may provide a printed list of data stored and / or generated by the computer system.

[0080] A computer system performs some or all of the processing steps of the present invention in response to a processor executing one or more sequences of one or more instructions contained in memory, such as main memory. Such instructions may be read into main memory from a computer-readable medium, such as a hard disk or a removable media drive. One or more processors in a multiprocessing configuration may also be used to execute sequences of instructions contained in main memory. In alternative embodiments, hardwired circuitry may be used instead of or in combination with software instructions. Thus, the embodiments are not limited to any particular combination of hardware circuitry and software.

[0081] As described above, a computer system includes at least one computer-readable medium or memory for holding instructions programmed in accordance with the teachings of the present invention and for housing data structures, tables, records, or other data described herein. Examples of computer-readable mediums include compact disks, hard disks, floppy disks, tapes, magneto-optical disks, PROMs (EPROMs, EEPROMs, flash EPROMs), DRAMs, SRAMs, SDRAMs, or any other magnetic medium, compact disks (e.g., CD-ROMs), or any other optical medium, punch cards, paper tapes, or other physical mediums having a pattern of holes, carrier waves (described below), or any other medium that is computer-readable.

[0082] V. Tangible non-temporary computer-readable storage media Another aspect of the present invention, when executed by a computer processor, includes receiving a personal information profile via a user interface of an application running on one or more computer processors, wherein the personal information profile includes inputting personal information relating to a plurality of types of information selected from types of information including comorbidities, engineered features, pre-hospital interventions, injury mechanisms, and numerical variables; (b) storing the personal information of the subject in a database accessible by the application and accessible by the subject via the user interface of the application via one or more computer processors; and (c) receiving a data visualization dashboard via the user interface of the application running on one or more computer processors, wherein the dashboard displays predictions of healthcare resource requests corresponding to the personal information stored in the database. d) Assigning a computable value for any of the multiple types of information in the personal information stored in the database and a computable value point matrix stored in a memory device accessible by one or more computer processors, wherein the computable value point matrix is ​​generated and continuously updated by a machine learning module; (e) Determining in real time, based on an evaluation via one or more computer processors, the total computable value of the types of information in the personal information of the subject, wherein a predictive allocation of medical resources is generated and determined if the total computable value of the types of information in the personal information of the subject exceeds or falls below a pre-selected computable value; (f) Receiving a goal-setting module via a user interface of an application running on one or more computer processors, wherein the goal-setting module isThe present invention relates to a tangible non-temporary computer-readable storage medium that includes instructions to perform the following actions: (g) receiving input from a healthcare professional regarding computable values ​​to be assigned to types of information within the personal information that are the focus of clinical care based on the personal information stored in the database; and (b) notifying a healthcare professional providing medical care to the subject via the user interface about the predicted allocation of medical resources to the subject. In some embodiments, the tangible non-temporary computer-readable storage medium described herein, when executed by a computer processor, further includes instructions to cause the processor to perform the following actions: (i) enable one or more computing devices to communicate data with one or more computer processors, a data communication connection, and one or more tangible non-temporary computer-readable media accessible by one or more computer processors; (ii) store a personal information database; and (iii) input into a machine learning module, the personal information database and the machine learning module being stored in one or more tangible non-temporary computer-readable media, respectively.

[0083] Computer programs (also called programs, software, software applications, applications, scripts, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and can be deployed in any form, such as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. Computer programs do not necessarily correspond to files in a file system. A program can be stored in part of a file that holds other programs or data (for example, one or more scripts stored in a markup language document), in a single file dedicated to the program, or in a set of coordinated files (for example, a file storing one or more modules, subprograms, or parts of code). Computer programs can be deployed to run on a single computer, or on multiple computers located in one site or distributed across multiple sites and interconnected by a communication network.

[0084] Furthermore, certain methods and / or corresponding actions that support the steps and corresponding functions described herein may also be used to implement corresponding software structures and algorithms, as well as their equivalents. The processes described herein can be performed by one or more programmable processors (computing device processors) that run one or more computer applications or programs to perform the functions by manipulating input data to produce outputs.

[0085] Stored on any one or combination of computer-readable media, the present invention includes software for controlling a computer system, software for driving one or more devices for carrying out the present invention, and software for enabling the computer system to interact with a human user. Such software may include, but is not limited to, device drivers, operating systems, development tools, and application software. Such computer-readable media further include computer program products of the present invention for performing all or part of the processing performed when carrying out the present invention (if the processing is distributed).

[0086] The computer code or software code of the present invention may be any interpretable or executable code mechanism, including but not limited to scripts, interpretable programs, dynamic link libraries (DLLs), Java classes, and complete executable programs. Furthermore, some of the processing of the present invention may be distributed for better performance, reliability, and / or cost.

[0087] Various forms of computer-readable media may be involved in executing one or more sequences of one or more instructions to a processor for execution. For example, instructions may initially be executed on a magnetic disk of a remote computer. The remote computer may load instructions for remotely implementing all or part of the present invention into dynamic memory and transmit the instructions wirelessly (for example, via a wireless cellular network or WiFi network). A modem local to the computer system may receive data wirelessly and convert the data into an infrared signal using an infrared transmitter. An infrared detector coupled to a bus may receive the data carried by the infrared signal and place the data on the bus. The bus carries the data to main memory, from which the processor retrieves and executes the instructions. Instructions received by main memory may optionally be stored in a storage device before or after execution by the processor.

[0088] A computer system also includes a bus-coupled communication interface. The communication interface provides bidirectional data communication coupling to a network link connected to another communication network, such as a local area network (LAN) or the internet. For example, the communication interface could be a network interface card for connecting to any packet-switched LAN. Other examples include an asymmetric digital subscriber line (ADSL) card, an integrated services digital network (ISDN) card, or a modem providing data communication connectivity to a corresponding type of communication line. Wireless links may also be implemented. In any such implementation, the communication interface transmits and receives electrical, electromagnetic, or optical signals carrying digital data streams representing various types of information.

[0089] A network link typically provides data communication to a cloud over one or more networks to other data devices. For example, a network link may provide connection to other computers or remotely located presentation devices over a local network (e.g., a LAN) or through equipment operated by a service provider that provides communication services over a communication network. In preferred embodiments, the local network and communication network preferably use electrical, electromagnetic, or optical signals to carry digital data streams. Signals over various networks, as well as signals over network links and communication interfaces, carry digital data to and from computer systems, but are exemplary forms of carrier waves that transmit information. Computer systems can send and receive data, including program code, over networks, network links, and communication interfaces. Furthermore, a network link may provide, for example, connection to client devices or LAN-based connections to client devices such as personal digital assistants (PDAs), laptop computers, tablet computers, smartphones, or mobile phones. LAN communication networks and other communication networks, such as cellular wireless networks and Wi-Fi networks, may use electrical, electromagnetic, or optical signals to carry digital data streams. The processor system can send notifications and receive data, including program code, via networks, network links, and communication interfaces.

[0090] The present invention will be further illustrated by the following examples, which should not be construed as limiting. All references, patents, and published patent applications cited herein and throughout the figures and tables are incorporated herein by reference.

[0091] example Example 1: Machine learning predictions of trauma trigger levels in children differ depending on ground truth. Methods: Retrospective data were collected from the in-house trauma registry of all pediatric patients who triggered trauma team activation at a pediatric trauma center in western New York between January 2014 and January 2020. No patients were excluded from the analysis.

[0092] The collected data included patient demographics, injury mechanism, medical comorbidities, pre-hospital interventions, pre-hospital Glasgow Coma Score (GCS), pre-hospital revised trauma score (RTS), pre-hospital systolic blood pressure (SBP), initial GCS at the emergency department (ED), and initial SBP at the ED. Additional features were derived from the original dataset based on current in-facility trauma team activation criteria. Furthermore, trauma severity scores (ISS) and six NFTI criteria were collected, which included: 1) transfusion of packed red blood cells within 4 hours of arrival, 2) transfer from the emergency department (ED) to the operating room within 90 minutes, 3) transfer from the ED to the imaging-assisted treatment room (IR), 4) transfer from the ED to the intensive care unit (ICU) with an ICU stay of ≥3 days, 5) mechanical ventilation without procedural anesthesia within 3 days, and 6) death within 60 hours of arrival. We modeled the Trauma Team Activation (TTA) at two levels: full activation and partial activation.

[0093] Three ground truths were constructed for predictive modeling: 1) Cribari method: ISS > 15 required full activation; 2) NFTI: affirmation of any of the six NFTI criteria required full activation; and 3) Cribari + NFTI: ISS > 15 or affirmation of any of the six NFTI criteria required full activation. To determine over and under triage, each patient was assigned a TTA level based on each ground truth, and the ground truth TTA levels were compared to the actual activation levels assigned by the ED staff.

[0094] For machine learning, the dataset was randomly split into training data (80%), validation data (10%), and test data (10%). Two explainable conventional machine learning models (logistic regression and random forest) were trained / tested 1000 times in separate trials using each of the three ground truths. Hyperparameters for each model were tuned using model performance on the validation set. Continuous data are presented as mean (standard deviation [SD]) or median (interquartile range) where necessary. Categorical variables are presented as percentages (number [n]). Model performance was evaluated using the area under the receiver operating characteristic curve (AUC), and 95% confidence intervals (CI) were estimated over 1000 trials.

[0095] Results: Of the 1366 patients included, 230 (16.8%) triggered a full activation. Table 1 shows the percentage of patients identified as undertriaged, overtriaged, and properly triaged for each ground truth.

[0096] [Table 1]

[0097] Cribari and NFTI identified distinct but overlapping sets of undertriaged patients. For example, of 210 patients identified as undertriaged by NFTI, only 38.1% (80 patients) were also identified as undertriaged by Cribari (Figure 5). Therefore, Cribari + NFTI captured significantly more undertriaged patients compared to either method alone. Similarly, distinct but overlapping sets of patients were identified as overtriaged. Patients identified as undertriaged were compared among the three groups (Table 2). NFTI detected significantly more undertriaged children with penetrating injury mechanisms compared to Cribari. Furthermore, the Cribari + NFTI combination captured more patients with penetrating injury mechanisms compared to either method alone (Table 2).

[0098] [Table 2]

[0099] When specific injury mechanisms were evaluated, NFTI detected more undertriaged children with stab wounds compared to Cribari. There were no other significant differences for individual injury mechanisms. Patients with stab wounds undertriaged by NFTI had a mean ISS of 6.2 (8.6), indicating why more patients were missed by Cribari. Furthermore, 75% of these patients were transported to the operating room within 90 minutes of arrival. The mortality rates of undertriaged patients captured by all three ground truth models were similar (Table 2). However, of the 46 patients who died (mortality rate of 3.4% overall), Cribari captured only 67.4% (31 patients), while NFTI and Cribari+NFTI captured 97.8% (45 patients). All machine learning models outperformed TTA decisions made by ED staff. Classification by ED staff showed modest accuracy with significant variability compared to machine learning models, with the highest accuracy achieved when NFTI was used as ground truth (Table 3).

[0100] [Table 3]

[0101] In terms of pure model performance, logistic regression using Cribari as the ground truth performed best. However, as mentioned above, each ground truth identified a different subset of patients that met the criteria for full activation.

[0102] Trauma systems are reducing mortality after severe trauma and rely on efficient systems that accurately match the needs of injured patients with appropriate hospital resources and services. The results of this study highlight the challenges associated with optimizing TTA criteria even within a single institution. This effort demonstrates that the two most common methods of retrospective triage assessment differ in their ability to detect specific injury characteristics. Specifically, NFTI is more sensitive to penetrating injury mechanisms when the cumulative injury load is not sufficient to raise the ISS above 15.

[0103] However, due to the significant damage load, 75% of patients with stab wounds designated as undertriaged by the NFTI required surgical intervention within 90 minutes of arrival.

[0104] Furthermore, NFTI captured 97.8% of undertriage-related deaths in this pediatric population, compared to only 67.4% by Cribari. Since undertriage is more likely to have adverse effects than overtriage, these results indicate that using Cribari + NFTI for ground truth is superior to using either method alone.

[0105] Reflecting the complexity of the triage process, all machine learning models in this study performed better than ED staff, regardless of the ground truth used.

[0106] This demonstrates that machine learning tools can be useful in hospital triage situations.

[0107] The Cribari+NFTI ground truth reduced the performance of machine learning models compared to each method alone. However, this combination captured more clinically important information, resulting in a more ideal ground truth. This was an unexpected outcome.

[0108] The logistic regression model performed best when using Cribari as ground truth. This may be because logistic regression better estimates a continuous measure of trauma severity compared to the six binary responses provided by the NFTI. While the logistic regression model works well with binary data (e.g., those used in Cribari's ISS score), pure model performance is not equivalent to clinical significance.

[0109] In conclusion, the ideal TTA guideline is concise, captures all patients who truly need the highest level of trauma activation, and captures no one who does not.

[0110] These results demonstrate that machine learning can be used to help optimize TTA decisions in emergency departments, but clinical judgment and experience are crucial even during the model-building phase to ensure that performance metrics are grounded in the real world.

[0111] Example 2: A deep learning model for predicting trauma initiation levels The results demonstrate the feasibility of using a machine learning approach for assigning trauma threshold levels, as incorporating deep learning improves the algorithm's robustness to missing data, a challenge often encountered in real-world clinical scenarios. While the results show that the interpretable algorithms function within acceptable standards, no comparisons have been made between the deep learning models and existing triage accuracy, or with the results of conventional interpretable ML models. Therefore, a retrospective study compared the performance of three deep learning models [1. Convolutional Neural Network (CNN), 2. Generative Adversarial Network (GAN), 3. Autoencoder] with the interpretable ML model developed herein. Model performance was evaluated using accuracy, recall, precision, area under the receiver operating characteristic curve (AUC), and 95% confidence interval (CI). Based on the data, successful systems achieved AUC ≥ 80 and 95% CI < ±0.04 while minimizing undertriage.

[0112] Retrospective data were collected from the in-house trauma registry of all pediatric patients who triggered trauma (January 2014 to January 2022). The collected data included patient demographics, mechanism of injury, ICD-9 / 10 code, medical comorbidities, prehospital interventions, prehospital Glasgow Coma Score (GCS), prehospital Modified Trauma Score (RTS), prehospital systolic blood pressure (SBP), initial GCS at the ED, initial SBP at the ED, ISS, and six NFTI criteria (1. transfusion of packed red blood cells within 4 hours of arrival, 2. transfer from the ED to the operating room within 90 minutes, 3. transfer from the ED to the imaging-assisted treatment room (IR), 4. transfer from the ED to the intensive care unit (ICU) with an ICU stay of ≥3 days, 5. mechanical ventilation without procedural anesthesia within 3 days, and 6. death within 60 hours of arrival). ML features (Table 4) were extracted from the collected variables, and additional features were engineered based on the criteria for triggering intra-facility trauma.

[0113] [Table 4] JPEG2026509718000006.jpg87170

[0114] Trauma onset levels were modeled using two levels: full onset and partial onset. The following three ground truths were used for predictive modeling: 1) Cribari method: ISS > 15 indicates full onset, 2) NFTI: affirmation of any of the six NFTI criteria indicates full onset, and 3) Cribari + NFTI: ISS > 15 or affirmation of any of the six NFTI criteria indicates full onset. To determine over and under triage, each patient was assigned a trauma onset level based on each ground truth and then compared to the actual onset level assigned by the ED staff.

[0115] Specifically, when applying it to the algorithm, the results of three criteria sets—Cribari, NFTI, and Cribari+NFTI—were applied as labels to each patient included in the training data. These labels then communicated the "correct" answer, or ground truth, for each patient to the computer. While ground truth is used to adjust the decision threshold so that the computer can optimize the threshold to obtain the correct answer as much as possible, ground truth is not actually used as a criterion for making trauma team activation decisions. Essentially, ground truth is used to check how the algorithm works, which is similar to how it would be used in a clinical setting.

[0116] Furthermore, since we use the labels provided by Cribari to check the algorithm's behavior in cases where Cribari fails due to incomplete or missing ISS, it is crucial to include patients with incomplete or missing ISS. Because the goal is to determine the best criterion for training the algorithm, if we choose Cribari, the algorithm needs to be able to handle these edge cases (i.e., adjust to these situations). Removing such outliers increases the likelihood of the algorithm failing when faced with such cases in a prospective manner. If these outliers cause the model to produce triage-level mispredictions, then Cribari is not the ideal criterion to use for training. Based on the research described herein, combining Cribari + NFTI mitigates this concern to some extent, creating the best criterion for training the algorithm to handle such missing, inaccurate, or outlier data.

[0117] For machine learning, the dataset was randomly split into a training set (80%), a validation set (10%), and a test set (10%) for each trial. Three deep learning models (CNN, GAN, and autoencoder) were each trained / tested 1000 times in separate trials using each of the three ground truths. The validation set was used to tune the hyperparameters of each model, balancing pure algorithmic performance against clinical performance metrics (e.g., minimizing undertriage). Algorithmic performance was evaluated using accuracy, recall, precision, AUC, and 95% CI built up over 1000 trials. Based on preliminary data, the optimal system achieved AUC ≥ 80 and 95% CI < ±0.04 while keeping the triage rate below 12%.

[0118] Finally, the performance of all three deep learning algorithms was compared to the same performance metrics generated for existing interpretable ML algorithms, as well as the actual performance of human clinicians. The best algorithm, which balanced both clinical metrics and pure algorithmic performance, was selected for further testing. Based on the data, the successful deep learning algorithms outperformed human clinicians in terms of accuracy and variability in assigning trauma trigger levels, while meeting the model performance standards described above.

[0119] Example 3: Graphical user interface for implementation While a wide variety of predictors can be used for initial model development and tuning, the performance of the algorithm depends most heavily on a weighted subset of variables or features. For implementation purposes, it is unrealistic to expect clinicians to input a large number of variables into the model prior to obtaining predicted trauma-initiated levels. A weighted feature map was generated for each model tested herein. The top 20 features of each model were used as predictors to perform retrospective predictions of trauma-initiated level assignments and evaluate model performance. Clinical concerns were balanced by selecting and fine-tuning the optimal subset of features for each model to optimize predictive performance. Some decrease in model performance with a limited subset of features is expected. Therefore, successful systems continued to outperform human clinicians, achieving AUC > 85 and 95% CI < 0.04 while keeping the undertriage rate below 12%.

[0120] Example 4: Interpretable ML models vs. Acceptability of deep learning models The ultimate success of an algorithm implementation depends on both model performance and its acceptability to the target end-user. An algorithm acceptability survey was conducted using standardized triage scenarios (survey design), comparing three groups: 1. no decision support, 2. deep learning support, and 3. interpretable model support. Multiple clinician types (trauma surgeons, emergency physicians, and communication nurses) were surveyed, and the support models were evaluated for overall acceptability, preference, and reliability of model outputs. Additional trials demonstrated the feasibility of model implementation and generated preliminary data for larger trials. Accuracy in trauma trigger levels was compared with and without decision support using a randomized design in which triagers were not presented with either decision support or model support, balancing this with clinician feedback on the model and other characteristics (e.g., acceptability, ease of use). Implementation feasibility is demonstrated when the selected model improves the accuracy of trauma triggers, reduces variability, and is considered acceptable for clinical use.

[0121] In collaboration with survey design experts, we developed 12 standardized triage scenarios that represent a variety of patient presentations, differing in trauma severity, availability of pre-hospital information, and the presence or absence of other missing information (Table 5).

[0122] [Table 5]

[0123] A randomized controlled trial was conducted using the standardized triage scenario developed in this invention. Three groups (1. no decision support, 2. deep learning support, 3. interpretable model support) were compared among multiple clinician types (e.g., trauma surgeons, emergency physicians, and communication nurses). Clinicians evaluated the support models in terms of overall acceptability, preference, and reliability of model outputs. An ideal model has high reliability of model outputs, is accepted and preferred by end-users.

[0124] Additional data identified opportunities for process / quality improvement in existing trauma triggering workflows that could be leveraged to support implementation. Trials demonstrated the feasibility of model implementation and generated data for larger, prospective trials. A randomized design was created in which triage personnel were not presented with either decision support or model support. Accuracy in determining trauma trigger levels was compared with and without decision support, and balanced with clinician feedback on support and other characteristics (e.g., acceptability, ease of use). Implementation is considered feasible if the selected model improves accuracy in determining trauma triggers, reduces variability, and is considered acceptable for clinical use.

[0125] Neither the Cribari nor the NFTI criteria can be used for prospective prediction. In this study, we will not use Cribari or the NFTI prospectively to assess patients. Rather, we will ask the algorithm to assess patients using its internal criteria, and the algorithm will check its performance against the NFTI and Cribari (much like in a qualitative review). As an example, the algorithm will predict a specific trauma team activation level for a patient based on a specific variable, and then check that prediction against the NFTI. If the patient was not properly triaged, the algorithm will adjust the “weight” of that variable to mark it as less important in future iterations. This training process will be repeated until the decision threshold is optimized (as much as possible) and the available pre-hospital predictor variables are weighted according to their importance in the prediction. These variables, weights, and decision threshold will then be used prospectively to predict the appropriate level of trauma team activation.

[0126] Example 5: Demonstrative Improvement in Triage Decision Performance Using Machine Learning Models This study investigated the variation in undertriage rates by ED staff at different patient arrival times, as shown in Panel B of Figure 6. This study showed that, despite ground truth activation levels remaining nearly constant throughout the day, undertriage rates appeared inconsistent depending on arrival time, as shown in Figure 7. A two-sample proportion test comparing undertriage rates in the ED during the day (6 AM to 6 PM) and at night (6 PM to 6 AM) showed a statistically significant difference at a significance level of α=0.05 (two-tailed test), indicating that undertriage was significantly higher during the day.

[0127] To identify where classification errors occurred in the ED, this study considered under and overtriage rates in different groups within the data. A significance level of α=0.05 was used to determine whether the differences in these rates between different groups were statistically significant. In evaluating the model's performance, this study took into account any discrepancies found.

[0128] The data showed that the undertriage rate among ED staff was 65.5%, and the overtriage rate was 16.5%. For reference, Table 6 below shows the under / overtriage targets and the current ED staff levels.

[0129] [Table 6]

[0130] The mechanisms of injury were divided into eight categories (Panel A in Figure 6).

[0131] ·Motor vehicle collision (MVC)

[0132] • Injuries caused by falling (falls)

[0133] ·Puncture wound (puncture wound)

[0134] Gunshot wound (GSW)

[0135] • Sports injuries (sports)

[0136] • Vehicle hitting a pedestrian (pedestrian)

[0137] • Injuries caused by assault (assault)

[0138] • Injuries unrelated to the above categories (other)

[0139] A two-sided ratio test of one versus the other at a significance level of α=0.05 revealed significant differences in undertriage rates in six of the eight injury mechanisms. These six mechanisms are listed below, along with whether their associated undertriage rates were higher or lower than those of the remaining mechanisms grouped together.

[0140] 1) Gunshot wounds - Low undertriage rate

[0141] 2) High rate of fall-and-under triage.

[0142] 3) Low rate of undertriage in car collisions

[0143] 4) Vehicles hitting pedestrians - Low undertriage rate

[0144] 5) Sports injuries - High rate of undertriage

[0145] 6) Stab wounds - Low undertriage rate

[0146] Similar tests comparing overtriage rates across various injury mechanisms showed that each overtriage rate differed significantly from the remaining mechanisms. This is expected, given that undertriage and overtriage are generally inversely proportional, considering that undertriage rates differ depending on the injury mechanism.

[0147] Patient age was another factor that influenced under and overtriage rates in the ED. Figure 8 shows under and overtriage rates for patients of various ages. These plots show a general trend that undertriage rates generally decrease with increasing age, and overtriage rates generally increase with increasing age.

[0148] A two-sample test comparing undertriage rates for patients above and below the median age of 47.57 years in the dataset showed that the undertriage rate was significantly higher for patients above the median age (Figure 9).

[0149] The data includes 10,959 records of patients who were admitted to the ED between 2014 and 2021. The raw data has over 200 features related to the patient's medical history and pre-hospital interventions. These features include patient demographic information, mechanism of injury, pre-hospital interventions, patient comorbidities, ED procedures performed on the patient, and many other variables.

[0150] This data includes two ground truth variables, Cribari and NFTI, related to the level of treatment required by the patient. Ground truth variables are determined using various criteria after the patient's treatment. The Cribari variable is available for all records in the data, while the NFTI variable is not available for records from 2014 to 2019.

[0151] Both ground truth variables are binary. "1" is "fully activated," meaning the patient requires a high level of treatment. "0" is "partially activated," meaning the patient does not require a high level of treatment (see panel A in Figure 10).

[0152] Regarding features with missing data that were not discarded, this study found several sets of concurrently occurring missing values ​​in individual samples. These concurrently occurring missing values ​​were related to vital sign measurements, and each set of missing features consisted of different summary values ​​within the same set of values. Because these groups of missing features occurred concurrently, and the missing data was not random but rather due to underlying circumstances concerning the patients, this study used the MICE (Multivariate Imputation By Chained Equations) algorithm to impute the missing values. This algorithm randomly imputes the missing values ​​and then iteratively updates the imputed values ​​using a consistency metric across the entire dataset as an optimization metric. Once the missing values ​​are imputed, the data can be saved for future use.

[0153] Feature selection is an essential step in model building because it helps reduce data dimensionality, mitigate overfitting, and improve model performance. In this study, we used the Recursive Feature Elimination (RFE) method for the feature selection task.

[0154] The RFE method iteratively removes the least important features from the dataset based on the importance ranking assigned by the estimator. The algorithm continues until the desired number of features is reached or the performance metrics reach a satisfactory level.

[0155] In this study, we used XGBooster and the Random Forest algorithm as estimators in the RFE method. XGBooster is a powerful machine learning algorithm widely used for classification and regression problems. It is popular because it can handle missing values, scale to large datasets, and provide excellent accuracy.

[0156] Furthermore, to evaluate the effectiveness of recursive feature removal using XGBooster as an estimator, this study also used the random forest algorithm, a versatile machine learning algorithm known for its robustness against noise and outliers and suitable for both regression and classification problems, as an alternative estimator for feature selection.

[0157] As shown in Panel B of Figure 10, this study analyzed the distribution of ground truth variables for each selected feature. The features are O2 (oxygen), BVM (bag-valve-mask ventilation), ETT (intubation), and Suck (airway suctioning). For example, the set of samples that received O2 had a higher percentage of patients in full activation. When a patient's feature was observed to be "1" in this study, the patient was more likely to require full activation, as shown in Panel B of Figure 10.

[0158] When this model is used as an aid tool for healthcare professionals, they need to be able to trust the model, understand the degree of reliability of each prediction, and check how various features of a given sample (i.e., the attributes of the arriving patient) influence decision-making. To meet these needs, this study implements a LIME (Local Interpretable Model-agnostic Explanations) explainer using the model [M. Ribeiro, S. Singh, C. Guestrin, "Why Should I Trust You?: Explaining the Predictions of Any Classifier," Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, California, USA, August 13-17, 2016].

[0159] The LIME explainer works by taking a single input (data from one patient) and performing small perturbations around feature values ​​to see how changes in these values ​​affect the output classification. The explainer provides information only about how individual classifications were determined, rather than providing a general explanation of how the model makes predictions. This is well-suited to these needs when implemented as an aid tool, as it only needs to explain one decision at a time.

[0160] A visualization based on the output of the LIME explainer is shown in Figure 11. The trigger level determined by the model is shown at the top of the figure, and the confidence level of the reported classification is shown below it. The confidence level is thought to be the probability assigned to the trigger level selected by the model. In the bar graph in the figure, there is one bar for each feature, and they are arranged in order of influence (the top bar has the greatest influence on partial trigger, and the bottom bar has the greatest influence on full trigger). The color of each bar indicates how much the value of the feature it represents influenced the model's decision toward full or partial trigger, and the length of the bar indicates how strong that influence was. The example shown in the figure is a patient classified by the model as requiring full trigger, and the feature that had the greatest influence on the determination of the direction of full trigger was stab wound (STAB).

[0161] A series of two-sample proportion tests at a significance level of α=0.05 showed that the model's overall accuracy, undertriage rate, and overtriage rate were all significantly improved compared to the classification performance of the ED staff. The performance indicators for the model and ED staff are shown in Table 7 below.

[0162] [Table 7]

[0163] By comparing multiple metrics using the Cribari method as the ground truth, it has been shown that the model outperforms human classification in terms of accuracy and false positive and false negative error rates. Success criteria of overtriage error rate <5% and undertriage error rate 25-35% are also nearly achieved, and given that the model only utilizes pre-treatment features, it can be replicated with minimal effort at any trauma center. This is in contrast to the Cribari ground truth, which considers a combination of pre- and post-treatment features for modeling purposes.

[0164] Example 6: Machine learning improves the accuracy of assigning trauma team activation levels to pediatric patients. This study retrospectively collected data from the pediatric trauma center's in-house trauma registry and electronic medical records for all patients (under 18 years of age) who triggered trauma team activation (January 2014 to December 2021). Data included demographics, injury mechanism, comorbidities, pre-hospital interventions, numerical variables, and six "Need for Trauma Intervention (NFTI)" criteria. Using the union of the Cribari and NFTI indices as ground truth (full activation occurring when trauma severity score > 15 or any of the six NFTI criteria were affirmative), three machine learning models (logistic regression, random forest, and support vector machine) were tested 1000 times in separate trials. Model performance was quantified and compared to that of emergency department (ED) staff.

[0165] To mimic the information available to ED staff at or near patient arrival, predictor variables, or features, were identified based on pre-hospital conditions or their availability immediately after patient arrival. Multiple features were extracted from text data of International Classification of Diseases (ICD) 9 / 10 codes, pre-hospital interventions, and comorbidities, and encoded into variables. Additional features were engineered from the original dataset based on the latest intra-institutional trauma center triage guidelines. These features were then used as input to machine learning algorithms (see Table 4 above for exemplary predictors used by the machine learning models).

[0166] Table 8 below shows the top 10 features for each of the logistic regression, random forest, and support vector machine models. Features in bold are those shared across all models, and features underlined are those shared by two out of the three models.

[0167] [Table 8]

[0168] Missing values ​​were identified in pre-hospital SBP, GCS, and RTS. To leverage the relationships between each clinical variable, this study utilized multivariate iterative imputation with chained equations (MICE) [Getz K, Hubbard RA, Linn KA, "Performance of multiple imputation using modern machine learning methods in electronic health records data," Epidemiology, 2023, 34, 206e15]. MICE is the most widely used imputation method for clinical data and works by iteratively learning how to imputate a given variable using other variables until convergence occurs.

[0169] The primary outcome of interest was the predicted trauma activation level, defined as the “level” assigned to the patient upon arrival at the ED. In this exemplary embodiment, three levels of resource activation are possible: Level 1 (full trauma activation), Level 2 (partial trauma activation), and Trauma Consultation (partial trauma activation). Each patient is assigned an actual activation level by the ED staff upon arrival. However, this actual activation level cannot be used as ground truth for predictive modeling due to the possibility of over / undertriage caused by inherent human biases and errors.

[0170] Therefore, in this study, we chose to use the union of two retrospective assessment tools: 1) the Cribari matrix and 2) the NFTI criteria. To assess over and undertriage, each patient was assigned a predicted trigger level, and full trigger was triggered by an ISS > 15 or an affirmative response to any of the six NFTI criteria. Partial trigger was predicted if all six NFTI criteria were negative and the ISS < 15. For each patient, the predicted trigger level was then compared to the actual trigger level to determine whether there was overtriage, proper triage, or undertriage. This approach is also referred to in this application as the "ISS and NFTI combination."

[0171] We trained three common, state-of-the-art, explainable, conventional machine learning models: logistic regression, random forest, and support vector machines. Each model was trained on all 104 features, including patient characteristics, comorbidities, and outcomes. Hyperparameters for each model were tuned according to grid search, and the best version of each model was compared to the ED staff. The training data was labeled using the union of Cribari+NFTI ground truths.

[0172] To train the predictive model, the dataset was randomly split into training data (80%) and test data (20%). For example, in the training set, 209 patients received trigger level 1 (full) and 1015 patients received trigger level 0 (partial / consultative). In the test set, 41 patients received trigger level 1 (full) and 101 patients received trigger level 0 (partial / consultative). All analyses were performed using Python 3.7.0, and the model was trained and evaluated using the Scikit-Learn 1.0.1 package.

[0173] Logistic regression was used to learn a linear combination of input variables, which was then scaled to probabilities within the final logistic function. Essentially, this type of machine learning is interpretable and explainable in that the learned linear coefficients can be used to understand which clinical variables had the greatest impact on the final model predictions. For reproducibility, the chosen parameters were balanced class weights and a maximum of 2e iterations. 3 The solution used was C=10, and it was a Newton-CG solver.

[0174] Random Forest is an ensemble-based machine learning model that improves accuracy and prevents overfitting by training several simple decision trees on subsamples of a dataset and combining the results of the decision trees using averaging. Its decision tree backbone allows this study to clearly understand which binary decisions are being made and which are most important for the final prediction. For reproducibility, the chosen parameters are balanced class weights and a maximum number of estimators of 1e. 3 The criteria were a maximum depth of 5, an entropy criterion, and determination of the maximum number of features by square root.

[0175] A support vector machine finds a boundary between two binary classes that enables separation by maximizing the distance between this boundary and the data points. This model uses a linear support vector machine, where a linear boundary is learned, allowing the user to identify which of the clinical variable coefficients is weighted most heavily in constructing the linear boundary, thus enabling clearer interpretation. For reproducibility, the chosen parameters are balanced class weights, a linear kernel, and a maximum of 1e iterations. 8 , and C=0.95.

[0176] In this study, we use the Area Under the Receiver Operating Characteristic Curve (AUC) and the F1 score as primary performance metrics because they are typically used to assess the predictive strength of binary classification models in imbalanced data situations [Jeni LA, Cohn JF, De La Torre F, "Facing imbalanced data-recommendations for the use of performance metrics," Humane Society Conference on Sentimental Computing and Intelligent Interaction, 2013, p.245e51]. AUC considers the entire range of the classification threshold for the probability of the model's prediction, thus capturing the model's ability to correctly classify both positive and negative cases regardless of imbalance. F1 assesses a model that balances correctly identifying positive cases with minimizing false positives, thus providing a more balanced metric that highlights model performance in minority classes. Both of these metrics range from 0 to 1, where 1 is best and 0 is worst. In contrast, accuracy is an insufficient evaluation metric when given imbalanced data, as it has low sensitivity to minority classes and may be determined solely by the majority class. Nevertheless, for completeness, accuracy metrics are also reported. 95% confidence intervals were created after 1000 resampling, retraining, and retesting (with different random initializations) for each model. Univariate analysis was performed using MATLAB (R2022b, The MathWorks Inc, Natick, MA).

[0177] For sufficient patient demographics and injury characteristics, 1366 patients were included in this analysis. The mean ± standard deviation (SD) of ISS across the group was 9.39 ± 10.1 [median (IQR) = 6.00 (1.0e13.0)], with 18.01% (n=246) having an ISS > 15. Overall, 26.65% (n=364) had an affirmative result on at least one of the NFTI criteria. ED staff triggered full trauma team activation in 16.84% (n=230) of patients (actual activation level).

[0178] When the union of Cribari+NFTI metrics was used as the ground truth, the accuracy of the ED staff was 75%, but the AUC was 0.73±0.04 and the F1 score was only 0.485 (see Table 9 below (Machine learning performance metrics compared to the ED staff using Cribari+NFTI as the ground truth)).

[0179] [Table 9]

[0180] All machine learning models outperformed the ED staff in all performance metrics. Of the three machine learning models tested, the support vector machine model performed best, with the highest AUC of 0.813±4.12e-5, the lowest F1 score of 0.800±3.33e-16, and the lowest variance (95% CI). The relevant receiver operating characteristic curve is shown in Figure 12.

[0181] Over- and under-triage also occur in model predictions. Therefore, it is essential to understand the over- and under-triage rates and compare them to those of ED staff. As mentioned above, the overall over-triage rate for ED staff was 5.05% (n=69), and the under-triage rate was 19.99% (n=273). Table 10 below shows the over- and under-triage rates (the percentage of patients over- and under-triaged using the test dataset) for all three machine learning models and ED staff using the same test dataset (excluding model training data). The over-triage rate was calculated by dividing the number of patients predicted by each model to require full activation but who did not require full activation according to the Cribari+NFTI ground truth by the total number of patients in the test set (n+142). Similarly, the undertriage rate was calculated by dividing the number of patients who required full triage (though not all models predicted full triage) by the total number of patients in the test set, according to the Cribari+NFTI ground truth.

[0182] [Table 10]

[0183] The support vector machine model minimized undertriage at a rate of 11.27% (n=16) while maintaining overtriage at a similar rate of 8.45% (n=12).

[0184] The top five features of importance in the decision-making process of the support vector machine model were (1) blood transfusion (i.e., the patient had received or was receiving a blood transfusion before arrival), (2) gunshot wound to the trunk, (3) intraosseous access before hospitalization, (4) gunshot wound to the head, and (5) Glasgow Coma Scale score <8 (Figure 13).

[0185] All machine learning models outperformed ED staff across all performance metrics. These results demonstrate that data-driven methods can optimize trauma team activation in the ED while improving both patient safety and hospital resource utilization.

[0186] The ED staff model had an accuracy of 75%, an area under the curve (AUC) of 0.73±0.04, and an F1 score of 0.49. The support vector machine, which performed best among all machine learning models, had an accuracy of 80%, an AUC of 0.81±4.1e-5, and an F1 score of 0.80, with smaller variance compared to other models and the ED staff model.

[0187] The results of this study demonstrate that machine learning-based tools can be used to improve accuracy and reduce variability in trauma team activation decisions in emergency medical systems (EDs). Specifically, the support vector machine model used in this study improved the AUC by 8.3% and reduced variability by several orders of magnitude. These performance improvements were achieved using a fully explainable model while reducing undertriage rates and maintaining overtriage rates. Overall, these results indicate that data-driven methods can be used to optimize trauma team activation in EDs, thereby improving both patient safety and hospital resource utilization.

[0188] Interestingly, six of the ten most heavily weighted features by the model were engineered features derived from in-house triage criteria, indicating that the model's predictions of trigger levels are consistent with those used by healthcare providers. Furthermore, pre-hospital interventions that may suggest a patient has sustained more severe injury (i.e., intraosseous access, airway suctioning, bleeding control, oxygen administration, and needle thoracotomy) accounted for five of the top 15 features (Figure 13).

[0189] During the training process, the algorithm adjusts this weighting each time it checks its performance against ground truth. In certain embodiments, an ablation study is used to sequentially remove individual variables and determine their impact on predictive accuracy. This allows for the removal of selected variables with minimal impact, ultimately resulting in fewer than 10 predictor variables available to accurately assign trauma team activation levels.

[0190] Example 7: When prioritizing patient safety, Cribari+NFTI appears to be ideal for training machine learning algorithms that predict trauma team activation levels. Trained ED staff triggered full activation in 230 out of 1,366 patients (16.84%) and partial activation in 1,136 patients (83.16%, Panel A in Figure 14). Cribari resulted in full activation in 246 patients (18.01%) and partial activation in 1,120 patients (81.99%). NFTI resulted in full activation in 364 patients (26.65%) and partial activation in 1,002 patients (73.35%). NFTI+Cribari resulted in full activation in 434 patients (31.77%) and partial activation in 932 patients (68.23%). The breakdown of full and partial activation for each injury mechanism is shown in Panel B of Figure 14.

[0191] Cribari and NFTI identified a set of patients with incomplete overlap who were predicted to require full activation (Panel A in Figure 15). Full agreement was observed among Cribari, NFTI, and ED staff in only 7.03% of patients (n=96) (Panel B in Figure 15), but agreement with ED staff was observed in 11.79% of patients (n=161) using the combined Cribari+NFTI index. To better understand the behavior of the three ground truths, patient characteristics were compared among the three groups with respect to undertriage and overtriage.

[0192] Under Triage When using Cribari, 143 patients (10.47%) were considered undertriaged, compared to 210 patients (15.37%) when using NFTI. The combined Cribari+NFTI index resulted in an undertriage rate of 19.99% (n=273). There were no demographic differences among patients undertriaged according to each of the three ground truths. NFTI and Cribari+NFTI were more sensitive to undertriage in patients with penetrating injury mechanisms (p=0.006) and detected more undertriaged children with puncture wounds compared to Cribari (p=0.014). There were no differences between groups regarding pre-hospital interventions or vital signs at ED arrival.

[0193] puncture wound Across the cohort, 95 patients were stabbed, with a mean ISS of 4.96 (7.05). Fourteen patients (14.74%) with stab wounds triggered full activation by ED staff. Cribari indicated that only 8 patients (8.42%) required full activation, while NFTI and Cribari+NFTI estimated that 32 patients (33.68%) and 34 patients (35.79%), respectively, required full activation. Of the 34 patients identified as requiring full activation by the Cribari+NFTI index, 8 (23.53%) had an ISS > 15, 8 (23.53%) received blood transfusions within 4 hours of arrival, 4 (11.76%) were placed on a ventilator without procedural anesthesia within 3 days of arrival, 23 (67.65%) were transferred directly from the ED to the operating room within 90 minutes of arrival, 3 (8.82%) were transferred directly from the ED to the imaging-assisted treatment room, 6 (17.85%) were transferred from the ED to the ICU with an ICU LOS of 3 days or more, and 2 (5.88%) died within 60 hours of arrival. Patients with stab wounds considered undertriaged by NFTI (n=24, 25.26%) had a mean ISS of 6.2 (8.6), which explains why these patients were missed by Cribari. However, 75% (18 / 24) of stab wounds undertriaged by NFTI were transported directly from the emergency department to the operating room.

[0194] Overtriage 126 patients (9.22%) were considered overtriaged using Cribari, compared to 75 patients (5.49%) using NFTI. The combined Cribari+NFTI index showed an overtriage rate of 4.98% (n=68). There were no demographic differences in patients overtriaged according to each of the three ground truths. Regardless of ground truth, overtriage was particularly prevalent in gunshot wounds. Compared to NFTI and Cribari+NFTI, Cribari showed overtriage in more patients with abuse as the mechanism of injury (p<0.001), more patients requiring prehospital airway management (p<0.001), and more patients receiving CPR in a prehospital setting (p=0.017). Furthermore, compared to NFTI and Cribari+NFTI, Cribari showed overtriage in patients with lower mean GCS scores at ED (p<0.001).

[0195] result Overall, the mortality rate in this dataset was 3.37% (n=46). Of the patients who died, 35 (76.09%) were triggered by a fully activated ED by trained ED staff, 32 (69.57%) required a fully activated ED according to Cribari, and 46 (100.00%) required a fully activated ED according to both NFTI and Cribari+NFTI (p<0.001). Of the 14 patients “missed” on the Cribari index, 7 did not have an ISS recorded in the trauma registry, 5 were recorded as dying at arrival or dying in the ED, 3 more were transported directly from the ED to the operating room, and 6 were transported directly to the ICU. There were no significant differences in mortality among the three ground truths for undertriaged patients (p=0.803). However, mortality was significantly higher in the Cribari overtriage group (7.14%, n=9) compared to the NFTI and Cribari+NFTI groups (0.00%, n=0, p=0.005). The mean length of stay across the entire dataset was 4.26 (7.76) days. In undertriaged patients, there was no significant difference among the three ground truths regarding LOS (p=0.664). However, Cribari showed overtriage in patients with significantly longer LOS compared to the NFTI and Cribari+NFTI groups (p=0.017).

[0196] This study demonstrated that the NFTI and Cribari+NFTI indices are more sensitive to penetrating injury mechanisms than Cribari alone, particularly detecting more undertriaged children with stab wounds. This is the first time this has been demonstrated. This is the first study to demonstrate a difference in the detection of a specific injury mechanism. The difference in detecting stab wounds is significant in the study population, as the Cribari index uses a minimum threshold of ISS > 15 to indicate the need for full trauma team activation, and the mean ISS for patients with stab wounds is only 4.95 ± 7.05, often involving a limited body area. Patients with stab wounds demonstrate why the Cribari+NFTI index is superior to either index alone from a patient safety perspective.

[0197] Regarding stab wounds in particular, approximately 68% of patients identified as requiring full activation by the combined indicators were transported directly from the ED to the operating room within 90 minutes of arrival, indicating a significant need for available resources upon arrival. The NFTI index alone captured most patients requiring full trauma team activation and all patients transported directly to the OR, while the Cribari index captured two more patients with stab wounds and elevated ISS.

[0198] The characteristics of overtriaged patients also highlight the limitations of the Cribari index. These results demonstrate that Cribari frequently overtriages patients with a high risk of serious injury, as patients requiring pre-hospital airway management or CPR, and those with low mean GCS scores at the ED, are more frequently overtriaged according to the Cribari index.

[0199] Perhaps most importantly, the Cribari+NFTI and NFTI indicators captured 100% of the mortality in this dataset, compared to only 69.6% by the Cribari indicator alone. In this study's comparison of indicators, the sensitivity of this result is nearly 100% because one of the criteria for NFTI includes death within 60 hours of arrival. However, from a patient safety perspective, it is a concern to note that the mortality rate in the Cribari overtriage group (7.14%) was significantly higher than the mortality rate for NFTI and the Cribari+NFTI indicator (0%).

[0200] One concern regarding the use of these metrics is that, because each metric is designed to determine triage quality in a retrospective manner, it may not be accurately applicable in pre-hospital or hospital triage settings. However, when used as ground truth for predictive models, these metrics can be used for prospective hospital triage to train models to identify and weight the most predictable pre-hospital variables (Liu CW, Chacon M, Crawford L, Polydore H, Ting T, Wilson NA, "Machine Learning Improves the Accuracy of Trauma Team Activation Level Assignments in Pediatric Patients," J Pediatr Surg., September 22, 2023, S0022-3468(23)00551-1). Therefore, in this study, prioritizing sensitivity to patient safety rather than hospital resource conservation, we found that the Cribari+NFTI metrics are ideal ground truth for training predictive models.

[0201] This endeavor is highly significant because determining a suitable ground truth for training machine learning algorithms is challenging. Furthermore, the pediatric trauma system is essentially derived from the adult trauma system, and there is limited pediatric literature exploring how these indicators specifically behave in pediatric patients. In addition to the usefulness of predictive modeling, this exploration of appropriate ground truth can lead to tangible quality improvement efforts in clinical practice and could bring similar benefits to other pediatric trauma programs.

[0202] While various embodiments have been described above, it should be understood that such disclosures are presented only as examples and are not limiting. Therefore, the breadth and scope of the subject matter and methods should not be limited by any of the exemplary embodiments described above, but should be defined solely in accordance with the following claims and their equivalents.

[0203] The above description is intended to teach those skilled in the art how to put the invention into practice and is not intended to detail all obvious modifications and variations that would be apparent to those skilled in the art upon reading this description. However, all such obvious modifications and variations are included within the scope of the invention as defined by the following claims. The claims are intended to cover components and steps in any order that is effective in achieving the purpose intended therein, unless the context specifically indicates otherwise.

Claims

1. A method for training a machine learning model to predict the allocation of medical resources needed to treat patients with traumatic injuries, (a) To establish a ground truth, the ground truth corresponding to a standardized level of trauma onset for a standardized patient having a standardized set of clinical conditions, (b) Assigning a first trauma trigger level to individual patients identified by a plurality of clinical conditions selected from the standardized set of clinical conditions, wherein a machine learning model assigns the first trauma trigger level by an algorithm including a plurality of relative computable values, each relative computable value being assigned to one or more clinical conditions within the standardized set of clinical conditions, (c) A human medical review to determine which of the set of clinical conditions each individual patient presented with, (d) Determining a second trauma-inducing level that matches the set of clinical conditions presented by each individual patient through a human medical review, (e) Inputting the second trauma trigger level, determined by a human medical review based on the set of clinical conditions presented by each individual patient, into the machine learning model, (f) Comparing the second trauma-inducing level determined by a human medical review with the first trauma-inducing level assigned by the machine learning model to individual patients, based on the set of clinical conditions exhibited by each patient, for each patient identified as having multiple clinical conditions selected from a standardized set of clinical conditions, (g) Based on the comparison, determine whether the first trauma trigger level assigned by the machine learning model corresponds to undertriage, overtriage, or an appropriate level of triage for the individual patient in the clinical setting, (h) Comparing the second trauma-inducing level, determined by a human medical review based on the set of clinical conditions presented by each individual patient, with the ground truth, (i) Based on the comparison, determine whether the second trauma trigger level corresponds to undertriage, overtriage, or an appropriate level of triage for the individual patient in the clinical setting, (j) Adjusting one or more relative computable values ​​used by the machine learning model when assigning the first trauma trigger level, in accordance with the determination of whether each individual patient received undertriage, overtriage, or an appropriate level of triage in the clinical setting. (k) Repeat steps (b) to (j) for multiple patients, each with a set of clinical conditions selected from the set of clinical conditions, until the machine learning model reaches an appropriate level of triage within the desired parameters. Methods that include...

2. The aforementioned desired parameters include the machine accuracy rate, and the machine accuracy rate is (1) The machine learning model assigns a third trauma trigger level to multiple test patients, each of whom is identified as having multiple clinical conditions selected from a standardized set of clinical conditions, (2) Compare the third trauma trigger level for each test patient with the ground truth, and based on the comparison, determine whether the third trauma trigger level assigned by the machine learning model corresponds to undertriage, overtriage, or an appropriate level of triage for each test patient. (3) The following equation, that is, Machine accuracy = (Number of test patients with the appropriate level of triage ÷ Total number of test patients) × 100% Based on this, the machine accuracy rate for the test patient is calculated, The method according to claim 1, as determined by...

3. The method according to claim 2, wherein the desired parameters further include an undertriage rate and / or an overtriage rate.

4. The method according to any one of claims 1 to 3, wherein the ground truth is based on the Crivari Matrix method (trauma severity score (ISS)) or the method for determining the need for trauma intervention (NFTI).

5. The Ground Truth is based on a combination of ISS and NFTI, according to any one of claims 1 to 3.

6. The method according to any one of claims 1 to 5, wherein the machine learning model is a supervised learning model.

7. The method according to any one of claims 1 to 5, wherein the machine learning model is a deep learning model.

8. The method according to any one of claims 1 to 7, wherein the set of clinical conditions includes one or more conditions selected from the group including acute injury associated with known or suspected child abuse, blood transfusion, hypotension, multiple rib fractures, pelvic fracture, severe head injury with trauma to the trunk or limbs, stab wounds to the neck, trunk, head, or groin and buttocks, traumatic amputation of the proximal wrist or ankle, suspected proximal fractures in two or more places, a Glasgow coma score of 8 or less, and gunshot wounds to the neck, trunk, head, arms, legs, or groin and buttocks.

9. The method according to any one of claims 1 to 8, wherein the set of clinical conditions includes one or more conditions selected from the group including asthma, hemorrhagic disorder, cirrhosis or ascites, current anticoagulation, current smoking, current steroid use, diabetes, drug use disorder, drug abuse disorder, drug abuse, functional dependence (pre-hospital), history of congenital anomalies, history of prematurity, history of mental disorder, attention deficit disorder, personality disorder, major mental disorder, transplant history, inflammatory bowel disease, Crohn's disease, ulcerative colitis, obesity, pre-existing spinal cord injury, pre-hospital cardiac arrest, renal failure or current dialysis, and seizures.

10. The method according to any one of claims 1 to 9, wherein the machine learning model assigns the first trauma trigger level based on a limited subset of the set of clinical conditions.

11. The method according to any one of claims 1 to 10, wherein the desired parameters of the machine learning model include an undertriage rate.

12. A method for allocating medical resources to a person for the treatment of traumatic injury, (a) A step of receiving a personal information profile via a user interface of an application running on one or more computer processors, wherein the personal information profile includes inputting personal information relating to a plurality of types of information selected from types of information including comorbidities, engineered features, pre-hospital interventions, injury mechanisms, and numerical variables, (b) The step of storing the subject's personal information in a database accessible by the application and accessible by the subject through the user interface of the application via one or more computer processors, (c) A step of assigning, via one or more computer processors, a computable value relating to any of a plurality of types of information in the personal information stored in the database and a computable value point matrix stored in a memory device accessible by the one or more computer processors, wherein the computable value point matrix is ​​generated by a machine learning module using a machine learning model trained by the method of claim 1 and is continuously updated. (d) A step of determining, via one or more computer processors, in real time based on an evaluation, a total computable value of the type of information in the subject's personal information, wherein a predictive allocation of medical resources is generated if the total computable value of the type of information in the subject's personal information exceeds or falls below a pre-selected computable value; (e) The step of notifying a healthcare professional who provides medical care to the subject via the user interface about the predicted allocation of medical resources to the subject, Methods that include...

13. The method according to claim 12, wherein the user interface is accessed by the medical professional on a mobile device.

14. The method according to claim 12, wherein the user interface is accessed by the medical professional on a wireless monitor.

15. The method according to any one of claims 12 to 14, wherein the engineered features include one or more features selected from the group including acute injuries associated with known or suspected child abuse, blood transfusions, hypotension, multiple rib fractures, pelvic fractures, significant head injuries with trauma to the trunk or limbs, stab wounds to the neck, trunk, head, or groin and buttocks, traumatic amputation of the proximal wrist or ankle, suspected proximal fractures in two or more places, a Glasgow coma score of 8 or less, and gunshot wounds to the neck, trunk, head, arms, legs, or groin and buttocks.

16. The method according to any one of claims 12 to 15, wherein the comorbidities include one or more conditions selected from the group including asthma, hemorrhagic disorder, cirrhosis or ascites, current anticoagulation, current smoking, current steroid use, diabetes, drug use disorder, drug abuse disorder, drug abuse, functional dependence (pre-hospital), history of congenital anomalies, history of prematurity, history of mental disorders, attention deficit disorder, personality disorder, major mental disorder, transplant history, inflammatory bowel disease, Crohn's disease, ulcerative colitis, obesity, pre-existing spinal cord injury, pre-hospital cardiac arrest, renal failure or current dialysis, and seizures.

17. The method according to any one of claims 12 to 16, wherein the pre-hospital intervention includes one or more interventions selected from the group including airway management, bag-valve-mask ventilation, intubation, airway suctioning, laryngeal mark airway placement, supraglottic support device placement, oral airway placement, bleeding control, tourniquets, manual compression, pelvic binder placement, hemostatic bandaging, cardiopulmonary resuscitation, electrocardiogram, intraosseous access, intravenous fluid administration, limb immobilization, medication administration, needle thoracotomy, oxygen, and spinal fixation.

18. The method according to any one of claims 12 to 17, wherein the injury mechanism includes one or more injury mechanisms selected from the group including abuse, assault, fall, gunshot wound, car collision, pedestrian hit-and-run, sports-related injury, and stab wound.

19. The method according to any one of claims 12 to 18, wherein the numerical variable includes one or more variables selected from a group including Glasgow Coma Scale score, modified trauma scale, mean pre-hospital systolic blood pressure, diastolic blood pressure, heart rate, and age.

20. The method according to any one of claims 12 to 19, wherein the type of information in the personal information is classified as either present or absent.

21. The method according to any one of claims 12 to 20, wherein the calculable value for each of the types of information in the personal information is selected in advance.

22. The method according to claim 12, wherein the pre-selected calculable value for each of the types of information in the personal information is selected to be incorrect in that it chooses to overtriage the subject.

23. (h) receiving a message inviting the healthcare professional to join a group of other healthcare professionals using the application via the user interface of the application running on one or more computer processors, wherein the group can communicate with each other via the user interface; The method according to any one of claims 12 to 22, further comprising:

24. The method according to any one of claims 12 to 23, further comprising the step of tracking over time the relative effective allocation or lack of allocation of medical resources related to the activation of a trauma team by the medical professionals, via the one or more computer processors and the summable values.

25. The further step includes preparing the system, and the system is (i) One or more computing devices that communicate data with each other, each having one or more computer processors, a data communication connection, and one or more tangible non-temporary computer-readable media accessible by the one or more computer processors, (ii) Personal information database and (iii) Machine learning module and Includes, The method according to any one of claims 12 to 24, wherein the personal information database and the machine learning module are each stored in one or more tangible, non-temporary computer-readable media.

26. The method according to claim 25, wherein the plurality of databases further include a medical resource database.

27. The method according to claim 25, wherein the medical resource database includes trauma team activation information, and the trauma team activation information is accessible by the medical professional via the user interface.

28. A system for allocating medical resources to individuals for the treatment of their traumatic injuries, One or more computer processors, One or more tangible computer-readable media accessible by the one or more computer processors, The one or more tangible computer-readable media, when executed by the one or more processors, (a) Receiving a personal information profile through a user interface of an application running on one or more computer processors, the personal information profile including inputting personal information relating to multiple types of information selected from types of information including comorbidities, engineered features, pre-hospital interventions, injury mechanisms, and numerical variables, (b) storing the personal information of the subject via one or more computer processors in a database that is accessible by the application and accessible by the subject via the user interface of the application, (c) Assigning, via one or more computer processors, a computable value relating to any of the multiple types of information in the personal information stored in the database and a computable value point matrix stored in a memory device accessible by the one or more computer processors, wherein the computable value point matrix is ​​generated and continuously updated by a machine learning module using a machine learning model trained by the method of claim 1. (d) Determining in real time, based on an evaluation via one or more computer processors, a total computable value of the type of information in the subject's personal information, and determining, if the total computable value of the type of information in the subject's personal information exceeds or falls below a pre-selected computable value, a predictive allocation of medical resources is generated and determined. (e) Notifying healthcare professionals who provide medical care to the subject via the user interface about the predicted allocation of medical resources to the subject, A system that includes instructions to execute something.

29. When executed by a computer processor, the processor: (a) Receiving a personal information profile through a user interface of an application running on one or more computer processors, the personal information profile including inputting personal information relating to multiple types of information selected from types of information including comorbidities, engineered features, pre-hospital interventions, injury mechanisms, and numerical variables, (b) storing the personal information of the subject via one or more computer processors in a database that is accessible by the application and accessible by the subject via the user interface of the application, (c) Assigning, via one or more computer processors, a computable value relating to any of the multiple types of information in the personal information stored in the database and a computable value point matrix stored in a memory device accessible by the one or more computer processors, wherein the computable value point matrix is ​​generated and continuously updated by a machine learning module using a machine learning model trained by the method of claim 1. (d) Determining in real time, based on an evaluation via one or more computer processors, a total computable value of the type of information in the subject's personal information, and determining, if the total computable value of the type of information in the subject's personal information exceeds or falls below a pre-selected computable value, a predictive allocation of medical resources is generated and determined. (e) Notifying healthcare professionals who provide medical care to the subject via the user interface about the predicted allocation of medical resources to the subject, A tangible, non-temporary, computer-readable storage medium containing instructions for executing a command.

30. When executed by a computer processor, the processor: (i) To enable one or more computing devices to communicate with each other, wherein each device comprises one or more computer processors, a data communication connection, and one or more tangible, non-temporary computer-readable media accessible by the one or more computer processors. (ii) Storing personal information databases, (iii) Inputting into the machine learning module, It further includes instructions to execute, The personal information database and the machine learning module are each stored in one or more tangible non-temporary computer-readable storage media according to claim 29.

31. A method for training a machine learning model to predict the allocation of medical resources needed to treat patients with traumatic injuries, (a) To establish a ground truth, the ground truth corresponding to a standardized level of trauma onset for a standardized patient having a standardized set of clinical conditions, (b) Assigning a mechanical trauma trigger level to an individual patient identified by a set of clinical conditions selected from the standardized set of clinical conditions, wherein a machine learning model assigns a first trauma trigger level by an algorithm including a set of relative computable values, each relative computable value being assigned to one or more clinical conditions within the standardized set of clinical conditions, (c) Comparing the mechanical trauma initiation level with the ground truth, (d) Based on the comparison, determine whether the level of mechanical trauma initiation corresponds to undertriage, overtriage, or an appropriate level of triage for the individual patient, (e) If the mechanical injury trigger level in step (d) corresponds to undertriage or overtriage, adjust one or more relative computable values ​​used by the machine learning model when assigning the mechanical injury trigger level, (f) Repeat steps (b) to (e) for multiple patients, each of which is identified, with multiple clinical conditions selected from the set of clinical conditions, until the machine learning model reaches an appropriate level of triage within the desired parameters. Methods that include...

32. The method according to claim 31, wherein each of the patients has a previous trauma trigger level assigned by a human healthcare provider.

33. The aforementioned desired parameter includes the machine accuracy rate, which is given by the following formula, i.e., Machine accuracy = (Number of patients requiring appropriate triage level ÷ Total number of patients) × 100% The method according to claim 31 or 32, determined based on the above.

34. The Ground Truth is based on the Clibari Matrix method (Trauma Severity Score (ISS)), the Method for Determining the Need for Trauma Intervention (NFTI), or a combination of ISS and NFTI, according to any one of claims 31 to 33.

Images