Systems and methods for nursing intuition using advanced digital intelligence

Abstract

Systems and methods for utilizing nursing intuition to support early clinical intervention are provided. The systems and methods may include a vector database stored on one or more memory devices, the vector database comprising one or more vector embeddings positioned in a vector field, wherein each of the one or more vector embeddings is indicative of medical data of one or more individuals, and wherein the positioning of the one or more vector embeddings in the vector field is based on the similarity of the one or more vector embeddings to each other. The systems and methods may also include at least one machine learning model. The machine learning model may be trained to establish a likelihood of clinical deterioration of one or more individuals based on the vector database.

Description

SYSTEMS AND METHODS FOR NURSING INTUITION USING ADVANCED DIGITAL INTELLIGENCECROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of, and priority to, U.S. Provisional Patent Application No. 63 / 688,840, filed on August 29, 2024, which is hereby incorporated by reference herein in its entirety.TECHNICAL FIELD

[0002] The present disclosure relates generally to systems and methods for analyzing patient data for determining deterioration risks, and more particularly, to systems and methods for analyzing patient data using multiple large language models to determine a deterioration score indicative of the risk of suffering from a deterioration event.BACKGROUND

[0003] Clinical early warning systems (EWS) utilize data from patients’ electronic health records (EHR) to identify unanticipated clinical deterioration, potentially alerting clinicians to facilitate early intervention. In many cases, these early warning systems are sensitive and lack specificity, and can generate thousands of false positive alerts. The resulting alarm fatigue leads to clinical staff desensitization and burnout, often leading nurses to mute or ignore alerts, potentially delaying response time. The present disclosure is directed to systems and methods for implemented improved early warning systems, and other systems and methods for analyzing patient deterioration.SUMMARY

[0004] A method for determining a deterioration score of a patient associated with a plurality of potential deterioration events comprises receiving medical data associated with the patient; inputting the medical data into a plurality of large language models (LLMs), each of the plurality of LLMs trained to: analyze at least a respective portion of the medical data; identify a plurality of tokens within the respective portion of the medical data; determine a patientpatient specific relevance of each of the plurality of tokens to the plurality of potential deterioration events; and output a patient-specific coefficient for one or more of the plurality of tokens indicative of the relevance of the one or more plurality of tokens to the plurality of4930-7714-2882 1065472-000993WOPT 1deterioration events; and determining the deterioration score based at least in part on the coefficients outputted by the plurality of LLMs, the deterioration score being indicative of a percentage chance that the patient will suffer at least one of the plurality of deterioration events.

[0005] A system for determining a deterioration score of a patient associated with a plurality of potential deterioration events comprises a plurality of large language models (LLMs) and a scoring algorithm. Each of the plurality of LLMs is trained to analyze at least a respective portion of medical data associated with the patient; identify a plurality of tokens within the respective portion of the medical data; determine a patient-patient specific relevance of each of the plurality of tokens to the plurality of potential deterioration events; and output a patientspecific coefficient for one or more of the plurality of tokens indicative of the relevance of the one or more plurality of tokens to the plurality of deterioration events. The scoring algorithm is configured to determine the deterioration score based at least in part on the coefficients outputted by the plurality of LLMs, the deterioration score being indicative of a percentage chance that the patient will suffer at least one of the plurality of deterioration events.

[0006] The above summary is not intended to represent each implementation or every aspect of the present disclosure. Additional features and benefits of the present disclosure are apparent from the detailed description and figures set forth below.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The disclosure, and its advantages and drawings, will be better understood from the following description of representative embodiments together with reference to the accompanying drawings. These drawings depict only representative embodiments and are therefore not to be considered as limitations on the scope of the various embodiments or claims.

[0008] FIG. 1 is a block diagram of a system for determining a deterioration score for a patient, according to aspects of the present disclosure.

[0009] FIG. 2 is a block diagram of an example of the system of FIG. 1 that includes five large language models, according to aspects of the present disclosure.

[0010] FIG. 3 is a block diagram showing details of the five large language models of FIG. 2, according to aspects of the present disclosure.

[0011] FIG. 4 is a flowchart of a method for determining a deterioration score for a patient, according to aspects of the present disclosure.

[0012] FIG. 5 is a block diagram of a system implementing the method of FIG. 4, according to aspects of the present disclosure.

[0013] While the present disclosure is susceptible to various modifications and alternative4930-7714-2882 1065472-000993WOPT 2forms, specific implementations and embodiments have been shown by way of example in the drawings and will be described in detail herein. It should be understood, however, that the present disclosure is not intended to be limited to the particular forms disclosed. Rather, the present disclosure is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure as defined by the appended claims.DETAILED DESCRIPTION

[0014] Disclosed herein are systems and methods for analyzing patient data and determining the risk of deterioration that allow for faster and deeper clinical intelligence. Machine learning models (e.g., artificial neural networks implementing large language models) are trained on patient data to generate coefficients for tokens in the patient data that are associated with the relevance of the tokens to deterioration events. The coefficients can be used to estimate a deterioration score for the patient that is indicative of a percentage chance that the patient will suffer at least one of the plurality of deterioration events.

[0015] FIG. 1 is a block diagram of a system 100 for estimating a deterioration score for a patient. The system 100 includes an Advanced Digital Intelligence (ADI) model 110. The ADI model includes a plurality of large language models (LLMs) I I 2A- I I 2 / / , with each of the LLMs 112A-112 / / being associated with one of a corresponding plurality of vector databases 114A-114 / / . The ADI model 110 can receive inputs from a variety of different sources regarding a new patient to generate a deterioration score of 116, which is indicative of the chances that a patient will deteriorate within a given timeframe (e.g., 24 hours, 48 hours, etc.). As used herein, the term “deterioration” generally refers to the patient either having to be transferred to the Intensive Care Unit (ICU), the patient suffering from cardiac arrest, the patient suffering from respiratory arrest, the patient suffering from a stroke, the activation of a rapid response team to attend to the patient, or the patient dying.

[0016] In general, the ADI model 110 can be used to determine the chance that a patient (which may be a patient at an outpatient visit, a patient admitted to an inpatient unit, etc.) will deteriorate during a specific encounter with the patient. A given encounter may be a specific instance of the patient visiting a healthcare facility. In some implementations, different ones of the LLMs I I 2A- I I 2 / / can analyze different data associated with the patient. For example, as discussed in more detail herein, a first LLM of the LLMs I I 2A- I I 2 / / can analyze data all available historical data in the patient’s electronic health record (EHR) (e.g., data from the patient’s EHR from prior to the encounter; a second LLM of the LLMs 112A-1 12 / / can analyze data associated with specific tests and treatments that healthcare providers (e.g., doctors,4930-7714-2882 1065472-000993WOPT 3nurses, etc.) order to be performed for the patient during the encounter; and a third LLM of the LLMs I I 2A- I I 2 / / can analyze associated with the actual outcomes of tests and treatments undergone by the patient during the encounter.

[0017] In FIG. 1, the ADI model 110 has a three inputs 118A, 118B, and 118C. Input 118A is received from a healthcare provider, such as a nurse or a physician, and may include a natural language query regarding a new patient. Input 118B is received from the EHR system, and may include an update or change to the patient’s EHR, such as the addition of a new test result, a note from the healthcare provider (e.g., a comment added to the EHR by a nurse or doctor), etc. Input 118B may also include the creation of the patient’s EHR in the EHR system.

[0018] Input 118C is received from a separate machine learning model that acts as an Al agent for the ADI model 110 that generates a query regarding the new patient that the ADI model 110 can be used to generate the deterioration score 116. The Al agent may itself receive the input 118A from the healthcare provider and / or the input 118B from the EHR, and then generate a query for the ADI model 110. In some implementations, this Al agent is an LLM model separate from the ADI model 110 that has been trained to generate search queries regarding the clinical condition of the patient.

[0019] Once the ADI model 110 has received any of the inputs 118 A- 118C, the ADI model 110 can review the patient’s EHR (e.g., from input 118B) and identify tokens within the patient’s EHR that are highly correlated with deterioration. The separate LLMs I I 2A- I I 2 / / , which may be trained on different data, can review different portions of the patient’s EHR to identify tokens, and to assign coefficients to identified tokens. The coefficients of the identified tokens can be used to generate the deterioration score 116. In some implementations, receiving any of the inputs 114A-114C can trigger the ADI model 110 to identify the tokens and generate the deterioration score 116. In other implementations, only a specific request in the form of a query (e.g., a natural language query) from input 114A (healthcare provider) or input 114C (Al agent) triggers the identification of the tokens and the generation of the deterioration score 116.

[0020] Each of the vector databases 1 14A- I I4 / / includes vector embeddings representing a large number of possible tokens that are arranged based on their relevance to clinical deterioration. The vector databases 1 14A- I I 4 / / and the vector embeddings are generated during the training of the LLMs I I 2A- I I 2 / / , and the position of two vector embeddings within the vector database relative to each other is indicative of the similarity of the two corresponding tokens in relation to the clinical deterioration of the patients. The LLMs I I 2A- I I 2 / / can compare tokens in the patient’s EHR data to the vector embeddings representing tokens within the corresponding vector database I I4A- I I4 / / , for example using a ^-nearest neighbor4930-7714-2882 1065472-000993WOPT 4algorithm, an approximate nearest neighbor algorithm, and / or a Euclidian distance algorithm. The LLMs 112A-112 / / will generally identify the / / / -most relevant tokens, and output a coefficient indicative of the relevance of each respective one of the m tokens indicative of the respective token’ s relevance to clinical deterioration. These coefficients can be used to generate the deterioration score 116. In other implementations, the LLMs I I 2A- I I 2 / / do not actually search the vector databases 114A-1 14 / / , but will generate coefficients for the tokens in the input data based on the training undergone to generate the vector databases 114A-114 / ?. In these implementations, when the LLMs 112A-112 / / operate on new data, they effectively generate a new series of vector embeddings based on tokens identified in the new data.

[0021] In some implementations, at least one of the LLMs 112A-112 / ? does not identify tokens within the patient’s EHR data, but instead is trained on clinical reference data (e.g., textbooks, medical principles, hospital policies, etc.), and identifies the / / / -most relevant tokens from this data. In these implementations, the coefficients outputted by the LLM are generally always the same, even for patients with different medical data (so long as this LLM itself is not updated). In some implementation, at least one of the LLMs 112A-112 / / does not identify tokens within the patient’s EHR data, but is instead trained on the output of other ones of the LLMs 112A-112 / / . As discussed in more detail herein, this LLM can receive the coefficients output by other ones of LLMs 112A-1 12 / / and generate additional coefficients.

[0022] In some implementations, each of the LLMs 112A-112 / / is a neural network. For example, at least one of the LLMs 112A-112 / / may be a deep learning transformer neural network that is trained using backpropagation gradient descent methods. In another example, at least one of the LLMs 112A-112 / / may be a deep neural network trained with forwardforward techniques. In a further example, at least one of the LLMs 112A-112 / / may be a reinforcement learning model.

[0023] FIG. 2 shows a block diagram of one implementation of the ADI model 110 of FIG. 1. In FIG. 1, the ADI model 110 includes a first LLM 202 A, a second LLM 202B, a third LLM 202C, a fourth LLM 202D, and a fifth LLM 202E. The second, third, and fourth LLMs 202B- 202D are communicatively coupled with the EHR 204 and receive patient data 206 as inputs, while the first LLM 202A is not coupled to the EHR 204. The first LLM 202A generates a first set of coefficients 208 A and sends them to the fifth LLM 202E and to a scoring algorithm 210. The second LLM 202B generates a second set of coefficients 208B and sends them to the fifth LLM 202E and to the scoring algorithm 210. The third LLM 202C generates a third set of coefficients 208C and sends them to the fifth LLM 202E and to the scoring algorithm 210. The fourth LLM 202D generates a fourth set of coefficients 208A and sends them to the fifth LLM4930-7714-2882 1065472-000993WOPT 5202E and to the scoring algorithm 210. The fifth LLM 202E generates a fifth set of coefficients 208E and sends them to the scoring algorithm 210. The scoring algorithm 210 generates the deterioration score 212 based on all of the sets of coefficients 208A-208E.

[0024] FIG. 3 shows a more detailed version of the implementation of the ADI model 110 of FIG. 2. As shown, the first LLM 202A is a deep learning transformer neural network architecture that is trained using backpropagation gradient descent methods on clinical reference data associated with general (non patient-specific) principles of internal medicine and acute internal medicine. This clinical reference data can include data obtained from medical textbooks, medical journals, and / or any other written source of medical information; data associated with policies of the healthcare provider and / or of the healthcare facility where the patient is located; or both. In some cases, the first LLM 202A may also be trained on EHR data for patients. Training the first LLM 202A creates a first vector database that includes vector embeddings representing tokens present in the clinical reference data, where the vector embeddings are positioned within the vector database relative to how the tokens affect the chance that the patient will deteriorate. Once the first LLM 202A is trained, it can be used to generate the coefficients of the m-most relevant tokens from the clinical reference data, which can generally be used during the analysis of any new patient during a real encounter. In some implementations, training of the first LLM 202A generates the coefficients for each of the tokens. The coefficients may be in some cases the internal attention weights assigned to each of the tokens by the encoder portion of the first LLM 202A.

[0025] The second LLM 202B is also a deep learning transformer neural network architecture that is trained using backpropagation gradient descent methods, but is trained on specific patient data and on principles of specific subspecialties (in the illustrated implementation, 21 subspecialties), such as emergency medicine, otorhinolaryngology, immunology, neurology, cardiology, critical care, endocrinology, gastroenterology, radiology, pain management, psychiatry, orthopedics, ophthalmology, geriatrics, hematology, oncology, obstetrics and gynecology, hospice and palliative medicine, urology, pathology, gastroenterology, infectious diseases, pulmonology, nephrology, rheumatology, or any combination thereof.

[0026] The second LLM 202B analyzes past data from the patient’s EHR from prior to the current encounter. Thus, the training data for the second LLM 202B will generally include preencounter data for a plurality of historical patients. In some implementations, the training data also includes, for each historical patient, a manually-assigned deterioration score assigned by clinical reviewers based on (i) the pre-encounter EHR data for the historical patient or (ii) all4930-7714-2882 1065472-000993WOPT 6of the EHR data for the historical patient. Training the second LLM 202B creates a second vector database that includes vector embeddings representing tokens present in the preencounter EHR data, where the vector embeddings are positioned within the vector database relative to how the tokens impact the chances of clinical deterioration. Once the second LLM 202B is trained, it can be used during the analysis of a specific new patient during a real encounter to generate the coefficients of the m-most relevant tokens in that new patient’s preencounter EHR data.

[0027] In some implementations, use of the second LLM 202B on the new patient’s data generates a new series of vector embeddings for the tokens found in the new patient’s data, which can be used to generate the coefficients of each of the tokens. In some cases, the coefficients are the internal attention weights assigned to each of the tokens in the new patient’ s data by the encoder portion of the second LLM 202B.

[0028] The third LLM 202C is a deep neural network that is trained using forward-forward techniques on patient EHR data from encounter that is associated with actions of the healthcare provider (e.g., nurses, doctors), including measurements / tests ordered by the doctor, imaging ordered by the doctor, medications prescribed / ordered by the doctor, notes input into the patient’s EHR by the healthcare provider, etc. In a specific example, a token associated with the third LLM 202C may be a token indicating that the doctor ordered the patient to undergo a CT scan during the encounter. The third LLM 202C is thus trained on patient-specific data and more general principles associated with the training of the healthcare providers, nursing intuition, historical clinical practice, and past experience of the healthcare providers, and is focused on the patient presentation during the encounter from the perspective of the healthcare provider. The training data for the third LLM 202C will thus include, for each historical patient, EHR data from the encounter associated with the actions of the healthcare provider. In some implementations, the training data also includes, for each historical patient, a manually- assigned deterioration score assigned by clinical reviewers based on (i) the EHR data for the historical patient associated with the healthcare provider’s actions during the encounter or (ii) all of the EHR data for the historical patient.

[0029] Training the third LLM 202C creates a third vector database that includes vector embeddings representing tokens present in the EHR data associated with the healthcare provider’s actions during the encounter, where the vector embeddings are positioned within the vector database relative to how the tokens impact the chances of clinical deterioration. Once the third LLM 202C is trained, it can be used during the analysis of a specific new patient4930-7714-2882 1065472-000993WOPT 7during a real encounter to generate the coefficients of the m-most relevant tokens in that new patient’s EHR data associated with the healthcare provider’s actions during the real encounter.

[0030] In some implementations, use of the third LLM 202C on the new patient’s data generates a new series of vector embeddings for the tokens found in the new patient’s data, which can be used to generate the coefficients of each of the tokens. In some cases, the coefficients are the internal attention weights assigned to each of the tokens in the new patient’ s data by the encoder portion of the third LLM 202C.

[0031] The fourth LLM 202D is a deep neural network that is trained using forwardforward techniques on patient EHR data from encounter that is associated with non- deterioration-related outcomes occurring during the encounter, including tests results, images generated following scans undergone by the patient, etc. In a specific example, a token associated with the fourth LLM 202D may be a token associated with the resulting CT scan that the patient underwent during the encounter. The fourth LLM 202D is thus trained on patient-specific data and on more specific principles of the training, practice, and experience of different subspecialties, and is focused on the patient presentation during the encounter from an outcome perspective (outcomes other than whether the patient deteriorated). The training data for the fourth LLM 202D will thus include, for each historical patient, EHR data from the encounter associated with the outcomes occurring during the encounter. In some implementations, the training data also includes, for each historical patient, a manually- assigned deterioration score assigned by clinical reviewers based on (i) the EHR data for the historical patient associated with outcomes during the encounter or (ii) all of the EHR data for the historical patient.

[0032] Training the fourth LLM 202D creates a fourth vector database that includes vector embeddings representing tokens present in the EHR data associated with outcomes occurring during the encounter, where the vector embeddings are positioned within the vector database relative to how the tokens impact the chances of clinical deterioration. Once the fourth LLM 202D is trained, it can be used during the analysis of a specific new patient during a real encounter to generate the coefficients of the m-most relevant tokens in that new patient’s EHR data associated with outcomes occurring during the real encounter.

[0033] In some implementations, use of the fourth LLM 202D on the new patient’s data generates a new series of vector embeddings for the tokens found in the new patient’s data, which can be used to generate the coefficients of each of the tokens. In some cases, the coefficients are the internal attention weights assigned to each of the tokens in the new patient’ s data by the encoder portion of the fourth LLM 202B.4930-7714-2882 1065472-000993WOPT 8

[0034] The fifth LLM 202E is a reinforcement learning model that is trained on the coefficients generated by the other LLMs 202A-202D, EHR data, and actual deterioration outcomes (e.g., deterioration or no deterioration). The fifth LLM 202E is thus trained on patient-specific data and patient-specific outcomes, including diagnoses, baseline clinical values, vital signs, etc. The training data for the fifth LLM 202E will thus include, for each historical patient, the coefficients generated by the other LLMs 202A-202D, and an indication of whether the patient actually deteriorated during the encounter (e.g., a 0 representing no deterioration and a 1 representing deterioration). In some implementations, the training data also include, for each historical patient, the EHR data for the historical patient.

[0035] Training the fifth LLM 202E creates a fifth vector database that includes vector embeddings representing tokens present in the EHR data, where the vector embeddings are positioned within the vector database relative to how the tokens affect the actual deterioration outcomes for each of the historical patients. Once the fifth LLM 202E is trained, it can be used during the analysis of a specific new patient during a real encounter to generate the coefficients of the / 71-most relevant tokens in that new patient’s EHR.

[0036] In some implementations, use of the fifth LLM 202E on the new patient’s data generates a new series of vector embeddings for the tokens found in the new patient’s data, which can be used to generate the coefficients of each of the tokens. In some cases, the coefficients are the internal attention weights assigned to each of the tokens in the new patient’ s data by the encoder portion of the fifth LLM 202E.

[0037] Lastly, the coefficients generated by each of the LLMS 202A-202E are used to generate the deterioration score 212, which in this implementation is a number between 0 and 100 representing the percentage chance that the new patient will deteriorate during the encounter (or within some timeframe after the deterioration score 212 is generated). In some implementations, each coefficient generated by each of the LLMs 202A-202E is a number between 0 and 1. The scoring algorithm 210 multiplies together the values of all of the coefficients and divides the result by the total number of coefficients, resulting in a number between 0 and 50. That number is then multiplied by 2 to place it on the 0-100 scale representing the percentage chance of deterioration.

[0038] The EHR data that is used to train any of the LLMs and for analysis of a new patient can generally include any type of data, including text, images, videos, etc. The medical data may include the age of each patient, the height of each patient, the weight of each patient, the sex of each patient, the treatment history of each patient, tests results for each patient (e.g., vital signs, hemoglobin, lactate, electrolytes), imaging results for each patient (e.g., x-ray images,4930-7714-2882 1065472-000993WOPT 9MRI images, CT images, etc.), physiological characteristics of each patient, an indication of whether each patient has an implanted device (e.g., a pacemaker, a stimulator for treatment of sleep apnea, an artificial organ or bone, etc.), notes from a healthcare provider (e.g., nursing assessments), medical telemetry data of the patient, the patient’s electronic health / medical record or portions thereof, subjective judgments regarding each patient (such as physiology or quality of life), other data, and any combination thereof.

[0039] FIG. 4 is a flowchart of a method 400 for analyzing a patient for risk of suffering from a deterioration events. The deterioration event may include the patient being transferred to the intensive care unit, the patient suffering from cardiac arrest, the patient suffering from respiratory arrest, the patient suffering a stroke, a rapid response team being activated to attend to the patient, the patient dying etc.

[0040] Step 402 includes receiving medical data associated with the patient, for example from the patient’s EHR. Step 404 includes inputting the medical data or a portion thereof into a plurality of LLMs. Each of the plurality of LLMs is trained to analyze the medical data or portion thereof, identify tokens with the medical data, determine the patient-specific relevance of the tokens to potential deterioration events, and output patient-specific coefficients for at least some of the tokens, where the coefficients for each respective token are indicative of the relevance of the respective token to the plurality of deterioration event.

[0041] In some implementations, each of the plurality of LLMs is configured to analyze a distinct portion of the patient’s medical data. In one example, a first LLM is trained to analyze a first portion of the medical data associated with a past history of the patient, a second LLM is trained to analyze a second portion of the medical data associated with actions of a healthcare provider during a current encounter with the patient, and a third LLM is trained to analyze a third portion of the medical data associated with outcomes associated with the actions of the healthcare provider during the current encounter with the patient. Some of these LLMs (such as the first LLM) may be trained using b ackpropagation techniques, while others of these LLMs (such as the third LLM and the fourth LLM) may be trained using forward-forward techniques.

[0042] In some implementations, the plurality of LLMs includes a fourth LLM that is trained to analyze the medical data (and / or a portion thereof) and coefficients generated by the other LLMs, and generate its own coefficients indicative of the relevance of tokens in the medical data to deterioration events. This fourth LLM may be a reinforcement learning model. In some implementations, a reference LLM may be used. The reference LLM is trained on non- patient-specific clinical reference data associated with the plurality of deterioration events, and is trained to output non-patient specific coefficients for a plurality of reference tokens in the4930-7714-2882 1065472-000993WOPT 10clinical reference data that are indicative of the relevance of the plurality of reference tokens to the plurality of deterioration events. The reference LLM may be trained using backpropagation techniques.

[0043] In some implementations, the coefficients generated by any of the plurality of LLMs and / or the reference LLM are the internal attention weights assigned to the plurality of tokens / reference tokens by the encoder portion of the respective LLM.

[0044] Step 406 includes receiving the coefficients from the LLMs, and step 408 includes determining a deterioration score indicative of the percentage chance that the patient will suffer from one of the deterioration events based on the coefficients.

[0045] In some implementations, step 406 includes receiving a coefficient for every single token identified by the plurality of LLMs. In these implementations, the deterioration score may be based on all of the coefficients from each of the plurality of LLMs (and in some cases additionally on all the coefficients from the reference LLM), or be based only on the m-highest coefficients from each of the plurality of LLMs (and in some cases additionally on the m- highest coefficients from the reference LLM). In other implementations, the plurality of LLMs (and / or the reference LLM) are trained to output only their m-highest coefficients, instead of all of the coefficients. In these implementations, the deterioration score is based on the / ??- highest coefficients from each of the plurality of LLMs (and in some cases additionally on the / 77-highest coefficients from the reference LLM). m may generally be any suitable number (which may be based on the processing power of the system that is used to implement method 400), and in some cases m=10.

[0046] In some implementations, step 408 includes determining the product and / or average of all of the coefficients and then scaling that result to be on a scale of 0 to 100. For example, if 50 total coefficients are used to determine the deterioration score (e.g., the 10-highest coefficients from five different LLMs), then the deterioration score may be equal to the average of these 50 coefficients scaled to a 0-100 scale. In some implementations, each coefficient has a value between 0 and 1, in which case the average of all the coefficients being used can be multiplied by 100 to place the deterioration score on the 0-100 scale.

[0047] In some implementations, the deterioration score is indicative of the percentage chance that the patient will suffer at least one of the plurality of deterioration events within a future timeframe, which may be, for example, 1 hour, 2 hours, 4 hours, 6 hours, 8 hours, 12 hours, 24 hours, 48 hours, 5 days, 1 week, etc. In some implementations, method 400 further includes transmitting to a healthcare provider (e.g., a doctor or a nurse at the healthcare facility where the patient is located an indication of the deterioration score; an indication of one or4930-7714-2882 1065472-000993WOPT 11more of the tokens identified by any of the LLMs (e.g., the most relevant token based on the coefficient, the / / -most relevant tokens based on the coefficient, the most relevant token from each LLM based on the coefficient, etc.); an indication of the coefficients themselves (e.g., the highest coefficient, the / / -highest coefficients, the highest coefficient from each LLM based on the coefficient, etc.); and any other relevant information. Thus, method 400 can include generating and transmitting potential differential diagnoses and / or factors contributing to the risk of deteriorating.

[0048] A system for determining the deterioration score of the method 400 can thus include a plurality of LLMs and a scoring algorithm. Each LLM is trained to analyze at least a respective portion of medical data associated with the patient; identify a plurality of tokens within the respective portion of the medical data; determine a patient-patient specific relevance of each of the plurality of tokens to the plurality of potential deterioration events; and output a patient-specific coefficient for one or more of the plurality of tokens indicative of the relevance of the one or more plurality of tokens to the plurality of deterioration events. The scoring algorithm is configured to determine the deterioration score based at least in part on the coefficients outputted by the plurality of LLMs, the deterioration score being indicative of a percentage chance that the patient will suffer at least one of the plurality of deterioration events.

[0049] FIG. 5 illustrates a block diagram of system 500 that can be used to implement method 400 and any other principles, techniques, features, etc. disclosed herein. System 500 can be used to train and / or implement any of the machine learning models discussed herein. The system 500 can include one or more processing devices 510, which can each include any one or more of a processor 512, a memory 514, a display 516, a user input device 518, and / or other components. The memory 514 can include machine-readable instructions for executing the methods disclosed herein, and / or other methods. The processor 512 can execute these instructions to implement these methods. The memory 514 can also store data associated with the methods.

[0050] The processing device 510 can include any suitable processing device, such as general purpose computer systems, microprocessors, digital signal processors, microcontrollers, application specific integrated circuits (ASICs), programmable logic devices (PLDs) field programmable logic devices (FPLDs), programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), mobile devices such as mobile telephones, personal digital assistants (PDAs), or tablet computers, local servers, remote servers, wearable computers, or the like. The memory 514 can include any suitable memory device and / or machine-readable medium that is capable of storing, encoding, and / or carrying a set of instructions for execution4930-7714-2882 1065472-000993WOPT 12by a processing device and that cause the processing device to perform and / or implement any of the features discussed herein, including solid-state memories, optical media, magnetic media, random access memory (RAM), read only memory (ROM), a floppy disk, a hard disk, a CD ROM, a DVD ROM, flash memory, or other computer readable medium that is read from and / or written to by a magnetic, optical, or other reading and / or writing system that is coupled to the processing device, can be used for the memory or memories. In some implementations, the system 500 may include multiple processors and multiple memories, such that the machine- readable instructions are distributed across multiple memories and executed by one or more of the processors.

[0051] The display 516 can be used to display any information associated with the features disclosed herein. The display 516 can be any known display technology, including but not limited to display devices using Liquid Crystal Display (LCD) or Light Emitting Diode (LED) technology. The user input device 518 can be used to allow the user to interact with the system 500 for any suitable purpose, including initiating, pausing, or terminating the analysis by the model; adjusting any parameters of the analysis, etc.

[0052] In some implementations, the system 500 includes an electronic health record (EHR) 520. The EHR 520 can contain all available data about the patient, including medical history, test results, doctor’s notes, other charting notes about the patient’s condition, etc. The EHR 520 can also generally be updated at any time with new data associated with the patient as it is generated and / or received, model. The EHR 520 can generally be any suitable type of electronic health record.

[0053] In general, a system for implementing any of the disclosed methods, principles, techniques, features, etc. may comprise one or more memory devices storing machine-readable instructions, and one or more processors configured to execute the machine-readable instructions. Such a system may comprise one or more processors configured to implement any of the disclosed methods, principes, techniques, features, etc. Such a system may include a computer program product (e.g., a non-transitory computer readable medium) comprising instructions which when executed by a computer, implement any of the disclosed methods, principes, techniques, features, etc.

[0054] Examples

[0055] Disclosed herein are several examples associated with the present disclosure.

[0056] Example 1

[0057] Disclosed is a method for generating a vector database based on the EHR data of a plurality of patients. The vector database will include a large amount of vector embeddings,4930-7714-2882 1065472-000993WOPT 13where each vector embedding represents a token from the medical data, and the position of two vector embeddings within the vector database relative to each other is indicative of the similarity of the two corresponding tokens in relation to the clinical condition (e.g., deterioration percentage / score) of the patients. Method 400 can be used in implementations of the present disclosure

[0058] A first step of the method includes preparing a training dataset from the EHR data. In some implementations, preparing the training dataset may also include preprocessing some of the data, such as removing noise, resizing images and / or videos, etc. Preparing the training dataset may in some cases include assigning a manual deterioration score to each patient based on clinical expertise. For example, healthcare providers (e.g., nurses, doctors, etc.) can review medical data for each patient and manually assign the patient a deterioration score, which in some cases is the percentage change that the patient will deteriorate within a given timeframe (e.g., within the next 24 hours, the next 48 hours, etc.).

[0059] Once prepared, the training dataset will consist of a large amount of tokens that each represent some individualized piece of data, whether it be a portion of text, a portion of or a whole image, a portion of or a whole video, etc. Each token will generally have the same format, so that medical data of different types (e.g., text and images) can be analyzed alongside each other.

[0060] A second step of the method includes training a machine learning model using the training dataset. During the training process, the first machine learning model learns connections between the various tokens, and begins to group the tokens according to how similar the tokens are with respect to the deterioration score of the patient they are associated with. In some implementations, the machine learning model that is trained using the training dataset is part of the ADI model 110 discussed herein, and may specifically be any of the LLMs 112A-1 12 / / and / or any of the LLMs 202A-202E.

[0061] In some implementations, this step includes training the machine learning model using forward-forward techniques. For example, the machine learning model may be an LLM that is trained without backpropagation. In these implementations, the machine learning model is trained by moving through the machine learning model forward twice, instead of once forward and once backward. Training data that is correct (also referred to as positive data) can be input into the first machine learning model during the first forward pass, and training data that is incorrect (also referred to as negative data) can be input into the machine learning model during the second forward pass. The weights and other parameters of the layers are adjusted in order to maximize the activity of the machine learning model when the correct data is input4930-7714-2882 1065472-000993WOPT 14into the machine learning model, and to minimize the activity of the machine learning model when the incorrect data is input into the machine learning model.

[0062] The machine learning model trained at this step is trained using data (e.g., data associated with nursing intuition and clinical experience) that is more subjective. Thus, the deterioration scores generated by the first machine learning model are generally more reflective of a healthcare provider’s intuition regarding which factors lead to deterioration, instead of some standard reference literature indicating which factors lead to deterioration. Because of this subjectivity, it is generally much more difficult to train the first machine learning model via backpropagation. Training the first machine learning model using the forward-forward training technique includes defining a local loss function for each nodes, so that only the nodes themselves (or a subset of the nodes) need to be retrained, instead of the entire model. This eliminates the need for training via error back-propagation, which generally requires perfect or near-perfect characterization. This also allows the first machine learning model to scale efficiently to large numbers of parameters (e.g., tokens).

[0063] A third step of this method includes generating a vector database of the tokens using the parameters of the trained first machine learning model. In some implementations, the first machine learning model is a neural network where each token is a separate input into the neural network, and the outputs of the neural network are indicative of the deterioration score. At the input side, each token is connected to a plurality of activation functions. For a given token, the weights of each connection between that token and the plurality of activation functions form the elements of the vector embedding for that token. In other implementations, parameters of the first machine learning model may be used in a different manner to generate the elements of the vector embedding for each token.

[0064] In any implementation, the vector embeddings in the vector database are arranged within the vector database based on their similarity to each other relative to how the tokens represented by the vector embeddings contribute to the deterioration score in patients in the training datasets. Thus, clusters of vector embeddings within the vector database generally represent a patient “phenotype” associated with a certain deterioration score.

[0065] Once the vector database is generated, the vector database can be used to estimate a deterioration score of a new patient. Disclosed herein is an example method for analyzing a patient using the vector database. A first step of the method includes receiving a search query associated with the patient. In some implementations, the search query is a natural language query provided by a healthcare practitioner and / or an Al agent.4930-7714-2882 1065472-000993WOPT 15

[0066] A second step of the method includes generating one or more query vector embeddings based on the search query. In some implementations, this step is performed using a second machine learning model, which may be an LLM (e.g., the LLM 118A of the ADI model 110). The second machine learning model is trained to receive a natural language query and generate one or more query vector embeddings based on the natural language query. The second machine learning model will generally have access to a patient’s electronic medical / health record, and based on the search query, will be able to extract relevant tokens from the patient’s electronic medical / health record. Based on the tokens, the second machine learning model can generate the query vector embeddings, which generally are vector embeddings representing tokens associated with the patient. In general, the second machine learning model is trained on the vector embeddings in the vector database, and thus is able to generate the query vector embeddings based on the input search query.

[0067] A third step of the method includes identifying the ^-nearest vector embeddings within the vector database using the query vector embeddings. In some implementations, this step includes using a ^-nearest neighbor (kNN) to identify the nearest vector embeddings. In additional or alternative implementations, this step includes determining the Euclidian distance between each of the query vector embeddings and one or more vector embeddings in the vector database.

[0068] Once the ^-nearest vector embeddings are identified, a fourth step of the method includes determining the deterioration score based on the identified vector embeddings. In some implementations, this step includes inputting the identified vector embeddings back into the first machine learning model (e.g., the neural network). This may include setting as inputs to the first machine learning model the tokens represented by the identified vector embeddings. The output of the first machine learning model represents the deterioration score. In other implementations, the second machine learning model (e.g., the LLM) receives as inputs the identified vector embeddings and outputs the deterioration score. In either implementation, the second machine learning model may also output additional context associated with the deterioration score, such as query vector embeddings determined to be more impactful to the deterioration score, specific tokens from the patient’s electronic medical / health record determined to be more impactful to the deterioration score, etc.

[0069] The deterioration score itself may be any suitable format. For example, in some cases, the deterioration score may be a percentage between 0% and 100%, where the percentage is the chance of deteriorating within a given time frame. In other cases, the deterioration score may be a percentage between 0 and 1. In some cases, the deterioration score is a binary output4930-7714-2882 1065472-000993WOPT 16indicating that the patient is deteriorating (or will deteriorate) or is not deteriorating (or will not deteriorate). In some of these cases, the first machine learning model and / or the second machine learning model may be trained to determine only the binary output. In others of these cases, the first machine learning model and / or the second machine learning model may be trained to determine a deterioration percentage, but only generate the binary output based on a threshold deterioration percentage (e.g., a deterioration percentage of greater than or equal to 50% results in an output of deteriorating, and a deterioration percentage of less than 50% results in an output of not deteriorating).

[0070] As discussed herein, the search query received by the second machine learning model at the first step of the method may be generated by an Al agent. In some implementations, the Al agent is an LLM that is trained to generate search queries regarding the clinical condition of a patient based on natural language search queries received from a healthcare practitioner. For example, the Al agent may be trained to receive basic queries such as “Is this patient deteriorating?,” and then generate a search query for the second machine learning model that passes in simple real-time variables for the patient such as respiratory rate, systolic pressure, etc.

[0071] In some implementations, multiple stages of Al agent queries can be used to generate the final search query that is received by the second machine learning model at the first step of the method. For example, the Al agent can generate an initial search query from a natural language query received from a healthcare provider, and then generate an updated search query based on the initial search query. This updated search query may be the final search query that is received by the second machine learning model at the first step of the method. This updated search query may also be an intermediate search query that can again be input into the Al agent to generate a further search query. In some cases, multiple Al agents can be used. A first Al agent can receive the initial search query and generate an intermediate search query. A second Al agent can receive the intermediate search query and generate a final search query or another intermediate search query. In any implementation, the final search query that is generated can be input into the second machine learning model at the first step of the method, so that the second machine learning model can generate the one or more query vector embeddings.

[0072] Example 2

[0073] A system for storing data comprises one or more memory devices each storing machine-readable instructions, one or more storage devices, one or more processors, and a vector database stored in the one or more storage devices. The vector database may also be4930-7714-2882 1065472-000993WOPT 17stored on a computing system or storage devices connected to the system by a wireless or wired network connection. The vector database may include one or more vector embeddings positioned in a vector field, wherein each of the one or more vector embeddings is indicative of medical data of one or more individuals, and wherein the positioning of the one or more vector embeddings in the vector field is based on the similarity of the one or more vector embeddings to each other. The vector embeddings may be created by a machine learning algorithm, a mathematical algorithm, or another process. These embeddings may be indicative of medical information relating to one or more individuals, such as an age of each of the one or more individuals, a height of each of the one or more individuals, a weight of each of the one or more individuals, a sex of each of the one or more individuals, a treatment history of each of the one or more individuals, data generated by one or more tests performed on each of the one or more individuals, data generated by one or more procedures undergone by each of the one or more individuals, image data associated with imaging performed on each of the one or more individuals, physiological characteristics of each of the one or more individuals, an indication of whether each of the one or more individuals has one or more implanted devices, or any combination thereof. The vector embeddings may represent key features of the data in a vector format. The vector embeddings may also be used to convert data of different formats into a numerical vector form suitable for storage in the vector database. For example, the medical data may be video, audio, image, numerical, or other data. The vector embeddings may represent these different data types in the vector database.

[0074] The vector database can be searched. A method of searching the system of claim 1 may comprise receiving a search query; analyzing, by a machine learning (ML) model, the search query to establish one or more query vector embeddings indicative of the search query; searching the vector database using the one or more query vector embeddings indicative of the search query; and generating one or more search results based on a similarity between the one or more vector embeddings in the vector database and the one or more query vector embeddings. The search query may be a natural language query, a semantic search, one or more vector embeddings, or a prompt from an Al agent. The search query may be a final search query. The final search query may be generated by inputting an initial search query into an Al agent and generating, by the Al agent, the final search query based on the initial search query. The Al agent may be configured to receive one or more vector embeddings. The Al agent may also be configured to receive an intermediate search query based on the initial search query. The intermediate search query may be generated by the Al agent itself. In some example embodiments, the initial search query may be generated by a first Al agent, and the intermediate4930-7714-2882 1065472-000993WOPT 18search query may be generated by inputting the initial search query into a second Al agent configured to generate the intermediate search query. In some other example embodiments, the method may include inputting an initial search query into a first Al agent, generating a plurality of intermediate search queries, a first of the plurality of intermediate search queries being based on the initial search query, and each subsequent one of the plurality of intermediate search queries being based on a preceding one of the plurality of intermediate search queries, inputting a last one of the plurality of intermediate search queries into the first Al agent or a second Al agent, and generating by the first Al agent or the second Al agent, the final search query based on the last one of the plurality of intermediate search queries. The method may analyze the search query to establish query vector embeddings that are indicative of the query. By comparing the query vectors to the one or more vector embeddings in the vector database, suitable search results are returned. Search result vector embeddings from the vector database can be generated by comparing the query vectors to the one or more vector embeddings in the vector database using k nearest neighbor searching, Euclidean distance, or other vector comparison and search methods. Machine learning, Al, and large language models can also be used to generate vector embeddings from the vector database to the query.

[0075] A method of analyzing an individual comprises receiving a search query regarding a clinical condition of the individual. The method may then train a machine learning (ML) model to determine a score representing the likelihood that the clinical condition of the individual will deteriorate based on at least the search query and one or more vector embeddings positioned in a vector field of a vector database, the one or more vector embeddings being indicative of medical data of one or more other individuals, and determine, using the one or more vector embeddings in the vector database and the trained machine learning model, the score representing the likelihood that the clinical condition of the individual will deteriorate. The ML model may be an artificial neural network (ANN), an LLM, a forwardforward algorithm (FFA), a reinforcement learning model, or another ML or Al model. Training the ML model may comprise using the vector embeddings indicative of medical data for one or more individuals to train the ML model. The ML model may find patterns and causal relationships in these data. Further, the ML model may be trained on external data sources or real-time data from sources like medical telemetry. The trained model can then recognize patterns and relationships in the data and be used to establish a score representing the likelihood that the clinical condition of the one or more individuals will deteriorate. This score can be a numerical score on a scale, a natural language assessment, a symbol, or another means of scoring.4930-7714-2882 1065472-000993WOPT 19

[0076] A system for analyzing an individual comprises one or more memory devices storing machine-readable instructions, and one or more processors configured to execute the machine-readable instructions to: receive a search query regarding a clinical condition the individual; training a machine learning (ML) model to determine a score representing a likelihood that the clinical condition of the individual will deteriorate based on at least the search query and one or more vector embeddings positioned in a vector field of a vector database, the one or more vector embeddings being indicative of medical data of one or more other individuals; and establishing, using the one or more vector embeddings in the vector database and the trained machine learning model, a score representing the likelihood that the clinical condition of the individual will deteriorate. The vector embeddings indicative of medical data of one or more individuals may be stored on a storage of the system, on the memory, or on the storage or memory of another computing system connected by wired or wireless network. The vector embeddings indicative of medical data of one or more individuals may also be within a vector database positioned according to their similarity to each other.

[0077] Example 3

[0078] Introduction

[0079] NIDIA (Nursing Intuition and Advanced Digital Intelligence) is a high-performing early warning system that assigns every patient a risk score at admission and continuously adjusts it based on the patient’s unique clinical trajectory, enabling detection of clinical deterioration up to two hours before visible signs or symptoms appear. Traditional early warning systems use outdated technology built with fixed thresholds that fail to adapt to patient-specific changes. This results in highly sensitive systems with high false positive rates that cause frontline staff to disable warning systems and places patients at risk of stroke, cardiac arrest, sepsis, ICU admission, or death. To address this problem, NIDIA was built. NIDIA is a machine learning model that merges acute medicine and nursing intuition and includes clinically focused large language models (LLM), transformers for sequence-to-sequence tasks, a novel forward-forward Model (FFM) that avoids traditional backpropagation, and multidimensional vector embeddings stored in a vector database. The system was trained and internally validated using a 70 / 30 data split and k-fold cross-validation. It achieved 87.3% accuracy, 93% precision, 84% recall, an Fl-score of 0.78, and an ROC AUC of 0.83, demonstrating strong discrimination and fewer false positives. NIDIA mimics human cognitive processes, learns continuously from new data, and has high predictive value. It offers the potential to lower alert fatigue and improve clinical outcomes.

[0080] Failure to identify and intervene before a patient shows signs and symptoms of clinical4930-7714-2882 1065472-000993WOPT 20deterioration is a failure of the early warning system. Early identification remains a persistent challenge despite efforts to deploy detection systems to effectively solve this worldwide problem. In-hospital cardiac arrest and unanticipated intensive care unit (ICU) transfers are associated with high mortality. Arrests are predominantly respiratory and metabolic with high mortality post-arrest that suggests efforts to predict could prove beneficial.

[0081] Traditional early warning systems (EWS) rely on fixed thresholds and isolated data points that often overlook many of the subtle patient-specific signs of clinical deterioration. The inability of EWS to balance sensitivity and specificity leads to increased workload and alarm fatigue. Excessive false and unactionable alarms desensitize nurses leading to alarm indifference that results in missing critical alarms. These misleading alarms are largely created by a mismatch between the default threshold for the alarm and what is relevant for the patient based on their size, age, condition, or context, or introduced through poor connectivity between sensors and the patient. In intensive care and cardiac units, nurses experience over 100 alarms per patient bed per day, with some units reporting up to 350 alarms per patient per day 562. The majority of these alarms are false or non-acti enable, leaving nurses to judge necessity based on context and available resources. Early warning systems can have up to a 77% false positive rate, resulting in frequent alerts for patients who are not actually at risk.

[0082] The primary goal of this project is to use a nursing intuition scale to reduce alarm false positives by establishing an appropriate sensitivity and increasing specificity. A clinical decision support (CDS) tool was developed that extracts nurses’ intuition using clinical suspicion based on both documented findings and behaviors from the EHR and transform into observable data that supports early prediction of clinical decline in hospitalized patients. To use near real-time data from the patient’s EHR and utilize artificial intelligence (Al) and machine learning (ML) algorithms to calculate a percentage value that is more specific and less sensitive and predicts the patient’s likelihood of deterioration and requiring an escalation of care with more accuracy, reducing the volume of false positives, and thereby reducing alarm fatigue and staff burnout. To include in this machine learning algorithm a variable that is based on intuition; based on instinctive feeling rather than conscious reasoning. The machine learning algorithm will be based on data science but include human insight; (1) vital signs, (2) nursing assessments, (3) laboratory test results (laboratory tests that do not change that fast; hemoglobin’s, lactates, electrolytes), and (4) nursing intuition.

[0083] The ML model is continuously learning from alarm patterns to optimize alert algorithms and successfully analyze and prioritize alarms based on patient data, thereby reducing unnecessary alerts and mitigating nurse desensitization. This paper emphasizes the4930-7714-2882 1065472-000993WOPT 21pivotal role of Al and ML in reshaping early warning systems to address alarm fatigue in nursing. By leveraging advanced technologies, healthcare institutions can enhance patient safety and improve nurse responsiveness to critical alarms. The potential benefits of integrating Al and ML in early warning systems underscore the need for further research and implementation in clinical practice. This review aims to inspire ongoing academic discourse and practical initiatives to combat alarm fatigue and elevate the standard of care in nursing.

[0084] Methods

[0085] 718 patients admitted into the inpatient units at a hospital were examined. The study cohort consisted of adult patients over the age of 18, they were not pregnant, not diagnosed with mental illness, and not admitted into intensive care units. It was counted the number of times that these patients met the Systemic inflammatory response syndrome (SIRS) criteria and activated a Best Practice Alert / Operational Practice Alter (BPA / OPA). 5,862 BPA / OPA’s were activated. Out of these alert activations 45% were accurate and 51% of them were false positives, meaning that the patient was not clinically deteriorating. The role of the Rapid Response Team is to respond to each alarm activation.

[0086] An Advanced Digital Intelligence (ADI), named NIDIA, was created which united two Machine Learning approaches, bringing together Large Language Model (LLM) and a Forward-Forward Algorithm (FFA) to design a vector-based database that allows for faster learning and deeper intelligence. The ADI learned at 7x the speed of conventional ANN models. The accuracy of the ADI model increased to 84.7%, meeting our gold standard for bedside Clinical Decision Support (CDS) tools, the false positives were reduced to 7%.

[0087] Over time training the model, NIDIA became more accurate and reduced even further the number of false positives. In one example, measured were 260 patients, 2,634,194 flowsheets, 22,321 diagnoses, 149,769 lab results 80,325, 52,617 procedures, and 3,35 imaging studies, consuming over 18,109 input parameters and super-parameters. This analysis resulted in an accuracy of 89.1% and reducing false positives to 5%.

[0088] The nursing intuition score was developed by collecting nurses’ intuition of patients at time of admission and during their head-to-toe evaluation. Inspired by the pain scale and several focus group sessions involving frontline nursing staff from the Cedars Sinai Medical Center (CSMC) in units 6-NE, 6-NW, 6- SE, and 6-SW. Approximately 25 staff members were involved in the initial prototype, obtained by in from participating nurses.

[0089] Collection of Data and Outcomes

[0090] Generally, these nursing intuition scores would be entered or updated at least three times per 24 hours, morning, evening, and night. The nurse's years of experience was not4930-7714-2882 1065472-000993WOPT 22included in this study to test the hypothesis, but it may show that more experienced nurses would be more accurate in their judgment of patient risk.

[0091] The outcome of this scientific study is to provide clinicians with a near real-time patient’s deterioration index percentage value every fifteen minutes or whenever data is updated or inserted into the patients EHR. This tool will enhance decision-making in the clinical workflow and generate computerized alerts. This tool will also increase quality, enhance health outcomes, assist in avoiding errors and attempt to reduce adverse events, improving efficiency, cost-benefit, and increase provider and patient satisfaction.

[0092] All data related to an escalation of care (Rapid Response Team (RRT) activation, transfer to ICU, or Code Blue (cardiac or respiratory resuscitation), or mortality) were obtained from the Electronic Medical Record (EHR). The transfers to the ICU were obtained indirectly through the EHR based on the patient location data. Cases in which the charge nurse was requested to the bedside were considered an escalation care and considered possible signs of a deterioration event and were reviewed manually.

[0093] Potential deterioration was evaluated by either a physician or a nurse practitioner who did not have any contact with the patient. Reviews were all performed retrospectively, and reviewers were blinded to the nursing intuition score. All outcome events included RRT activations, ICU transfers, and codes (call for resuscitation after cardiorespiratory arrest). These outcomes were collected from the electronic health record system (EHR).

[0094] Model Development and Evaluation

[0095] The training of NIDIA followed a hybrid Al training framework, combining unsupervised pretraining with supervised fine-tuning using backpropagation and forwardforward methods. The training process of NIDIA involved initially unsupervised backpropagation, and then repeating forward propagation, loss calculation, backpropagation, and optimization over many epochs. By iteratively adjusting weights and biases based on loss gradients, the neural network has gradually learned after relationships were determined between input and output data through supervised learning to minimize prediction errors and generalize effectively to new data. With each cycle, the artificial neural network has become more efficient at recognizing patterns in the data, refining its parameters to make more accurate predictions.

[0096] NIDIA trained on an input dataset 5,966,350 rows of data (for 718 patients) made up of data from the EHR (multiples of two 11,932,700 a forward pass and a backward pass), examine the neural network node activity level on a forward pass, and look at the dimensional output and the final gradient weight (coefficient of 0.25).4930-7714-2882 1065472-000993WOPT 23

[0097] Statistical Analysis

[0098] The area under the receiver operating characteristic curve (AUROC) was calculated by treating the nursing intuition score as an alert. Any index alert: if the patient had an outcome of interest in the 24 hours after that index alert, it is considered a true positive. False negatives were instances in which an outcome occurred that was not preceded by an index alert. Data acquisition was done using SQL scripts and extracted from Epic’s Caboodle. Data Cleansing and analysis was done using PyCharm and the Python programming language version 3.9. Feature engineering was done using Python. The first pass of the linear regression modeling was run against DataRobots cloud-based service. The second pass of linear regression modeling was conducted using Python and scikit-learn, an open-source machine learning library in Python that provides simple and efficient tools for predictive data analysis tool built on NumPy, SciPy, and matplotlib. The use of scikit- learn allowed for engineering machine learning algorithms; (1) classification, (2) regression, (3) clustering, and (4) dimensionality reduction. Data visualization was created using Python and matplotlib.

[0099] The first calculation was based on true positive alerts where the patient meets the criteria of conventional alert system for an alert and there is an escalation of care. False positives are when a patient meets the criteria of conventional alert systems for an alert, but there is no escalation of care required. False negative is when a patient requires an escalation of care but there is no alert with conventional alert systems. A true negative is when a patient does not require an escalation of care and there is no alert with conventional alert systems.

[0100] Results

[0101] The 718 unique patient dataset expands out to 1,220 encounters in different settings, and 8,540 alerts, computing to 107,700 events to be observed by the model. Prior to the use of the NIDIA Al model the legacy false positives would be 54,927, using the NIDIA Al ADI model the false positives have been reduced to 7,539 events, that is a 44% reduction in false alarms (false positives) thereby protecting valuable resources, saving money, increasing quality, reducing burn out, balancing capacity, and improving outcomes for patients, and reducing risk.

[0102] One or more elements or aspects or steps, or any portion(s) thereof, from one or more of any of the claims below can be combined with one or more elements or aspects or steps, or any portion(s) thereof, from one or more of any of the other claims or combinations thereof, to form one or more additional implementations and / or claims of the present disclosure.4930-7714-2882 1065472-000993WOPT 24

[0103] While the present disclosure has been described with reference to one or more particular embodiments or implementations, those skilled in the art will recognize that many changes may be made thereto without departing from the spirit and scope of the present disclosure. Each of these implementations and obvious variations thereof is contemplated as falling within the spirit and scope of the present disclosure. It is also contemplated that additional implementations according to aspects of the present disclosure may combine any number of features from any of the implementations described herein.4930-7714-2882 1065472-000993WOPT 25

Claims

CLAIMSWHAT IS CLAIMED IS:

1. A method for determining a deterioration score of a patient associated with a plurality of potential deterioration events, the method comprising: receiving medical data associated with the patient; inputting the medical data into a plurality of large language models (LLMs), each of the plurality of LLMs trained to: analyze at least a respective portion of the medical data; identify a plurality of tokens within the respective portion of the medical data; determine a patient-patient specific relevance of each of the plurality of tokens to the plurality of potential deterioration events; and output a patient-specific coefficient for one or more of the plurality of tokens indicative of the relevance of the one or more plurality of tokens to the plurality of deterioration events; and determining the deterioration score based at least in part on the coefficients outputted by the plurality of LLMs, the deterioration score being indicative of a percentage chance that the patient will suffer at least one of the plurality of deterioration events.

2. The method of claim 1, wherein at least some of the plurality of LLMs are configured to analyze a distinct portion of the medical data.

3. The method of claim 2, wherein the plurality of LLMs includes: a first LLM trained to analyze a first portion of the medical data associated with a past history of the patient; a second LLM trained to analyze a second portion of the medical data associated with actions of a healthcare provider during a current encounter with the patient; and a third LLM trained to analyze a third portion of the medical data associated with outcomes associated with the actions of the healthcare provider during the current encounter with the patient.

4. The method of claim 3, wherein the deterioration score is determined by obtaining a product and / or an average of (i) the m-highest coefficients of the first LLM, (ii) the m-highest coefficients of the second LLM, and (iii) the / ^-highest coefficients of the third LLM.4930-7714-2882 1065472-000993WOPT 265. The method of claim 3, wherein the first LLM is trained using backpropagation techniques, and wherein the second LLM and the third LLM are trained using forward-forward techniques.

6. The method of claim 3, wherein the plurality of LLMs further includes a fourth LLM trained to analyze (i) the coefficients of at least the first LLM, the second LLM, and the third LLM and (ii) the medical data.

7. The method of claim 6, wherein the fourth LLM is a reinforcement learning model.

8. The method of claim 7, wherein the deterioration score is determined by obtaining a product and / or an average of (i) the m-highest coefficients of the first LLM, (ii) the m-highest coefficients of the second LLM, (iii) the m-highest coefficients of the third LLM, and (iv) the / 77-highest coefficients of the fourth LLM.

9. The method of claim 3, wherein the deterioration score is further based on coefficients output by a reference LLM trained on non-patient-specific clinical reference data associated with the plurality of deterioration events, the reference LLM trained to output non-patient specific coefficients for a plurality of reference tokens in the clinical reference data that are indicative of the relevance of the plurality of reference tokens to the plurality of deterioration events.

10. The method of claim 9, wherein the reference LLM is trained using backpropagation techniques.

11. The method of claim 9, wherein the coefficients output by the reference LLM are internal attention weights assigned to the plurality of reference tokens by an encoder portion of the reference LLM.

12. The method of claim 9, wherein the deterioration score is determined by obtaining a product and / or an average of (i) the m-highest coefficients of the first LLM, (ii) the m-highest coefficients of the second LLM, (iii) the m-highest coefficients of the third LLM, (iv) the m-4930-7714-2882 1065472-000993WOPT 27highest coefficients of the fourth LLM, and (v) the m-highest coefficients of the reference LLM.

13. The method of claim 1, wherein the coefficients output by each respective LLM of the plurality of LLMs are internal attention weights assigned to the plurality of tokens by an encoder portion of the respective LLM.

14. The method of claim 1, wherein each respective LLM of the plurality of LLMs is trained to output the / ^-highest coefficients among the coefficients for all of the plurality of tokens identified by the respective LLM.

15. The method of claim 1, wherein the deterioration score is based at least in part on the / 77-highest coefficients output by each respective LLM of the plurality of LLMs among all of the coefficients output by each respective LLM.

16. The method of claim 13, wherein the deterioration score is determined by obtaining a product and / or an average of the m -highest coefficients of each of the plurality of LLMs.

17. The method of claim 13, wherein the deterioration score is determined by obtaining a product and / or an average of (i) the m-highest coefficients of each of the plurality of LLMs and (ii) the m-highest coefficients output by a reference LLM trained to identify reference tokens within clinical reference data.

18. The method of any one of claims 4, 8, 12, 16, and 17, wherein the product and / or the average is scaled to be on a scale between 0 and 100 to determine the deterioration score.

19. The method of any one of claims 4, 8, 12, and 14 to 17, wherein m=10.

20. The method of claim 1, wherein the plurality of deterioration events includes the patient being transferred to an intensive care unit of a healthcare facility, the patient suffering from cardiac arrest, the patient suffering from respiratory arrest, the patient suffering a stroke, a rapid response team being activated to attend to the patient, the patient dying, or any combination thereof.4930-7714-2882 1065472-000993WOPT 2821. The method of claim 1, wherein the deterioration score is indicative of the percentage chance that the patient will suffer at least one of the plurality of deterioration events within a future timeframe.

22. The method of claim 21, wherein the future timeframe is 1 hour, 2 hours, 4 hours, 6 hours, 8 hours, 12 hours, 24 hours, 48 hours, 5 days, or 1 week.

23. The method of claim 1, further comprising transmitting to a healthcare provider of the patient an indication of the deterioration score.

24. The method of claim 23, further comprising transmitting to the healthcare provider of the patient an indication of at least one token identified by at least one of the plurality of LLMs.

25. The method of claim 23, further comprising transmitting to the healthcare provider of the patient an indication of a coefficient for at least one token identified by at least one of the plurality of LLMs.

26. The method of claim 24 or claim 25, wherein the at least one token includes the token among all of the plurality of tokens identified by the plurality of LLMs having the highest coefficient.

27. The method of claim 24 or claim 25, wherein the at least one token includes the n tokens among all of the plurality of tokens identified by the plurality of LLMs having the n- highest coefficients.

28. A system for determining a deterioration score of a patient associated with a plurality of potential deterioration events, the system comprising: one or more memory devices storing machine-readable instructions; and one or more processors, each of the one or more processors configured to execute the machine-readable instructions of at least one of the one or more memory devices to implement the method of any one of claims 1 to 27.4930-7714-2882 1065472-000993WOPT 2929. A system for determining a deterioration score of a patient associated with a plurality of potential deterioration events, the system including one or more processors configured to implement the method of any one of claims 1 to 27.

30. A computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 27.

31. The computer program product of claim 30, wherein the computer program product is a non-transitory computer readable medium.

32. A system for determining a deterioration score of a patient associated with a plurality of potential deterioration events, the system comprising: a plurality of large language models (LLMs), each of the plurality of LLMs being trained to: analyze at least a respective portion of medical data associated with the patient; identify a plurality of tokens within the respective portion of the medical data; determine a patient-patient specific relevance of each of the plurality of tokens to the plurality of potential deterioration events; and output a patient-specific coefficient for one or more of the plurality of tokens indicative of the relevance of the one or more plurality of tokens to the plurality of deterioration events; and a scoring algorithm configured to determine the deterioration score based at least in part on the coefficients outputted by the plurality of LLMs, the deterioration score being indicative of a percentage chance that the patient will suffer at least one of the plurality of deterioration events.

33. The system of claim 32, wherein at least some of the plurality of LLMs are configured to analyze a distinct portion of the medical data of the patient.

34. The system of claim 32, wherein the plurality of LLMs includes: a first LLM trained to analyze a first portion of the medical data associated with a past history of the patient; a second LLM trained to analyze a second portion of the medical data associated with actions of a healthcare provider during a current encounter with the patient; and4930-7714-2882 1065472-000993WOPT 30a third LLM trained to analyze a third portion of the medical data associated with outcomes associated with the actions of the healthcare provider during the current encounter with the patient.

35. The system of claim 34, wherein the first LLM is trained using backpropagation techniques, and wherein the second LLM and the third LLM are trained using forward-forward techniques.

36. The system of claim 34, wherein the plurality of LLMs includes a fourth LLM trained to analyze (i) the coefficients of at least the first LLM, the second LLM, and the third LLM and (ii) the medical data.

37. The system of claim 36, wherein the fourth LLM is a reinforcement learning model.

38. The system of claim 32, further comprising a reference LLM trained on non-patient- specific Oclinical reference data associated with the plurality of deterioration events, the reference LLM trained to output non-patient specific coefficients for a plurality of reference tokens in the clinical reference data that are indicative of the relevance of the plurality of reference tokens to the plurality of deterioration events.

39. The system of claim 38, wherein the reference LLM is trained using backpropagation techniques.

40. The system of claim 39, wherein the coefficients output by the reference LLM are internal attention weights assigned to the plurality of reference tokens by an encoder portion of the reference LLM.

41. The system of claim 32, wherein the coefficients output by each respective LLM of the plurality of LLMs are internal attention weights assigned to the plurality of reference tokens by an encoder portion of the respective LLM.

42. The system of claim 32, wherein the scoring algorithm is configured to determine the deterioration score based at least in part on the m-highest coefficients output by each respective LLM of the plurality of LLMs among all of the coefficients output by each respective LLM.4930-7714-2882 1065472-000993WOPT 3143. The system of claim 42, wherein the scoring algorithm is configured to determine the deterioration score by obtaining a product and / or an average of the m-highest coefficients of each of the plurality of LLMs.

44. The system of claim 32, wherein the scoring algorithm is configured to determine the deterioration score based at least in part on (i) the m-highest coefficients output by each respective LLM of the plurality of LLMs among all of the coefficients output by each respective LLM and (ii) the m-highest coefficients output by a reference LLM trained on non- patient-specific clinical reference data associated with the plurality of deterioration events, the reference LLM trained to output non-patient specific coefficients for a plurality of reference tokens in the clinical reference data that are indicative of the relevance of the plurality of reference tokens to the plurality of deterioration events.

45. The system of claim 44, wherein the scoring algorithm is configured to determine the deterioration score by obtaining a product and / or an average of the m-highest coefficients of each of the plurality of LLMs and the m-highest coefficients of the reference LLM .

46. The system of claim 43 or claim 45, wherein the scoring algorithm is configured to scale the product and / or the average on a scale between 0 and 100 to determine the deterioration score.

47. The system of any one of claims 42 to 44, wherein m=10.

48. The system of claim 32, wherein the plurality of deterioration events includes the patient being transferred to an intensive care unit of a healthcare facility, the patient suffering from cardiac arrest, the patient suffering from respiratory arrest, the patient suffering a stroke, a rapid response team being activated to attend to the patient, the patient dying, or any combination thereof.

49. The system of claim 32, wherein the deterioration score is indicative of the percentage chance that the patient will suffer at least one of the plurality of deterioration events within a future timeframe.4930-7714-2882 1065472-000993WOPT 3250. The system of claim 49, wherein the future timeframe is 1 hour, 2 hours, 4 hours, 6 hours, 8 hours, 12 hours, 24 hours, 48 hours, 5 days, or 1 week.

51. A system for storing data, the system comprising: one or more memory devices each storing machine-readable instructions; one or more storage devices; one or more processors; and a vector database stored in the one or more storage devices, the vector database including: one or more vector embeddings positioned in a vector field, wherein each of the one or more vector embeddings is indicative of medical data of one or more individuals, and wherein the positioning of the one or more vector embeddings in the vector field is based on the similarity of the one or more vector embeddings to each other.

52. The system of claim 51, wherein the data includes an age of each of the one or more individuals, a height of each of the one or more individuals, a weight of each of the one or more individuals, a sex of each of the one or more individuals, a treatment history of each of the one or more individuals, data generated by one or more tests performed on each of the one or more individuals, data generated by one or more procedures undergone by each of the one or more individuals, image data associated with imaging performed on each of the one or more individuals, physiological characteristics of each of the one or more individuals, an indication of whether each of the one or more individuals has one or more implanted devices, or any combination thereof.

53. The system of claim 51 , wherein the medical data includes an electronic medical record of each of the one or more individuals.

54. The system of claim 51, wherein the medical data includes medical telemetry data of each of the one or more individuals.

55. The system of claim 51, wherein the medical data includes a score indicative of the intuition of one or more medical professionals regarding a physiology or quality of life of each of the one or more individuals.4930-7714-2882 1065472-000993WOPT 3356. A method of searching the system of claim 51, the method comprising: receiving a search query; analyzing, by a machine learning (ML) model, the search query to establish one or more query vector embeddings indicative of the search query; searching the vector database using the one or more query vector embeddings indicative of the search query; and generating one or more search results based on a similarity between the one or more vector embeddings in the vector database and the one or more query vector embeddings.

57. The method of claim 56, wherein the one or more search results include at least one of the one or more vector embeddings in the vector database.

58. The method of claim 57, wherein the one or more search results include text, image, video, audio, or numerical data indicative of one or more vector embeddings in the vector database, or any combination thereof.

59. The method of claim 56, wherein generating the one or more search results based on the similarity between the one or more vector embeddings in the vector database and the one or more query vector embeddings includes: positioning the one or more query vector embeddings in the vector field of the vector database; and determining the similarity using a k-nearest neighbor (kNN) search algorithm.

60. The method of claim 56, wherein generating the one or more search results based on the similarity between the one or more vector embeddings in the vector database and the one or more query vector embeddings includes: positioning the one or more query vector embeddings in the vector field of the vector database; and determining a Euclidean distance between each of the one or more vector embeddings in the vector database and each of the one or more query vector embeddings.4930-7714-2882 1065472-000993WOPT 3461. The method of claim 56, wherein the similarity between the one or more vector embeddings in the vector database and the one or more query vector embeddings is determined by a large-language model (LLM) trained on the one or more vector embeddings in the vector database.

62. The method of claim 56, wherein the similarity between the one or more vector embeddings in the vector database and the one or more query vector embeddings is determined by an ML model trained on the one or more vector embeddings in the vector field of the vector database.

63. The method of claim 61 or claim 62, wherein the similarity between the one or more vector embeddings in the vector database and the one or more query vector embeddings is established by refining the search results through one or more subsequent applications of an ML model trained on the one or more vector embeddings in the vector field of the vector database to the search results.

64. A method of analyzing an individual, the method comprising: receiving a search query regarding a clinical condition of the individual; generating, via a trained machine learning (ML) model, one or more query vector embeddings based on the search query; training a machine learning (ML) model to determine a score representing a likelihood that the clinical condition of the individual will deteriorate based on at least the search query and one or more vector embeddings positioned in a vector field of a vector database, the one or more vector embeddings being indicative of medical data of one or more other individuals; and determining, using the one or more vector embeddings in the vector database and the trained machine learning model, the score representing the likelihood that the clinical condition of the individual will deteriorate.

65. The method of claim 64, wherein determining the score representing the likelihood that the clinical condition of the individual will deteriorate includes: establishing one or more query vector embeddings based on the search query; and comparing the one or more query vector embeddings with the one or more vector embeddings in the vector database.4930-7714-2882 1065472-000993WOPT 3566. The method of claim 65, wherein each respective one of the one or more vector embeddings in the vector database is positioned in the vector field based on a similarity between the respective one of the one or more vector embeddings and all others of the one or more vector embeddings.

67. The method of claim 65, wherein comparing the query vector embeddings with the one or more vector embeddings in the vector database includes determining a similarity using a k- nearest neighbor (kNN) search algorithm.

68. The method of claim 65, wherein comparing the query vector embeddings with the one or more vector embeddings in the vector database includes determining a similarity using a Euclidean distance.

69. The method of claim 64, wherein the search query is generated by an artificial intelligence (Al) agent.

70. The method of claim 69, wherein the Al agent is a large language model (LLM) trained to generate one or more search queries regarding the clinical condition of the one or more individuals.

71. The method of claim 64, wherein the search query is a final search query, and wherein the method further comprises: inputting an initial search query into an Al agent; and generating, by the Al agent, the final search query based on the initial search query.

72. The method of claim 64, wherein the search query is a final search query, and wherein the method further comprises: inputting an initial search query into an Al agent; generating, by the Al agent, an intermediate search query based on the initial search query; inputting the intermediate search query back into the Al agent; and generating, by the Al agent, the final search query based on the intermediate search query.4930-7714-2882 1065472-000993WOPT 3673. The method of claim 64, wherein the search query is a final search query, and wherein the method further comprises: inputting an initial search query into a first Al agent; generating, by the first Al agent, an intermediate search query based on the initial search query; inputting the intermediate search query into a second Al agent; and generating, by the Al agent, the final search query based on the initial search query.

74. The method of claim 64, wherein the search query is a final search query, and wherein the method further comprises: inputting an initial search query into a first Al agent; generating a plurality of intermediate search queries, a first of the plurality of intermediate search queries being based on the initial search query, and each subsequent one of the plurality of intermediate search queries being based on a preceding one of the plurality of intermediate search queries; inputting a last one of the plurality of intermediate search queries into the first Al agent or a second Al agent; and generating, by the first Al agent or the second Al agent, the final search query based on the last one of the plurality of intermediate search queries.

75. The method of claim 74, wherein at least one of the plurality of intermediate search queries is generated by the first Al agent.

76. The method of claim 75, wherein all of the plurality of intermediate search queries are generated by the first Al agent.

77. The method of claim 74 or claim 75, wherein at least one of the plurality of intermediate search queries is generated by the second Al agent.

78. The method of claim 77, wherein all of the plurality of intermediate search queries are generated by the second Al agent.4930-7714-2882 1065472-000993WOPT 3779. The method of claim 74, wherein each of the plurality of intermediate search queries is generated by an Al agent other than the first Al agent and the second Al agent.

80. The method of claim 79, wherein each of the plurality of intermediate search queries is generated by a third Al agent.

81. The method of claim 79, wherein at least one of the plurality of intermediate search queries is generated by a third Al agent and at least one of the plurality of intermediate search queries is generated by a fourth Al agent.

82. The method of claim 79, wherein each of the plurality of intermediate search queries is generated by a different Al agent.

83. The method of claim 64, wherein the medical data includes an age of the one or more individuals, a height of the one or more individuals, a weight of the one or more individuals, a sex of the one or more individuals, a treatment history of the one or more individuals, data generated by one or more tests performed on the one or more individuals, data generated by one or more procedures on the one or more individuals, image data associated with imaging performed on the one or more individuals, physiological characteristics of the one or more individuals, an indication of whether the one or more individuals has one or more implanted devices, or any combination thereof.

84. The method of claim 64, wherein the medical data includes an electronic medical record of the one or more individuals.

85. The method of claim 64, wherein the ML model is a neural network.

86. The method of claim 64, wherein the ML model is a large language model (LLM).

87. A system for analyzing an individual, the system comprising: one or more memory devices storing machine-readable instructions; and one or more processors configured to execute the machine-readable instructions to: receive a search query regarding a clinical condition of the individual;4930-7714-2882 1065472-000993WOPT 38train a machine learning (ML) model to determine a score representing a likelihood that the clinical condition of the individual will deteriorate based on at least the search query and one or more vector embeddings positioned in a vector field of a vector database, the one or more vector embeddings being indicative of medical data of one or more other individuals; and determine, using the one or more vector embeddings in the vector database and the trained machine learning model, a score representing the likelihood that the clinical condition of the individual will deteriorate.

88. The system of claim 87, wherein the medical data includes an electronic medical record of each of the one or more other individuals.

89. The system of claim 87, wherein the ML model is a neural network.

90. The system of claim 87, wherein the ML model is a large language model (LLM).4930-7714-2882 1065472-000993WOPT 39

Images