Systems and methods for classifying the severity of illness of a patient

Abstract

A method of classifying a severity of illness of a patient comprises receiving data associated with the patient, inputting at least the received data into a first machine learning model and executing the first machine learning model. The first machine learning model is trained to identify a plurality of tokens within the data associated with a characteristics of the patient, and assign a weight to each token. The method further comprises inputting the tokens into a second machine learning model and executing the second machine learning model. The second machine learning model is trained to assign a rank to each token. The method further comprises generating an indication of the severity of illness of the patient based on (i) the plurality of tokens, (ii) the weight assigned to each respective token of the plurality of tokens, and (iii) the rank assigned to each respective token of the plurality of tokens.

Description

SYSTEMS AND METHODS FOR CLASSIFYING THE SEVERITY OF ILLNESS OFA PATIENTCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of, and priority to, U.S. Provisional Patent Application No. 63 / 560,557 filed on March 1, 2024, which is hereby incorporated by reference herein in its entirety.TECHNICAL FIELD

[0002] The present disclosure relates generally to systems and methods for analyzing illness of a patient, and more particularly, to systems and methods for determining the severity of illness of a patient using at least one machine learning model trained using a back- propagation method, and at least one machine learning model trained using a forward-forward method.BACKGROUND

[0003] Patients with serious illnesses may benefit from advance care planning, palliative care such as symptom management, emotional and psychological support for them, their families, and their loved ones, goals of care conversations, and / or end of life care, etc. However, it can be difficult for one single healthcare provider to easily recognize during a visit with the patient whether the patient’s illness is serious enough to warrant a recommendation for palliative care or other types of supportive care. Thus, new systems and methods quickly identifying the severity of illnesses in patients are needed.SUMMARY

[0004] A method of classifying the severity of illness of a patient includes receiving data associated with the patient, inputting at least the received data into a first machine learning model and executing the first machine learning model. The first machine learning model is trained to identify a plurality of tokens within the data associated with a characteristics of the patient and assign a weight to each token. The method further includes inputting the tokens into a second machine learning model and executing the second machine learning model. The second machine learning model is trained to assign a rank to each token. The method further includes generating an indication of the severity of illness of the patient based on (i) theplurality of tokens, (ii) the weight assigned to each respective token of the plurality of tokens, and (iii) the rank assigned to each respective token of the plurality of tokens.

[0005] A system for classifying the severity of illness of a patient includes one or more memory devices and one or more processors. The one or more memory devices store machine- readable instructions. The one or more processors are configured to execute the machine- readable instructions to implement a method that includes receiving data associated with the patient, inputting at least the received data into a first machine learning model and executing the first machine learning model. The first machine learning model is trained to identify a plurality of tokens within the data associated with a characteristics of the patient and assign a weight to each token. The method further includes inputting the tokens into a second machine learning model and executing the second machine learning model. The second machine learning model is trained to assign a rank to each token. The method further includes generating an indication of the severity of illness of the patient based on (i) the plurality of tokens, (ii) the weight assigned to each respective token of the plurality of tokens, and (iii) the rank assigned to each respective token of the plurality of tokens.

[0006] A system for classifying the severity of illness of a patient includes a first machine learning model, a second machine learning model, and one or more processors. The first machine learning model is configured to receive data associated with the patient, and is trained to (i) identify a plurality of tokens within the data and (ii) assign a weight to each respective token. Each token is associated with a characteristic of the patient. The second machine learning model is configured to receive the plurality of tokens and / or the weight assigned to each token, and is trained to assign a rank to each respective token. The one or more processors are configured to generate an indication of the severity of the illness of the patient based on the plurality of tokens, the weight assigned to reach respective token, and the rank assigned to reach respective token.

[0007] The above summary is not intended to represent each implementation or every aspect of the present disclosure. Additional features and benefits of the present disclosure are apparent from the detailed description and figures set forth below.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The disclosure, and its advantages and drawings, will be better understood from the following description of representative embodiments together with reference to the accompanying drawings. These drawings depict only representative embodiments and are therefore not to be considered as limitations on the scope of the various embodiments or claims.

[0009] FIG. 1 is a block diagram of a system for classifying the severity of illness of a patient, according to aspects of the present disclosure.

[0010] FIG. 2 is a flow chart of a method for classifying the severity of illness of a patient, according to aspects of the present disclosure.

[0011] FIG. 3 is a block diagram representing the method of FIG. 2, according to aspects of the present disclosure.

[0012] FIG. 4 is a block diagram of machine learning models of an implementation of the system of FIG. 1, according to aspects of the present disclosure.

[0013] While the present disclosure is susceptible to various modifications and alternative forms, specific implementations and embodiments have been shown by way of example in the drawings and will be described in detail herein. It should be understood, however, that the present disclosure is not intended to be limited to the particular forms disclosed. Rather, the present disclosure is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure as defined by the appended claims.DETAILED DESCRIPTION

[0014] Patients with serious and / or complex illnesses typically consume a large amount of healthcare resources, including visits with healthcare providers and other interactions with the healthcare system. The National Institutes of Health (NIH) defines serious illness as one that carries a high risk of mortality to the patient (generally at least a 70% risk of death in the next year), and either negatively impacts the patient’s daily function and / or quality of life, or imposes a high degree of caregiver burden. However, due to the number and variety of interactions that patients with a serious illness tend to experience (e.g., visiting different healthcare providers and / or facilities) and the varied manners in which relevant factors are documented in the electronic healthcare record, it can be difficult for any single provider at any moment in time to determine if that patient has a serious illness and thus if they would benefit from advance care planning, palliative care including symptom management, emotional and psychological management for the patient, their family, and their loved ones, goals of care conversations, and / or end of life care, etc. Disclosed herein are systems and methods for identifying which patients are seriously ill and thus who would benefit from appropriate interventions. This classification can be performed based on objective data associated with the patient’s current health, as well as subjective and / or semi-subjective information found in the electronic healthcare record. As used herein, the term “illness” generally does not refer to aspecific disease or condition, but instead refers to the general state of being ill (e.g., suffering from a disease or condition, or multiple diseases or conditions).

[0015] FIG. 1 illustrates a block diagram of system 100 that can be used to classify the severity of illness. The system 100 can include one or more processing devices 100, which can each include any one or more of a processor 112, a memory 114, a display 116, a user input device 118, and / or other components. The memory 114 can include machine-readable instructions for executing the methods disclosed herein, and / or other methods. The processor 112 can execute these instructions to implement these methods. The memory 114 can also store data associated with the methods.

[0016] The processing device 110 can include any suitable processing device, such as general purpose computer systems, microprocessors, digital signal processors, microcontrollers, application specific integrated circuits (ASICs), programmable logic devices (PLDs) field programmable logic devices (FPLDs), programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), mobile devices such as mobile telephones, personal digital assistants (PDAs), or tablet computers, local servers, remote servers, wearable computers, or the like. The memory device 114 can include any suitable memory device and / or machine- readable medium that is capable of storing, encoding, and / or carrying a set of instructions for execution by a processing device and that cause the processing device to perform and / or implement any of the features discussed herein, including solid-state memories, optical media, magnetic media, random access memory (RAM), read only memory (ROM), a floppy disk, a hard disk, a CD ROM, a DVD ROM, flash memory, or other computer readable medium that is read from and / or written to by a magnetic, optical, or other reading and / or writing system that is coupled to the processing device, can be used for the memory or memories. In some implementations, the system 100 may include multiple processors and multiple memories, such that the machine-readable instructions are distributed across multiple memories and executed by one or more of the processors.

[0017] The display 116 can be used to display any information associated with the features disclosed herein. The display device 116 can be any known display technology, including but not limited to display devices using Liquid Crystal Display (LCD) or Light Emitting Diode (LED) technology. The user input device 118 can be used to allow the user to interact with the system 100 for any suitable purpose, including initiating, pausing, or terminating the analysis by the model; adjusting any parameters of the analysis, etc.

[0018] In some implementations, the system 100 includes an electronic health record (EHR) 120. The EHR 120 can contain all available data about the patient, including medicalhistory, test results, doctor’s notes and other charting notes about the patient’s condition, etc. The EHR 120 may include data associated with the patient’s functional status and / or any caregiver burden, which could include, for example, doctor’s notes regarding the same. The electronic health record system 120 can also generally be updated at any time with new data associated with the patient as it is generated and / or received, model. The EHR 120 can generally be any suitable type of electronic health record.

[0019] FIG. 2 shows a flow chart of a method 200 for classifying the severity of illness of a patient. Method 200 is described as being executed using system 100, but could be executed in whole or in part, by any suitable system. Step 210 of method 200 includes receiving data associated with the patient. This data will generally be any data that is in the patient’s EHR 120, and / or that is being newly added to the patient's EHR 120. In some implementations, the data received in step 210 is the first set of data associated with the patient that is received. In other implementations however, the data received in step 210 is added to previous data that has been received for the patient.

[0020] In some implementations, the act of receiving the data comprises automatically receiving the data in the memory 114 of the system 100 after the patient’s EHR 120 is updated with the data. This can include data being added to the patient’s EHR 120 for the first time, or data being added to the data that is already in the patient’ s EHR 120. For example, if the patient visits with a healthcare provider, any new data input into the patient's EHR 120 by the healthcare provider can be automatically sent to the memory 114 of the system 100. In some cases, the memory 114 already stores previous data that was input into the patient’s EHR 120. In other cases, the act of adding new data to the patient's EHR 120 may cause all of the data (including old data) to be transmitted to the memory 114.

[0021] In other implementations, the act of receiving the data comprises receiving the data in the memory 114 manually after the patient’s EHR 120 is updated with the data. This can include data being added to the patient’s EHR 120 for the first time, or data being added to the data that is already in the patient’s EHR 120. For example, if the patient visits with a healthcare provider and the healthcare provider inputs new data into the patient's EHR 120, the healthcare provider can later cause the data to be automatically to the memory 114. In some cases, the memory 114 already stores previous data that was received from the patient’s EHR 120. In other cases, manually sending data to the memory 114 from the patient’s EHR 120 includes causing all of the data in the EHR 120 (including old data) to be transmitted to the memory

[0022] In further implementations, the act of receiving the data may include receiving the data in the memory 114 separately from any actions related to the patient’s EHR 120. For example, the memory 114 can be used to store data associated with the patient (which may also be stored in the patient’ s EHR). If new data associated with the patient is generated (e.g., during a visit with a healthcare provider), that new data can be directly input into the memory 114.

[0023] Step 220 includes inputting at least the received data into a first machine learning model and the executing the first machine learning model. The data that is input into the first machine learning model can include only data that is newly added to the patient’s EHR 120 (and / or was most recently added to the patient’s EHR 120), only data that was already stored in the patient’s EHR 120, or both. The first machine learning model is trained to identify tokens within the inputted data, and assign a weight to each of the tokens. These tokens can be a word, a phrase containing a plurality of words, a word stem, a word root, a word base, etc., which may include any combination of letters, numbers, symbols, etc. The first machine learning model will generally have been trained to look for a specific list of tokens within the inputted data.

[0024] In general, each token is related to an aspect of determining whether the patient meets the definition of being seriously ill. Thus, the tokens can includes tokens that are associated with the health and / or physiology of the patient (e.g., an age of the patient, a height of the patient, a weight of the patient, a sex of the patient, a treatment history, other physiological statistics and / or characteristics of the patient, data generated by one or more tests performed on the patient (e.g., test results), data generated by and / or as a result of one or more procedures performed on the patient, image data associated with imaging performed on the patient, measurements, types and amounts of medication being used, whether any devices are implanted, etc.). The tokens can also include tokens associated with the functional level of the patient and / or the burden on a caregiver of the patient. The tokens can include tokens that are more qualitative-focused, such as the phrases “high blood pressure,” “rapid heart rate,” “difficulty getting out of bed,” “gait speed,” “assistive device,” “wheelchair,” “cost burden,” “expensive,” etc. The tokens can also include tokens that are more quantitative-focused, such as “blood pressure 150 mm Hg,” “heart rate 120 bpm,” “adult bmi 50.0 59.9 kg sq m,” “adult bmi 60.0 69.9 kg sq m,” etc.

[0025] Additionally or alternatively, the tokens can be specific ranges or classes of various different variables. For example, tokens related to the patient’s ejection fraction can include an ejection fraction between 50% and 70%, an ejection fraction between 41% and 49%, an ejection fraction greater than or equal to 71%, an ejection fraction less than or equal to 40%,etc. If the patient’s ejection fraction is identified within the EHR data as being 55%, then the token of the ejection fraction between 50% and 70% is determined to have been identified, and the other tokens related to the ejection fraction are determined to not have been identified. But if the patient’s ejection fraction is identified within the EHR data as being 37%, then the token of the ejection fraction less than or equal to 40% is determined to have been identified, and the other tokens related to the ejection fraction are determined to have not been identified.

[0026] In another example, tokens related to any specific characteristic (e.g., blood pressure, body mass index, blood glucose level, ejection fraction, hemoglobin level, total cholesterol, etc.) could include any combination of classes such as “normal,” “mild,” “moderate,” “severe,” etc. After the value of the specific characteristic is identified in the patient’s EHR data, whichever token is applicable for that characteristic can be determined to have been identified, and the other tokens are determined to not have been identified.

[0027] In general, the tokens can include any suitable word or phrase. Example tokens includes “air embolus,” “cancer pancreas body,” “renal glycosuria,” “angina preinfarctional,” “history of lung transplant,” “impulse ridden personality,” “substance addiction,” “internal orthopedic device with infection or inflammatory reaction,” “neoplasm of uncertain behavior of brain and spinal cord,” “congenital anomaly optic nerve right,” “opioid abuse episodic use,” “ketosis,” “afib,” “history of liver transplant,” “infection of automatic implantable cardioverter defibrillator lead,” “fracture hip,” “t 12 burst fracture,” “amputation of great toe left traumatic,” “varicose veins of lower extremities with ulcer,” and “intentional self harm by airgun.” Thus, the tokens can be related to a wide variety of different diseases, conditions, patient characteristics, test results, diagnoses, etc.

[0028] The weight that is assigned to each respective token by the first machine learning model is indicative of the amount of abnormality represented by the respective token. The first machine learning model is trained using reference data, so that once any tokens are identified, they can be assigned a weight indicative of the abnormality. The reference data can include data obtained medical textbooks, medical journals, and / or any other written source of medical information; data associated with policies of the healthcare provider and / or of the healthcare facility where the patient is located; or both. In some implementations, the reference data can include reference data associated with specific subspecialties, such as addition medicine, emergency medicine, otorhinolaryngology, immunology, neurology, cardiology, critical care, endocrinology, gastroenterology, radiology, pain management, psychiatry, orthopedics, ophthalmology, geriatrics, hematology, oncology, obstetrics and gynecology, hospice and palliative medicine, urology, pathology, gastroenterology, infectious diseases, pulmonology,nephrology, rheumatology, or any combination thereof. In general, the first machine learning model that assigns weights to each of the tokens is trained using primarily objective data that indicates now normal or abnormal various different tokens are. In general, the first machine learning model can be trained on principles of internal medicine, clinical expertise, best research evidence, patient values and preferences, and combinations thereof, and can then be fine-tuned using information related to any of the above subspecialties and / or other subspecialties.

[0029] The first machine learning model in some implementations is an artificial neural network (e.g., a large language model) that is trained partially or wholly using backpropagation. In these implementations, training data is input into the input layer of the first machine learning model, and the output received from the output layer of the first machine learning model is compared to the desired output. The weight and / or other parameters of the layers of the first machine learning model are adjusted working backwards from the output layer to the input layer to minimize the cost function of the first machine learning model.

[0030] In some implementations, the abnormality is specific to the patient and not the general population. Thus, if one of the tokens identified is indicative of an elevated heart rate, but an elevated heart rate is generally normal for the patient, then the weight assigned to that token will be lower than if an elevated heart rate is not normal for the patient.

[0031] In some implementations, the normality of each respective identified token is determined by viewing the history of that token in the patient’s data. For example, the data in the patient’s EHR may be time stamped, so there may be multiple instances of the same token in the data, with each instance of the token being associated with a specific visit with a healthcare provider and / or an update to the patient’s EHR. If it is determined that the same token has repeatedly appeared in the patient’s EHR in the past, then the weight assigned to the token during the present execution of method 200 may be lower. But if this is the first time that the token has appeared in the patient’s EHR, then the weight assigned to the token during the present execution of method 200 may be higher. In other implementations, quantitative tokens may be compared to a baseline normality threshold, which could be a baseline value and / or range of values. For example, if a token related to a numeric value of some measurement or test result is identified (e.g., heart rate 150 bpm), then that token can be compared to a baseline value or range of values for that measurement or test. In general, because method 200 can be re-executed each time additional data is added to the patient’s EHR, the first machine learning model can view patterns in the patient data as compared to standard medical reference literature to determine the abnormality of each token.

[0032] The output of the first machine learning model is an indication of which tokens were identified in the inputted data, and the weight assigned to each respective tokens. In some implementations, the first machine learning model outputs a two-element vector. The first element of each respective vector is an identifier (e.g., an ID number) of one of the identified tokens, and the second element of each respective vector is the weight assigned to the identified token corresponding to the respective vector. Any suitable format can be used for the identifiers of each token and for the weights. In some implementations, the identifier of each token is a decimal number between 0 and 1. In some implementations, the weight of each token is a decimal number between 0 and 1. In some implementations, both the identifier and the weight of each token is a decimal number between 0 and 1. In some implementations, the first machine learning model outputs a vector only for the identified tokens. In other implementations, the first machine learning model outputs a vector for all of the tokens it has been trained to identify. Vectors corresponding to tokens identified in the data will include the identifier of the token as the first element, and the assigned weight as the second element. Vectors corresponding to tokens that were not identified in the data will include the identifier of each token as the first element, and a weight of 0 as the second element. In some implementations, each possible token has a pre-determined weight. The first machine learning model assigns to any identified token its pre-determined weight, and assigns to any unidentified token a weight of 0.

[0033] Step 230 of method 200 includes inputting at least the plurality of tokens into a second machine learning model that has been trained to assign a rank to each of the tokens. The rank that is assigned to each respective token by the second machine learning model is indicative of the magnitude of the affect that the respective token has on the determination of the severity of the patient’s illness. In other words, the rank of each respective is indicative of the size of the contribution that the respective token has in determining the severity of the patient’s illness.

[0034] In general, the weight and the rank of each token are independent. In some cases, a given token may have a larger weight (indicating that the token is abnormal for the patient) but a smaller rank (indicating that the abnormality of the token is not very relevant in determining the severity of the patient’s illness). In other cases, a given token may have a smaller weight (indicating that the token is normal for the patient) but a larger rank (indicating that even though the token is normal, the token is still very relevant in determining the severity of the patient’s illness). In further cases, a given token may have a larger weight (indicating that the token is abnormal for the patient) and a larger rank (indicating that the abnormality of the token is very relevant in determining the severity of the patient’s illness). In still other cases, a given tokenmay have a smaller weight (indicating that the token is normal for the patient) and a smaller rank (indicating that the normality of the token is not very relevant in determining the severity of the patient’s illness).

[0035] In some implementations, the second machine learning model outputs only the rank for each token. The ranks can then be added to the two-element vectors output by the first machine learning model to form three-element vectors, where the first element is the identifier, the second element is the weight, and the third element is the rank. In other implementations, the second machine learning model outputs the entirety of the three-element vectors. The rank of each of the identified tokens can be a value between 0 and 1, similar to the weight.

[0036] In some implementations, only the identified tokens are ranked. In these implementations, if there are n possible tokens that can be identified from the EHR data but only k<n tokens that were identified, the second machine learning model can output three- element vectors where the lowest-ranked token will have a rank of r / (which will generally be 0), the highest-ranked token will have a rank of (which will generally be 1), and the k-2 tokens in between will have a rank of between V2 and k-i.

[0037] In other implementations, all tokens are ranked, even those that were not identified in the EHR data. In these implementations, if there are n possible tokens that can be identified from the EHR data but only k<n tokens that were identified, the second machine learning model can output three-element vectors where the lowest-ranked token (which may or may not be a token that was identified in the EHR data) will have a rank of n (which will generally be 0), the highest-ranked token (which may or may not be a token that was identified in the EHR data) will have a rank of r„ (which will generally be 1), and the n-2 tokens in between (which will generally include both tokens not identified in the EHR data and tokens identified in the EHR data) will have a rank of between V2 and rw-y.

[0038] In some implementations, the second machine learning model is trained to assign the ranks to each of the respective tokens based only on the tokens themselves. In these implementations, only the identifiers of the identified tokens need to be input into the second machine learning model. However, the assigned weights (and / or other data) could also be input into the second machine learning model as part of the two-element vectors generated by the first machine learning model. In other implementations, the second machine learning model is trained to assign the ranks to each of the respective tokens based on both the tokens and the weights assigned to each token. In these implementations, the entirety of the two-element vectors generated by the first machine learning model are input into the second machine learning model. In further implementations, the second machine learning model is trained toassign the ranks to each of the respective tokens based on the tokens, the weights assigned to each token, and the same EHR data that is input into the first machine learning model. In these implementations, the entirety of the two-element vectors generated by the first machine learning model are input into the second machine learning model, along with the EHR data.

[0039] In some implementations, the second machine learning model is a forward-forward neural network that is trained without using backpropagation. In these implementations, the second machine learning model is trained by moving through the second machine learning model forward twice, instead of once forward and once backward. Training data that is correct (also referred to as positive data) can be input into the second machine learning model during the first forward pass, and training data that is incorrect (also referred to as negative data) can be input into the second machine learning model during the second forward pass. The weights and other parameters of the layers are adjusted in order to maximize the activity of the second machine learning model when the correct data is input into the second machine learning model, and to minimize the activity of the second machine learning model when the incorrect data is input into the second machine learning model.

[0040] In contrast to the first machine learning model, the second machine learning model is trained using data that is more subjective. This data can include data associated with past experiences and “gut feelings” of healthcare providers, risk assessments used in the past by healthcare providers, and other more subjective types of data. Thus, the ranks assigned by the second machine learning model are generally more reflective of a healthcare provider’s intuition regarding which factors lead to meeting the NTH definition of serious illness, instead of some standard reference literature indicating which factors lead to meeting the NIH definition of serious illness. Because the ranks are not rigorously quantifiable like the weights and there is generally no standard reference literature for determining what combination of weights and ranks might result in a serious illness or not, it is generally much more difficult to train the second machine learning model via backpropagation. Training the second machine learning model using the forward-forward training technique includes defining a local loss function for each nodes, so that only the nodes themselves (or a subset of the nodes) need to be retrained, instead of the entire model. This eliminates the need for training via error back- propagation, which generally requires perfect or near-perfect characterization. This also allows the second model to scale efficiently to large numbers of parameters (e.g., tokens).

[0041] Step 240 of method 200 includes generating an indication of the severity of illness based on the identified tokens, the weight assigned to each token, the rank assigned to each token, or any combination thereof. The generation of the indication is generally based on boththe weight of the tokens (which could include only the identified tokens, or both the identified tokens and the non-identified tokens (which will generally have a weight of 0)) and the rank of the tokens (which could include only the identified tokens, or both the identified tokens and the non-identified tokens). In some implementations however, the generation of the indication may be based on only the weights of the identified tokens, the weights of all tokens, the rank of the identified tokens, the ranks of all tokens, or even just which tokens were identified.

[0042] The generation of the indication of the severity of illness can be done by inputting the three-element vectors (or whatever form that the data to be used is in) into a scoring algorithm. In some implementations, the scoring algorithm adds together the weight of each identified token, and divides that number by the number of identified tokens. Similarly, the scoring algorithm adds together the rank of each identified token, and divides that number by the number of identified tokens. The two resulting numbers can be added together and divided by two, which results in the percentage chance that the patient has a serious illness (e.g., the chance that one or more healthcare providers reviewing the patient and the patient’s EHR would conclude that the patient’s condition meets the NTH definition of a serious illness). The percentage chance can be represented as a specific percentage of a scale between 0 and 100, a number between 0 and 1, an integer number 1 to 6, or in any other suitable fashion.

[0043] In other implementations, the indication of the severity of illness is a binary indication of whether the patient meets the definition of being seriously ill. In these implementations, the binary indication can be based on the percentage chance that the patient has a serious illness, where a percentage chance that satisfies a predetermined threshold (e.g., greater than or equal to 70%) leading to an indication of “yes” or “1”, and a percentage chance that does not satisfy the predetermined threshold leading to an indication of “no” “0”.

[0044] In some implementations, the second machine learning model is trained so that the initial use of the second machine learning model (which may be the first use ever, or the first execution for a specific patient) uses an initial set of rules to rank each of the tokens. After the indication of the severity of illness of the patient is generated, that indication can be validated by one or more independent healthcare providers (doctors, nurses, etc.). If indication was correct, the data can be labeled as correct training data and the second machine learning model can be updated with this new correct training data, and if output of the second machine learning model was incorrect, that data can be labeled as incorrect training data and the second machine learning model can be updated with this incorrect training data.

[0045] In some implementations, the second machine learning model is trained so that the initial use of the second machine learning model (which may be the first use ever, or the firstexecution for a specific patient) assigns an initial rank to each of the tokens. However, after every use of the second machine learning model (either separately for each individual patient or for a use with any patient), second machine learning model can be updated. The output of the second machine learning model can be compared to the opinion of one or more independent healthcare providers (doctors, nurses, etc.). If the output of the second machine learning model was correct, that data can be re-input into the second machine learning model as correct training data, and if output of the second machine learning model was incorrect, that data can be reinput into the second machine learning model as incorrect training data.

[0046] FIG. 3 shows a block diagram 300 representing the method 200 illustrated in FIG. 2. The block diagram 300 shows the first machine learning model 310 generating vectors 312 containing the weights, the second machine learning model 320 generating vectors 314 containing the ranks, and the scoring algorithm 330 that generates the final indication of the severity of illness of the patient, indicated as the score 332. As shown, the first machine learning model can be an artificial neural network that is pre-trained using data such as including principles of internal medicine, clinical expertise, best research evidence, patient values and preferences, etc.; and fine-tuned using data associated with a variety of different subspecialties. The second machine learning model can also be an artificial neural network, and is designed to represent physician intuition regarding the severity of illness of the patient. The second machine learning model can be pre-trained using data include medical training and practice, past experiences and “gut feelings” of healthcare providers, explicit and intangible judgements of healthcare providers, risk assessments, hospital policies and protocols, etc.; and fine-tuned using data associated with a variety of different subspecialties.

[0047] Referring now to FIG. 4, in some implementations, the system may include an additional machine learning model that acts as a reinforcement agent for the first machine learning model (generating the weights) and the second machine learning model (generating the ranks). FIG. 4 is a block diagram of a system 400 that includes the first machine learning model 410 and the second machine learning model 420. In the illustrated implementation, the first machine learning model 410 is labeled as a large language model and the second machine learning model 420 is labeled as a forward-forward model. However, as discussed herein, the first machine learning model 410 and the second machine learning model 420 may be different types of models as well.

[0048] As shown, the first machine learning model 410 receives as an input data 405, which will generally include patient data from the patient’s health record. The second machine learning model 420 also receives as an input the data 405. As discussed herein, in someimplementations the specific data received by each of the models 410, 420 may be the same or may be different, and may include all of the data in the patient’s health record or only a portion. The first machine learning model 410 identifies tokens 415 A in the data 405 and generates weights 415B of the tokens 415 A. The second machine learning model 420 generates ranks 425 of the tokens 415A. The scoring algorithm 430 generates the severity score 435 (e.g., the indication of the severity of the illness) based on the weights 415B and the ranks 425.

[0049] Those of skill in the art will note that the block diagram 400 does not necessarily show every connection between the first machine learning model 410, the second machine learning model 420, and the scoring algorithm 430. Instead, the block diagram 400 simply indicates that the first machine learning model 410 identifies the tokens 415 A and generates the weights 415B based on the data 405, the second machine learning model 420 generates the ranks 425 based on the tokens 415 A and the weights 415B, and the scoring algorithm 430 generates the severity score 435 based on at least the weights 415B and the ranks 425. In various implementations of the system 400, the actual connections (e.g., data transmissions) may differ. For example, the scoring algorithm 430 may receive nothing directly from the first machine learning model 410, but instead receive the weights 415B and the ranks 425 from the second machine learning model 420. In another example, the first machine learning model 410 and / or the second machine learning model 420 may also send the tokens 415A to the scoring algorithm 430.

[0050] The severity score 435 is then compared to a separate severity score generated by the reinforcement agent 440. The reinforcement agent 440 is a machine learning model that determines whether the first machine learning model 410 and / or the second machine learning model 420 should re-analyze the data 405.

[0051] In some implementations, the reinforcement agent 440 is a machine learning model that is trained using knowledge from physician reviewers. In some implementations, the reinforcement agent 440 is a forward-forward model. Similar to the second machine learning model 420, utilizing a forward-forward model can reduce errors introduced during back- propagation.

[0052] In one example implementation, the reinforcement agent 440 was trained using training data comprising patient data for a plurality of patients, and a severity score given to each of the plurality of patients by one or more physician reviewers. Each of the reviewers manually reviewed the patient data for each patient and gave the patient a score indicating one of six severity levels. For each patient, the scores were combined to generate the final severity score for that patient. In the example implementation, two physician reviewers each gave thepatient a score from 1-6, with 6 being the most severe. If both reviewers for a patient were on the same side of the scale (e.g., both reviewers with a 1, 2, or 3; or both reviewers with a 4, 5, or 6), then the average score was assigned to the patient, if the reviewers were on opposite sides of the scale, then a third reviewer scored the patient as a tiebreaker. The score for a patient requiring a tiebreaker was the average score of the third reviewer’s score and the score of the original reviewer on the same side of the scale as the tiebreaker. The reinforcement agent 440 may be trained in other manners as well, including different numbers of reviewers, different scales for scoring the patients, etc.

[0053] In any implementation, the reinforcement agent 440 receives the same input data 405 as the first machine learning model 410 and the second machine learning model 420 and generates its own severity score, which is an estimate of how the physician reviewers would have scored the severity of the patient. Once the scoring algorithm 430 generates the severity score 435, a comparison is performed between the severity score 435 from the scoring algorithm 430 and the severity score from the reinforcement agent 440. If the two severity scores differ enough, then a re-analysis request 445A can be sent to the first machine learning model 410 and / or a re-analysis request 445B can be sent to the second machine learning model 420.

[0054] In some implementations, the severity score 435 generated by the scoring algorithm 430 and the severity score generated by the reinforcement agent 440 are both on the same scale (e.g., a score from 1 to 6, with 1 being the least severe and 6 being the most severe), and are directly compared. If the two severity scores are different, and / or if the difference between the two scores satisfies a predetermined threshold, then the re-analysis request 445 A and / or the reanalysis request 445B can be sent. In some implementations, if the two severity scores are on opposite sides of a scale (e.g., the severity score 435 is a 1, 2, or 3 and the severity score 445 is a 4, 5, or 6), then the re-analysis request 445A and / or 445B are sent, regardless of the magnitude of the difference. In some implementation, the re-analysis request 445A and / or 445B is sent only if the two severity scores are on opposite sides of the scale, and the magnitude of the difference between the two scores satisfies a threshold. In other implementations however, the severity scores may have different forms or scale, in which case one or both of the scores may need to be scaled or adjusted to allow for the comparison.

[0055] In the illustrated implementation, the reinforcement agent 440 receives the severity score 435 from the scoring algorithm 430 and performs the comparison, sending out the reanalysis request 445A and / or 445B if the comparison so requires. In other implementations however, a different module or processor may perform the actual comparison, after receivingthe severity score 435 from the scoring algorithm 430 and the severity score from the reinforcement agent 440. This separate module or processor may itself send out the re-analysis requests 445A and / or 445B, or may and cause the reinforcement agent 440 to send out the reanalysis requests 445A and / or 445B. In general, the re-analysis requests 445A and / or 445B do not indicate to the first machine learning model 410 and the second machine learning model 420 what the actual error was with the generation of the severity score 435 was (e.g., what went wrong in the analysis of the input data 405), and this could introduce bias and / or fitting issues. Instead, the re-analysis request 445A and 445B indicate to the first machine learning model 410 and the second machine learning model 420 that some error was present in the determination of the severity score 435, and that they should perform their analysis again. In this manner, the first machine learning model 410 and / or the second machine learning model 420 can be trained by using the output of the reinforcement agent 440.

[0056] In some implementations, the re-analysis request 445A is sent to the first machine learning model 410, and the first machine learning model 410 re-analyzes the input data 405 to generate an updated plurality of tokens, and to assign a weight to each of the updated plurality of tokens. The updated tokens and the weights for the updated tokens can be sent to the second machine learning model 420, which generates the ranks for each of the updated tokens. The weights and the ranks for the updated plurality of tokens are then sent to the scoring algorithm 430 to generate the severity score 435.

[0057] In some implementations, the re-analysis request 445A is sent to the first machine learning model 410, and the first machine learning model 410 re-analyzes the input data 405 to assign updated weights to each of the original plurality of tokens. The updated weights (and in some cases the original tokens again) can be sent to the second machine learning model 420, which generates the ranks for each of the original tokens based on the updated weights. The updated weights and the updated ranks for the original plurality of tokens are then sent to the scoring algorithm 430 to generate the severity score 435.

[0058] In some implementations, the re-analysis request 445A is sent to the first machine learning model 410, and the first machine learning model 410 re-analyzes the input data 405 to assign updated weights to each of the original plurality of tokens. The updated weights and the original ranks for the original plurality of tokens are then sent to the scoring algorithm 430 to generate the severity score 435.

[0059] In some implementations, only re-analysis request 445B is sent to the second machine learning model 420, and the second machine learning model 420 re-analyzes the original plurality of tokens and / or the weights assigned to each of the original plurality oftokens to generate updated ranks for each of the original tokens. The weights and the updated ranks for the original plurality of tokens is then sent to the scoring algorithm 430 to generate the severity score 435.

[0060] Thus, the reinforcement agent 440 can be used to cause an updated severity score to be generated. The updated severity score can be based on (i) an updated plurality of tokens, weights assigned to the updated plurality of tokens, and ranks assigned to the updated plurality of tokens; (ii) the original plurality of tokens, updated weights assigned to the original plurality of tokens, and updated ranks assigned to the original plurality of tokens; (iii) the original plurality of tokens, updated weights assigned to the original plurality of tokens, and the original ranks assigned to the plurality of tokens, or (iv) the original plurality of tokens, the original weights assigned to the original plurality of tokens, and updated ranks assigned to original plurality of tokens.

[0061] ALTERNATIVE IMPLEMENTATIONS

[0062] Alternative Implementation 1. A method of classifying a severity of illness of a patient, the method comprising: receiving data associated with the patient; inputting at least the received data into a first machine learning model and executing the first machine learning model, the first machine learning model being trained to identify a plurality of tokens within the data and assign a weight to each respective token of the plurality of tokens, each token being associated with a characteristic of the patient; inputting the plurality of tokens into a second machine learning model and executing the second machine learning model, the second machine learning model being trained to assign a rank to each respective token of the plurality of tokens; and generating an indication of the severity of illness of the patient based on (i) the plurality of tokens, (ii) the weight assigned to each respective token of the plurality of tokens, and (iii) the rank assigned to each respective token of the plurality of tokens.

[0063] Alternative Implementation 2. The method of Alternative Implementation 1, wherein the first machine learning model is trained using backpropagation.

[0064] Alternative Implementation 3. The method of Alternative Implementation 1 or Alternative Implementation 2, wherein the first machine learning model is trained on reference data that includes (i) data obtained from medical textbooks, medical journals, or both; (ii) data associated with one or more policies of a healthcare provider of the patient, a healthcare facility of the patient, or both; or (iii) any combination of (i) and (ii).

[0065] Alternative Implementation 4. The method of any one of Alternative Implementations 1 to 3, wherein each token of the plurality of tokens is associated with a healthof the patient, a functional status of the patient, a burden on a caregiver of the patient, or any combination thereof.

[0066] Alternative Implementation 5. The method of Alternative Implementation 4, wherein the weight assigned to each respective token is indicative of an amount of abnormality of represented by the respective token, the amount of abnormality being specific to the patient.

[0067] Alternative Implementation 6. The method of Alternative Implementation 5, wherein the amount of abnormality represented by each respective token is specific to the patient.

[0068] Alternative Implementation 7. The method of any one of Alternative Implementations 4 to 6, wherein the rank of each respective token is indicative of a magnitude of an effect of the respective token on the severity of illness of the patient.

[0069] Alternative Implementation 8. The method of any one of Alternative Implementations 4 to 7, wherein the plurality of tokens includes one or more tokens associated with a physiology of the patient, one or more tokens associated with a level of function of the patient, one or more tokens associated with a quality of life of the patient, one or more tokens associated with a caregiver of the patient, or any combination thereof.

[0070] Alternative Implementation 9. The method of Alternative Implementation 8, wherein the one or more tokens associated with the physiology of the patient includes an age of the patient, a height of the patient, a weight of the patient, a sex of the patient, a treatment history of the patient, data generated by one or more tests performed on the patient, data generated by one or more procedures on the patient, image data associated with imaging performed on the patient, physiological characteristics of the patient, an indication of whether the patient has one or more implanted devices, or any combination thereof.

[0071] Alternative Implementation 10. The method of any one of Alternative Implementations 1 to 9, wherein the data includes an electronic medical record of the patient, and wherein the first machine learning model is trained to extract the plurality of tokens from the electronic medical record of the patient.

[0072] Alternative Implementation 11. The method of any one of Alternative Implementations 1 to 10, wherein the second machine learning model is a forward-forward machine learning model.

[0073] Alternative Implementation 12. The method of any one of Alternative Implementations 1 to 11, wherein the second machine learning model is trained without using backpropagation.

[0074] Alternative Implementation 13. The method Alternative Implementation 11 or Alternative Implementation 12, wherein the indication of the severity of illness of the patient can be validated by a healthcare provider, and wherein the second machine learning model can be updated based on the validated indication and (i) the plurality of tokens, (ii) the weight assigned to each of the plurality of tokens, or (iii) both (i) and (ii).

[0075] Alternative Implementation 14. The method of any one of Alternative Implementations 1 to 13, wherein the second machine learning model is trained to assign the plurality of ranks based on (i) only the plurality of tokens identified within the data by the first machine learning model, or (ii) the plurality of tokens identified within the data by the first machine learning model and the plurality of weights.

[0076] Alternative Implementation 15. The method of any one of Alternative Implementations 1 to 14, further comprising inputting the plurality of weights into the second machine learning model, the second machine learning model being trained to assign the plurality of ranks based on both the plurality of tokens identified within the data by the first machine learning model and the weight assigned to each respective token of the plurality of tokens.

[0077] Alternative Implementation 16. The method of any one of Alternative Implementations 1 to 15, wherein the weight of each token is indicative of an amount of abnormality of the token that is specific to the patient, and wherein the rank of each token is indicative of a magnitude of an effect of the corresponding characteristic on the severity of illness of the patient.

[0078] Alternative Implementation 17. The method of any one of Alternative Implementations 1 to 16, wherein the first machine learning model is trained to generate a plurality of two-element vectors, each respective vector corresponding to one of the plurality of tokens.

[0079] Alternative Implementation 18. The method of Alternative Implementation 17, wherein a first element of each respective two-element vector is an identifier of the corresponding token, and wherein a second element of each respective two-element vector is the weight assigned to the corresponding token.

[0080] Alternative Implementation 19. The method of Alternative Implementation 18, wherein the second machine learning model is trained to convert each respective two-element vector to a three-element vector, the third element of each respective three-element vector being the weight assigned to the corresponding token.

[0081] Alternative Implementation 20. The method of Alternative Implementation 17, wherein the second machine learning model is trained to generate a plurality of three-element vectors, each respective vector corresponding to one of the plurality of tokens.

[0082] Alternative Implementation 21. The method of Alternative Implementation 20, wherein a first element of each respective three-element vector is the identifier of the corresponding token, a second element of each respective three-element vector is the weight assigned to the corresponding token, and a third element of each respective three-element vector is the weight assigned to the corresponding token.

[0083] Alternative Implementation 22. The method of any one of Alternative Implementations 1 to 21, wherein the indication of the severity of illness is either an indication that the illness meets a predetermined threshold for severity or that the illness does not meet the predetermined threshold for severity.

[0084] Alternative Implementation 23. The method of any one of Alternative Implementations 1 to 21, wherein the indication of the severity of illness is a percentage chance that the illness meets a predetermined threshold for severity.

[0085] Alternative Implementation 24. The method of any one of Alternative Implementations 1 to 23, wherein the generated indication of the severity of the illness is a first indication of the severity of the illness, and wherein the method further comprises inputting the received data and the first indication of the severity of the illness into a reinforcement agent that is trained to: generate a second indication of the severity of the illness; compare the first indication of the severity of the illness with the second indication of the severity of the illness; and in response to the comparison satisfying a threshold, transmit a re-analysis request to the first machine learning model, the second machine learning model, or both.

[0086] Alternative Implementation 25. The method of any one of Alternative Implementations 1 to 23, wherein the generated indication of the severity of the illness is a first indication of the severity of the illness, and wherein the method further comprises: inputting the received data into a reinforcement agent; receiving a second indication of the severity of the illness from the reinforcement agent; compare the first indication of the severity of the illness with the second indication of the severity of the illness; and in response to the comparison satisfying a threshold, transmit a re-analysis request to the first machine learning model, the second machine learning model, or both.

[0087] Alternative Implementation 26. The method of Alternative Implementation 24 or Alternative Implementation 25, wherein the re-analysis request causes the first machinelearning model to re-analyze the received data and generate an updated plurality of tokens and a weight assigned to each respective token of the updated plurality of tokens.

[0088] Alternative Implementation 27. The method of Alternative Implementation 26, wherein the re-analysis request further causes the second machine learning model to re-analyze the plurality of tokens and the weight assigned to reach respective token of the updated plurality of tokens to generate a rank assigned to each respective token of the updated plurality of tokens.

[0089] Alternative Implementation 28. The method of Alternative Implementation 27, further comprising generating an updated indication of the severity of the illness of the patient based on (i) the updated plurality of tokens, (ii) the weight assigned to each respective token of the updated plurality of tokens, and (iii) the rank assigned to each respective token of the updated plurality of tokens.

[0090] Alternative Implementation 29. The method of any one of Alternative Implementations 24 to 28, wherein the reinforcement agent is a forward-forward machine learning model.

[0091] Alternative Implementation 30. The method of any one of Alternative Implementations 24 to 29, wherein the reinforcement agent is trained without using backpropagation.

[0092] Alternative Implementation 31. A system for classifying a severity of illness of a patient, the system comprising: one or more memory devices storing machine-readable instructions; and one or more processors, each of the one or more processors configured to execute the machine-readable instructions of at least one of the one or more memory devices to implement the method of any one of Alternative Implementations 1 to 30.

[0093] Alternative Implementation 32. A system for classifying a severity of illness of a patient, the system including one or more processors configured to implement the method of any one of Alternative Implementations 1-30.

[0094] Alternative Implementation 33. A computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the method of any one of Alternative Implementations 1-30.

[0095] Alternative Implementation 34. The computer program product of Alternative Implementation 33, wherein the computer program product is a non-transitory computer readable medium.

[0096] Alternative Implementation 35. A system for classifying a severity of illness of a patient, the system comprising: a first machine learning model configured to receive data associated with the patient, the first machine learning model being trained to: identify aplurality of tokens within the data; and assign a weight to each respective token of the plurality of tokens, each token being associated with a characteristic of the patient; a second machine learning model configured to receive the plurality of tokens, the weight assigned to each respective token of the plurality of tokens, or both, the second machine learning model being trained to assign a rank to each respective token of the plurality of tokens based on the plurality of tokens, the weight assigned to each respective token of the plurality of tokens, or both; and one or more processors configured to generate an indication of the severity of the illness of the patient based on (i) the plurality of tokens, (ii) the weight assigned to each respective token of the plurality of tokens, and (iii) the rank assigned to each respective token of the plurality of tokens.

[0097] Alternative Implementation 36. The system of Alternative Implementation 35, wherein generated indication of the severity of the illness generated by the one or more processors is a first indication of the severity of the illness, and wherein the system further comprises a reinforcement agent trained to: generate a second indication of the severity of the illness based at least in in part on the data associated with the patient; compare the first indication of the severity of the illness with the second indication of the severity of the illness; and in response to the comparison satisfying a threshold, transmit a re-analysis request to the first machine learning model, the second machine learning model, or both.

[0098] Alternative Implementation 37. The system of Alternative Implementation 36, wherein the reinforcement agent is a forward-forward machine learning model.

[0099] Alternative Implementation 38. The system of Alternative Implementation 36 or 37, wherein the reinforcement agent is trained without using backpropagation.

[0100] Alternative Implementation 39. The system of any one of Alternative Implementations 35 to 38, wherein the first machine learning model is trained using backpropagation.

[0101] Alternative Implementation 40. The system of any one of Alternative Implementations 35 to 39, wherein the first machine learning model is trained on reference data that includes (i) data obtained from medical textbooks, medical journals, or both; (ii) data associated with one or more policies of a healthcare provider of the patient, a healthcare facility of the patient, or both; or (iii) any combination of (i) and (ii).

[0102] Alternative Implementation 41. The system of any one of Alternative Implementations 35 to 40, wherein each token of the plurality of tokens is associated with a health of the patient, a functional status of the patient, a burden on a caregiver of the patient, or any combination thereof.

[0103] Alternative Implementation 42. The system of Alternative Implementation 41, wherein the weight assigned to each respective token is indicative of an amount of abnormality of represented by the respective token, the amount of abnormality being specific to the patient.

[0104] Alternative Implementation 43. The system of Alternative Implementation 42, wherein the amount of abnormality represented by each respective token is specific to the patient.

[0105] Alternative Implementation 44. The system of any one of Alternative Implementations 41 to 43, wherein the rank of each respective token is indicative of a magnitude of an effect of the respective token on the severity of illness of the patient.

[0106] Alternative Implementation 45. The system of any one of Alternative Implementations 41 to 44, wherein the plurality of tokens includes one or more tokens associated with a physiology of the patient, one or more tokens associated with a level of function of the patient, one or more tokens associated with a quality of life of the patient, one or more tokens associated with a caregiver of the patient, or any combination thereof.

[0107] Alternative Implementation 46. The system of Alternative Implementation 45, wherein the one or more tokens associated with the physiology of the patient includes an age of the patient, a height of the patient, a weight of the patient, a sex of the patient, a treatment history of the patient, data generated by one or more tests performed on the patient, data generated by one or more procedures on the patient, image data associated with imaging performed on the patient, physiological characteristics of the patient, an indication of whether the patient has one or more implanted devices, or any combination thereof.

[0108] Alternative Implementation 47. The system of any one of Alternative Implementations 35 to 46, wherein the data includes an electronic medical record of the patient, and wherein the first machine learning model is trained to extract the plurality of tokens from the electronic medical record of the patient.

[0109] Alternative Implementation 48. The system of any one of Alternative Implementations 35 to 47, wherein the second machine learning model is a forward-forward machine learning model.

[0110] Alternative Implementation 49. The system of any one of Alternative Implementations 35 to 48, wherein the second machine learning model is trained without using backpropagation.[OHl] Alternative Implementation 50. The system Alternative Implementation 48 or Alternative Implementation 49, wherein the indication of the severity of illness of the patient can be validated by a healthcare provider, and wherein the second machine learning model canbe updated based on the validated indication and (i) the plurality of tokens, (ii) the weight assigned to each of the plurality of tokens, or (iii) both (i) and (ii).

[0112] Alternative Implementation 51. The system of any one of Alternative Implementations 35 to 50, wherein the second machine learning model is trained to assign the plurality of ranks based on (i) only the plurality of tokens identified within the data by the first machine learning model, or (ii) the plurality of tokens identified within the data by the first machine learning model and the plurality of weights.

[0113] Alternative Implementation 52. The system of any one of Alternative Implementations 35 to 51, further comprising inputting the plurality of weights into the second machine learning model, the second machine learning model being trained to assign the plurality of ranks based on both the plurality of tokens identified within the data by the first machine learning model and the weight assigned to each respective token of the plurality of tokens.

[0114] Alternative Implementation 53. The system of any one of Alternative Implementations 35 to 52, wherein the weight of each token is indicative of an amount of abnormality of the token that is specific to the patient, and wherein the rank of each token is indicative of a magnitude of an effect of the corresponding characteristic on the severity of illness of the patient.

[0115] Alternative Implementation 54. The system of any one of Alternative Implementations 35 to 53, wherein the first machine learning model is trained to generate a plurality of two-element vectors, each respective vector corresponding to one of the plurality of tokens.

[0116] Alternative Implementation 55. The system of Alternative Implementation 54, wherein a first element of each respective two-element vector is an identifier of the corresponding token, and wherein a second element of each respective two-element vector is the weight assigned to the corresponding token.

[0117] Alternative Implementation 56. The system of Alternative Implementation 55, wherein the second machine learning model is trained to convert each respective two-element vector to a three-element vector, the third element of each respective three-element vector being the weight assigned to the corresponding token.

[0118] Alternative Implementation 57. The system of Alternative Implementation 54, wherein the second machine learning model is trained to generate a plurality of three-element vectors, each respective vector corresponding to one of the plurality of tokens.

[0119] Alternative Implementation 58. The system of Alternative Implementation 57, wherein a first element of each respective three-element vector is the identifier of the corresponding token, a second element of each respective three-element vector is the weight assigned to the corresponding token, and a third element of each respective three-element vector is the weight assigned to the corresponding token.

[0120] Alternative Implementation 59. The system of any one of Alternative Implementations 35 to 58, wherein the indication of the severity of illness is either an indication that the illness meets a predetermined threshold for severity or that the illness does not meet the predetermined threshold for severity.

[0121] Alternative Implementation 60. The system of any one of Alternative Implementations 35 to 59, wherein the indication of the severity of illness is a percentage chance that the illness meets a predetermined threshold for severity.

[0122] One or more elements or aspects or steps, or any portion(s) thereof, from one or more of any of the Alternative Implementations and / or claims herein can be combined with one or more elements or aspects or steps, or any portion(s) thereof, from one or more of any of the other Alternative Implementations and / or claims and / or combinations thereof, to form one or more additional implementations and / or claims of the present disclosure.

[0123] While the present disclosure has been described with reference to one or more particular embodiments or implementations, those skilled in the art will recognize that many changes may be made thereto without departing from the spirit and scope of the present disclosure. Each of these implementations and obvious variations thereof is contemplated as falling within the spirit and scope of the present disclosure. It is also contemplated that additional implementations according to aspects of the present disclosure may combine any number of features from any of the implementations described herein.

Claims

CLAIMSWHAT IS CLAIMED IS:

1. A method of classifying a severity of illness of a patient, the method comprising: receiving data associated with the patient; inputting at least the received data into a first machine learning model and executing the first machine learning model, the first machine learning model being trained to identify a plurality of tokens within the data and assign a weight to each respective token of the plurality of tokens, each token being associated with a characteristic of the patient; inputting the plurality of tokens into a second machine learning model and executing the second machine learning model, the second machine learning model being trained to assign a rank to each respective token of the plurality of tokens; and generating an indication of the severity of illness of the patient based on (i) the plurality of tokens, (ii) the weight assigned to each respective token of the plurality of tokens, and (iii) the rank assigned to each respective token of the plurality of tokens.

2. The method of claim 1, wherein the first machine learning model is trained using backpropagation.

3. The method of claim 1 or claim 2, wherein the first machine learning model is trained on reference data that includes (i) data obtained from medical textbooks, medical journals, or both; (ii) data associated with one or more policies of a healthcare provider of the patient, a healthcare facility of the patient, or both; or (iii) any combination of (i) and (ii).

4. The method of any one of claims 1 to 3, wherein each token of the plurality of tokens is associated with a health of the patient, a functional status of the patient, a burden on a caregiver of the patient, or any combination thereof.

5. The method of claim 4, wherein the weight assigned to each respective token is indicative of an amount of abnormality of represented by the respective token, the amount of abnormality being specific to the patient.

6. The method of claim 5, wherein the amount of abnormality represented by each respective token is specific to the patient.

7. The method of any one of claims 4 to 6, wherein the rank of each respective token is indicative of a magnitude of an effect of the respective token on the severity of illness of the patient.

8. The method of any one of claims 4 to 7, wherein the plurality of tokens includes one or more tokens associated with a physiology of the patient, one or more tokens associated with a level of function of the patient, one or more tokens associated with a quality of life of the patient, one or more tokens associated with a caregiver of the patient, or any combination thereof.

9. The method of claim 8, wherein the one or more tokens associated with the physiology of the patient includes an age of the patient, a height of the patient, a weight of the patient, a sex of the patient, a treatment history of the patient, data generated by one or more tests performed on the patient, data generated by one or more procedures on the patient, image data associated with imaging performed on the patient, physiological characteristics of the patient, an indication of whether the patient has one or more implanted devices, or any combination thereof.

10. The method of any one of claims 1 to 9, wherein the data includes an electronic medical record of the patient, and wherein the first machine learning model is trained to extract the plurality of tokens from the electronic medical record of the patient.

11. The method of any one of claims 1 to 10, wherein the second machine learning model is a forward-forward machine learning model.

12. The method of any one of claims 1 to 11, wherein the second machine learning model is trained without using backpropagation.

13. The method claim 11 or claim 12, wherein the indication of the severity of illness of the patient can be validated by a healthcare provider, and wherein the second machine learningmodel can be updated based on the validated indication and (i) the plurality of tokens, (ii) the weight assigned to each of the plurality of tokens, or (iii) both (i) and (ii).

14. The method of any one of claims 1 to 13, wherein the second machine learning model is trained to assign the plurality of ranks based on (i) only the plurality of tokens identified within the data by the first machine learning model, or (ii) the plurality of tokens identified within the data by the first machine learning model and the plurality of weights.

15. The method of any one of claims 1 to 14, further comprising inputting the plurality of weights into the second machine learning model, the second machine learning model being trained to assign the plurality of ranks based on both the plurality of tokens identified within the data by the first machine learning model and the weight assigned to each respective token of the plurality of tokens.

16. The method of any one of claims 1 to 15, wherein the weight of each token is indicative of an amount of abnormality of the token that is specific to the patient, and wherein the rank of each token is indicative of a magnitude of an effect of the corresponding characteristic on the severity of illness of the patient.

17. The method of any one of claims 1 to 16, wherein the first machine learning model is trained to generate a plurality of two-element vectors, each respective vector corresponding to one of the plurality of tokens.

18. The method of claim 17, wherein a first element of each respective two-element vector is an identifier of the corresponding token, and wherein a second element of each respective two-element vector is the weight assigned to the corresponding token.

19. The method of claim 18, wherein the second machine learning model is trained to convert each respective two-element vector to a three-element vector, the third element of each respective three-element vector being the weight assigned to the corresponding token.

20. The method of claim 17, wherein the second machine learning model is trained to generate a plurality of three-element vectors, each respective vector corresponding to one of the plurality of tokens.

21. The method of claim 20, wherein a first element of each respective three-element vector is the identifier of the corresponding token, a second element of each respective three-element vector is the weight assigned to the corresponding token, and a third element of each respective three-element vector is the weight assigned to the corresponding token.

22. The method of any one of claims 1 to 21, wherein the indication of the severity of illness is either an indication that the illness meets a predetermined threshold for severity or that the illness does not meet the predetermined threshold for severity.

23. The method of any one of claims 1 to 21 , wherein the indication of the severity of illness is a percentage chance that the illness meets a predetermined threshold for severity.

24. The method of any one of claims 1 to 23, wherein the generated indication of the severity of the illness is a first indication of the severity of the illness, and wherein the method further comprises inputting the received data and the first indication of the severity of the illness into a reinforcement agent that is trained to: generate a second indication of the severity of the illness; compare the first indication of the severity of the illness with the second indication of the severity of the illness; and in response to the comparison satisfying a threshold, transmit a re-analysis request to the first machine learning model, the second machine learning model, or both.

25. The method of any one of claims 1 to 23, wherein the generated indication of the severity of the illness is a first indication of the severity of the illness, and wherein the method further comprises: inputting the received data into a reinforcement agent; receiving a second indication of the severity of the illness from the reinforcement agent; compare the first indication of the severity of the illness with the second indication of the severity of the illness; and in response to the comparison satisfying a threshold, transmit a re-analysis request to the first machine learning model, the second machine learning model, or both.

26. The method of claim 24 or claim 25, wherein the re-analysis request causes the first machine learning model to re-analyze the received data and generate an updated plurality of tokens and a weight assigned to each respective token of the updated plurality of tokens.

27. The method of claim 26, wherein the re-analysis request further causes the second machine learning model to re-analyze the plurality of tokens and the weight assigned to reach respective token of the updated plurality of tokens to generate a rank assigned to each respective token of the updated plurality of tokens.

28. The method of claim 27, further comprising generating an updated indication of the severity of the illness of the patient based on (i) the updated plurality of tokens, (ii) the weight assigned to each respective token of the updated plurality of tokens, and (iii) the rank assigned to each respective token of the updated plurality of tokens.

29. The method of any one of claims 24 to 28, wherein the reinforcement agent is a forwardforward machine learning model.

30. The method of any one of claims 24 to 29, wherein the reinforcement agent is trained without using backpropagation.

31. A system for classifying a severity of illness of a patient, the system comprising: one or more memory devices storing machine-readable instructions; and one or more processors, each of the one or more processors configured to execute the machine-readable instructions of at least one of the one or more memory devices to implement the method of any one of claims 1 to 30.

32. A system for classifying a severity of illness of a patient, the system including one or more processors configured to implement the method of any one of claims 1 to 30.

33. A computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 30.

34. The computer program product of claim 33, wherein the computer program product is a non-transitory computer readable medium.

35. A system for classifying a severity of illness of a patient, the system comprising: a first machine learning model configured to receive data associated with the patient, the first machine learning model being trained to: identify a plurality of tokens within the data; and assign a weight to each respective token of the plurality of tokens, each token being associated with a characteristic of the patient; a second machine learning model configured to receive the plurality of tokens, the weight assigned to each respective token of the plurality of tokens, or both, the second machine learning model being trained to assign a rank to each respective token of the plurality of tokens based on the plurality of tokens, the weight assigned to each respective token of the plurality of tokens, or both; and one or more processors configured to generate an indication of the severity of the illness of the patient based on (i) the plurality of tokens, (ii) the weight assigned to each respective token of the plurality of tokens, and (iii) the rank assigned to each respective token of the plurality of tokens.

36. The system of claim 35, wherein generated indication of the severity of the illness generated by the one or more processors is a first indication of the severity of the illness, and wherein the system further comprises a reinforcement agent trained to: generate a second indication of the severity of the illness based at least in in part on the data associated with the patient; compare the first indication of the severity of the illness with the second indication of the severity of the illness; and in response to the comparison satisfying a threshold, transmit a re-analysis request to the first machine learning model, the second machine learning model, or both.

37. The system of claim 36, wherein the reinforcement agent is a forward-forward machine learning model.

38. The system of claim 36 or 37, wherein the reinforcement agent is trained without using backpropagation.

39. The system of any one of claims 35 to 38, wherein the first machine learning model is trained using b ackpropagation.

40. The system of any one of claims 35 to 39, wherein the first machine learning model is trained on reference data that includes (i) data obtained from medical textbooks, medical journals, or both; (ii) data associated with one or more policies of a healthcare provider of the patient, a healthcare facility of the patient, or both; or (iii) any combination of (i) and (ii).

41. The system of any one of claims 35 to 40, wherein each token of the plurality of tokens is associated with a health of the patient, a functional status of the patient, a burden on a caregiver of the patient, or any combination thereof.

42. The system of claim 41, wherein the weight assigned to each respective token is indicative of an amount of abnormality of represented by the respective token, the amount of abnormality being specific to the patient.

43. The system of claim 42, wherein the amount of abnormality represented by each respective token is specific to the patient.

44. The system of any one of claims 41 to 43, wherein the rank of each respective token is indicative of a magnitude of an effect of the respective token on the severity of illness of the patient.

45. The system of any one of claims 41 to 44, wherein the plurality of tokens includes one or more tokens associated with a physiology of the patient, one or more tokens associated with a level of function of the patient, one or more tokens associated with a quality of life of the patient, one or more tokens associated with a caregiver of the patient, or any combination thereof.

46. The system of claim 45, wherein the one or more tokens associated with the physiology of the patient includes an age of the patient, a height of the patient, a weight of the patient, a sex of the patient, a treatment history of the patient, data generated by one or more tests performed on the patient, data generated by one or more procedures on the patient, image data associated with imaging performed on the patient, physiological characteristics of the patient,an indication of whether the patient has one or more implanted devices, or any combination thereof.

47. The system of any one of claims 35 to 46, wherein the data includes an electronic medical record of the patient, and wherein the first machine learning model is trained to extract the plurality of tokens from the electronic medical record of the patient.

48. The system of any one of claims 35 to 47, wherein the second machine learning model is a forward-forward machine learning model.

49. The system of any one of claims 35 to 48, wherein the second machine learning model is trained without using backpropagation.

50. The system claim 48 or claim 49, wherein the indication of the severity of illness of the patient can be validated by a healthcare provider, and wherein the second machine learning model can be updated based on the validated indication and (i) the plurality of tokens, (ii) the weight assigned to each of the plurality of tokens, or (iii) both (i) and (ii).

51. The system of any one of claims 35 to 50, wherein the second machine learning model is trained to assign the plurality of ranks based on (i) only the plurality of tokens identified within the data by the first machine learning model, or (ii) the plurality of tokens identified within the data by the first machine learning model and the plurality of weights.

52. The system of any one of claims 35 to 51, further comprising inputting the plurality of weights into the second machine learning model, the second machine learning model being trained to assign the plurality of ranks based on both the plurality of tokens identified within the data by the first machine learning model and the weight assigned to each respective token of the plurality of tokens.

53. The system of any one of claims 35 to 52, wherein the weight of each token is indicative of an amount of abnormality of the token that is specific to the patient, and wherein the rank of each token is indicative of a magnitude of an effect of the corresponding characteristic on the severity of illness of the patient.

54. The system of any one of claims 35 to 53, wherein the first machine learning model is trained to generate a plurality of two-element vectors, each respective vector corresponding to one of the plurality of tokens.

55. The system of claim 54, wherein a first element of each respective two-element vector is an identifier of the corresponding token, and wherein a second element of each respective two-element vector is the weight assigned to the corresponding token.

56. The system of claim 55, wherein the second machine learning model is trained to convert each respective two-element vector to a three-element vector, the third element of each respective three-element vector being the weight assigned to the corresponding token.

57. The system of claim 54, wherein the second machine learning model is trained to generate a plurality of three-element vectors, each respective vector corresponding to one of the plurality of tokens.

58. The system of claim 57, wherein a first element of each respective three-element vector is the identifier of the corresponding token, a second element of each respective three-element vector is the weight assigned to the corresponding token, and a third element of each respective three-element vector is the weight assigned to the corresponding token.

59. The system of any one of claims 35 to 58, wherein the indication of the severity of illness is either an indication that the illness meets a predetermined threshold for severity or that the illness does not meet the predetermined threshold for severity.

60. The system of any one of claims 35 to 59, wherein the indication of the severity of illness is a percentage chance that the illness meets a predetermined threshold for severity.

Images