Learn and apply background similarities between entities

Abstract

The techniques disclosed herein relate to learning and applying background patient similarity. In various embodiments, a first value for a query entity may be displayed (702) on an interface. The first value may be related to a first background. A first trained similarity function may be selected (704) from a plurality of trained similarity functions. The first trained similarity function may be associated with the first background. The first selected trained similarity function may be applied (706) to a set of features associated with the query entity and corresponding sets of features associated with a plurality of candidate entities. One or more sets of similar candidate entities may be selected (708) from the plurality of candidate entities based on the application of the first trained similarity function. Information associated with the first set of one or more similar candidate entities may be displayed (710) on the interface.

Description

Technical Field

[0001] The various embodiments described herein generally relate to entity data analysis. More specifically, but not exclusively, the various methods and apparatuses disclosed herein relate to techniques for learning and applying entity background similarity. Background Art

[0002] Various clinical decision support (“CDS”) algorithms have been developed to provide risk scores for recent and / or long-term patient deterioration. These techniques help to better identify high-risk patients and give clinical time for appropriate planning of intervention processes. Sometimes, such treatment decision-making steps are determined by clinical guidelines. However, the guidelines are not personalized and cannot consider every possible situation. Instead, it is often necessary for a clinician to make a decision and he / she must rely on past experience. Summary of the Invention

[0003] The present disclosure relates to techniques for learning and applying entity background similarity. For example, in various embodiments, the techniques described herein can be used by a clinician (e.g., a physician, a nurse) handling a particular patient entity, a caregiver, etc. to identify other similar patients, especially other patients similar in a particular medical background. By identifying other patients with similar backgrounds, the clinician can learn what treatments are effective or ineffective, what treatments tend to produce a particular outcome, etc. Various point-of-care treatment decision support tools (e.g., software run by one or more processors) are described herein that provide the clinician with access to various information about the patient being treated (also referred to herein as the “query patient”), including other patients similar to the query patient in various medical backgrounds (e.g., cohorts of common characteristics).

[0004] In various embodiments, techniques such as artificial intelligence (e.g., deep learning, machine learning, kernel classification, multi-kernel learning, etc.) and / or statistical techniques can be employed to facilitate the identification of patients with similar backgrounds. For example, in some embodiments, a plurality of “template similarity functions” (or a “pool” of “template similarity functions”) can be generated. Each template similarity function in the pool of template similarity functions can compare a certain subset of the feature vectors associated with the query patient and the corresponding subset of the (one or more) feature vectors associated with one or more other patients (referred to herein as “candidate patients”). Entities such as patients can have a state that changes over time. Thus, an entity feature vector such as a query patient feature vector can be regarded as a “snapshot” of the entity state at a particular moment or during a time window. For example, as time passes and as a patient undergoes more tests, treatments, measurements, etc., the patient's feature vector can also change over time similarly.

[0005] In some embodiments, each template similarity function can be designed or "tuned" to determine how similar two patients are with respect to a given subset of features of the feature vectors associated with the patients. Thus, the shape of each template similarity function can be guided by population statistics (e.g., distributions) associated with the subset of features being compared. These population statistics can be derived, for example, from a retrospective patient database. In some embodiments, one or more of the template similarity functions can be tuned to highlight or amplify the similarity between patients sharing outliers, such as outliers that fall into the "tails" of the distribution of a particular feature. Overall, the pool of template similarity functions can provide a diverse view of the similarity between two patients.

[0006] In some embodiments, the output from the pool of template similarity functions can be provided as input (e.g., applied thereon) to an item that will be referred to herein as a "composite similarity function". In some embodiments, the composite similarity function can compute an item referred to herein as a "composite similarity score" for a query patient and a candidate patient based on the output from the pool of template similarity functions. In various embodiments, the composite similarity function can take the form of a machine learning model, a deep learning model, a statistical model, etc. In some embodiments, the composite similarity function can compute a convex combination of the output from the pool of template similarity functions.

[0007] In some embodiments, the composite similarity function can take the form of a weighted combination of the respective outputs of multiple template similarity functions. Different sets of weights can be applied to the output of the template similarity functions in different contexts. For example, a first set of weights can be applied in a hemodynamic instability context, a second set of weights can be applied in an acute kidney injury context, and so on. The weights applied in a particular medical context can be tuned to amplify the output of individual template similarity functions that are relatively important for that context. Outputs that are less important in the medical context can be weighted relatively less heavily.

[0008] The weights can be learned in various ways. In some embodiments, one or more kernel learning techniques (e.g., kernel smoothing algorithms) can be used to learn the weights. These weights can be learned based on training data labeled for a particular medical context, or in other words, a context-specific version of the composite similarity function can be trained. For example, to learn the weights for a hemodynamic instability context, training data labeled with some measure of hemodynamic instability can be applied. To learn the weights for an acute kidney injury context, training data labeled with some measure of acute kidney injury can be applied, and so on.

[0009] Once the weights for various medical backgrounds have been learned, the pool of template similarity functions and the composite similarity function can be applied to the feature vectors of the query patient and one or more candidate patients to identify candidate patients that are similar across various backgrounds. For example, a ranked list of candidate patients that are most similar to the query patient in the context of hemodynamic instability can be determined and provided. In some embodiments, this ranked list can be used, for example, to identify "common feature groups" of patients with similar backgrounds. The clinician can then evaluate the treatments applied to the common feature groups of similar patients and the resulting outcomes to more intelligently select a treatment for the query patient. Additionally or alternatively, in some embodiments, the pool of template similarity functions and the composite similarity function can be used, for example, by selecting the weights associated with the background of interest, to predict the clinical state of the query patient in a particular background.

[0010] While the examples described herein relate to healthcare, this is not meant to be limiting. The techniques described herein can be applied to a variety of fields outside of healthcare. For example, the techniques described herein can be used to identify background-similar entities for individuals in need of rehabilitation for drug and / or alcohol abuse, such as to enable learning and leveraging the outcomes for background-similar individuals to select a rehabilitation program. The techniques described herein can also be used in other fields, such as, travel (e.g., identifying others with similar tastes to select the most likely enjoyable itinerary), sports (e.g., comparing athletes for team selection), etc.

[0011] In addition, the techniques described herein yield various technical advantages. For example, by tuning the template similarity functions as described herein, the need for imputation methods for missing data can be avoided because if a value exists, the template similarity function may only contribute to the output of the composite similarity function. Eliminating the need for data imputation can reduce inaccuracies and / or save computational resources such as processor cycles, memory, etc. Further, employing the composite similarity function (especially with different weights learned for different backgrounds) can effectively impose a sparse regularization term (e.g., L1-norm), which allows ignoring template similarities that do not improve performance. In the healthcare domain, accurately identifying common feature groups of background similarity, and in particular being able to evaluate the treatments and / or outcomes of the common feature groups, can facilitate more intelligent and / or effective clinical decision-making.

[0012] Generally, in one aspect, a method may include the following operations: displaying a first value for a query entity on an interface, where the first value is related to a first context; selecting a first trained similarity function from a plurality of trained similarity functions, where the first trained similarity function is associated with the first context; applying the first selected trained similarity function to a feature set associated with the query entity and corresponding feature sets associated with a plurality of candidate entities; selecting a set of one or more similar candidate entities from the plurality of candidate entities based on the application of the first trained similarity function; and displaying information associated with the first set of one or more similar candidate entities on the interface.

[0013] In various embodiments, the method may further include: displaying a second value for the query entity on the interface, where the second value is related to a second context; selecting a second trained similarity function from the plurality of trained similarity functions, where the second trained similarity function is associated with the second context; applying the second selected trained similarity function to the feature set associated with the query entity and the corresponding feature sets of the plurality of candidate entities; selecting a second set of one or more similar candidate entities from the plurality of candidate entities based on the application of the second trained similarity function; and displaying information associated with the second set of one or more similar candidate entities on the interface.

[0014] In various embodiments, displaying information associated with the first set of one or more similar candidate entities may include: grouping the first set of one or more similar candidate entities into groups according to corresponding values of at least one attribute of the first set of one or more similar candidate entities; obtaining a corresponding first value related to the first context for each group in the groups; and displaying information about each group associated with the corresponding first value related to the first context on the interface.

[0015] In various embodiments, the first value related to the first context is a score obtained by a clinical decision support algorithm. In various embodiments, the at least one attribute is an administered treatment. In various embodiments, the at least one attribute is a patient outcome. In various embodiments, the information about each group associated with the corresponding first value related to the first context may include patient outcome statistics.

[0016] It should be understood that all combinations of the foregoing concepts and additional concepts discussed in more detail below (assuming such concepts are not mutually inconsistent) are considered to be part of the inventive subject matter disclosed herein. In particular, all combinations of the claimed subject matter that appear at the end of this disclosure are considered to be part of the inventive subject matter disclosed herein. It should also be understood that the terms expressly employed herein may also appear in any disclosure incorporated by reference, and such terms should be accorded a meaning most consistent with the particular concepts disclosed herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In the drawings, like reference numerals generally refer to like parts throughout the different views. Also, the drawings are not necessarily to scale, but rather generally focus on illustrating the various principles of the embodiments described herein.

[0018] Figure 1 Schematically illustrates an environment in which selected aspects of the present disclosure according to various embodiments may be implemented.

[0019] Figure 2 Illustrates techniques for generating a template similarity function according to various embodiments.

[0020] Figure 3 、 Figure 4 and Figure 5 Depict exemplary graphical user interfaces that may present information determined using the techniques described herein.

[0021] Figure 6 and Figure 7 Depict example methods for implementing selected aspects of the present disclosure.

[0022] Figure 8 Schematically depicts an example computing system architecture. DETAILED DESCRIPTION

[0023] Various clinical decision support (“CDS”) algorithms have been developed to provide risk scores for recent and / or long-term patient deterioration. These techniques help to better identify high-risk patients and give appropriate clinical times for planning interventions. Sometimes, such treatment decision steps are determined by clinical guidelines. However, the guidelines are not personalized and cannot take into account every possible situation. Instead, it is often up to the clinician to make the decision and he / she must rely on past experience. Generally, it would be beneficial to be able to determine background similarities between entities such as patients for various purposes in various fields.

[0024] In view of the foregoing, various embodiments and implementations of the present disclosure relate to learning and applying entity background similarity. Referring to Figure 1, schematically depicts an environment in which selected aspects of the present disclosure may be implemented. One or more client devices 102, an entity similarity system 104, a retrospective patient database 106, and a training database 108 are shown in network communication via one or more networks 110 such as the Internet. In various embodiments, one or more of components 102 - 108 may be omitted, combined with other components, and other components may or may not be added.

[0025] One or more client devices 102 may include, for example, one or more of the following: a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device of a user's vehicle (e.g., an in - vehicle communication system, an in - vehicle entertainment system, an in - vehicle navigation system), a stand - alone interactive speaker, a smart appliance (e.g., a smart TV), and / or a wearable device of the user that includes a computing device (e.g., a user's watch with a computing device, a user's glasses with a computing device, a virtual reality computing device, or an augmented reality computing device). Additional and / or alternative client computing devices may be provided.

[0026] In various embodiments, (one or more) client devices 102 may operate a variety of different applications, such as, for example, a web browser, an email client, a messaging client, a social media client, etc. Of most importance for the present disclosure, the client device 102 (which will be referred to herein in the singular) may operate a clinician decision application 112. The clinician decision application 112 may be software that can be operated by a clinician or other individual interested in a particular patient condition (e.g., a caregiver, a researcher, etc.) to evaluate information determined using various aspects of the present disclosure. Figures 3 - 5 An example graphical user interface (“GUI”) that may be generated and / or presented by the clinician decision application 112 is depicted in accordance with various embodiments.

[0027] The entity similarity system 104 may include one or more computing devices that may operate together to collect, generate, and / or compute data that can be used to identify background - similar entities, where the background - similar entities in this example and other background - similar entities described herein may be medical patients. In some embodiments, the entity similarity system 104 may include one or more modules or engines, any of which may be implemented using any combination of hardware and / or software. For example, in Figure 1In [the context], the entity similarity system 104 includes a similarity training engine 114 and a similarity detection engine 116. In other embodiments, the similarity training engine 114 and the similarity detection engine 116 can be combined into a single engine or module. In some embodiments, and as will be described in more detail below, the entity similarity system 104 can employ multiple similarity template functions 118 and / or one or more machine learning models 120 to calculate a background similarity between entities such as patients.

[0028] The retrospective patient database 106 can include information about patients, such as, for example, age, weight, diagnosis, vital signs, tests performed, test results, treatments administered / applied, medications, etc. In some embodiments, the retrospective patient database 106 can take the form of a conventional hospital information system (“HIS”) for storing, for example, electronic medical records (“EMR”) associated with multiple patients. As will be described in more detail shortly, the patient records in the retrospective patient database 106 and, in some cases, the feature vectors generated / extracted from these records can be used to represent candidate patients for performing the techniques described herein. Although depicted as a single database in [the context], in practice any number of databases (which can be operated by one or more computing systems, such as a group of computing systems that cooperate to provide a so-called “cloud” computing system or architecture) can be used to implement the retrospective patient database 106 (and any other database or index described herein). Figure 1 In [the context], the training database 108 can store one or more background training data sets 122 for, for example, training one or more machine learning models 120

[0029] In some embodiments, the training database 108 and the retrospective patient database 106 can be combined into a single logical and / or physical database. In some embodiments, multiple background training data sets 122 1-N can be stored in the training database 108. As will be described in more detail shortly, in some embodiments, each background training data set 122 can include individual training examples labeled with a specific background label. These labels can facilitate the training of different instances of a machine learning model 120 that can be used in different backgrounds. As an example, the individual training examples of a first background training data set 122 1-N can be labeled to indicate a measure or indication of hemodynamic instability. The individual training examples of a second background training data set 122 1 2Individual training examples can be labeled to indicate a measure of acute kidney injury. And so on. In some embodiments, the label can be binary, for example, to indicate the presence or absence of a particular medical condition. Additionally or alternatively, the label can be non-binary and can alternatively indicate a measure of a particular feature value (e.g., within a continuous range).

[0030] Entity similarity system 104 can be configured to perform various aspects of the present disclosure, for example, by similarity training engine 114 and / or similarity detection engine 116. For example, in some embodiments, entity similarity system 104 can be configured to provide / obtain a plurality of template similarity functions 118. Each template similarity function among the plurality of template similarity functions 118 can compare a corresponding subset of features of a query entity feature vector (e.g., a query patient feature vector) with a corresponding subset of features of a candidate entity feature vector (e.g., a candidate patient feature vector).

[0031] In some embodiments, entity similarity system 104 may further include the above-mentioned machine learning model(s) 120, which receive the output of template similarity function 118 as input and calculate a composite similarity score based on these values. In some embodiments, each machine learning model 120 can take the form of a composite similarity function, which in some embodiments can be a weighted combination of the corresponding outputs of a plurality of template similarity functions 118.

[0032] In various embodiments, similarity training engine 114 can be configured to obtain, for example, a first plurality of labeled entity vectors from training database 108 as first background training data. For example, if machine learning model 120 is being trained to calculate similarity between patients in the context of hemodynamic instability, similarity training engine 114 can obtain background training data set 122, which includes training examples labeled to indicate hemodynamic stability (or lack thereof). As will be described in more detail below, similarity training engine 114 can use these training examples to train machine learning model 120 so as to tune machine learning model 120 to calculate similarity between patients in the context of hemodynamic instability. In various embodiments, multiple machine learning models 120 can be trained, for example, one machine learning model for each desired background. For example, one machine learning model 120 can be trained to calculate similarity between patients in the context of hemodynamic instability. Another can be trained to calculate similarity between patients in the context of acute kidney injury. And so on.

[0033] The similarity detection engine 116 can be configured to apply a query entity feature vector (which can include multiple features extracted from the entity) to one or more candidate entity feature vectors using, for example, a similarity template function 118 and one or more trained machine learning models 120. In a medical context, for example, the similarity detection engine 116 can be configured to use the similarity template function 118 and the machine learning model 120 to compare the features of a query patient (e.g., vital signs, age, weight, disposition, etc. (which can be obtained in real time and / or from a retrospective patient database 106)) with the corresponding features of candidate patients obtained from the retrospective patient database 106. As an output, the similarity detection engine 116 can provide only a background similarity score between the query entity and the candidate entity, or, it can provide a list of candidate entities ranked based on their similarity to the query entity.

[0034] The output of the similarity detection engine 116 can be used by the clinician decision application 112 to provide information and / or tools to a clinician or other person that enable the clinician to make an informed decision about the query patient. As a non-limiting example, a clinician may be able to view groups of common features of background similar patients to see what dispositions were applied and the (e.g., statistical) outcomes of those dispositions. Based on this information, the clinician can decide on a course of action. Additionally or alternatively, in some embodiments where, for example, a disposition is automatically administered by a robot, the applied disposition can be automatically selected at least in part based on the information provided by the similarity detection engine 116. As another example, the clinician decision application 112 can use the techniques described herein to predict the clinical state of a query patient in a particular context.

[0035] Template similarity function

[0036] A technique for generating the template similarity function 118 will now be described. It should be understood that this description is not meant to be limiting, and other techniques and / or template similarity functions can be employed. Also, although the phrase "template similarity function" is used herein, this is not meant to be limiting. These functions can also be referred to as "kernels". For purposes of discussion, a query patient (e.g., a patient for whom a clinician wants to make a decision about a disposition, etc.) can be represented as p q , and each candidate patient being compared to p q can be represented as p c .

[0037] In some embodiments, m template similarity functions S 1 (p q , p c ), …, S m (p q , pc )'s pool. These functions can be used as a basis for the subsequent background similarity learning phase described below. Each template similarity function S in the template similarity function i (p q , p c ) takes two feature vectors (one from the query patient p q , and the other from the candidate patient p c ) as input and returns an output (e.g., a score) quantifying the degree of similarity between p q and p c . Each template similarity function judges similarity in a different way by considering different subsets of the patient's feature vectors. For example, each template similarity function can be tuned to different features (e.g., a similarity function based on heart rate or blood pressure). Small groups of features can also be considered to address the interaction between features. The resulting pool of template similarity functions S 1 (p q , p c ), …, S m (p q , p c ) can provide a diverse view of similarity. The background-specific machine learning model 120 (described in more detail below) can be used to fuse this diverse view into a single score based on the clinical background.

[0038] Figure 2 illustrates an example of how a single template similarity function can be applied to a specific patient feature (i.e., heart rate). Other template similarity functions that evaluate other types of similarity between patient vector features can operate similarly. In various embodiments, a population distribution for the patient's heart rate can be determined. This can be achieved, for example, by calculating a heart rate histogram based on the training data set 120 in the training database 108.

[0039] In this example, the heart rate (“HR”) of the query patient p q is 70, while the heart rate of the candidate patient p c is 120. Arrows from the query patient and the candidate patient point to a histogram illustrating the distribution of the patient's heart rate, which resembles a bell curve (but this is not required). Such a histogram can provide a probability distribution P HR (x). In some embodiments, the unnormalized template similarity function can be calculated using, for example, the following formula:

[0040]

[0041] In some embodiments, the function f can be a monotonically decreasing function, e.g., f(x) = (1 - x). In this example, P HR (xq ≤HR≤x c ) can be the cumulative probability of patients where the heart rate is somewhere between p q and p c Since the function f is monotonically decreasing, as the cumulative probability increases (or decreases), the similarity between the corresponding heart rates of p q and p c decreases (or increases). The cumulative probability is represented by the area A under the bell curve in Figure 2 .

[0042] In some embodiments, the template similarity function can be normalized using a formula such as the following:

[0043]

[0044] In formula (2), the two denominator terms respectively represent the outputs of the expected average non-normalized template similarity function for all other patients in the retrospective patient database 106 for p q and p c . E represents the expectation (mean). This normalization can be used to normalize the output of the template similarity function to a common range.

[0045] The method uses the population distribution of features or the population distribution of a common feature group to quantify similarity. The effect of this process is that the similarity between the two heart rate values for p q and p c depends not only on their proximity but also on their degree of abnormality. Since the similarity score is inversely proportional to the area under the probability distribution between these two values ( Figure 2 A in), outliers located closer to the tails of the distribution will receive higher scores. This has the benefit of highlighting outliers that clinicians may be concerned about. Intuitively, in the case of two patients (or more generally, two entities), their similarity is inversely correlated with the expected number of patients (or more generally, entities) located between them.

[0046] Although this heart rate example applies to a single feature (heart rate) being compared between patients, this is not meant to be restrictive. These steps can be generalized to a template similarity metric that considers multiple features of a patient (or more generally, an entity) vector. In particular, a multi-dimensional probability distribution can be employed.

[0047] Generally speaking, the template similarity vector or "kernel" can be generated in various ways. For example, let x j and z j be represented as the corresponding feature values for two entities with state vectors x and z. Then, having a range of [min(x j , zj ), max(x j , z j ))] The expected number of entities with values within the range is given by the area under the population distribution P(X j ) of X within that range. In some embodiments, the following kernel can be applied to the feature X j : j :

[0048] k j，c (x, z) = (1 - P(min(x j , z j ) ≤ X j ≤ max(x j , z j ))) c (A)

[0049] In various embodiments, the kernel can be applied to binary or ordered discrete features. For example, X j can be a Bernoulli random variable that characterizes whether a patient exhibits a symptom or the presence of a rare condition or complication. In this case, formula (A) simplifies to:

[0050]

[0051] Thus, if two patients x and z both have or both do not have a clinical condition, then the similarity between the two patients can be inversely correlated with the prevalence or absence of that condition, and if their condition states are different, there can be no similarity. The above kernel assumes an ordered relationship with the values of the random variable. However, it can be extended to a nominal categorical variable through one-hot encoding, which converts a nominal variable over c categories into c Bernoulli random variables.

[0052] Machine learning model training

[0053] Now, example techniques for training one or more machine learning models 120 to generate a composite similarity score based on the combined output of m template similarity functions 118 will be described. In some embodiments, the machine learning model 120 can be a composite similarity function S c (p q , p c ), which can be a convex combination of the outputs of m template similarity functions. For example, in some embodiments, the composite similarity function can be modeled as follows:

[0054]

[0055] Subject to a i ≥ 0, i = 1,..., m (4)

[0056]

[0057] Thus, in this example, S C is the weighted average of the outputs of m individual template similarity functions, where the weights are represented as a 1 , a 2 , …, a m .

[0058] To train the machine learning model 120 (which can mean learning the weights a 1 , a 2 , …, a m ) in various embodiments, for example, n pairs of paired training examples (p (1) , y (1) ), …, (p (n) , y (n) ) can be provided from the training database 108. p (i) can be a vector of input features (vital signs, laboratory, demographic data, etc.) for a given patient, and y (i) can be a label indicating the clinical background state of the patient. Depending on the clinical state reflected by the clinical background, the clinical background can be a binary label or a real-valued number. For example, y (i) can be a binary label indicating which of two disease states the patient (i) belongs to.

[0059] In some embodiments, a multi-kernel learning algorithm (such as a kernel smoothing algorithm) executed on a training set of labels (p (1) , y (1) ), …, (p (n) , y (n) ) can be used to train the weights a 1 , a 2 , …, a m . For example, in some embodiments, the following approximation function can be employed to calculate the approximate label for each patient

[0060]

[0061] Intuitively, formula (6) attempts to approximate the label for the i-th patient by taking a weighted average of the "true" labels (y) of the adjacent terms of the i-th patient, where the adjacent terms are defined by the composite similarity S C . Thus, if the output of S C indicates that two adjacent patients are similar, the "true" label (y) of one adjacent term will have a greater impact on the approximate label of the other adjacent term.

[0062] Once the approximate label is calculated, the difference between the approximate label and the "true" label can be used to determine the weights a 1 , a 2 , …, a m . For example, the loss function can be used to measure the difference between the true label y and its approximation . For example, if the label is binary, binary cross-entropy can be used. If the label is a continuous value, mean squared error can be used as the loss. In either case, a formula such as the following can be used to minimize the total loss over all training examples:

[0063]

[0064] Note that this objective is implicitly a function of the weights a C of S 1 , a 2 , …, a m . Then minimization can be performed, for example, using gradient descent (e.g., stochastic, batch, etc.) to learn the optimal weights in this context.

[0065] In some embodiments, Equation (6) can also be used to predict the clinical context (i.e., their label) of a particular object based on the "true" label (y) associated with similar objects. For example, the label approximation determined for an object with an unknown clinical status can be influenced by the corresponding true label y associated with the object determined using, for example, one or more of Equations (3)-(5) above.

[0066] Example usage

[0067] Figures 3 - 5 depicts an example graphical user interface ("GUI") that can be presented on a display by the clinician decision application 112 in Figure 1 . Now referring to Figure 3 , for example, when a clinician pulls up the record of a patient being queried, the clinician can view the dashboard GUI 300. In this example, the name of the patient being queried is "John Doe", as indicated in the title bar of the dashboard GUI 300. The dashboard GUI 300 enables the clinician to obtain an overview of the current clinical status of the patient being queried. Many different panels (four panels (330 Figure 3 are depicted in 1-4 )) can each convey context-specific aspects of the status of the patient being queried, e.g., an overview of a particular anatomical system, the status of a particular condition, etc. For example, the first panel 330 1 gives an overview of the cardiovascular health status. The second panel 330 2Provide an overview of the cardio-renal syndrome (“CRS”) of the queried patient. The third panel 330 3 Provide an overview of the renal system of the queried patient. The fourth panel 330 4 Provide an overview of the respiratory system of the queried patient. These panels 330 1-4 These are merely examples; more or fewer panels providing overviews of other clinical contexts may be provided.

[0068] Also depicted is a fifth panel 330 providing an overview of various clinician decision support (“CDS”) statistics of the queried patient 5 . In Figure 3 , wherein the fifth panel 330 5 includes an overview of the hemodynamic instability index or “HII” of the queried patient and overviews of the acute kidney injury (“AKI”) status and acute respiratory distress syndrome (“ARDS”) of the queried patient. In Figure 3 , additional information 332 regarding the HII of the queried patient is depicted as the HII of the queried patient has been elevated (78).

[0069] A clinician can select any one of the panels 330, for example, using a mouse or by touching a touch screen, to obtain more detailed information regarding the corresponding clinical context. For example, in the case where the HII of the queried patient has been elevated, the clinician may select the fifth panel 330 5 to obtain additional information regarding the portion related to the HII. In doing so, another GUI such as the GUI 400 depicted in Figure 4 may be presented.

[0070] In Figure 4 , the GUI 400 includes a more detailed overview of the hemodynamic instability index of the queried patient, including the individual data contributing to the HII score 78 (e.g., age, CVP, heart rate, etc.). The GUI 400 may also include a graph 438 depicting the HII of the queried patient over time. Additionally, the GUI 400 may include a button 440 or other selectable element that a clinician can select to view similar patients. When selected, this may trigger a cross-multiple-template similarity function( Figure 1The application of the feature vectors associated with the query patient 118) to compute the corresponding outputs. As described above, these outputs can be applied as inputs to the context-specific machine learning model 120, which, as described above, can be a composite similarity function (e.g., equation (3) above), that applies weights learned for the current context to the output of the template similarity function to compute a composite context similarity score. In particular, weights learned using the various equations described previously using the context training data set 122 from the training database 108 can be applied to the corresponding outputs of the template similarity function. In some embodiments, this technique can be applied to compare multiple candidate patient feature vectors, such as using information from the retrospective patient database 106, with the feature vector of the candidate patient, and a list of candidate patients ranked by similarity to the query patient can be returned. In some embodiments, only the x most similar patients can be returned, where x is a positive integer that is, for example, manually selected or determined based on the number or percentage of candidate patients that meet a certain similarity threshold.

[0071] Figure 5 Depicts an example GUI 500 that can be presented in response to a selection of the button 440. In Figure 5 it, the HII score 78 of the query patient is depicted, where a line connects the score to a plurality of treatment option groups. In Figure 5 it, these options include no treatment, infusion treatment, blood transfusion, inotropic agents, and vasopressor treatment. However, these are not meant to be limiting. For each treatment option group, a number of similar patients to whom the corresponding treatment option has been applied and the corresponding average outcome, in this example the average HII post-treatment, are also depicted.

[0072] For example, no treatment was applied to twenty contextually similar patients, resulting in an increased average HII score of 85. Infusion treatment was applied to fifteen contextually similar patients, resulting in a slightly decreased average HII score of 74. Blood transfusion was applied to thirty contextually similar patients, resulting in a slightly decreased average HII score of 73. Inotropic agent treatment was applied to thirty contextually similar patients, resulting in a slightly decreased average HII score of 69. Vasopressor treatment was applied to the majority of contextually similar patients (i.e., one hundred of them), resulting in a sharply decreased average HII score of 45. Thus, the clinician can easily see that vasopressor is by far the most effective treatment option applied to contextually similar patients. In various embodiments, each treatment option group itself can be selectable to view more information (e.g., more granular statistics) about the contextually similar patients in that group and the treatments they received.

[0073] Although not depicted in the figures, in various embodiments, the techniques described herein can be implemented to provide a clinician with additional information other than the Figure 5 information depicted in. For example, in some embodiments, some of the most background-similar patients can be presented to the clinician, for example, in the form of a list. In some such embodiments, the clinician may be able to select a given background-similar patient to learn more information about the background-similar patient, such as their medical history, specific measurements (e.g., vital signs, laboratory results), applied / administered treatments, family history, etc.

[0074] Figure 6 FIG. illustrates an example method 600 for practicing selected aspects of the present disclosure according to various embodiments. For convenience, these operations are described with reference to a system that performs the operations of the flowchart. The system may include various components of various computer systems, including the entity similarity system 104. Additionally, although the operations of method 600 are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, or added. Additionally, other intermediate or long-term results may also be displayed, such as, for example, the incidence of organ failure, length of hospital stay, mortality rate, etc.

[0075] At block 602, the system may provide a plurality of template similarity functions, such as, for example, Figure 1 118 in. The plurality of template functions may be provided in various ways. In some embodiments, the plurality of template functions may be manually created by one or more clinicians, for example. As described above, each of the plurality of template similarity functions may be designed to compare a corresponding subset of features of a query entity feature vector associated with a query patient and a corresponding subset of features of a candidate entity feature vector associated with a candidate patient. At block 604, the system may provide a machine learning model, such as, for example, a composite similarity function that includes a weighted combination of the respective outputs of the plurality of template similarity functions. Equation (3) above is a non-limiting example of a composite similarity function.

[0076] At block 606, the system may provide and / or obtain, for example, from the training database 108 a first plurality of labeled entity vectors as first background training data (e.g., Figure 1 1201 in). In some embodiments, the first background training data may be specifically selected to train the weights of a composite similarity function to be applied to find similar entities (e.g., patients) in a particular background. For example, if the goal is to find patients similar to any patient diagnosed with type II diabetes, the first background training data may include training examples in the form of patient feature vectors labeled to indicate the presence or absence of type II diabetes.

[0077] At block 608, the system, for example via the similarity training engine 114, may apply an approximation function such as equation (6) above to approximate the first background label for the corresponding labeled entity vector data for each corresponding labeled entity vector of the first background training data. In some embodiments, the first background label can be approximated based on the output of the composite similarity function and the corresponding "true" first background label (y) of the entity vectors of other labels of the first background training data. As described above, intuitively, this can mean that the more similar two patients are, the closer the true label (y) of one patient is to the approximated label of the other patient. The greater the contribution, the greater the

[0078] At block 610, the system can train a first context-specific composite similarity function based on the composite similarity function (e.g., the aforementioned formula (3)). This can include, for example, using a first loss function (e.g., )) to learn similarity functions for multiple templates (e.g., Figure 1 The first background weight a of 118) 1 ,a 2 ,…,a m In various embodiments, the first weight may be stored for later use as part of a first context-specific composite similarity function.

[0079] As previously described, in various embodiments, different context-specific composite similarity functions (or more generally, machine learning models 120) may be learned for different entity contexts. For example, a first context-specific composite similarity function may be learned for hemodynamic instability, a second context-specific composite similarity function may be learned for acute kidney injury, a third context-specific composite similarity function may be learned for one type of cancer, a fourth context-specific composite similarity function may be learned for another type of cancer, a fifth context-specific composite similarity function may be learned for type I diabetes, a sixth context-specific composite similarity function may be learned for type II diabetes, and so on. Therefore, at block 612, similar to block 606, the system may provide a second plurality of labeled entity vectors as second context training data.

[0080] At box 614, the system can apply an approximation function (e.g., the aforementioned formula (6)) to approximate the second context label for the corresponding labeled entity vector data based on the output of the composite similarity function and the corresponding second context labels of other labeled entity vectors of the second context training data for each corresponding labeled entity vector of the second context training data. This operation can be similar to box 608. At box 616, similar to box 610, the system can train a second context-specific composite similarity function based on the composite similarity function. In some embodiments, boxes 606-610 can be repeated for as many different contexts as desired, thereby generating a "library" of context-specific composite similarity functions that can be selectively applied later to find context-similar entities and / or predict the state of a query entity in a specific context.

[0081] Figure 7 Describes selected aspects for practicing the present disclosure (i.e., applying template similarity functions and using methods such as Figure 6 The operations depicted in the flowchart are described in detail in an example method 700 of using (one or more) context-specific machine learning models (e.g., composite similarity functions) to identify context-similar entities. For convenience, the operations are described with reference to a system that performs the operations of the flowchart. The system may include various components of various computer systems, including the entity similarity system 104 and / or the clinician decision application 112. Moreover, although the operations of method 700 are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, or added.

[0082] At block 702, the system may display a first value for a query entity on an interface. In various embodiments, the first value may be related to a first context, and the first query entity may be an entity of interest, such as a query patient being examined by a clinician. Figures 3 - 4 An example of a first value associated with a first context is depicted in FIG, where the context is hemodynamic instability and the value is an HII score of 78. At block 704, the system can select a first trained similarity function associated with the first context from a plurality of trained similarity functions. The trained similarity functions can include, for example, using Figure 6 As an example, when the clinician selects Figure 4 , this can trigger the selection of a context-specific composite similarity function trained for hemodynamic instability. In particular, the selected context-specific composite similarity function can include weights specific to hemodynamic instability learned using formulas such as the aforementioned formulas (6)-(7).

[0083] At block 706, the system may apply a first selected trained similarity function to a set of features associated with a query entity and corresponding sets of features associated with a plurality of candidate entities. For example, a query patient feature vector may include features such as demographics (e.g., age, weight, gender, etc.), complications, vital signs (e.g., heart rate, systolic blood pressure, etc.), and / or laboratory results (e.g., sodium, lactate, magnesium, etc.). Candidate patient feature vectors may be selected from the retrospective patient database 106 and may include similar features. In some embodiments, (query or candidate) patient feature vectors may include features extracted from a latent variable model, e.g., features extracted from a hidden layer in a deep neural network. In various embodiments, the query patient feature vector and candidate patient feature vectors may be applied as inputs to a pool of template similarity functions 118. The corresponding outputs of these functions may be applied as inputs to a machine learning model 120, which as previously described, may be a similarity function that has been trained for the selected context, e.g., the composite similarity function of formula (3).

[0084] At block 708, the system may select a set of one or more similar candidate entities from the plurality of candidate entities based on the application of the first trained similarity function at block 706. For example, in some embodiments, the system may return a list of candidate patients ranked by background similarity to the query patient. At block 710, the system, e.g., via the clinician decision application 112, may display information associated with the first set of one or more similar candidate entities on an interface.

[0085] The information displayed at block 710 may take various forms. In some embodiments, the information may include, for example, different panels for each returned background-similar candidate patient. Each panel may display various background-specific information about the corresponding candidate patient. In some embodiments, a clinician may select a panel to display more detailed information about the corresponding candidate patient. Additionally or alternatively, the information may include various statistics about background-similar patients, e.g., statistics related to treatments, outcomes, etc. grouped by attributes such as outcomes, applied treatments, etc. in groups of background-similar patients. An example of such statistics is depicted in Figure 5 In.

[0086] Figure 8is a block diagram of an example computer system 810. The computer system 810 generally includes at least one processor 814 that communicates with a plurality of peripheral devices via a bus subsystem 812. These peripheral devices can include a storage subsystem 824 (including, for example, a memory subsystem 825 and a file storage subsystem 826), a user interface output device 820, a user interface input device 822, and a network interface subsystem 816. The input and output devices allow a user to interact with the computer system 810. The network interface subsystem 816 provides an interface to an external network and is coupled to corresponding interface devices in other computer systems.

[0087] The user interface input device 822 can include a keyboard, a pointing device (e.g., a mouse, trackball, touchpad, or graphics tablet), a scanner, a touchscreen incorporated in a display, an audio input device (e.g., a speech recognition system, a microphone), and / or other types of input devices. In general, the use of the term "input device" is intended to include all possible types of devices and ways of inputting information into the computer system 810 or onto a communication network.

[0088] The user interface output device 820 can include a display subsystem, a printer, a fax machine, or a non-visual display (e.g., an audio output device). The display subsystem can include a cathode ray tube (CRT), a flat panel device (e.g., a liquid crystal display (LCD)), a projection device, or some other mechanism for creating a visible image. The display subsystem can also provide a non-visual display, for example, via an audio output device. In general, the use of the term "output device" is intended to include all possible types of devices and ways of outputting information from the computer system 810 to an object or another machine or computer system.

[0089] The storage subsystem 824 stores programming and data structures that provide some or all of the functionality described in the modules / engines herein. For example, the storage subsystem 824 can include logic for performing selected aspects of methods 600 and / or 700 and / or for implementing one or more components depicted in the various figures. The memory subsystem 825 used in the storage subsystem 824 can include a plurality of memories, including a main random access memory (RAM) 830 for storing instructions and data during program execution and a read-only memory (ROM) 832 in which fixed instructions are stored. The file storage subsystem 826 is capable of providing permanent storage for program and data files and can include a hard disk drive, a CD-ROM drive, an optical disk drive, or a removable media cartridge. Modules implementing the functionality of certain embodiments can be stored in the storage subsystem 824 by the file storage subsystem 826 or in other machines accessible by the processor(s) 814.

[0090] The bus subsystem 812 provides a mechanism for enabling the various components and subsystems of the computer system 810 to communicate with one another as desired. Although the bus subsystem 812 is schematically shown as a single bus, alternative implementations of the bus subsystem may use multiple buses.

[0091] The computer system 810 can be of various types, including workstations, servers, computing clusters, blade servers, server farms, smartphones, smartwatches, smart glasses, set-top boxes, tablet computers, laptop computers, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, for the purpose of illustrating some embodiments, the description of the computer system 810 depicted in Figure 8 is only intended as a specific example. Many other configurations of the computer system 810 may have more or fewer components than the computer system depicted in Figure 8

[0092] Although several inventive embodiments have been described and illustrated herein, those of ordinary skill in the art will readily conceive of various other units and / or structures for performing the functions and / or obtaining the results and / or one or more of the advantages described herein, and each such variation and / or modification is considered to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily understand that all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and the actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications of the teachings of the present invention. Those skilled in the art will recognize or be able to use, without more than routine experimentation, many equivalents to the specific inventive embodiments described herein. Accordingly, it should be understood that the foregoing embodiments are presented by way of example only, and that within the scope of the claims and their equivalents, embodiments of the present invention may be practiced in a manner different from that specifically described and claimed. The inventive embodiments of the present disclosure relate to each and every separate feature, system, product, material, kit, and / or method described herein. Additionally, any combination of two or more such features, systems, products, materials, kits, and / or methods can be included within the scope of the inventive disclosure of the present invention if such features, systems, products, materials, kits, and / or methods are not mutually inconsistent.

[0093] All definitions defined and used herein should be understood to control over dictionary definitions, definitions in incorporated by reference documents, and / or ordinary meanings of the defined terms.

[0094] Unless the context clearly dictates otherwise, as used in the specification and claims herein, the words "a" and "an" should be understood to mean "at least one".

[0095] ​As used herein in the specification and claims, the phrase "and / or" shall be understood to mean "any one or both" of the elements so combined, i.e., elements that are present together in some cases and separate in other cases. Multiple elements listed with "and / or" shall be construed in the same manner, i.e., "one or more" of the elements so combined. Other elements may optionally be present in addition to the elements specifically identified by the "and / or" clause, whether related or unrelated to those specifically identified. Thus, as a non-limiting example, when used in conjunction with open-ended language such as "comprising", a reference to "A and / or B" can in one embodiment refer only to A (optionally including elements other than B); in another embodiment only to B (optionally including elements other than A); in yet another embodiment to both A and B (optionally including other elements); and so on.

[0096] As used herein in the specification and claims, "or" shall be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" shall be interpreted inclusively, i.e., including at least one, but also including more than one element or elements in a list of elements and (optionally) additional unlisted items. Only terms that explicitly indicate the contrary (e.g., "only one of" or "exactly one of" or when used in a claim "consisting of") will refer to including exactly one element of a plurality of elements or elements in a list. In general, when the term "or" as used herein is preceded by an exclusive term (e.g., "any one of", "one of", "only one of" or "exactly one of"), the term shall be interpreted only to mean an exclusive alternative (i.e., "one or the other but not both"). The term "consisting essentially of" when used in a claim shall have its ordinary meaning as used in the field of patent law.

[0097] As used herein in the specification and claims, the phrase "at least one" referring to a list of one or more elements should be understood to mean at least one element selected from any one or more elements in the list of elements, but does not necessarily include at least one element of each element specifically listed in the list of elements, and does not exclude any combination of elements in the list of elements. This definition also allows that elements may optionally be present in addition to the elements specifically identified within the list of elements to which the phrase "at least one" refers, whether related or unrelated to those specifically identified elements. Thus, as a non-limiting example, "at least one of A and B" (or equivalently, "at least one of A or B", or equivalently, "at least one of A and / or B") can refer to at least one A, optionally including more than one A, and no B (and optionally including elements other than B) in one embodiment; and at least one B, optionally including more than one B, and no A (and optionally including elements other than A) in another embodiment; and at least one A, optionally including more than one A, and at least one B, optionally including more than one B (and optionally including other elements) in yet another embodiment; and so on.

[0098] It should also be understood that in any method claimed herein that includes more than one step or action, the order of the steps or actions of the method is not necessarily limited to the order in which the steps or actions of the method are recorded unless explicitly stated to the contrary.

[0099] In the claims and the description above, all transitional phrases (e.g., "comprising," "including," "carrying," "having," "containing," "involving," "having," "composed of," etc.) should be construed as open-ended, i.e., meaning including, but not limited to, "including." Only the transitional phrases "consisting of" and "consisting essentially of" shall be closed or semi-closed transitional phrases, respectively, as set forth in Section 2111.03 of the U.S. Patent Office Manual of Patent Examining Procedures. It should be understood that certain expressions and reference numerals used in the claims do not limit the scope pursuant to Rule 6.2(b) of the Patent Cooperation Treaty ("PCT").

Claims

1. A computer-implemented method for presenting one or more entities with similar backgrounds to a user, the method being implemented by one or more processors and comprising: displaying, on an interface, a first value for a query entity, wherein the first value is related to a first background; selecting a first trained similarity function from a plurality of trained similarity functions, wherein the first trained similarity function is a background-specific machine learning model, and wherein the first trained similarity function is selected based on the first background; applying the first selected trained similarity function to a query entity feature vector associated with the query entity and to candidate entity feature vectors associated with a plurality of candidate entities, wherein the first selected trained similarity function is configured to compare a feature subset of the query entity feature vector in the first background with a corresponding feature subset of the candidate entity feature vectors in the first background; selecting, based on the application of the first selected trained similarity function, a first set of one or more similar candidate entities in the first background from the plurality of candidate entities; and displaying, on the interface, information associated with the first set of one or more similar candidate entities.

2. The computer-implemented method according to claim 1, further comprising: displaying, on the interface, a second value for the query entity, the second value being related to a second background; selecting a second trained similarity function from the plurality of trained similarity functions, wherein the second trained similarity function is a background-specific machine learning model, and wherein the second trained similarity function is selected based on the second background; applying the second selected trained similarity function to the query entity feature vector associated with the query entity and to the candidate entity feature vectors associated with the plurality of candidate entities, wherein the second selected trained similarity function is configured to compare another feature subset of the query entity feature vector in the second background with a corresponding other feature subset of the candidate entity feature vectors in the second background; selecting, based on the application of the second selected trained similarity function, a second set of one or more similar candidate entities in the second background from the plurality of candidate entities; and displaying, on the interface, information associated with the second set of one or more similar candidate entities.

3. The method according to claim 1, wherein, displaying information associated with the first set of one or more similar candidate entities comprises: grouping the first set of one or more similar candidate entities according to corresponding values of at least one attribute of the first set of one or more similar candidate entities; obtaining, for each group in the group, a corresponding first value related to the first background; displaying, on the interface, information about each group associated with the corresponding first value related to the first background.

4. The method according to claim 3, wherein, The first value related to the first context is a score obtained by a clinical decision support algorithm.

5. The method according to claim 4, wherein, the at least one attribute is the administered treatment.

6. The method according to claim 4, wherein, the at least one attribute is the patient outcome.

7. The method according to claim 3, wherein, the information about each group associated with the respective first value related to the first context includes patient outcome statistics.

8. A system for presenting to a user one or more contextually similar entities, the system comprising one or more processors and a memory operatively coupled to the one or more processors, wherein, the memory stores instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to perform the following operations: display a first value for a query entity on an interface, wherein the first value is related to a first context; select a first trained similarity function from a plurality of trained similarity functions, wherein the first trained similarity function is a context-specific machine learning model, and wherein the first trained similarity function is selected based on the first context; apply the first selected trained similarity function to a query entity feature vector associated with the query entity and to candidate entity feature vectors associated with a plurality of candidate entities, wherein the first selected trained similarity function is configured to compare a subset of features of the query entity feature vector in the first context with corresponding subsets of features of the candidate entity feature vectors in the first context; select a first set of one or more similar candidate entities in the first context from the plurality of candidate entities based on the application of the first selected trained similarity function; and display information associated with the first set of one or more similar candidate entities on the interface.

9. The system according to claim 8, further comprising instructions for performing the following operations: display a second value for the query entity on the interface, the second value being related to a second context; select a second trained similarity function from the plurality of trained similarity functions, wherein, the second trained similarity function is a context-specific machine learning model, and wherein the second trained similarity function is selected based on the second context; apply the second selected trained similarity function to the query entity feature vector associated with the query entity and to the candidate entity feature vectors associated with the plurality of candidate entities, wherein the second selected trained similarity function is configured to compare another subset of features of the query entity feature vector in the second context with corresponding other subsets of features of the candidate entity feature vectors in the second context; select a second set of one or more similar candidate entities in the second context from the plurality of candidate entities based on the application of the second selected trained similarity function; and Display information associated with the second set of one or more similar candidate entities on the interface.

10. The system according to claim 8, wherein, displaying information associated with the first set of one or more similar candidate entities includes: grouping the first set of one or more similar candidate entities according to the respective values of at least one attribute of the first set of one or more similar candidate entities; obtaining a corresponding first value related to the first background for each group in the groups; displaying on the interface information about each group associated with the corresponding first value related to the first background.

11. The system according to claim 10, wherein, the first value related to the first background is a score obtained by a clinical decision support algorithm.

12. The system according to claim 11, wherein, the at least one attribute is the administered treatment.

13. The system according to claim 11, wherein, the at least one attribute is the patient outcome.

14. The system according to claim 10, wherein, the information about each group associated with the corresponding first value related to the first background includes patient outcome statistics.

15. A computer program product, comprising computer program code units, which are adapted to implement the method according to any one of claims 1 to 7 when the program runs on a computer.

Images