Assessment and management system for rehabilitative conditions and related methods

Abstract

To provide systems and methods for measuring patient outcomes in a rehabilitation setting.SOLUTION: A method disclosed herein provides an assessment relating to self-care, an assessment relating to mobility, and an assessment relating to cognition, where the assessments are preselected using the item response theory.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 563,960, filed Sep. 27, 2017, the entire contents of which are incorporated herein by reference.

[0002] Technical Field

[0002] The present disclosure relates generally to rehabilitation techniques, and more particularly to computer-assisted methods for assessing a patient. [Background technology]

[0003] background

[0003] An "outcome measure", also known as an "outcome assessment tool", is a set of items used to account for a patient's changing medical or functional status. One outcome measure is the Functional Independence Measure (FIM®), which provides a method for measuring functional status. The assessment includes 18 items consisting of motor tasks (13 items) and cognitive tasks (5 items). Tasks are scored by clinicians on a 7-point ordinal scale ranging from complete assistance to complete independence. Scores range from 7 (lowest) to 91 (highest) for motor skills and 7 to 35 for cognitive skills. Items include feeding, grooming, bathing, upper body dressing, lower body dressing, toileting, bladder management, bowel management, bed to chair transfer, toilet transfer, shower transfer, mobility (walking or wheelchair level), stairs, understanding, expression, social interaction, problem solving, and memory.

[0004]

[0004] The FIM assessment uses a scoring scale ranging from a score of 1 (reflecting complete assistance) to a score of 7 (reflecting complete independence). A score of 7 is intended to reflect that the patient is completely independent. A score of 1 is intended to reflect that the patient can perform less than 25% of the tasks or requires assistance from two or more people. As a result of this scoring system, many patients who improve in a free-standing inpatient rehabilitation facility or inpatient rehabilitation department within a hospital do not necessarily show an improvement in their outcome scores during their rehabilitation. For example, a patient with a spinal cord injury may significantly improve their fine finger skill motor skills (allowing the patient to use a computer or smartphone) during rehabilitation. However, their FIM score does not improve in this situation. Summary of the Invention [Problem to be solved by the invention]

[0005]

[0005] There is a need for outcome measures that more accurately capture an assessment of a patient's medical or functional status. In addition, there is a need for outcome measures that help patients, such as rehabilitation patients, better identify areas for improvement. [Means for solving the problem]

[0006]

[0006] An "item" is a question or other type of assessment used in an outcome measure. For example, one item on an outcome measure known as the Berg Balance Scale instructs a patient to "stand up. Do not use your hands for support." A "score" is the result of a score or other assessment in response to the assessment of an item. For example, scores for items on the Berg Balance Scale are as follows: a score of 4 reflecting that the patient can stand and stabilize themselves independently without using their hands, a score of 3 reflecting that the patient can stand independently using their hands, a score of 2 reflecting that the patient can stand using their hands after several attempts, a score of 1 reflecting that the patient requires minimal assistance from another person to stand or stabilize, and a score of 0 reflecting that the patient requires moderate or maximal assistance from another person to stand.

[0007]

[0007] Classical test theory is a psychometric theory that involves many predicting outcomes of educational evaluations and psychological tests, such as item difficulty or test-taker ability. Classical test theory is a test theory based on the idea that a person's observed or obtained score on a test is the sum of a true score (a score without errors) and an error score. Classical test theory assumes that each person has a true score T that would be obtained if there were no errors in the measurement. A person's true score is defined as the expected number-correct score across an infinite number of independent administrations of the test. Unfortunately, test users never observe a person's true score, only the observed score X. It is assumed that observed score = true score + some error, that is, X = T + E, where X is the observed score, T is the true score, and E is the error. The reliability of the observed score X of a test, i.e., the overall consistency of the measurement, is defined as the ratio of the variance of the true score to the variance of the observed score. Since the variance of the observed scores can be shown to be equal to the sum of the variance of the true scores and the variance of the error scores, this formulates the signal-to-noise ratio and the reliability of a test score increases when the proportion of the error variance of the test score decreases and vice versa. Reliability is equal to the proportion of the variance of the test score that can be explained if the true score is known. The square root of reliability is the correlation between the true score and the observed score. An estimate of reliability can be obtained by various means such as parallel testing or a measure of internal consistency known as Cronbach's alpha coefficient. Cronbach's alpha can be shown to provide a lower limit for reliability, and therefore the reliability of a test score in a population will always be higher than the value of Cronbach's alpha in that population.

[0008] overview

[0008] The problem of accurately measuring improvement in rehabilitation patients is solved by developing an outcome assessment that incorporates factor analysis and item response theory.

[0009]

[0009] The problem of measuring improvement in a rehabilitation patient is solved by asking the patient a series of questions and returning domain-specific and / or overall scores.

[0010]

[0010] The problem of improving care for rehabilitation patients is solved by predicting domain-specific and / or overall scores on patient outcome measures and providing clinical intervention when the domain-specific and / or overall scores fall below the predicted scores.

[0011] drawing

[0011] The accompanying drawings set forth in detail the features of the present techniques, the techniques together with their objects and advantages can be best understood from the following detailed description taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0012] [Figure 1] 1 shows a flow diagram of a particular exemplary method for preparing a preliminary outcome measurement. [Diagram 2]

[0013] 1 shows a flow diagram for electronically collecting item scores in a preliminary outcome measure. [Diagram 3]

[0014] 1 shows an exemplary scoring method for the IRT compared to the FIM® score known in the prior art. [Figure 4]

[0015] 1 shows an exemplary plot of certain data regarding patient scores in the self-care, cognition, and mobility domains. [Diagram 5]

[0016] It further shows the patient's current and expected functional status for each of the items / tasks in the self-care domain. [Figure 6A]

[0017] A “FIM Explorer” section with functionality that allows clinicians to select and / or set goals for each FIM-specific task. [Figure 6B]

[0017] A "FIM Explorer" section with functionality that allows the clinician to select and / or set goals for each FIM-specific task. [Figure 7]

[0018] A comparison chart is shown. [Figure 8]

[0019] 1 shows various plots of the self-care domain compared to the patient's FIM score. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Detailed Description

[0020] A "bi-factor model" is a structural model in which items cluster on specific factors while simultaneously loading on a general factor.

[0014]

[0021] The term "categorical" is used to describe response options that have no explicit or implicit order or ranking.

[0015]

[0022] The Comparative Fit Index (CFI) compares the performance of a constructed structural model against the performance of a model that does not assume relationships between variables.

[0016]

[0023] A "composite structure" is a CFA structural model in which at least one item loads on multiple factors.

[0017]

[0024] "Confirmatory factor analysis" (CFA) is a type of factor analysis used when psychometricians know how latent traits and items should be grouped and related. A structural model is developed and fit to the data. The goal of this model is to achieve a good fit to the data.

[0018]

[0025] A "constraint" is a restriction placed on a model for mathematical stability or the application of content area theory. For example, if two factors in confirmatory factor analysis are expected to have no relationship between them, a constraint on their correlation (requiring it to be equal to 0.00) can be added to the model.

[0019]

[0026] A "continuous" variable is one that is measured without categories, such as time, height, weight, etc.

[0020]

[0027] "Covariates" are variables in the model that are not on measurement but may still have some explanatory power. For example, in rehabilitation studies, it may sometimes be useful to include covariates for age, sex, length of stay, diagnostic group, etc.

[0021]

[0028] "Dichotomous" refers to a response option that is an ordinal with two categories (e.g., low vs. high), or it may refer to correctly scored items vs. incorrectly scored items, which is also conceptually an ordinal response with two categories.

[0022]

[0029] "Differential item functioning" (DIF) is a measure in item response theory of how parameter estimates may behave differently across groups (within different samples) or across observations (over time).

[0023]

[0030] "Difficulty" is, in item response theory, the required minimum level of a latent trait required to respond in a particular way. In dichotomous response measures, there is a single difficulty level (i.e., the minimum level of the latent trait that raises the probability of responding correctly to 50% or higher). In polytomous response measures, "difficulty" is better expressed as "severity", since there are usually no right or wrong answers. In polytomous response measures, the number of estimated difficulty levels is k-1, where k is the number of response options. These difficulty levels represent the level of the latent trait required to endorse the next higher category. They are sometimes called thresholds.

[0024]

[0031] "Dimensionality" refers to the number of latent traits that a measure addresses: a measure that records one trait is said to be unidimensional, while a measure that records several traits is said to be multidimensional.

[0025]

[0032] "Discrimination" is the ability of a test to separate people with high ability on a latent trait from those with low ability. Similarly, discrimination indicates the magnitude of the relationship between items and the latent trait. Discrimination is conceptually very similar to factor loadings and can be mathematically converted to factor loadings.

[0026]

[0033] "Agree" means to select a response option.

[0027]

[0034] An "equality constraint" is a mathematical requirement in item response theory and confirmatory factor analysis to constrain discriminatory power or factor loadings to be equal when only two items load on a factor.

[0028]

[0035] "Equating" refers to the use of item response theory to obtain similarity between scores on various measures that record levels of the same latent trait. Equating can also be used to compare alternative forms of the same measurement.

[0029]

[0036] "Error" refers to a term used to describe the amount of uncertainty surrounding a model. Models with parameter estimates that are very close to the observed data will have a small amount of error, while those that are very different will have a large amount of error. Error can also indicate the amount of uncertainty surrounding a particular parameter estimate itself.

[0030]

[0037] "Estimation" refers to the statistical process of deriving parameter estimates from data. These procedures can be performed using specialized psychometric software known in the art.

[0031]

[0038] "Exploratory factor analysis" is a type of factor analysis that clusters items according to their correlations. Exploratory factor analysis is often done without any direction from the analyst, except for how many factors to extract. The groupings are then "rotated." Rotation methods attempt to find factor loadings that exhibit a simple structure by ensuring that factor loadings are pushed toward -1.00, 0.00, or 1.00.

[0032]

[0039] A "factor" refers to a latent trait in factor analysis. Unlike latent traits in item response theory, factors do not usually have scores associated with them.

[0033]

[0040] "Factor analysis" is a statistical method for determining the strength and direction of relationships between factors and items. The data on which factor analysis is based are correlations between items. Factor analysis can accommodate ordinal or continuous data, but not unordered categorical. Scores can be calculated from factor analysis, but IRT scores are more reliable. It can be exploratory or confirmatory.

[0034]

[0041] "Factor correlation" refers to the correlation between two factors. A CFA model with correlated factors is called "oblique."

[0035]

[0042] "Factor loading" refers to the magnitude of the relationship between an item and a factor in factor analysis. Although it is not mathematically the same as a correlation, its scale and interpretation are similar; values ​​range (usually) from -1.00 to 1.00. Strong negative factor loadings indicate a strong inverse correlation between the item and the latent trait, whereas strong positive loadings have the opposite interpretation. A factor loading of 0.00 indicates the absence of any relationship.

[0036]

[0043] "Goodness-of-fit statistics" or "goodness-of-fit indices" refer to metrics used to quantify how well a model performs. Popular goodness-of-fit metrics, from confirmatory factor analysis, and structural equation modeling, include the root mean square error (RMSEA), comparative fit index (CFI), Tucker-Lewis index (TL1), and weighted root mean-square residual / standardized root mean-square residual (WRMR / SRMR).

[0037]

[0044] "General factor" refers to the factor on which all items load in the bifactor model.

[0038]

[0045] The "Graded response model" (GRM) is an extension of the two-parameter logistic model that allows for ordinal responses. Instead of only one difficulty level, the graded response model yields k-1 difficulties, where k is the number of response categories.

[0039]

[0046] A "hierarchical model" is a structural model in which latent traits load on other latent traits to form a hierarchy.

[0040]

[0047] "Higher-order / lower-order factors" are a type of latent variable on which higher-order factors load in a hierarchical model.

[0041]

[0048] "Index" is a term used to refer to a goodness of fit index / statistic (e.g., a comparative goodness of fit index) or as a synonym for "measurement."

[0042]

[0049] An "item" refers to a question, task, or score on a measurement that is addressed by a respondent or a representative of the respondent (eg, a clinician).

[0043]

[0050] An "item characteristic curve (ICC)" is a graph that plots the probability of selecting various response options given the level of a latent trait. It is sometimes called a "trace line."

[0044]

[0051] "Item response theory" (IRT) is a collection of statistical models used to determine the behavior of items, which obtain scores according to a structural model. In one form of use, IRT uses the response patterns of all people in a sample to derive estimates of those items and scores. IRT uses data that is ordinal or categorical. Mathematically speaking, item response theory uses characteristics of items and people to predict the probability that a person will select a particular response option for a given item.

[0045]

[0052] An "IRT score" is a score specific to IRT analysis that is given on a standardized scale. An IRT score is similar to a z-score. In some IRT scoring methods, a score of 0.00 implies having an average level of the latent trait, a large negative score implies a low level of the latent trait, and a large positive value implies a high level of the latent trait.

[0046]

[0053] "Latent traits" are similar to factors in factor analysis but are more frequently used in item response theory. A latent trait is what a set of related items is supposed to measure. They can be used interchangeably with factors, domains, or dimensions.

[0047]

[0054] "Latent variables" is the term for variables that are not directly measured. Latent variables contain latent traits.

[0048]

[0055] "Linking" is similar to equating but for item parameter estimates rather than scores.

[0049]

[0056] "Loading" is a verb used to describe what an item does on a factor. For example: "In this model, item 4 loads on both the local dependent factor as well as the general factor."

[0050]

[0057] "Local dependence" (LD) is a violation of the local independence assumption in which items are related for some reason other than the latent trait. If local dependence appears to be in the data, it can be addressed by modeling correlations between items or by creating a locally dependent factor. This can be due to a number of reasons, including similar wording, nearly identical content, and the location of the item on the measure (this last example occurs frequently with the last items on a long measure).

[0051]

[0058] "Local independence" refers to an assumption in psychometrics that states that item behavior is due solely to latent traits in the model and item-specific error. When items violate this assumption, they are said to be locally dependent.

[0052]

[0059] "Manifest variables" is a general term for variables that are measured directly, and includes items, covariates, and other such variables.

[0053]

[0060] "Measure" refers to a collection of items that attempts to measure the level of some latent trait. It can be used interchangeably with assessment, test, questionnaire, index, or scale.

[0054]

[0061] A "model" is a combination of a response model and a structural model in psychometrics. Roughly speaking, a model represents both the form of the data and how the data recorded in the model's variables should be related.

[0055]

[0062] "Model fit" is a term used to indicate how well a model represents the data. This can be done in a variety of ways, such as comparing the observed data to the predictions made by the model or comparing the selected model to a null model (a model in which none of the variables are related). The metrics used to evaluate model fit are called goodness-of-fit statistics.

[0056]

[0063] "Multidimensional" is a term used to describe measures that record multiple latent traits.

[0057]

[0064] "Multigroup analysis" refers to the process in IRT whereby the sample can be divided into different groups and parameter estimates specific to each group can be estimated.

[0058]

[0065] "Nominal models" are similar to graded response models except for items with response options that are categorical rather than ordinal.

[0059]

[0066] "Oblique" is an adjective used to describe factors that are correlated.

[0060]

[0067] "Order" refers to the way in which an item records data, for example possible responses to an item are a series of categories ordered from low to high or high to low.

[0061]

[0068] "Orthogonal" refers to factors that are restricted to zero correlation.

[0062]

[0069] "Parameter estimates" are statistically derived values ​​estimated by psychometric software. It is a general term that may include item discrimination power, factor loadings, or factor correlations, etc.

[0063]

[0070] "Path diagrams" are diagrams intended to show relationships between items, latent traits, and covariates. In a path diagram, rectangles / squares represent observed variables (i.e. items, covariates, or any modeled variables for which there is explicitly recorded information), ellipses / circles represent latent traits or variables for which there is no explicitly recorded information, single arrows reflect unidirectional relationships (as in regression), and double arrows reflect correlations / covariances between modeled variables.

[0064]

[0071] "Multiple-valued" is a term for items that have multiple response options, which can be ordinal or categorical.

[0065]

[0072] A "pseudo-bifactor model" is a bifactor model in which not all items cluster onto a specific factor. Instead, some items may only load onto a general factor.

[0066]

[0073] A "psychometrician" is a type of statistician who specializes in measurement.

[0067]

[0074] "Psychometric" refers to statistics used in creating or expressing measurements.

[0068]

[0075] The "Rasch model" is a response model that assumes that all item discrimination powers are equal to 1.00. It is not typically used unless this assumption is true or nearly true. This assumption makes it easier to interpret scores and difficulty levels, and allows item response theory to be used for (relatively) small sample sizes, but it is very rare that all item discrimination powers behave identically. It is a simplified case of the two-parameter logistic model that allows item discrimination powers to vary. Therefore, the Rasch model is sometimes called the one-parameter logistic model (1PL). It can be used when the response is dichotomous.

[0069]

[0076] A "responder" is a person who answers items on a measurement.

[0070]

[0077] A "response" is a respondent's answer to an item.

[0071]

[0078] "Response categories" are the different options a respondent can select as a response to an item. If the item results in a dichotomous response, the data is recorded as true (1) or false (0).

[0072]

[0079] "Response model" refers to the way in which a measurement model handles the format of responses in item response theory. Popular response models include the Rasch model, the two-parameter logistic model, the three-parameter logistic model, the graded response model, and the nominal model.

[0073]

[0080] A "response pattern" is a sequence of numbers representing a respondent's answer to each question in a measurement.

[0074]

[0081] The root mean square error of fit (RMSEA) is a goodness-of-fit statistic in applied psychometrics. It measures the closeness of expected data (the data the model produces) to observed data. An RMSEA of less than 0.08 is usually desirable, although some practitioners prefer an RMSEA of less than 0.05.

[0075]

[0082] A "score" is a numerical value intended to represent the level or amount of a latent trait possessed by a respondent. Classical test theory calculates a score as the sum of item responses, whereas item response theory uses both the response pattern and the quality of the items to estimate the score.

[0076]

[0083] "Sigmoid" (literally "S-shaped") is an adjective that is sometimes used to describe the shape of the TCC or ICC of 2PL items.

[0077]

[0084] A "simple structure" is a structural model in which all items load on one factor at a time.

[0078]

[0085] A "specific factor" is the factor on which an item set loads in a bifactor model.

[0079]

[0086] "Structural equation modeling" (SEM) is an extension of confirmatory factor analysis (CFA) that allows for relationships between latent variables, such as latent traits. When all latent variables in the model are latent traits, SEM and CFA are often used interchangeably.

[0080]

[0087] A "structural model" is a mathematical description of a system of hypotheses about the relationships between latent traits and items, depicted as a path diagram.

[0081]

[0088] "Total score" is the score calculated by adding up the numerical values ​​of all responses of a measure.

[0082]

[0089] "Sum score conversion" (SSC) is a table showing the relationship between sum scores and IRT scores.

[0083]

[0090] A "test characteristic curve" (TCC) is a plot of the relationship between total score and IRT score.

[0084]

[0091] A "testlet" is a small set of items that measure some component of an overall latent trait. When the definition of the latent trait is clearly specified in advance, creating a measure composed of testlets can result in a score that is more easily interpreted.

[0085]

[0092] "Threshold": See "Difficulty".

[0086]

[0093] "Tucker-Lewis Index" (TLI): A goodness-of-fit index that compares the performance of a constructed model to the performance of a model that makes no assumptions about relationships between the variables. A good-fitting model typically has a TLI above 0.95.

[0087]

[0094] "Three-parameter logistic model" (3PL): An extension of the two-parameter logistic model that also includes a "guessing" parameter. For example, in a multiple-choice item with four options, even a random guess would still have a 25% chance of answering correctly. The 3PL allows for this non-zero probability of answering correctly. It is used when the response is dichotomous.

[0088]

[0095] "Trace line": See "item characteristic curve."

[0089]

[0096] The "two-parameter logistic model" (2PL) is similar to the Rasch model but allows item discriminatory power to vary. It can be used when item responses are dichotomous.

[0090]

[0097] "Unidimensional" is a term used to describe measures that record only one latent trait.

[0091]

[0098] "Variable" is a general term used to describe a set of directly recorded data (manifest) or indirectly recorded data (latent) that measure a single thing.

[0092]

[0099] WRMR / SRMR (weighted root mean-square residual / standardized root mean-square residual) is a goodness-of-fit statistic that measures the size of the model residuals. Residuals are the difference between observed data and what the model predicts. Typical WRMR values ​​are recommended to be less than 1.00, but this recommendation may vary based on sample size or model complexity. WRMR is used when there is at least one categorical variable in the model, whereas SRMR is used when all variables are continuous.

[0093]

[0100] FIG. 1 illustrates a flow diagram of a particular exemplary method for preparing a preliminary outcome measure 100 for inclusion in an electronic medical record.

[0094]

[0101] At 101, an item set 200 is identified. In one embodiment, clinicians may be asked to provide their input for appropriate items for inclusion in item set 200 based on their training, education, and experience. Examples of clinicians may include physicians, physical therapists, occupational therapists, speech pathologists, nurses, and PCTs. The items in item set 200 may stem from a variety of outcome measures known in the art.

[0095]

[0102] At 102, the items in item set 200 may be grouped into one or more of a number of areas, referred to as "areas," related to treatment or clinical outcomes. These areas may be identified by a clinician. In one embodiment, the items in item set 200 may be grouped into three areas entitled "Self-care," "Mobility," and "Cognition." It should be understood that other groupings of additional and / or alternative areas are possible.

[0096]

[0103] At 103, relevant analysis steps can be performed. For example, the frequency with which an item in item set 200 is used in conventional practice to evaluate patients in a medical institution can be analyzed. Or the cost of equipment to perform the item can be evaluated. Clinical literature can be reviewed to identify outcome measures having items in item set 200 that are psychometrically acceptable and clinically useful. For example, the reliability and validity of an outcome measure having one or more items in item set 200 can be reviewed to ensure that it is psychometrically acceptable. As another example, each outcome measure and / or item can be reviewed to ensure that it is clinically useful. For example, there are many items available in the literature that are used to test a person's balance, but not all of them are appropriate for a patient in the context of rehabilitation. Based on these and similar factors, the initial item set can be narrowed to reduce the burden on patients, clinicians, and other medical personnel.

[0097]

[0104] At 104, a plurality of revised items are collected. A pilot study can be conducted on the plurality of items. The pilot study can be conducted by having clinicians evaluate patients on the revised items in a standardized manner, such that each clinician evaluates each patient using all of the revised items. In another embodiment, the clinician can select which items to use to evaluate the patient based on the patient's particular clinical characteristics. The decision regarding the selection of particular items can be based on information obtained during the patient's rehabilitation stay, for example, during an inpatient admission evaluation. The items can be administered at least twice during the patient's inpatient stay to determine the patient's improvement. The pilot study can be facilitated using an electronic medical record system, whereby the clinician enters the item scores into the electronic medical record.

[0098]

[0105] At 105, an analysis of the pilot study can be performed. For example, items that take too much time for a clinician to administer to a patient can be eliminated.

[0099]

[0106] At 106, the original paper-based items for the preliminary outcome measures 100 are implemented into the electronic medical record. Individual item-level scores can be recorded electronically. For example, the items to be implemented can be items that are the result of an analysis of a pilot study at 105, although a pilot study analysis is not required. Alternatively, the items in the preliminary outcome measures 100 can be implemented in an electronic system, such as a database, external to the electronic medical record. In one embodiment, the external electronic system can communicate with the electronic medical record using methods known in the art, such as database connection techniques. At 107, the items for the preliminary outcome measures 100 are programmed into the EMR using known methods to allow clinicians to enter their scores into the electronic medical record. In one embodiment, the EMR can provide prompts to alert, remind, and / or request that the clinician enter specific scores for specific items of the preliminary outcome measures 100. Such prompts can improve the reliability and completeness of clinical data entry into the EMR.

[0100]

[0107] 1 refers to selecting particular items from various outcome measures, it should be understood that a similar approach can be taken with respect to selecting the outcome measures themselves. For example, at 104, instead of selecting which items to use in assessing the patient, the entire outcome measure can be selected or ignored.

[0101]

[0108] FIG. 2 shows a flow diagram for electronically collecting ratings for items in the preliminary outcome measure 100. At 201, a clinician performs an assessment on a patient. In one embodiment, the clinician may perform the assessment using all items in the preliminary outcome measure 100. In another embodiment, the clinician may perform tests or items in the preliminary outcome measure 100 that are specific to the clinician's scope of practice. For example, a physical therapist may perform tests or items in the preliminary outcome measure 100 that are specific to physical therapy. In yet another embodiment, the clinician may use his or her clinical judgment based on his or her education, training, and experience to identify the tests in the preliminary outcome measure 100 that are most relevant to the patient. If the patient is very ill or has very limited function, the clinician knows not to perform certain items. For example, the clinician may not require a newly quadriplegic patient to perform a test that requires the patient to walk.

[0102]

[0109] The assessment may be an initial assessment conducted upon admission of the patient to the hospital or shortly thereafter. In one embodiment, each patient receiving care for a period of time, such as a month or a year, is assessed. In another embodiment, a majority of patients receiving care for a period of time are assessed. In yet another embodiment, multiple patients are assessed. In other embodiments, the patient population may be refined to include only inpatients, only outpatients, or a combination thereof.

[0103]

[0110] In various embodiments, certain tests within the preliminary outcome measures 100 may be performed once at or shortly after admission and again at or shortly before discharge. In various embodiments, certain tests within the preliminary outcome measures 100 may be performed weekly. In various embodiments, certain tests within the preliminary outcome measures 100 may be performed multiple times per week, such as twice per week.

[0104]

[0111] In one embodiment, the evaluation may be performed at a centralized location specific to performing the evaluation. The evaluation may be performed by a set of clinicians whose specific role is to perform the evaluation. The centralized location, with qualified staff and sufficient equipment to objectively evaluate the functional status of the patient, may be operated by a standardized process in a controlled and secure environment. In one embodiment, a clinician issues an order for the evaluation to a lab technician. For example, a clinician (physiotherapist, therapist, nurse, or psychologist) orders a specific test (such as gait and balance tests) or a group of tests. The test order may be sent electronically to the assessment unit ("AAL") and a hard copy may be printed for the patient. Once the AAL is ready, the patient may be transported to the AAL with assistance if necessary. Staff such as a technician performs the ordered tests. The results of the tests may be recorded and entered / transmitted into the electronic medical record. The results of the tests may be reviewed by the clinician to modify the care plan if necessary. This process may reduce the time it takes for a clinician to know how to perform the tests. One advantage of AAL is that other clinicians do not have to learn how to administer different tests each time a new test is introduced. Clinicians only need to learn how to read the results of the tests, not how to administer them. Tests can be administered by properly trained and qualified personnel. Clinical staff can focus on treatment, not on evaluation. More treatment sessions or additional time can be provided to improve results. Test equipment is kept centralized to reduce the demand and maintenance costs of multiple units. Tests can be performed in a well-controlled, standardized, and safe environment. Technologists can utilize standardized procedures to avoid potential bias caused by the rater (tendency for high scores to indicate improvement over time), thereby improving the quality of the data.

[0105]

[0112] The scores for each assessment may be stored in the EMR. For example, the scores for each assessment may be stored in a preliminary rating dataset 150. At 202, data analysis and cleanup may be performed on the preliminary rating dataset 150 to improve data quality. For example, outlying scores may be removed from the preliminary rating dataset 150. Data patterns in the preliminary rating dataset 150 from the same clinician may be examined and cleaned using methods known in the art. Scores in the preliminary rating dataset 150 of patients who show a large increase in scores from "dependent" to "independent" may also be discarded. Questionable data for a particular assessment may also be discarded.

[0106]

[0113] At 203, the rating data can be further extracted, cleaned, and prepared using methods known in the art to obtain data in a form that can be queried and analyzed. The data can be reviewed for quality, and various data options can be performed on the preliminary rating dataset 150, such as data pivoting, data merging, and creating a data dictionary. The data from the preliminary rating dataset 150 can be stored in different forms, such as in an EMR or in a data warehouse, for further analysis. Those skilled in the art will appreciate that there are many ways to organize the data in the preliminary rating dataset 150 for analysis. In one embodiment, the preliminary rating dataset 150 is organized such that item ratings can be utilized for analysis across multiple dimensions, such as time period and patient identification.

[0107]

[0114] Once the preliminary rating data set 150 is prepared for analysis, a psychometric evaluation can be performed on the preliminary rating data set 150. A psychometric evaluation evaluates how well the outcome measure actually measures what it is intended to measure. A psychometric evaluation can include a combination of classical test theory analysis, factor analysis, and item response theory to evaluate the preliminary rating data set 150 for various aspects that can include reliability, validity, responsiveness, dimensionality, item / test information, specific item functioning, and equating (score crosswalk). In one embodiment, a classical test theory analysis can be used to consider the reliability of items in the preliminary outcome measure 100 and how the preliminary outcome measure 100 and the domains work together.

[0108]

[0115] Item Reduction. The item reduction step 152 assists in reducing items from the preliminary outcome measure 100 that do not perform as expected. Factors may include reliability, validity, and responsiveness (also known as sensitivity to change). The purpose of the item reduction step 152 is to eliminate potential item content redundancy from the items in the preliminary outcome measure 100 down to a minimal subset of items in the IRT outcome measure 180 without sacrificing the psychometric properties of the data set. The item reduction step 152 may be performed using a computer or other computing device, for example using a computer program 125. The computer program 125 may be written in the R programming language or another suitable programming language. The computer program 125 provides options to allow the user to specify the number of items desired (as well as options for including specific items) and calculates reliability estimates of Cronbach's alpha coefficients for all possible combinations of items within those user-defined constraints. Acceptable ranges for Cronbach's alpha coefficients may also be defined within the computer program 125. Additionally, the computer program 125 can construct and execute a statistical modeling program such as Mplus (Muthen & Muthen, Los Angeles, CA, http: / / www.statmodel.com) to determine the fit of a one-factor confirmatory factor analysis (CFA) model for each reduced subset of items 155.

[0109]

[0116] The computer program 125 may be used to analyze some of the outcome measures explored for unidimensional subsets of 4 to 8 items included in the preliminary outcome measures 100 (FIST, BBS, FGA, ARAT, and MASA) and with Cronbach's alpha reliability between 0.70 and 0.95. Using these constraints, the number of items in many measures can be significantly reduced. For example, a measure can be reduced to at least half its original length while maintaining good psychometric properties. The resulting subset of items serves as components of a confirmatory factor analysis (CFA). In one embodiment, certain items, such as items from the FIM®, may not be included in the item reduction process. In one embodiment, the item reduction step 152 may be performed multiple times. For example, the item reduction step 152 may be performed for each outcome measure included in the preliminary outcome measures 100.

[0110]

[0117] In the item reduction step 152, the computer program 125 determines the extent to which items relate to each other. The computer program 125 can determine the extent to which items in the outcome measures in the preliminary outcome measure 100 relate to each other. In one embodiment, items relate to each other within an outcome measure if they have highly correlated responses. The analysis can begin by providing an initial core set of items, and the number of items can be determined by clinician input based on correlation between pairs of items. For example, the computer program 125 can determine how item A relates to item B, both of which are in the same outcome measure. If there is a high correlation, both items A and B are included in the core set. The computer program 125 can then determine how a new item C correlates to the itemset {A,B}. If there is a high correlation, item C is included in the core set. This method can be repeated with additional items D, E, F, etc. As above, the program evaluates the reliability (Cronbach's alpha) of all possible subsets of items. The program correlates responses of one itemset with responses of a second itemset. Cronbach's alpha is known in the art, but a brief example is provided here. The information used in calculating Cronbach's alpha is the correlation between all possible split-halves within a subset of items. For example, using three items {A,B,C}, Cronbach's alpha averages the correlations of A vs. BC, B vs. AC, and C vs. AB. In other words, it calculates the correlations between all pairs of unique subsets of a set. The purpose of correlation analysis is to help ensure that items are measuring the same underlying construct and improve reliability.

[0111]

[0118] Table 1 lists an example output of the item reduction step 152 for the Berg Balance Scale ("BBS") outcome measure setting the sample size equal to five items. The numbers in each cell in the "Item" column reflect the number of the question on the BBS (1: sit unsupported, 2: reposition-sit to stand, 3: reposition-stand to sit, 4: transfer movement, 5: stand unsupported, 6: stand with eyes closed, 7: stand with legs closed, 8: stand in tandem, 9: stand on one leg, 10: rotate (foot fixed)). Each reduced subset 155 is shown with an associated Cronbach's alpha value. The first reduced subset has the highest Cronbach's alpha of the reduced subsets in Table 1. In one embodiment, the reduced subset with the highest Cronbach's alpha is used as the first reduced subset for the CFA step 160, which is described in more detail below.

[0112] [Table 1]

[0113]

[0119] Confirmatory Factor Analysis. Factor analysis is a statistical method used to determine the number of underlying dimensions contained within a set of observed variables and to identify a subset of variables that correspond to each of the underlying dimensions. The underlying dimensions can be called continuous latent variables or factors. The observed variables (also known as items) are called indicators. Confirmatory Factor Analysis (CFA) can be used in situations where the dimensionality of a set of variables for a given population is already known from previous studies. CFA can be used to examine whether the established dimensionality and factor loading patterns fit new samples from the same population. This is the "confirmatory" aspect of the analysis. CFA can also be used to examine whether the established dimensionality and factor loading patterns fit samples from new populations. In addition, by examining the variances and covariances / correlations of factors, factor models can be used to study individual characteristics. Factor variance indicates the degree of heterogeneity of the factors. Factor correlation indicates the strength of association between the factors.

[0114]

[0120] Confirmatory factor analysis (CFA) can be performed using Mplus or other statistical software to examine how well items fit statistically within a pre-specified factor structure. CFA is characterized by constraints on factor loadings, factor variances, and factor covariances / correlations. CFA requires at least m^2 constraints, where m is the number of factors. CFA may include correlation residuals, which may be useful to represent minor factor effects on variables. A set of background variables may be included as part of the CFA.

[0115]

[0121] Mplus can estimate CFA models with background variables and CFA models for a single group or multiple groups. Factor indicators in a CFA model can be continuous, censored, binary, ordered categorical (ordinal), counts, or a combination of these variable types. When factor indicators are all continuous, Mplus has seven choices of estimation methods: maximum likelihood (ML), maximum likelihood with robust standard errors and chi-square (MLR, MLF, MLM, MLMV), generalized least squares (GLS), and weighted least squares (WLS), also known as ADF. When at least one factor indicator is binary or ordered categorical, Mplus has seven choices of estimation methods: weighted least squares (WLS), robust weighted least squares (WLSM, WLSMV), maximum likelihood (ML), maximum likelihood with robust standard errors and chi-square (MLR, MLF), and unweighted least squares (ULS). When at least one factor indicator is censored, unordered categorical, or count, Mplus has six choices of estimation methods: weighted least squares (WLS), robust weighted least squares (WLSM, WLSMV), maximum likelihood (ML), and maximum likelihood with robust standard errors and chi-square (MLR, MLF).

[0116]

[0122] Using a reliable subset of items from the measurement reduction step, a model can be defined in statistical software such as Mplus that assumes that all items in a domain are related to each other. The model can also measure specific constructs under the preview of the domain. For example, a subset of all items taken from the self-care measurement can be assumed to measure self-care, but also one of balance, upper limb function, and swallowing. Building a model in this manner allows for measuring both the entire domain (e.g., self-care) as well as a set of interrelated constructs that make up the domain (e.g., balance, UE function, and swallowing, which are constructs that make up self-care). Given the data, the structure of the model implies a set of expected correlations between each pair of items. However, those (maybe) correlations can be calculated directly from the data. Those correlations are the observed correlations. The validity of the constructed model, called "model fit" in statistics, can be determined using the root mean square error (RMSEA), which is a measure of the difference between the observed correlations and the expected correlations. In a preferred embodiment, if such difference values ​​are small (eg, less than 0.08) then the model has an acceptable fit.

[0117]

[0123] After applying CFA step 160 to the reduced subset 155, the output of CFA step 160 may include factor loadings, including a general factor loading. The general factor loading may range from -1 to 1, with a general factor loading value between 0.2 and 0.7 indicating whether a factor assesses the associated item well. The output of CFA step 160 may provide additional factor loadings for each item. In one embodiment, each item may have a factor loading for each sub-domain. For example, each item may have a factor loading value for balance, a factor loading value for upper extremity, a factor loading value for swallowing, and factor loading values ​​for each of the other sub-domains. In one embodiment, the factor loading value is non-zero if the item is associated with a sub-domain.

[0118]

[0124] In certain instances, applying the CFA step 160 to the reduced subset 155 may cause problems that require the selection of a new reduced subset 155. General factor loading values, for example, above 0.7, or especially values ​​close to 1.0, indicate redundancy. The way items are scored, for example based on the Action Research Arm Test (ARAT) outcome measures, necessarily requires too high reliability. A patient who achieves the highest score on the first (most difficult) item is allowed to obtain a score of 3 on all subsequent items on the scale. If the patient obtains a score below 3 on the first item, the second item is evaluated. This is the easiest item, and if the patient scores 0, the patient is unlikely to obtain a score higher than 0 on the remaining items, and 0 is given for the other items. This scoring method requires too high reliability. In other instances, if the factor loading values ​​are above 1, such values ​​reflect that the item pair has a (not possible) negative variance, and therefore the CFA step 160 needs to be performed on a new reduced subset 155. A new reduced subset 155 can be selected from the group of reduced subsets generated by item reduction step 152. For example, the new reduced subset with the next highest Cronbach's alpha can be selected and the CFA step 160 can be applied to the new reduced subset.

[0119]

[0125] In addition, during the process of performing CFA step 160, it may be apparent that items designated by the clinician as being included in one subdomain should be moved to a different subdomain (discussed further below) to improve the fit of the model used to generate IRT outcome measure 180. For example, during development of the embodiments described herein, items identified by the clinician as relating to "strength" were initially placed in the self-care domain. However, when performing CFA step 160, it was determined that those items did not fit the model. Moving those items to the "upper extremity function" subdomain improved the fit of the model.

[0120]

[0126] Table 2 below shows the goodness-of-fit statistics for a one-factor CFA including groups 1-10 as shown in Table 1. In the CFA step 160, an evaluation can be made as to whether the goodness-of-fit statistics listed in Table B meet the usual "good fit" criteria. In one embodiment, these criteria are RMSEA<0.08, CFI>0.95, TLI>0.95, and WRMR<1.00. One skilled in the art will appreciate that other good fit criteria can also be used.

[0121] [Table 2]

[0122] Although the above example is shown with respect to only one outcome measure, the Berg Balance Scale, it should be understood that the CFA step 160 is applied to each outcome measure within the preliminary outcome measures 100 .

[0123]

[0127] Item Response Theory. In one embodiment, the IRT outcome measure 180 can be configured to include multiple high-level domains. For example, the IRT outcome measure 180 can be configured to include a "Self-care" domain (including items determined to reflect the patient's ability to perform self-care), a "Mobility" domain (including items determined to reflect the patient's ability to be mobile), and a "Cognition" domain (including items determined to reflect the patient's cognitive abilities). Within each high-level domain, specific assessment domains, also referred to as "factors" or "clusters," can be identified. Table 3 reflects exemplary assessment domains associated with each high-level domain.

[0124] [Table 3]

[0125]

[0128] Because the measurement objectives of the IRT outcome measures 180 include measuring general domains (i.e., self-care, mobility, and cognition) as well as specific assessment domains within those domains, a bifactor structure for each of the domains can be targeted (general and domain-specific factors). The composition of the specific factors can be determined by the content of each item set. For example, items from the FIST, BBS, and FGA can be combined to form a "balance" assessment domain within the self-care domain. An RMSEA < 0.08 criterion was used to assess acceptable fit of the bifactor model to the data (Browne & Cudeck, 1992), and modification indices were also calculated to examine local item dependence and potential improvements to the model, such as additional cross-loadings (i.e., items contributing to several factors).

[0126]

[0129] Item response theory reflects a mathematical model that describes the relationship between a person's ability and item characteristics (such as difficulty). For example, individuals with higher ability are more likely to perform more difficult tasks, which may allow for more tailored interventions based on a series of questions. Other item characteristics may also be relevant, such as the "discrimination power" of an item, which is its ability to separate individuals with high or low levels of a characteristic.

[0127]

[0130] After constructing the CFA models for each of the domains, the final structures can be coded for implementation within an item response theory software package such as flexMIRT (Vector Psychometric Group, Chapel Hill, North Carolina, US). flexMIRT is a multilevel, multidimensional, and multigroup item response theory (IRT) software package for the analysis of items and scoring of tests. To address the ordinal categorical nature of item responses from clinician-rated achievement ratings, a multidimensional graded response model (M-GRM) can be selected. For example, the dimensions can be "self-care," "mobility," and "cognition." The subdomains of "self-care" can be "balance," "upper extremity function," "strength," "positioning," and "swallowing." The subdomains of "mobility" can be balance, wheelchair ("W / C") skills, positioning, bed mobility, and mobility. The subdomains of "cognition" can be "awareness," "arousal," "memory," "speech," and "communication."

[0128]

[0131] However, in a preferred embodiment, sub-areas can be pared down to focus on important sub-areas of ability. For example, in "Self-Care," the pared down sub-areas can be balance, UE function, and swallowing. In "Cognition," the pared down sub-areas can be cognition, memory, and communication. In "Mobility," there may be no sub-areas, or in other words, the sub-areas can all be clustered together.

[0129]

[0132] Analyses can also be multi-group in nature. For example, self-care and mobility samples can be divided into groups determined by the level of balance (sitting, standing, or walking). As another example, cognition samples can be divided into broad diagnostic categories (stroke, brain injury, neurological, or unrelated). In one embodiment, to address the complexity of the model, the Metropolis-Hastings-Robin-Monroe (MH-RM) algorithm (Cai, 2010) can be used for more efficient parameter estimation. MH-RM repeats the following three steps over and over again until the difference between two consecutive cycles falls below a selected criterion: In step 1 (imputation), a random sample of the latent trait is imputed from the distribution implied by the item parameter estimates taken from the previous cycle. If it is the first cycle, the distribution implied by the starting values ​​of the algorithm is used. This imputation can be done using the MH sampler. In step 2 (approximation), the log-likelihood of the imputed data is evaluated. In step 3 (Robin-Monroe update), new parameter estimates for the next cycle are calculated by using a Robin-Monroe filter on the log-likelihood of step 2. Step 1 is then repeated using the information from step 3. The slope can reflect the item discrimination power and the intercept can reflect the item difficulty.

[0130]

[0133] In addition to item slopes and intercepts, a maximum a posteriori (MAP) latent trait score can be calculated for each patient, reflecting the patient's level of performance.

[0131]

[0134] Principal coding for IRT focuses on transforming the mathematical structure chosen after CFA into a structure that can be evaluated using IRT. The data used for the analysis can be, for example, simply the patient's ratings on all items on which the patient was assessed. For consistency, the most recently available data for each patient on each item issued can be used. This consistency has the advantage of placing the patient's scores in a particular frame of reference, i.e., typical discharge levels. The MAP (maximum a posteriori) scoring method can be used, but other scoring methods are known, such as ML (maximum likelihood), EAP (expected a posteriori), or MI (multiple imputation), that can be used instead. In addition, there are various estimation methods that can be used. For example, the marginal maximum likelihood with expectation maximization algorithm (MML-EM) can be used. However, this method can be disadvantageous when dealing with a significant number of dimensions. In the preferred embodiment, the Metropolis-Hastings-Robin-Monroe (MH-RM) estimation is used.

[0132]

[0135] Maximum a posteriori (MAP) scoring requires two inputs: the population score density (usually taken as the standard normal for each dimension), and the IRT parameters for each item that the patient rated. Given what is known about the items and how the patient rated each of the items, multiplying the population density by the IRT function for each item results in what is known as the likelihood, a mathematical expression of the probability of various scores. The location of the maximum of that function is the patient's MAP score.

[0133]

[0136] Sometimes a response option for an item is chosen very rarely, which can cause problems associated with estimating the IRT parameters for that item (implying that the response option may not be necessary). In such cases, the responses can be collapsed into adjacent categories. For example, if an item has responses of {1,2,3,4}, and response 2 is very rarely observed in the data, one can recode the data {1,2,2,3}. It should be understood that the actual number of numbers is not important in an IRT analysis, but instead the order is important.

[0134]

[0137] Group Composition: To allow for more targeted assessment, the IRT analysis used here can be multi-group in nature. For self-care and mobility, patients can be grouped according to balance level (none, sitting, standing, and walking). Similarly, groups can be formed within the cognitive domain according to cognitive diagnosis (stroke, brain injury, neurological, or none). This method can result in multiple test forms that contain only the items appropriate for each patient. For example, such test forms can include test forms such as no balance, sitting balance, standing balance, and no balance constraints for "self-care" and "mobility", and stroke, brain injury, neurological, or no disorders for "cognition". Forms can be adapted according to group membership rather than assessment domain. For example, the patient's balance level can affect which balance measurement items appear on the self-care and mobility domains, while the patient's cognitive diagnosis (if any) can affect which measurements can appear on the form. For example, ABS is only used on the brain injury form of the cognitive measurement, and KFNAP is only used on the stroke measurement.)

[0135]

[0138] Item response theory yields separate scores for each domain. For example, a patient may score 1.2 in the "Self-care" domain, 1.4 in the "Mobility" domain, and 3 in the "Cognition" domain. In one embodiment, these scores may be reported separately to clinicians, patients, and others. In other embodiments, these scores may be combined into a single score. In one embodiment, a score of +1 means that the patient is one logit above the mean. A score of -1 means that the patient is one logit below the mean. Values ​​below -3 and above 3 are highly unlikely, since the mathematical assumption underlying IRT is that scores follow a mean distribution. It should be understood by those skilled in the art that other numbers reflecting standard deviations and logits may be used instead. For example, a score of 3 may mean that the patient is average, so that the score ranges from 0 to 6. As another example, a score of 50 may mean that the patient is average, and a score of +10 may mean that the patient is one logit above the mean, so that the score ranges from 20 to 80.

[0136]

[0139] An example of performing IRT step 170 for the self-care domain is given below. Seven factors are provided to IRT step 170: a self-care factor, a balance factor, a UE function factor, a swallowing factor, a hidden factor for ARAT, a hidden factor to overcome the negative correlation between FIST and FGA outcome measures, and a hidden factor specific to FIST so that FIST is not overweighted in the results. IRT step 170 (using, e.g., MH-RM estimation) returns a discrimination power matrix 172 and a difficulty matrix 174. For example, these matrices can be represented by a slope / intercept formulation, where the slopes reflect item discrimination power and the intercepts reflect item difficulty.

[0137]

[0140] Table 4 shows an exemplary discrimination matrix 172 for the self-care domain of an exemplary IRT outcome measure 180. Column headers a1-a7 in Table 4 show the following: a1: self-care, a2: (ARAT local dependence), a3: upper extremity function, a4: swallowing, a5: balance, a6: (reduced impact of FIST), a7: (inverse correlation of BBS and FGA), with "hidden" factors shown in parentheses. Table 4 lists the slope values ​​for each item in each factor a1-a7. The item names in Table 4 are also reflected in Table 6 in Appendix 1, which lists the items in an exemplary IRT outcome measure 180.

[0138] [Table 4]

[0139] Table 5 shows an exemplary difficulty matrix 174 for IRT outcome measures 180. Table 5 lists the intercept values ​​for each item in each factor d1-d6. The column headers d1-d6 in Table 5 show the following: (d1: self-care, d2: (ARAT local dependency), d3: upper extremity function, d4: swallowing, d5: balance, d6: (reduced impact of FIST)), with "hidden" factors shown in parentheses.

[0140] [Table 5]

[0141] It should be appreciated that a discrimination matrix 172 and difficulty matrix 174 can be prepared for each domain within the IRT outcome measure 180 .

[0142]

[0141] An exemplary score / probability response can be plotted, with the X-axis reflecting the score and the Y-axis reflecting the probability of response. The product of the curves results in a likelihood curve that looks somewhat like a bell curve. The peak of the curve can be used as the patient's score.

[0143] Input from Therapist to Ensure Clinical Relevance. Each item may be labeled with the cluster that best represents its role within the IRT outcome measure 180. This labeling may be done by the clinician based on their education, training, and experience. For example, the clinician may label items that measure balance, such as items that test function in a sitting position that are included in the "Mobility" domain and "Balance" factor of Table 1.

[0144]

[0143] Because item selection (retention or removal) in the item reduction step of the analysis is predicted based on psychometric and statistical evaluation, in one embodiment, clinical experts can review the content of items featured in the reduced item set for further feedback. For example, a panel of clinicians can be surveyed for input on whether items should be added to or removed from the subsets taken from each of the complete outcome measures. The clinician input can be used to build the final model for each domain to help ensure that the items retained are psychometrically valid and clinically relevant.

[0145]

[0144] Remodeling to derive final item set. After the pre-arranged item set is prepared, taking into account both psychometric evaluation and clinical judgment, CFA and IRT steps can be performed. Items that are not selected and have high clinical support can be added back into the model, while items that are included and have low support can be removed. Then, the root mean square error (RMSEA) calculated during CFA can be used to evaluate the fit of the model to the data, and new item parameter estimates and latent trait scores were calculated during IRT analysis. Table 6 in Appendix 1 of this specification lists the items in the preferred exemplary IRT outcome measure 180.

[0146]

[0145] Display

[0147]

[0146] Various aspects of the data regarding an individual patient's score can be displayed for the clinician and / or the patient.

[0148] FIG. 3 shows an exemplary scoring method of IRT compared to FIM® scores known in the prior art. IRT scores reflect the amount of ability a person, such as a patient, has. IRT scores can be continuously scaled scores across all functional categories. A score of exactly 0 means that the person has average ability at the time of discouragement. A score above 0 means that the person has above average ability. A score below 0 means that the person has below average ability. FIG. 3 shows the FIM scoring for the dressing item on the FIM and the continuum of attainable self-care IRT scores. The FIM score is reflected by the length of each patterned section. For example, a patterned section labeled "1" reflects an FIM score of 1, a section labeled "2" reflects an FIM score of 2, and so on. Scores and difficulty are presented on the same metric, i.e., if a person's IRT score is 1.50, the person is expected to score within the 6th category of the item.

[0149]

[0148] The value of the IRT score is evident by analyzing Figure 3. Assume a patient is admitted to an inpatient rehabilitation facility and the patient's IRT score improves from -1.00 to 0.00. The equivalent change in FIM level is +3. As a result, this is considered a good outcome for the patient as the patient has demonstrated functional gain.

[0150]

[0149] However, when improving within the FIM level, the FIM score is insufficient to indicate gain. Suppose another patient is admitted with a score of -2.00 and improves to -1.00. Even though this patient has improved to the same extent as the previous patient (+1.00), this patient's upper body dressing function level still appears to have not improved because the FIM change for this item is 0. As a result, one advantage of the IRT score is that it can detect improvements that the FIM cannot. In our experience, the expected changes in self-care for individuals with non-traumatic spinal cord injury and neurological injuries are quite dramatic when using IRT.

[0151]

[0150] Figure 4 shows an exemplary plot of certain data regarding the patient's scores in the self-care, cognition, and mobility domains. The percentage values ​​25%, 50%, 75%, and 100% reflect the percentage of scores on each domain. For example, a score of 100% in the self-care domain reflects a patient who achieved the highest possible score in that domain. The solid triangle reflects the patient's initial score, which can be tabulated based on an assessment at the time of admission or shortly after admission. The black triangle reflects the patient's current score. The dashed triangle reflects the patient's expected score. By reviewing the scores in this manner, the clinician can easily identify areas in which the patient has improved, and also areas in which additional treatment or other care may be useful. For example, after reviewing the plot of Figure 4, the clinician can determine that the self-care and mobility scores are below the expected scores in those domains, and therefore, further care should be focused in the areas of self-care and mobility.

[0152]

[0151] Predictive estimates can be derived in a variety of ways. In one embodiment, hierarchical linear modeling (HLM) can be used that incorporates information about past patient diagnoses, the severity of those diagnoses ("case mix group" which is a measure of the severity of the patient's condition within a diagnosis), the dates the patient was measured, and the scores on those dates. This modeling can output a prediction curve for all severities within all diagnoses up to 50 days of hospital stay. When plotting the information, the x-axis can be the number of days since admission and the y-axis can be the IRT (MAP) score.

[0153]

[0152] Other prediction methods can also be used, including data science methods such as neural networks and random forest models. Furthermore, additional patient information can be incorporated into the prediction process.

[0154] In one embodiment, a patient can be assessed using multiple-day IRT outcome measures 180. For example, a patient can be assessed on a first subset of questions from the IRT outcome measures 180 on a first day, and then on a second subset of questions on a second day. A data feed can be set up to collect the most recent item values.

[0155]

[0154] Adaptive testing can be used such that items in the IRT outcome measure 180 are selected for evaluation depending on the scores of items already evaluated. For example, a clinician can evaluate a patient using items in the IRT outcome measure 180 from the FIST test, calculate an initial IRT score based on the results, and select an optimal next item (or items) based on the initial IRT score. This process can be applied iteratively until the patient's score can be determined to be accurate within a predetermined level of uncertainty. For example, if the uncertainty is 0.3 or less, the adaptive testing method can stop providing additional items for evaluation and provide a final IRT score for review by the patient, clinician, or others.

[0156]

[0155] Figure 5 shows an example chart of certain data regarding a patient's scores in the self-care domain. Each row of the chart relates to a single item. For example, the top row relates to the wooden block grasping test. As described with respect to Figure 3, each row of this chart is divided into different shadings. The length of each section reflects how the score of that item relates to the AQ score. For example, section b1 reflects how the score of grasping item 1 relates to the AQ score.

[0157]

[0156] Figure 5 further shows the patient's current and expected functional status for each of the items / tasks in the self-care domain. It should be understood that this chart can display data from mobility, cognition, or other domains. The "Select Days of Stay" scroll bar allows the clinician to compare each performance level (e.g., current vs. expected) for all items for various days of stay. Such a comparison may allow the clinician to determine whether and, if so, to what extent an additional day of hospital stay would likely benefit the patient.

[0158]

[0157] To determine whether additional intervention is appropriate, clinicians can consider the IRT scores along with the patient's scores on specific FIM items. For example, if the patient's AQ score is 1, a score of 4 on the FIM Toileting measure is expected. However, if the FIM Toileting measure is lower, clinicians can use such measures as an indication to adjust treatment to specifically target improvements in toileting.

[0159]

[0158] Figures 6A and 6B are "FIM Explorer" sections with functionality that allows the clinician to select and / or set goals for each FIM-specific task. For example, in Figure 6A, "4-Minimal Assistance" is selected as the treatment goal for the feeding task. Once a task-specific goal is selected, a comparison chart as shown in Figure 7 may be displayed. This chart may allow the therapist or other clinician to compare whether the goal is set too high or too low when compared to a vertical line on the chart. The vertical line is derived from the rating of the selected goal and converted to an IRT score.

[0160]

[0159] Figure 8 shows various plots of the self-care domain compared to the patient's FIM score. As shown in Figure 8, areas of assessment within the self-care domain include balance, upper extremity function, and swallowing. In one embodiment, incomplete FIM performances can be omitted from the plot to avoid confusion over whether a score is low or simply incomplete.

[0161]

[0160] Prediction

[0162]

[0161] Prediction of AQ score may be based on various factors such as medical service group, case mix group (CMG), and / or length of stay. Within the CMG, age may be a factor used to aid in prediction.

[0163]

[0162] The data generated by the predictive model can be used in a variety of ways. For example, a patient's length of stay can be predicted by the patient's medical condition, injury level, and other demographic and clinical characteristics. As another example, if a patient falls short of his or her prediction in a given area, clinicians can target that area for more intensive treatment. As another example, if a patient's improvement in an area starts to taper off, clinicians can notice and prioritize balanced treatment in that area. As another example, given some monetary information, it becomes possible to evaluate the dollar value of expected improvement over a period of time and compare it to the inpatient care cost over the same time frame. The ratio of care value to care cost can be used to make discharge decisions. In addition, it is possible to predict success at other care institutions. Given similar evaluations at other levels and locations of care (e.g., outpatient, SNF, etc.), it is possible to determine the expected course of improvement at those institutions. In some cases, better decisions regarding care at those institutions can be made.

[0164] [Table 6]

[0165] [Table 7]

[0166] [Table 8]

[0167] [Table 9]

[0168] [Table 10]

[0169]

Table 11

[0170]

Table 12

[0171]

Table 13

[0172]

Table 14

[0173]

Table 15

[0174]

Table 16

[0175]

Table 17

[0176]

Table 18

[0177]

Table 19

[0178]

Table 20

[0179]

Table 21

[0180]

Table 22

[0181]

Table 23

[0182]

Table 24

[0183]

Table 25

[0184]

Table 26

[0185]

Table 27

[0186]

Table 28

[0187]

Table 29

[0188]

Table 30

[0189]

Table 31

[0190]

Table 32

[0191]

Table 33

[0192]

Table 34

[0193]

Table 35

[0194]

Table 36

[0195]

Table 37

Claims

1. 1. A computer-assisted method for assessing a patient, comprising: a. accepting input of a first assessment of the patient within a self-care domain having items within the areas of balance, upper extremity function, and swallowing; b. accepting input of a second assessment of the patient in a mobility domain having items in the areas of balance, wheelchair skills, positioning, bed mobility, and mobility; c. accepting input of a third assessment of the patient in a cognitive domain having items in the domains of cognition, memory, and communication; d. calculating a score based on the received ratings; and e. storing said calculated score in an electronic medical record. A method, comprising a computing device performing operations including:

2. obtaining the score from the electronic medical record; and adjusting a treatment plan based on said obtained score. The method of claim 1 further comprising:

3. The method of claim 1 , further comprising applying factor analysis to data derived from the electronic medical record.

4. The method of claim 1 , further comprising applying classical item analysis data derived from the electronic medical record.

5. 1. A method for measuring improvement in a rehabilitation patient, comprising: a. Using item response theory to preselect multiple ratings; b. accepting input of said plurality of ratings related to self-care; c. accepting input of the plurality of ratings related to mobility; d. accepting input of said plurality of cognition-related assessments; and e. storing said received input in an electronic medical record. A method, comprising a computing device performing operations including:

6. The method of claim 5 , wherein the plurality of ratings is further selected using factor analysis.

7. The method of claim 6 , wherein the plurality of ratings is further selected using classical item analysis.

8. 6. The method of claim 5, wherein each of the self-care, mobility, and cognition assessments is divided into factors.

9. 7. The method of claim 6, wherein the self-care factors are balance, upper extremity function, and swallowing.

10. The method of claim 5 , further comprising the computing device determining a score to measure the improvement.

11. The method of claim 10 further comprising the computing device displaying the score on a display screen.

12. The computing device, comparing said score to a predictive score to identify areas of treatment need; performing further operations including: The comparison is used to improve treatment. The method of claim 10.

Images