Assessment and management systems for rehabilitation status and related methods

Abstract

To provide systems and methods for measuring patient outcomes in a rehabilitation setting.SOLUTION: A method disclosed herein provides an assessment relating to self-care, an assessment relating to mobility, and an assessment relating to cognition, where the assessments are preselected using the item response theory.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] Cross - Reference to Related Applications

[0001] This application claims the benefit of priority of U.S. Provisional Patent Application No. 62 / 563,960, filed on September 27, 2017, which is hereby incorporated by reference in its entirety.

[0002] Technical Field

[0002] The present disclosure generally relates to rehabilitation techniques, and more particularly to computer - assisted methods for evaluating patients.

Background Art

[0003] Background

[0003] "Outcome measurement", also known as a "results assessment tool", is a set of items used to reveal a patient's changing medical or functional status. One outcome measurement is the Functional Independence Measure (FIM®), which provides a way to measure functional status. The assessment includes 18 items consisting of motor tasks (13 items) and cognitive tasks (5 items). The tasks are scored by a clinician based on a 7 - step ordinal scale ranging from total assistance to total independence. The scores range from 7 (lowest) to 91 (highest) for motor skills and from 7 to 35 for cognitive skills. The items include eating, grooming, bathing, upper body dressing, lower body dressing, toileting, bladder management, bowel management, transfer from bed to chair, transfer to toilet, transfer to shower, locomotion (walking or wheelchair level), stairs, comprehension, expression, social interaction, problem solving, and memory.

[0004]

[0004] The FIM assessment method uses a scoring system ranging from a score of 1 (reflecting complete assistance) to a score of 7 (reflecting complete independence). A score of 7 is intended to reflect that the patient is completely independent. A score of 1 is intended to reflect that the patient is able to perform less than 25% of the tasks or requires assistance from two or more people. As a result of this scoring method, many patients who show improvement in independent inpatient rehabilitation facilities or inpatient rehabilitation departments within hospitals do not necessarily show improvement in their outcome scores during their rehabilitation. For example, spinal cord injury patients may show significant improvement in fine motor skills (enabling the patient to use a computer or smartphone) during rehabilitation. However, in this situation, the patient's FIM score does not improve. [Overview of the project] [Problems that the invention aims to solve]

[0005]

[0005] There is a need for result measurements that more accurately capture the evaluation of a patient's medical or functional status. In addition, there is a need for result measurements that help patients, such as rehabilitation patients, better identify areas where improvement is possible. [Means for solving the problem]

[0006]

[0006] An “item” is a question or other type of assessment used in an outcome measurement. For example, an item in an outcome measurement known as the Berg Balance Scale instructs the patient as follows: “Stand up. Do not use your hands for support.” A “score” is the result of a score or other assessment corresponding to the assessment of an item. For example, scores for an item on the Berg Balance Scale are as follows: a score of 4 reflects that the patient can stand up without using their hands and be stable on their own; a score of 3 reflects that the patient can stand up on their own using their hands; a score of 2 reflects that the patient can stand up using their hands after several attempts; a score of 1 reflects that the patient needs minimal assistance from another person to stand or be stable; and a score of 0 reflects that the patient needs moderate or maximum assistance from another person to stand.

[0007]

[0007] Classical test theory is a number of psychometric theories that relate to predicting the results of educational assessments and psychological tests, such as the difficulty of items or the ability of the test taker. Classical test theory is a test theory based on the idea that a person's observed score or acquired score on a test is the sum of the true score (score without error) and the error score. Classical test theory assumes that each person has a true score T that would be obtained if there were no error in the measurement. A person's true score is defined as the expected number of correct scores across countless independent test administrations. Unfortunately, test users never observe a person's true score, only the observed score X. It is assumed that the observed score = true score + some error, i.e., X = T + E, where X is the observed score, T is the true score, and E is the error. The reliability of the observed score X of a test, i.e., the overall consistency of the measurement, is defined as the ratio of the variance of the true score to the variance of the observed score. The variance of an observed score can be expressed as equal to the sum of the variance of the true score and the variance of the error score. This allows for the formulation of the signal-to-noise ratio, where the reliability of a test score increases when the ratio of the error variance of the test score decreases, and decreases when the ratio decreases. Reliability is equal to the ratio of the variances of the test score that can be explained if the true score is known. The square root of reliability is the correlation between the true score and the observed score. Reliability can be estimated by various means, such as parallel testing or the measurement of internal consistency known as Cronbach's alpha coefficient. Cronbach's alpha can be expressed to provide a lower bound of reliability, and therefore the reliability of test scores within a given population is always higher than the value of Cronbach's alpha within that population.

[0008] overview

[0008] By developing an outcome evaluation that incorporates factor analysis and item response theory, the problem of accurately measuring the improvement of rehabilitation patients is solved.

[0009]

[0009] The problem of measuring the improvement of rehabilitation patients is solved by asking patients a series of questions and returning subject-specific scores and / or overall scores.

[0010]

[0010] The problem of improving the care of rehabilitation patients is solved by predicting the subject-specific scores and / or overall scores of the patient's outcome measurement, and by taking clinical intervention when the subject-specific scores and / or overall scores fall below the predicted scores.

[0011] drawing

[0011] The attached drawings describe the features of this technique in detail, but these techniques, as well as their purpose and advantages, can be best understood from the following detailed description, which should be interpreted in conjunction with the attached drawings. [Brief explanation of the drawing]

[0012] [Figure 1]

[0012] A flowchart of a specific exemplary method for preparing preliminary result measurements is shown. [Figure 2]

[0013] This diagram shows the flowchart for electronically collecting item scores in preliminary results measurements. [Figure 3]

[0014] This section presents an example scoring method for IRT compared to FIM® scores known from conventional technologies. [Figure 4]

[0015] This section presents illustrative plots of specific data regarding patient scores in the self-care, cognitive, and mobility areas. [Figure 5]

[0016] Further details the patient's current and expected functional status for each item / issue in the self-care area. [Figure 6A]

[0017] This is the "FIM Explorer" section, which has the functionality to allow clinicians to select and / or set goals for each FIM-specific task. [Figure 6B]

[0017] This is the "FIM Explorer" section, which has the function of allowing clinicians to select and / or set goals for each FIM-specific task. [Figure 7]

[0018] A comparison chart is shown. [Figure 8]

[0019] Shows various plots in the self-care area compared with the patient's FIM score.

Best Mode for Carrying Out the Invention

[0013] Detailed Description

[0020] The "two-factor model" is a structural model in which items cluster on a specific factor while simultaneously loading on a general factor.

[0014]

[0021] The term "categorical" is used to represent response options without an explicit or implicit order or ranking.

[0015]

[0022] The "Comparative Fit Index" (CFI) compares the performance of a constructed structural model against the performance of a model that does not assume relationships between variables.

[0016]

[0023] A "complex structure" is a CFA structural model in which at least one item loads on multiple factors.

[0017]

[0024] "Confirmatory factor analysis" (CFA) is a type of factor analysis used when psychometricians understand how latent traits and items should be grouped and related. A structural model is formulated and fitted to the data. The goal of this model is to achieve a good fit with the data.

[0018]

[0025] A "constraint" is a limitation imposed on a model for mathematical stability or the application of content area theory. For example, if it is expected that two factors in a confirmatory factor analysis have no relationship with each other, a constraint on their correlation (requiring it to be equal to 0.00) can be added to the model.

[0019]

[0026] A "continuous" variable is a variable that is measured without categories, such as time, height, or weight.

[0020]

[0027] "Covariance" refers to a variable within a model that is not directly measured but may still possess some explanatory power. For example, in rehabilitation research, it can sometimes be useful to include covariances related to age, sex, length of hospital stay, and diagnostic group.

[0021]

[0028] "Dichotomy" refers to a response choice that is ordered by having two categories (e.g., low versus high). Alternatively, it can refer to an ordered response that conceptually has two categories: correctly scored items versus incorrectly scored items.

[0022]

[0029] Differential item functioning (DIF) in item response theory is a measurement of how parameter estimations may behave differently for each group (within different samples) or for each observation (over a certain period of time).

[0023]

[0030] In item response theory, "difficulty" is the minimum level of latent trait required to respond in a particular way. In binary response measurements, there is a single difficulty level (e.g., the minimum level of latent trait required to raise the probability of answering correctly to 50% or higher). In multi-valued response measurements, there are usually no right or wrong answers, so "difficulty" is more appropriately expressed as "severity." In multi-valued response measurements, the estimated number of difficulty levels is k-1, where k is the number of response options. These difficulty levels represent the level of latent trait required to endorse the next higher category. They are sometimes called thresholds.

[0024]

[0031] "Dimension" refers to the number of latent properties that a measurement deals with. A measurement that records one property is said to be one-dimensional, while a measurement that records multiple properties is said to be multi-dimensional.

[0025]

[0032] "Discriminative power" is the ability of a test to differentiate between individuals with high and low ability in a latent trait. Similarly, discriminative power indicates the strength of the relationship between an item and a latent trait. Conceptually, discriminative power is very similar to factor loadings and can be mathematically converted into factor loadings.

[0026]

[0033] "Approval" means selecting a response option.

[0027]

[0034] An "equation constraint" is a mathematical requirement in item response theory and confirmatory factor analysis to restrict discriminative power or factor loadings so that they are equal when only two items load a factor.

[0028]

[0035] "Equalization" refers to using item response theory to obtain similarity between scores of various measurements that record the same level of latent trait. Equalization can also be used to compare alternative forms of the same measurement.

[0029]

[0036] "Error" is a term used to describe the magnitude of uncertainty surrounding a model. Models with parameter estimates that are very close to the observed data will have a small amount of error, while those that are completely different will have a large amount of error. Error can also indicate the magnitude of uncertainty surrounding a particular parameter estimate itself.

[0030]

[0037] "Estimation" refers to the statistical process of deriving parameter estimates from data. These procedures can be performed using specialized psychometric software known in the art.

[0031]

[0038] Exploratory factor analysis is a type of factor analysis that clusters items according to their correlations. Exploratory factor analysis is often performed without any instructions from the analyst, except for how many factors to extract. The groupings are then "rotated." The rotation method attempts to find factor loadings that exhibit a simple structure by ensuring that the factor loadings are pushed towards -1.00, 0.00, or 1.00.

[0032]

[0039] In factor analysis, a "factor" represents a latent characteristic. Unlike latent characteristics in item response theory, a factor typically does not have a score associated with it.

[0033]

[0040] Factor analysis is a statistical method for determining the strength and direction of relationships between factors and items. The data on which factor analysis is based is the correlation between items. Factor analysis can handle ordinal or continuous data, but not unordered categorical data. While it is possible to calculate a score from factor analysis, IRT scores are generally more reliable. It can be exploratory or confirmatory.

[0034]

[0041] "Factor correlation" refers to the correlation between two factors. A CFA model with correlated factors is called an "oblique" model.

[0035]

[0042] Factor loadings in factor analysis represent the strength of the relationship between an item and a factor. While not mathematically identical to correlation, their scale and interpretation are similar. Specifically, values ​​typically range from -1.00 to 1.00. Strong negative factor loadings indicate a strong inverse correlation between an item and a latent trait, while strong positive loadings have the opposite interpretation. A factor loading of 0.00 indicates no relationship whatsoever.

[0036]

[0043] "Goodness-of-fit statistics" or "goodness-of-fit indices" refer to metrics used to quantify how well a model performs. Popular goodness-of-fit metrics for confirmatory factor analysis and structural equation modeling include the mean squared error root (RMSEA), comparative goodness-of-fit index (CFI), Tucker-Lewis index (TL1), and WRMR / SRMR (weighted root mean-square residual / standardized root mean-square residual).

[0037]

[0044] The "general factor" refers to the factor that all items in a bifactor model load.

[0038]

[0045] The "Graded Response Model" (GRM) is an extension of the two-parameter logistic model that enables sequential responses. Instead of just one difficulty level, the graded response model introduces k-1 difficulties, where k is the number of response categories.

[0039]

[0046] A "hierarchical model" is a structural model in which latent characteristics load other latent characteristics to form a hierarchy.

[0040]

[0047] In a hierarchical model, "higher-order / lower-order factors" refer to a type of latent variable that is loaded by lower-order factors.

[0041]

[0048] The term "indicator" is used to refer to a goodness-of-fit index / statistic (e.g., a comparative goodness-of-fit index) or as a synonym for "measurement."

[0042]

[0049] "Item" refers to a question, task, or score related to the measurement that the responder or their representative (such as a clinician) undertakes.

[0043]

[0050] An "item characteristic curve (ICC)" is a graph that plots the probability of selecting various response options, given the level of latent characteristics. It is sometimes called a "trace line."

[0044]

[0051] Item response theory (IRT) is a collection of statistical models used to determine the behavior of items by obtaining scores according to a structural model. In some forms of use, IRT uses the response patterns of all people in a sample to obtain estimates of those items and scores. IRT uses data that is ordinal or categorical. Mathematically speaking, item response theory uses item and person characteristics to predict the probability that a person will choose a particular response option with respect to a given item.

[0045]

[0052] The "IRT score" is a score specific to IRT analysis, given on a standardized scale. The IRT score is similar to the z-score. In some IRT scoring methods, a score of 0.00 implies an average level of latent characteristic, a large negative score implies a low level of latent characteristic, and a large positive value implies a high level of latent characteristic.

[0046]

[0053] "Latent characteristics" are similar to factors in factor analysis, but are used more frequently in item response theory. Latent characteristics represent what a given set of items is supposed to measure. They can be used interchangeably with factors, domains, or dimensions.

[0047]

[0054] "Latent variable" is a term used for variables that are not directly measured. Latent variables include latent characteristics.

[0048]

[0055] "Linking" is similar to equalization, but it's for estimating item parameters rather than scoring.

[0049]

[0056] "Loading" is a verb used to describe what an item does to a factor. For example: "In this model, item 4 loads both local and general dependencies."

[0050]

[0057] Local dependence (LD) is a violation of the local independence assumption where items relate to each other for reasons other than latent characteristics. When local dependence appears to be present in the data, it can be addressed by modeling the correlations between items or by creating a local dependence factor. This can be due to a number of reasons, such as similar wording, nearly identical content, and the position of items in the measurement (this last example frequently occurs with the last item in a long measurement).

[0051]

[0058] "Local independence" refers to the psychometric assumption that the behavior of items is solely attributable to latent properties within the model and item-specific errors. If items violate this assumption, they are said to be locally dependent.

[0052]

[0059] "Manifest variables" is a general term for variables that are directly measured, and includes items, covariances, and other such variables.

[0053]

[0060] "Measurement" refers to a set of items that attempt to measure the level of some latent characteristic. It can be used interchangeably with assessment, test, questionnaire, index, or scale.

[0054]

[0061] In psychometrics, a "model" is a combination of a response model and a structural model. Roughly speaking, a model represents both the format of the data and how the recorded data should relate to the variables within the model.

[0055]

[0062] "Model fit" is a term used to indicate how well a model represents the data. This can be done in various ways, such as comparing observed data to predictions made by the model, or comparing a selected model to a null model (a model in which no variables are relevant). The metrics used to evaluate model fit are called goodness-of-fit statistics.

[0056]

[0063] "Multidimensional" is a term used to describe measurements that record multiple latent properties.

[0057]

[0064] "Multi-group analysis" in IRT refers to the process of dividing a sample into different groups, allowing for the estimation of parameters specific to each group.

[0058]

[0065] The "nominal model" is similar to the graded response model, except for items with response options that are categorical rather than ordinal.

[0059]

[0066] "Oblique" is an adjective used to describe correlated factors.

[0060]

[0067] "Order" describes the way an item records data. For example, possible responses to an item could be a series of categories ordered from low to high or high to low.

[0061]

[0068] "Orthogonal" refers to factors that are restricted to zero correlation.

[0062]

[0069] "Parameter estimation" refers to statistically derived values ​​estimated by psychometric software. It is a general term that may include item discrimination power, factor loadings, or factor correlations.

[0063]

[0070] A "path diagram" is a diagram intended to show the relationships between items, latent characteristics, and covariances. In a path diagram, rectangles / squares represent observed variables (i.e., items, covariances, or any modeled variables for which there is clearly recorded information), ellipses / circles represent latent characteristics or variables for which there is no clearly recorded information, single arrows reflect unidirectional relationships (such as in regression), and double arrows reflect correlations / covariances between modeled variables.

[0064]

[0071] "Multi-valued" is a term used for items that have multiple response options, and can be ordinal or categorical.

[0065]

[0072] A "pseudo-bifactor model" is a bifactor model in which not all items cluster to specific factors. Instead, some items may only be loaded onto general factors.

[0066]

[0073] A "psychometrist" is a type of statistician who specializes in measurement.

[0067]

[0074] "Psychometrics" refers to the statistics used when creating or representing a measurement.

[0068]

[0075] The "Rush model" is a response model that assumes all item powers are equal to 1.00. It is not typically used unless this assumption is true or nearly true. While this assumption simplifies the interpretation of scores and difficulty levels and allows the use of item response theory for (relatively) small sample sizes, it is extremely rare for all item powers to behave identically. This is a simplified example of a two-parameter logistic model that allows for variation in item powers. Therefore, the Rush model is sometimes called a one-parameter logistic model (1PL). It can be used when responses are dichotomous.

[0069]

[0076] The "respondent" is the person who answers the questions in the measurement.

[0070]

[0077] "Response" refers to the respondent's answer to an item.

[0071]

[0078] A "response category" is a set of options that a respondent can choose as their response to an item. If an item elicits a binary response, the data is recorded as either positive (1) or negative (0).

[0072]

[0079] In item response theory, a "response model" refers to the way a measurement model processes the format of a response. Popular response models include the Rush model, two-parameter logistic model, three-parameter logistic model, stepped response model, and nominal model.

[0073]

[0080] A "response pattern" is a sequence of numbers that represents the respondent's answers to each question in the measurement.

[0074]

[0081] The root mean squared error (RMSEA) is a goodness-of-fit statistic in applicable psychometrics. It measures the closeness of expected data (data generated by the model) to observed data. While an RMSEA of less than 0.08 is generally desirable, some articulates prefer an RMSEA of less than 0.05.

[0075]

[0082] A "score" is a numerical value intended to represent the level or quantity of latent traits held by the respondent. Classical test theory calculates the score as the sum of item responses, while item response theory estimates the score using both response patterns and item quality.

[0076]

[0083] "Sigmoid" (literally "S-shaped") is an adjective, sometimes used to describe the shape of the TCC or ICC of a 2PL item.

[0077]

[0084] A "simple structure" is a structural model in which all items load one factor at a time.

[0078]

[0085] A "special factor" is a factor loaded by a set of items in a bifactor model.

[0079]

[0086] Structural equation modeling (SEM) is an extension of confirmatory factor analysis (CFA) that enables relationships between latent variables, such as latent properties. When all latent variables in a model are latent properties, structural equation modeling (SEM) and CFA are often used interchangeably.

[0080]

[0087] A "structural model" is a mathematical description of a system of hypotheses regarding the relationship between latent characteristics and items. It is presented as a path diagram.

[0081]

[0088] The "total score" is calculated by summing up the numerical values ​​of all the responses in the measurement.

[0082]

[0089] The "Sum score conversion" (SSC) is a table that shows the relationship between the total score and the IRT score.

[0083]

[0090] A "Test characteristic curve" (TCC) is a graph that plots the relationship between the total score and the IRT score.

[0084]

[0091] A "testlet" is a small set of items that measure some components of an overall latent trait. When the definition of the latent trait is clearly defined beforehand, creating measurements composed of testlets can result in scores that are easier to interpret.

[0085]

[0092] "Threshold": See "Difficulty Level".

[0086]

[0093] The Tucker-Lewis Index (TLI) is a fitness index that compares the performance of a built model to the performance of a model that does not assume relationships between variables. A good fit model typically has a TLI of 0.95 or higher.

[0087]

[0094] The "three-parameter logistic model" (3PL) is an extension of the two-parameter logistic model that includes an "inference" parameter. For example, in a multiple-choice question with four options, there is a 25% probability of correctly answering even with random inference. The 3PL acknowledges this non-zero probability of correctly answering. It is used when the response is dichotomous.

[0088]

[0095] "Trace line": See "Item characteristic curve".

[0089]

[0096] The "two-parameter logistic model" (2PL) is similar to the Rush model but allows for variations in item discrimination power. It can be used when item responses are dichotomous.

[0090]

[0097] "One-dimensional" is a term used to describe a measurement that records only one latent characteristic.

[0091]

[0098] "Variable" is a general term used to describe a set of directly recorded (explicit) or indirectly recorded (latent) data that measures a single thing.

[0092]

[0099] WRMR / SRMR (weighted root mean-square residual / standardized root mean-square residual) is a goodness-of-fit statistic that measures the magnitude of a model's residuals. Residuals are the difference between the observed data and the data predicted by the model. A typical recommended WRMR value is less than 1.00, but this recommendation can vary depending on the sample size or model complexity. WRMR is used when the model contains at least one categorical variable, while SRMR is used when all variables are continuous.

[0093]

[0100] Figure 1 shows a flowchart of a specific exemplary method for preparing a preliminary result measurement 100 for inclusion in an electronic medical record.

[0094]

[0101] In 101, the item set 200 is identified. In one embodiment, a clinician may be asked to provide their input for appropriate items to include in the item set 200, based on their training, education, and experience. Examples of clinicians may include internists, physical therapists, occupational therapists, speech-language pathologists, nurses, and PCTs. The items in the item set 200 may arise from various outcome measures known in the art.

[0095]

[0102] In 102, items from item set 200 can be grouped into one or more domains, called “domains,” that relate to treatment or clinical outcomes. Clinicians can identify these domains. In one embodiment, items from item set 200 can be grouped into three domains titled “Self-Care,” “Mobility,” and “Cognition.” It should be understood that other groupings of additional and / or alternative domains are also possible.

[0096]

[0103] In 103, relevant analytical steps can be performed. For example, the frequency with which items in item set 200 are used in conventional practice to assess patients in a healthcare setting can be analyzed. Alternatively, the cost of the equipment required to perform the items can be evaluated. Clinical literature can be reviewed to identify outcome measures that have items in item set 200 that are psychometrically acceptable and clinically useful. For example, the reliability and validity of outcome measures that have one or more items in item set 200 can be reviewed to ensure they are psychometrically acceptable. As another example, each outcome measure and / or item can be reviewed to ensure it is clinically useful. For example, there are many items available in the literature used to test a person's balance, but not all of them are appropriate for a patient in a rehabilitation context. Based on these and similar factors, the initial item set can be narrowed to reduce the burden on patients, clinicians, and other healthcare professionals.

[0097]

[0104] In 104, multiple revised items are collected. A pilot study can be conducted on these multiple items. The pilot study can be conducted by having clinicians evaluate patients using the revised items in a standardized manner, so that each clinician evaluates each patient using all of the revised items. In another embodiment, clinicians can select which items to use to evaluate a patient based on the patient's specific clinical characteristics. Decisions regarding the selection of specific items can be made during the patient's rehabilitation stay, for example, based on information obtained from the admission evaluation of an inpatient. To determine the patient's improvement, the items may be administered at least twice during the patient's hospital stay. The pilot study can be facilitated using an electronic medical record system, which allows clinicians to enter the item scores into the electronic medical record.

[0098]

[0105] In section 105, experimental studies can be analyzed. For example, items that take too much time for clinicians to perform on patients can be removed.

[0099]

[0106] In 106, the original paper-based items for the preliminary result measurement 100 are implemented within the electronic medical record. Scores at the individual item level can be recorded electronically. For example, the items to be implemented may be the results of the analysis of the experimental study in 105, although the analysis of the experimental study is not mandatory. Alternatively, the items within the preliminary result measurement 100 can be implemented within an electronic system such as an external database of the electronic medical record. In one embodiment, the external electronic system can communicate with the electronic medical record using methods known in the art, such as database connection technology. In 107, the items for the preliminary result measurement 100 are programmed within the EMR using known methods, enabling clinicians to input their scores into the electronic medical record. In one embodiment, the EMR may provide prompts to warn, remind, and / or request that clinicians input specific scores for specific items of the preliminary result measurement 100. Such prompts can improve the reliability and completeness of clinical data input into the EMR.

[0100]

[0107] The above explanation regarding Figure 1 refers to selecting specific items from various outcome measurements, but it should be understood that a similar method can be used for selecting the outcome measurements themselves. For example, in 104, instead of selecting which items to use when evaluating a patient, the entire outcome measurement can be selected or ignored.

[0101]

[0108] Figure 2 shows a flowchart for electronically collecting scores for items within the preliminary outcome measurement 100. In 201, the clinician performs an assessment of the patient. In one embodiment, the clinician can perform the assessment using all items within the preliminary outcome measurement 100. In another embodiment, the clinician can perform tests or items within the preliminary outcome measurement 100 that are specific to the clinician's area of ​​practice. For example, a physical therapist can perform tests or items within the preliminary outcome measurement 100 that are specific to physical therapy. In yet another embodiment, the clinician can use their clinical judgment, based on their education, training, and experience, to identify the tests within the preliminary outcome measurement 100 that are most relevant to the patient. It can be seen that if the patient is extremely ill or has very limited function, the clinician will not perform certain items. For example, a clinician would not ask a newly quadriplegic patient to perform a test that requires the patient to walk.

[0102]

[0109] The assessment may be an initial assessment conducted when the patient is admitted to the hospital or shortly thereafter. In one embodiment, each patient receiving care for a certain period, such as one month or one year, is assessed. In another embodiment, the majority of patients receiving care for a certain period are assessed. In yet another embodiment, multiple patients are assessed. In other embodiments, the patient population can be refined to include only inpatients, only outpatients, or a combination thereof.

[0103]

[0110] In various embodiments, a specific test within the preliminary results measurement 100 can be performed once upon admission or immediately after admission, and again upon discharge or immediately before discharge. In various embodiments, a specific test within the preliminary results measurement 100 can be performed weekly. In various embodiments, a specific test within the preliminary results measurement 100 can be performed multiple times a week, such as twice a week.

[0104]

[0111] In one embodiment, the assessment may be conducted in a centralized location dedicated to performing the assessment. The assessment may be performed by a set of clinicians whose specific role is to perform the assessment. The centralized location, which has qualified staff and sufficient equipment to objectively assess the patient's functional status, may be operated by a standardized process within a controlled and safe environment. In one embodiment, the clinician gives instructions to the technician for the assessment. For example, the clinician (physiotherapist, therapist, nurse, or psychologist) instructs on specific tests (e.g., gait and balance tests) or sets of tests. The test instructions are transmitted electronically to the assessment department ("AAL") and a hard copy may be printed for the patient. When the AAL is ready, the patient can move to the AAL with assistance as needed. The technician or other staff performs the instructed tests. The test results can be recorded and entered / transmitted within the electronic medical record. The clinician can review the test results to modify the care plan as needed. This process can reduce the time it takes for the clinician to know how to perform the tests. One advantage of AAL is that other clinicians do not need to learn how to administer various tests each time a new test is introduced. Clinicians only need to learn how to interpret the test results, not how to administer the tests. Tests can be administered by properly trained and qualified personnel. Clinical staff can focus on treatment rather than assessment. More treatment sessions or additional time can be allocated to improve results. Test equipment is kept centrally to reduce the demand and maintenance costs of multiple units. Tests can be administered in a well-controlled, standardized, and safe environment. Technicians can use standardized procedures to avoid potential bias caused by evaluators (such as a tendency to give high scores to indicate improvement over time), thereby improving the quality of the data.

[0105]

[0112] The scores for each assessment can be stored within the EMR. For example, the scores for each assessment can be stored in a preliminary score dataset 150. In 202, data analysis and cleanup can be performed on the preliminary score dataset 150 to improve data quality. For example, out-of-range scores can be removed from the preliminary score dataset 150. Data patterns within the preliminary score dataset 150 from the same clinician can be examined and cleaned using methods known in the art. Scores in the preliminary score dataset 150 from patients showing a significant score increase from "dependent" to "independent" can also be discarded. Suspicious data for specific assessments can also be discarded.

[0106]

[0113] In 203, the score data can be further extracted, cleaned, and prepared using methods known in the art to obtain data in a format that can be queryed and analyzed. The data can be reviewed in terms of quality, and various data options can be performed on the preliminary score dataset 150, such as data pivoting, data merging, and the creation of a data dictionary. The data from the preliminary score dataset 150 can be stored in different forms, such as in an EMR or a data warehouse, for further analysis. Those skilled in the art will understand that there are many ways to configure the data in the preliminary score dataset 150 for analysis. In one embodiment, the preliminary score dataset 150 is configured so that the item scores can be used for analysis across multiple dimensions, such as time period and patient identification information.

[0107]

[0114] Once the preliminary score dataset 150 is prepared for analysis, a psychometric assessment can be performed on the preliminary score dataset 150. The psychometric assessment evaluates how well the outcome measure actually measures what it is intended to measure. The psychometric assessment can include a combination of classical test theory analysis, factor analysis, and item response theory, and evaluates the preliminary score dataset 150 for various aspects that may include reliability, validity, responsiveness, dimensionality, item / test information, singular item function, and equalization (score crosswalk). In one embodiment, classical test theory analysis can be used to examine the reliability of items within the preliminary outcome measure 100 and how the preliminary outcome measure 100 and the field interact.

[0108]

[0115] Item Reduction. Item reduction step 152 helps to reduce items from the preliminary outcome measure 100 that do not function as expected. Factors may include reliability, validity, and responsiveness (also known as sensitivity to change). The objective of item reduction step 152 is to eliminate potential item content redundancy from items in the preliminary outcome measure 100 to a minimum subset of items in the IRT outcome measure 180, without sacrificing the psychometric properties of the dataset. Item reduction step 152 can be performed using a computer or other computing device, for example, using computer program 125. Computer program 125 can be written in the R programming language or another suitable programming language. Computer program 125 provides options to specify the desired number of items (as well as options to include specific items) and calculates reliability estimates of Cronbach's alpha coefficient for all possible combinations of items within those user-defined constraints. A tolerance range for Cronbach's alpha coefficient can also be defined within computer program 125. In addition, the computer program 125 can construct and execute the syntax of a statistical modeling program such as Mplus (Muthen & Muthen, Los Angeles, CA, http: / / www.statmodel.com) to determine the goodness of fit of a one-factor confirmatory factor analysis (CFA) model for each reduced subset 155 of items.

[0109]

[0116] The computer program 125 can be used to analyze some of the result measurements that are explored for a one-dimensional subset of 4 to 8 items that are included in the preliminary result measurements 100 (FIST, BBS, FGA, ARAT, and MASA) and have a Cronbach's alpha reliability between 0.70 and 0.95. Using these constraints, the number of items in many measurements can be significantly reduced. For example, a measurement can be reduced to at least half its original length while maintaining good psychometric properties. The resulting subset of items can serve as components of a confirmatory factor analysis (CFA). In one embodiment, certain items, such as items from the FIM®, may not be included in the item reduction process. In one embodiment, the item reduction step 152 can be performed multiple times. For example, the item reduction step 152 can be performed for each result measurement included in the preliminary result measurements 100.

[0110]

[0117] In the item reduction step 152, the computer program 125 determines the degree to which the items relate to each other. The computer program 125 can determine the degree to which the items in the outcome measurements within the preliminary outcome measurement 100 relate to each other. In one embodiment, items relate to each other in the outcome measurement if they have highly correlated responses. The analysis can be started by providing an initial core set of items, the number of items may be determined by clinician input based on the correlation between item pairs. For example, the computer program 125 can determine how item A relates to item B, and both items A and B are in the same outcome measurement. If there is a high correlation, both items A and B are included in the core set. The computer program 125 can then determine how a new item C correlates with the item set {A,B}. If there is a high correlation, item C is included in the core set. This method can be repeated with additional items D, E, F, etc. As described above, the program evaluates the reliability (Cronbach's α) of all possible subsets of items. The program correlates the responses of one set of items with the responses of a second set of items. Cronbach's alpha is well known in the art, but a brief example is given here. The information used to calculate Cronbach's alpha is the correlation between all possible split-half pairs within a subset of items. For example, using three items {A, B, C}, Cronbach's alpha is the average of the correlations between A and BC, B and AC, and C and AB. In other words, it calculates the correlation between all pairs of unique subsets of a given set. The purpose of correlation analysis is to help ensure that the items measure the same underlying constructs and improve reliability.

[0111]

[0118] Table 1 lists exemplary outputs of item reduction step 152 for measuring results on the Berg Balance Scale ("BBS") with a sample size set to five items. The numbers in each cell in the "Item" column reflect the numbers of the questions on the BBS (1: Unsupported sitting, 2: Position change - sitting to standing, 3: Position change - standing to sitting, 4: Transfer, 5: Unsupported standing, 6: Eyes closed standing, 7: Feet together standing, 8: Tandem standing, 9: Single leg standing, 10: Rotation (foot fixed)). Each reduced subset 155 is shown along with its associated Cronbach's α value. The first reduced subset has the highest Cronbach's α of the reduced subsets in Table 1. In one embodiment, the reduced subset with the highest Cronbach's α is used as the first reduced subset for CFA step 160, which is described in more detail below.

[0112] [Table 1]

[0113]

[0119] Confirmatory Factor Analysis. Factor analysis is a statistical method used to determine the number of underlying dimensions contained within a set of observed variables and to identify subsets of variables corresponding to each of the underlying dimensions. The underlying dimensions can be called continuous latent variables or factors. The observed variables (also known as items) are called indicators. Confirmatory factor analysis (CFA) can be used when the number of dimensions of a set of variables for a given population is already known from previous research. CFA can be used to see if the established number of dimensions and factor loading patterns fit a new sample from the same population. This is the "confirmatory" aspect of the analysis. CFA can also be used to see if the established number of dimensions and factor loading patterns fit a sample from a new population. In addition, by examining the variances and covariances / correlations of factors, factor models can be used to study individual traits. The variances of factors indicate the degree of heterogeneity of the factors. The correlations of factors indicate the strength of the associations between factors.

[0114]

[0120] Confirmatory factor analysis (CFA) can be performed using Mplus or other statistical software to verify the statistically accurate fit of item compositions within a predefined factor structure. CFA is characterized by constraints on factor loadings, factor variances, and factor covariances / correlations. A CFA requires at least m^2 constraints, where m is the number of factors. A CFA may include correlation residuals, which can be useful in representing the minor influence of factors on a variable. A set of background variables can be included as part of the CFA.

[0115]

[0121] Mplus can estimate CFA models for a single group or multiple groups, and CFA models with background variables. Factor indicators for CFA models can be continuous, censored, binary, ordinal categorical, count, or a combination of these variable types. When all factor indicators are continuous, Mplus offers seven estimation options: maximum likelihood (ML), maximum likelihood with robust standard errors and chi-squared (MLR, MLF, MLM, MLMV), generalized least squares (GLS), and weighted least squares (WLS), also known as ADF. When at least one factor indicator is binary or ordinal categorical, Mplus offers seven estimation options: weighted least squares (WLS), robust weighted least squares (WLSM, WLSMV), maximum likelihood (ML), maximum likelihood with robust standard errors and chi-squared (MLR, MLF), and unweighted least squares (ULS). If at least one factor indicator is censored, unordered categorical, or count, Mplus offers six estimation options: weighted least squares (WLS), robust weighted least squares (WLSM, WLSMV), maximum likelihood (ML), and maximum likelihood with robust standard errors and chi-squared (MLR, MLF).

[0116]

[0122] A model can be defined in statistical software such as Mplus that uses a high-confidence subset of items from a measurement reduction step and assumes that all items within a domain are related to each other. The model can also measure specific constructs under a preview of that domain. For example, a subset of all items taken from a self-care measurement might be assumed to measure self-care, but simultaneously measure one of balance, upper limb function, and swallowing. Building a model in this way allows for the measurement of both the entire domain (e.g., self-care) and a set of related constructs that make up that domain (e.g., balance, upper limb function, and swallowing, which are constructs that make up self-care). Given the data, the structure of the model implies a set of expected correlations between each pair of items. However, these (possibly) correlations can be calculated directly from the data. These correlations are the observed correlations. The validity of a constructed model, called "model fit" in statistics, can be determined using the mean squared error root-squared (RMSEA), which is a measure of the difference between the observed correlation and the expected correlation. In a preferred embodiment, the model has an acceptable degree of fit when the value of such difference is small (e.g., less than 0.08).

[0117]

[0123] After applying the CFA step 160 to the reduced subset 155, the output of the CFA step 160 may include factor loadings, including general factor loadings. General factor loadings can range from -1 to 1, with values ​​between 0.2 and 0.7 indicating how well the factor evaluates the relevant item. The output of the CFA step 160 may provide additional factor loadings for each item. In one embodiment, each item may have factor loadings for each sub-domain. For example, each item may have factor loadings for balance, upper limb, swallowing, and each of the other sub-domains. In one embodiment, the factor loadings are non-zero if the item is related to a sub-domain.

[0118]

[0124] In certain cases, applying the CFA step 160 to a reduced subset 155 can lead to problems requiring the selection of a new reduced subset 155. For example, general factor loadings greater than 0.7, or especially values ​​close to 1.0, indicate redundancy. For example, the way items are scored based on the results of the Action Research Arm Test (ARAT) inevitably requires too much confidence. A patient who achieves the highest score on the first (most difficult) item is allowed to score 3 on all subsequent items on that scale. If the patient scores less than 3 on the first item, the second item is evaluated. This is the easiest item, and if the patient scores 0, it is unlikely that the patient will score higher than 0 on the remaining items, and 0 will be given for the other items. This scoring method requires too much confidence. In other cases, if a factor loading is greater than 1, such a value reflects that the item pair has an (impossible) negative variance, and therefore the CFA step 160 needs to be run on a new reduced subset 155. A new reduced subset 155 can be selected from the group of reduced subsets generated by the item reduction step 152. For example, a new reduced subset with the next highest Cronbach's α can be selected, and the CFA step 160 can be applied to that new reduced subset.

[0119]

[0125] In addition, during the process of performing CFA step 160, it may become apparent that items designated by the clinician as belonging to a certain subfield should be moved to a different subfield in order to improve the fit of the model used to generate the IRT result measurement 180 (as discussed further below). For example, during the development of the embodiments described herein, items identified by the clinician as relating to “strength” were initially placed in the self-care subfield. However, when performing CFA step 160, it was determined that these items did not fit the model. Moving these items to the “upper limb function” subfield improved the fit of the model.

[0120]

[0126] Table 2 below shows the goodness-of-fit statistics for one-factor CFA including groups 1-10 shown in Table 1. CFA step 160 allows for an evaluation of whether the goodness-of-fit statistics listed in Table B meet the usual "good fit" criteria. In one embodiment, these criteria are RMSEA < 0.08, CFI > 0.95, TLI > 0.95, and WRMR < 1.00. Those skilled in the art will understand that other good fit criteria can also be used.

[0121] [Table 2]

[0122] While the above example shows only one outcome measurement, the Berg Balance Scale, it should be understood that CFA step 160 applies to each outcome measurement within the preliminary outcome measurement 100.

[0123]

[0127] Item response theory. In one embodiment, the IRT outcome measure 180 can be configured to include multiple high-level domains. For example, the IRT outcome measure 180 may be configured to include a “self-care” domain (including items determined to reflect the patient’s ability to perform self-care), a “mobility” domain (including items determined to reflect the patient’s ability to move), and a “cognition” domain (including items determined to reflect the patient’s cognitive abilities). Within each high-level domain, specific assessment areas, also called “factors” or “clusters,” can be identified. Table 3 reflects exemplary assessment areas associated with each high-level domain.

[0124] [Table 3]

[0125]

[0128] The IRT outcome measure 180 includes measuring general domains (i.e., self-care, mobility, and cognition) as well as specific assessment areas within those domains, thus allowing for the targeting of a bifactorial structure for each domain (general factor and domain-specific factor). The composition of specific factors can be determined by the content of each item set. For example, items from FIST, BBS, and FGA can be combined to form the "balance" assessment area within the self-care domain. The acceptable goodness of fit of the bifactorial model to the data was assessed using the RMSEA < 0.08 criterion (Browne & Cudeck, 1992), and modification indices were also calculated to examine local item dependencies and potential improvements to the model, such as additional cross-loadings (i.e., items contributing to several factors).

[0126]

[0129] Item response theory reflects a mathematical model that describes the relationship between a person's abilities and item characteristics (such as difficulty). For example, a more capable person is more likely to be able to perform a more difficult task, which can enable more adaptive interventions based on a series of questions. Other item characteristics may also be relevant, such as the "discriminative power" of an item, which is the ability to distinguish between people with high-level and low-level characteristics.

[0127]

[0130] After constructing a CFA model for each domain, the final structure can be coded for execution within an item response theory software package such as flexMIRT (Vector Psychometric Group, Chapel Hill, North Carolina, US). flexMIRT is a multilevel, multidimensional, and multigroup item response theory (IRT) software package for item analysis and test scoring. To address the ordinal categorical nature of item responses from achievement ratings scored by clinicians, a multidimensional graded response model (M-GRM) can be selected. For example, the dimensions could be "self-care," "mobility," and "cognition." Subdomains of "self-care" could be "balance," "upper limb function," "strength," "positioning," and "swallowing." Subdomains of "mobility" could be balance, wheelchair ("W / C") skills, positioning, bed mobility, and mobility. Subdomains of "cognition" could be "consciousness," "excitement," "memory," "speech," and "communication."

[0128]

[0131] However, in a preferred embodiment, subfields can be reduced in order to focus on the most important subfields of the ability. For example, in "self-care," the subfields to be reduced may be balance, UE function, and swallowing. In "cognition," the subfields to be reduced may be cognition, memory, and communication. In "mobility," there may be no subfields, or in other words, all subfields can be clustered together.

[0129]

[0132] The analysis can also be multi-group by its nature. For example, samples of self-care and mobility can be divided into groups determined by their balance level (sitting, standing, or walking). As another example, samples of cognition can be divided into a wide range of diagnostic categories (stroke, brain injury, neurological, or unrelated). In one embodiment, to address the complexity of the model, the Metropolis-Hastings-Robin-Monroe (MH-RM) algorithm (Cai, 2010) can be used for more efficient parameter estimation. MH-RM repeats the following three steps until the difference between two consecutive cycles falls below a selected criterion: Step 1 (Completion) complements a random sample of latent traits from the distribution implied by the item parameter estimates taken from the previous cycle. If it is the first cycle, the distribution implied by the algorithm's starting values ​​is used. This completion can be done using the MH sampler. Step 2 (Approximate) evaluates the log-likelihood of the completed data. Step 3 (Robin-Monroe Update) computes new parameter estimates for the next cycle by using the Robin-Monroe filter on the log-likelihood from Step 2. Step 1 is then repeated using the information from Step 3. The slope can reflect the item's ability to identify items, while the intercept can reflect the item's difficulty level.

[0130]

[0133] In addition to the slope and intercept of the items, a maximum posterior (MAP) latent trait score, which reflects the patient's ability level, can be calculated for each patient.

[0131]

[0134] Principal coding for IRT focuses on transforming the mathematical structure selected after CFA into a structure that can be evaluated using IRT. The data used for analysis could be, for example, the patient's scores for all items that simply assessed the patient. To ensure consistency, the most recently available data for each patient for each item can be used. Maintaining this consistency has the advantage of placing the patient's score on a specific baseline, i.e., a typical discharge level. The MAP (Maximum Apocalypse) scoring method can be used, but other scoring methods such as ML (Maximum Likelihood), EAP (Expected Apocalypse), or MI (Multiple Complement) are known to be usable instead. In addition, there are various estimation methods that can be used. For example, marginal maximum likelihood (MML-EM) using the expectation maximization algorithm can be used. However, this method can be disadvantageous when dealing with a considerable number of dimensions. In a preferred embodiment, Metropolis-Hastings-Robin-Monroe (MH-RM) estimation is used.

[0132]

[0135] Maximum Post-Marketing (MAP) scoring requires two inputs: the population score density (usually considered standard normal for each dimension) and the IRT parameter for each item the patient was rated. Given what is known about an item and how the patient was rated for each item, multiplying the population density for each item by the IRT function yields what is known as the likelihood, i.e., a mathematical representation of the probability of various scores. The position of the maximum value of that function is the patient's MAP score.

[0133]

[0136] Occasionally, a response option for a particular item may be selected very rarely, which can cause problems in estimating the IRT parameter for that item (implying that the response option may be unnecessary). In such cases, those responses can be folded into adjacent categories. For example, if an item has responses {1,2,3,4} and the response 2 is observed very rarely in the data, the data can be recoded as {1,2,2,3}. It should be understood that in IRT analysis, the absolute number of a digit is not important; instead, order is what matters.

[0134]

[0137] Group Composition: To enable more targeted assessment, the IRT analysis used here can, by its nature, be multi-grouped. In self-care and mobility, patients can be grouped according to their balance level (none, sitting, standing, and walking). Similarly, groups can be formed within the cognitive domain according to cognitive diagnosis (stroke, brain injury, neurological, or none). This method may result in multiple test forms containing only the items appropriate for each patient. For example, such test forms might include "none," "sitting balance," "standing balance (up to)," and "no balance constraints" in "self-care" and "mobility," and "stroke, brain injury, neurological, or no disorder" in "cognition." Forms can be adapted according to group attribution rather than assessment domain. For example, a patient's balance level may influence which balance measurements appear in the self-care and mobility domains, while a patient's cognitive diagnosis (if any) may influence which measurements may appear on the form. For example, ABS is used only in the brain injury form of the cognitive measurement, and KFNAP is used only in the stroke measurement.

[0135]

[0138] Item response theory yields separate scores for each domain. For example, a patient might score 1.2 in the "Self-Care" domain, 1.4 in the "Mobility" domain, and 3 in the "Cognition" domain. In one embodiment, these scores may be reported separately to the clinician, the patient, and others. In other embodiments, these scores may be combined into a single score. In one embodiment, a score of +1 means the patient is 1 logit above average. A score of -1 means the patient is 1 logit below average. Values ​​below -3 and above 3 are highly unlikely because the mathematical assumption underlying IRT is that the scores follow a mean distribution. It should be understood by those skilled in the art that standard deviation and other numbers reflecting logits may be used instead. For example, a score of 3 may mean the patient is average, and thus the score ranges from 0 to 6. As another example, a score of 50 may mean the patient is average, and a score of +10 may mean the patient is 1 logit above average, and thus the score ranges from 20 to 80.

[0136]

[0139] The following is an example of performing IRT step 170 in the self-care area. Seven factors are given to IRT step 170: a self-care factor, a balance factor, a UE function factor, a swallowing factor, a hidden factor for ARAT, a hidden factor to overcome the negative correlation between FIST and FGA outcome measures, and a hidden factor specific to FIST so that FIST is not overemphasized in the outcome. IRT step 170 (using, for example, MH-RM estimation) returns a discriminative matrix 172 and a difficulty matrix 174. These matrices can be shown, for example, by a slope / intercept formula, where the slope reflects item discriminative power and the intercept reflects item difficulty.

[0137]

[0140] Table 4 shows an exemplary discriminative matrix 172 for the self-care domain of the exemplary IRT outcome measure 180. Column headers a1-a7 in Table 4 represent the following: (a1: Self-care, a2: (ARAT local dependence), a3: Upper limb function, a4: Swallowing, a5: Balance, a6: (Decreased FIST effect), a7: (Inverse correlation of BBS and FGA)), with "hidden" factors indicated in parentheses. Table 4 lists the slope values ​​for each item in factors a1-a7. The item names in Table 4 are also reflected in Table 6 of Appendix 1, which lists the items in the exemplary IRT outcome measure 180.

[0138] [Table 4]

[0139] Table 5 shows an exemplary difficulty matrix 174 for IRT outcome measurement 180. Table 5 lists the intercept values ​​for each item of factor d1 to d6. The column headers d1 to d6 in Table 5 are as follows: (d1: Self-care, d2: (ARAT local dependence), d3: Upper limb function, d4: Swallowing, d5: Balance, d6: (Decreased effect of FIST)), and "hidden" factors are shown in parentheses.

[0140] [Table 5]

[0141] It should be understood that a discriminative matrix 172 and a difficulty matrix 174 can be prepared for each domain within the IRT result measurement 180.

[0142]

[0141] Exemplary score / probability response can be plotted, with the X-axis reflecting the score and the Y-axis reflecting the probability of the response. The product of the curve yields a likelihood curve that looks somewhat like a bell curve. The peak of the curve can be used as the patient's score.

[0143]

[0142] Input from therapists to ensure clinical relevance. Each item may be labeled using the cluster that best represents its role within the IRT outcome measure 180. This labeling may be done by clinicians based on their education, training, and experience. For example, clinicians may label items that measure balance, such as items that test sitting function included in the “Mobility” field and “Balance” factor in Table 1.

[0144]

[0143] Since the selection (retention or removal) of items in the item reduction step of the analysis is predicted based on psychometric and statistical evaluations, in one embodiment, clinical experts can review the content of the items taken up in the reduced set of items for further feedback. For example, a group of clinicians can be surveyed for input on whether items should be added to or removed from the subset taken from each of the complete outcome measures. To help ensure that the retained items are psychometrically valid and clinically relevant, the inputs of these clinicians can be used to construct the final model for each area.

[0145]

[0144] Remodeling to derive the final item set. After the agreed-upon item set, which takes into account both psychometric assessments and clinical judgments, is ready, the CFA and IRT steps can be performed. Items that were excluded due to high clinical support can be added back to the model, while items with low support can be removed. The goodness of fit of the model to the data can then be evaluated using the mean squared error root (RMSEA) calculated during the CFA, and new item parameter estimates and latent trait scores were calculated during the IRT analysis. Table 6 in Appendix 1 of this specification lists items in a preferred exemplary IRT outcome measure 180.

[0146]

[0145] Display

[0147]

[0146] Various aspects of data regarding individual patient scores can be displayed for clinicians and / or patients.

[0148]

[0147] Figure 3 shows an exemplary scoring method of IRT compared to the FIM® score known in the prior art. The IRT score reflects the amount of ability a person, such as a patient, possesses. The IRT score can be a score that scales continuously across all functional categories. A score of exactly 0 means that the person has average ability at the time of dischare. A score greater than 0 means that the person has above-average ability. A score less than 0 means that the person has below-average ability. Figure 3 shows a continuum of FIM scoring and attainable self-care IRT scores for the dressing items on the FIM. The FIM score is reflected by the length of each patterned section. For example, a patterned section labeled "1" reflects an FIM score of 1, a section labeled "2" reflects an FIM score of 2, and so on. Scores and difficulty levels are presented on the same metric; that is, if a person's IRT score is 1.50, that person is expected to score in the sixth category of the items.

[0149]

[0148] The IRT score can be determined by analyzing Figure 3. Assume that the patient is admitted to a rehabilitation facility and that the patient's IRT score improves from -1.00 to 0.00. An equivalent change in FIM level is +3. As a result, the patient showed a functional gain, so this is considered a good outcome for the patient.

[0150]

[0149] However, when improvement is within the range of FIM levels, the FIM score is insufficient to indicate gain. Suppose another patient is admitted with a score of -2.00 and improves to -1.00. This patient appears to have not improved in upper body dressing function level, even though this patient has improved to the same extent as the previous patient (+1.00), because the change in FIM for this item is 0. Consequently, one advantage of the IRT score is that it can detect improvements that FIM cannot detect. In our experience, the expected changes in self-care for individuals with non-traumatic spinal cord injury and nerve injury are quite dramatic when using IRT.

[0151]

[0150] Figure 4 shows an illustrative plot of specific data regarding patient scores in the self-care, cognitive, and mobility domains. The percentage values ​​of 25%, 50%, 75%, and 100% reflect the percentage of the score in each domain. For example, a score of 100% in the self-care domain reflects a patient who achieved the highest possible score in that domain. Solid triangles reflect the patient's initial score, which can be tableed based on assessments at or immediately after admission. Black triangles reflect the patient's current score. Dashed triangles reflect the patient's expected score. By examining the scores in this way, clinicians can easily identify areas in which the patient has improved and areas where additional treatment or other care may be useful. For example, after examining the plots in Figure 4, since the self-care and mobility scores are below the expected scores in those domains, clinicians can determine that further care should be focused on the self-care and mobility domains.

[0152]

[0151] Predictive estimations can be derived in various ways. In one embodiment, hierarchical linear modeling (HLM) can be used that incorporates information about past patient diagnoses, the severity of those diagnoses (a "case mix group" which measures the severity of the patient's condition within a diagnosis), the date the patient was measured, and the scores for those dates. This modeling can output predictive curves for all severities within all diagnoses up to a 50-day hospital stay. When plotting the information, the x-axis can be the number of days since admission, and the y-axis can be the IRT(MAP) score.

[0153]

[0152] Other prediction methods, including data science methods such as neural networks and random forest models, can also be used. Furthermore, additional patient information can be incorporated into the prediction process.

[0154]

[0153] In one embodiment, a patient can be evaluated using IRT result measurements 180 over multiple days. For example, a patient can be evaluated on the first day based on a first subset of questions from the IRT result measurements 180, and then on the second day based on a second subset of questions. A data feed can be set up to collect the most recent item values.

[0155]

[0154] The adaptive test can be used so that items within the IRT result measurement 180 are selected for evaluation based on the scores of items that have already been evaluated. For example, a clinician can evaluate a patient using items within the IRT result measurement 180 from the FIST test, calculate an initial IRT score based on the results, and select the optimal next item (or multiple next items) based on the initial IRT score. This process can be repeated until the patient's score can be determined to be accurate within a given level of uncertainty. For example, if the uncertainty is 0.3 or less, the adaptive test method can stop providing additional items for evaluation and provide a final IRT score for consideration by the patient, clinician, or others.

[0156]

[0155] Figure 5 shows an illustrative chart of specific data regarding patient scores in the self-care area. Each row of the chart relates to a single item. For example, the top row relates to the wooden block grasping test. As explained with respect to Figure 3, each row of this chart is divided into different shaded sections. The length of each section reflects how the score of that item relates to the AQ score. For example, section b1 reflects how a score of 1 for the grasping item relates to the AQ score.

[0157]

[0156] Figure 5 further shows the patient's current and expected functional status for each item / task in the self-care area. It should be understood that this chart can display data from mobility, cognition, or other areas. The "Selection of Length of Stay" scroll bar allows clinicians to compare each level of competence (e.g., current vs. expected) for all items for various lengths of stay. Such comparisons can help clinicians determine whether, and if so, further hospitalization may be beneficial for the patient, and if so, to what extent.

[0158]

[0157] To determine whether additional intervention is appropriate, clinicians can review the IRT score along with the patient's score on specific FIM items. For example, if a patient's AQ score is 1, a score of 4 on the FIM toileting ability measure would be expected. However, if the FIM toileting ability measure is lower, clinicians can use such a measurement as a guide to adjust treatment to target improvement in toileting ability in particular.

[0159]

[0158] Figures 6A and 6B show the “FIM Explorer” section, which has the functionality to allow clinicians to select and / or set goals for each FIM-specific task. For example, in Figure 6A, “4 - Minimal assistance” is selected as the treatment goal for the feeding task. Once a task-specific goal is selected, a comparison chart, as shown in Figure 7, may be displayed. This chart may allow therapists or other clinicians to compare whether the goal is set too high or too low compared to the vertical line on the chart. The vertical line is derived from the score of the selected goal and converted into an IRT score.

[0160]

[0159] Figure 8 shows various plots of the self-care domain compared to the patient's FIM score. As shown in Figure 8, the assessment areas within the self-care domain include balance, upper limb function, and swallowing. In one embodiment, incomplete FIM executions can be omitted from the plot to avoid confusion between low scores and simply incomplete executions.

[0161]

[0160] Prediction

[0162]

[0161] AQ score predictions can be based on various factors such as healthcare service group, case mix group (CMG), and / or length of hospital stay. Within the CMG, age may be a factor used to aid in the prediction.

[0163]

[0162] The data generated by the predictive model can be used in various ways. For example, a patient's length of hospital stay can be predicted based on the patient's medical condition, level of injury, and other demographic and clinical characteristics. As another example, if a patient is underperforming in a given area, clinicians can focus on that area for more intensive treatment. As yet another example, if a patient's improvement in a certain area is gradually slowing down, clinicians can notice this and prioritize balanced treatment in that area. As yet another example, given some financial information, it becomes possible to evaluate the dollar value of the expected improvement over a certain period and compare it to the cost of inpatient care over the same timeframe. The ratio of care value to care cost can be used to make discharge decisions. In addition, it is possible to predict success in other treatment settings. Given similar evaluations at other levels and locations of treatment (e.g., outpatients, SNF, etc.), the expected course of improvement in those settings can be revealed. In some cases, better decisions regarding care in those settings can be made.

[0164] [Table 6]

[0165] [Table 7]

[0166] [Table 8]

[0167] [Table 9]

[0168] [Table 10]

[0169] Table 11

[0170] Table 12

[0171] Table 13

[0172] Table 14

[0173] Table 15

[0174] Table 16

[0175] Table 17

[0176] Table 18

[0177] Table 19

[0178] Table 20

[0179] Table 21

[0180] Table 22

[0181] Table 23

[0182] Table 24

[0183] Table 25

[0184] Table 26

[0185] Table 27

[0186] Table 28

[0187] Table 29

[0188] Table 30

[0189] Table 31

[0190] Table 32

[0191] Table 33

[0192] Table 34

[0193] Table 35

[0194] Table 36

[0195] Table 37

Claims

1. A method for evaluating a patient, wherein the method is performed by a computing device, a. Selecting multiple evaluation items for the first evaluation of the patient in the first evaluation domain, wherein the first evaluation domain is one of the self-care domain, mobility domain, and cognitive domain. b. Selecting multiple evaluation items for the second evaluation of the patient in the second evaluation domain, wherein the second evaluation domain is another domain different from the first evaluation domain, such as the self-care domain, mobility domain, and cognitive domain. c. Receiving input of the first and second evaluations of the patient, wherein the input of the evaluations includes data relating to the patient's performance with respect to each of the plurality of evaluation items, c. Generating a composite IRT score based on the received evaluation input using Item Response Theory (IRT) analysis, wherein the IRT analysis is based on a statistical model constructed for the first and second evaluation domains, d. Storing the composite IRT score in the patient's electronic medical record, Methods that include...

2. The method according to claim 1, wherein each of the plurality of evaluation items for the first and second evaluations is selected using IRT analysis.

3. The method according to claim 1, comprising generating a discriminant matrix for the first and second evaluation regions.

4. The method according to claim 1, comprising generating a difficulty matrix for the first and second evaluation regions.

5. The method according to claim 1, wherein the first evaluation area or the second evaluation area is the self-care field, and each of the plurality of evaluation items is within the areas of balance, upper limb function, and swallowing.

6. The method according to claim 1, wherein the first evaluation area or the second evaluation area is the mobility field, and each of the plurality of evaluation items is within the areas of balance, wheelchair skills, position changes, bed mobility, and mobility.

7. The method according to claim 1, wherein the first evaluation area or the second evaluation area is the cognitive field, and each of the plurality of evaluation items is within the domains of cognition, memory, and communication.

8. The method according to claim 2, wherein each of the plurality of evaluation items is further selected using factor analysis for the first and second evaluations.

9. The method of claim 2, wherein each of the plurality of evaluation items for the first and second evaluations is further selected using classical test theory analysis.

10. A method for measuring the improvement of a rehabilitation patient, wherein the method is performed by a computing device, a. Selecting assessment items for evaluating the patient in the first and second assessment domains based on item response theory (IRT) analysis performed on multiple statistical models constructed for the first and second assessment domains, wherein the first assessment domain is one of the self-care domain, mobility domain, and cognitive domain, and the second assessment domain is another of the self-care domain, mobility domain, and cognitive domain, distinct from the first assessment domain. b. Receiving input of the first evaluation of the patient, wherein the first evaluation of the patient is based on evaluation items selected in the first evaluation area, c. Receiving input of a second evaluation of the patient, wherein the second evaluation of the patient is based on evaluation items selected in the second evaluation domain. d. Determining a composite IRT score for the patient based on the input received for the first and second evaluations, e. Comparing the composite IRT score with the predicted composite IRT score for the patient in the corresponding first and second evaluation domains, Methods that include...

11. The method according to claim 10, wherein the first evaluation area or the second evaluation area is the self-care field, and each of the plurality of evaluation items is within the areas of balance, upper limb function, and swallowing.

12. The method according to claim 10, wherein the first evaluation area or the second evaluation area is the mobility field, and each of the plurality of evaluation items is within the areas of balance, wheelchair skills, position changes, bed mobility, and mobility.

13. The method according to claim 10, wherein the first evaluation area or the second evaluation area is the cognitive field, and each of the plurality of evaluation items is within the domains of cognition, memory, and communication.

14. The method according to claim 10, wherein the evaluation items are further selected using factor analysis.

15. The method according to claim 10, wherein the evaluation items are further selected using classical test theory analysis.

16. The method according to claim 10, further comprising generating a plot including the composite IRT score and the predicted composite IRT score in a format suitable for display on a display screen.

17. The method according to claim 10, wherein the comparison is used to identify areas of treatment need.

Images