Device for patient data availability analysis
By analyzing the availability of model variables and data sources in the clinical data repository and selecting an optimized subset of models, the data availability problem under the influence of noise is solved, efficient data analysis and hypothesis verification are achieved, and clinical research decisions are simplified.
Patent Information
- Application Number
- CN201880085081.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2017-12-26
- Filing Date
- 2018-11-20
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art is difficult to effectively analyze and verify research hypotheses when extracting variables from clinical data repositories due to noise when extracting variables from clinical data repositories, and data quality challenges affect subsequent modeling steps.
Provides a device and system for patient data availability analysis, including input modules, data and model analysis modules, and optimization modules. Users can specify multiple models and data sources by analyzing the attributes of model variables, source variables and data records. Identify the set of available data records and select an optimized subset of models using availability metrics to identify data quality issues and support research hypotheses.
By analyzing data availability among multiple hypotheses, simplifying subsequent analytical decisions, improving the effectiveness and reliability of data use, mitigating the impact of noise, and providing a self-service clinical research data processing tool.
Smart Images

Figure CN111602202B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an apparatus for patient data availability analysis, a system for patient data availability analysis, a method for patient data availability analysis and a computer program element. Background Art
[0002] The general background of the present invention is the field of identifying patient records. Cardiovascular information systems (CVIS) have been used in the health care industry for many years. An activity that utilizes information and preferably knowledge from CVIS or clinical data repositories (CDRs) is to find historical cases that match specific new patients or query criteria in order to provide valuable references for decision makers (e.g., caregivers or physicians). There is a trend to perform data analysis on data collected from CDRs, which can provide huge valuable data without spending a lot of dedicated costs and labor for formal clinical research. For typical clinical research problems, many variables (features) need to be extracted from CDRs, and there can be multiple methods to obtain these variables from different sources through different rules, but due to the presence of noise in the data, the availability of the data is different. CDR data contains noise, because clinical practice must balance the timely treatment of patients with the absolutely complete and accurate records made about patients and treatments. In addition, since humans are involved in the processing and recording of data, noise is introduced again due to this incomplete data.
[0003] US2016 / 0019265A1 describes a technique for generating automated suggestions for merging multiple sources. In one embodiment, a set of one or more parameters related to a proposed merge for multiple merged sources is received. In response to receiving the set of one or more parameters, a set of one or more suggestions for merging multiple merged sources is generated and stored on at least one of a volatile or non-volatile computer-readable storage medium. In some embodiments, the set of one or more recommendations may indicate how to improve performance associated with merging the multiple sources into a set of one or more targets based on a specific merge scenario. The set of one or more recommendations may be displayed during merge planning for the multiple merged sources.
[0004] WO 2014 / 201515 A1 discloses a system and process for processing medical data based on machine learning techniques (e.g., for determining the likelihood or risk of an adverse event or outcome for a person). The outcome may be, for example, the risk of attempting suicide, the likelihood of surviving cancer, multiple readmissions, etc. Summary of the invention
[0005] It would be advantageous to have an improved apparatus and associated method for patient data usability analysis and an improved system for patient data usability analysis to enable self-service of clinical research data records including noisy practice data.
[0006] The objects of the invention are solved by the subject matter of the independent claims, wherein further embodiments are incorporated in the dependent claims. It should be noted that the following described aspects and examples of the invention also apply to an apparatus for patient data availability analysis, a system for patient data availability analysis, a method for patient data availability analysis and a computer program element.
[0007] According to a first aspect, there is provided an apparatus for patient data availability analysis, comprising:
[0008] Input module;
[0009] Data and model analysis modules; and
[0010] Optimize module.
[0011] The input module is configured to enable a user to specify a plurality of models. Each model provides an output according to model variables. The input module is also configured to enable a user to define at least some model variables for the plurality of models. The input module is also configured to enable the user to specify source variables. Model variables can be derived from one or more source variables. The input module is also configured to enable a user to specify at least one data source. The input module is also configured to receive a plurality of data records from the at least one data source. Each data record includes at least one attribute. The data and model analysis module is configured to determine a set of available data records for each model according to the plurality of data records. The determination for a model includes utilizing model variables for the model, associated source variables, and the at least one attribute for the plurality of data records. The data and model analysis module is also configured to determine a plurality of availability metrics for the corresponding plurality of models, the determination including utilizing the determined set of available data records for each model. The optimization module is configured to sort the plurality of models and select a subset of models from the plurality of models as top models for data availability. The selection includes utilizing the plurality of availability metrics.
[0012] In this way, users can easily understand the trade-off between the required input variables and the data size to support the required variable set. They can obtain optimized data availability based on their requirements for data features. They can also identify potential data quality issues through variable availability analysis and check whether the data supports their research hypotheses for further verification through subsequent data analysis.
[0013] In other words, data availability is analyzed during the selection process between multiple hypotheses (models) so that further prediction and modeling steps using the available data and the selected model can be more efficient and useful.
[0014] Therefore, data availability selection will be done at the first level to determine the best availability between different variable sources and export choices to simplify and optimize decisions for subsequent analyses, otherwise each model will have a non-trivial combination to be selected and arbitrary selection will introduce data quality challenges in later modeling steps.
[0015] In an example, the input module is configured to enable a user to provide constraints on the desired data, and the data and model analysis module is configured to determine, for each of the plurality of models, statistics of the variables specified in the constraints on the desired data provided by the user. The selection of a subset of models may then include utilizing the determined statistics of the variables for the plurality of models.
[0016] In an example, the optimization module is configured to output information regarding whether any of the available data records for the top model are outside the constraints on desired data input by the user.
[0017] In other words, the device outputs information that compromises the user's expectations (constraint violations), allowing the user to better determine whether the model and its associated available data are acceptable.
[0018] In an example, the data and model analysis module is configured to determine at least one model property of at least one model of the plurality of models.Then, the selection of the subset of models may include utilizing the at least one model property.
[0019] In an example, the optimization module is configured to determine a best model of the subset of models.
[0020] In an example, the optimization module is configured to rank the subset of models.
[0021] In an example, the data and model analysis module is configured to determine a number of clean data records of available data records for each model, wherein each of the available data records includes a plurality of values. Clean data records may then be determined as those data records of the available data records that are missing less than a threshold number of values, and the usability metric for a model may include the number of clean data records for the model.
[0022] In an example, the availability measure for the model comprises a number of clean data records for the model divided by a total number of clean data records.
[0023] In an example, the threshold is a percentage of the number of values, which includes a zero percentage.
[0024] Therefore, clean data records can be those with no missing values or those with the number of missing values below a threshold percentage.
[0025] In an example, the availability measure for a model has a value weighted by the number of model variables for the model.
[0026] In an example, the data and model analysis module is configured to derive at least one model variable for at least one of the plurality of models. The deriving includes matching at least some of the model variables input by the user with corresponding at least one source variable.
[0027] In this way, the device can determine the correct model variables for the model based on the information provided by the user.
[0028] In an example, the input module is configured to enable the user to input a search query. The user specifying the plurality of models may include the input unit identifying at least one model according to the search query.
[0029] In other words, the user can explicitly define the model to be used, but can also provide a search query, and the input unit retrieves existing models, such as risk models, from a knowledge base in the clinical field based on the user search or keywords, thereby simplifying the input of model variables and variable derivation rules.
[0030] According to a second aspect, there is provided a system for patient data availability analysis, comprising:
[0031] at least one data source;
[0032] An apparatus for patient data availability analysis according to the first aspect and optionally any related example; and
[0033] Output unit.
[0034] The plurality of patient records are provided to the input unit from the at least one data source, and the output unit is configured to output information related to a top model of data availability.
[0035] According to a third aspect, there is provided a method for patient data availability analysis, comprising:
[0036] a) specifying multiple models, where each model provides output based on model variables;
[0037] b) defining at least some model variables for the plurality of models;
[0038] c) specifying source variables, wherein model variables can be derived from one or more source variables;
[0039] d) specifying at least one data source;
[0040] e) receiving a plurality of data records from the at least one data source, wherein each data record comprises at least one attribute;
[0041] f) determining a set of available data records for each model based on the plurality of data records, the determining of the model comprising using a model variable for the model, an associated source variable, and at least one attribute of the plurality of data records;
[0042] g) determining a plurality of availability measures for the corresponding plurality of models, said determining comprising utilizing the determined set of available data records for each model; and
[0043] h) selecting a set of models from the plurality of models as a top model of data availability, wherein the selecting includes utilizing a plurality of availability metrics.
[0044] Therefore, advanced data analytics methods can be direct integrated modules in cardiovascular information systems (CVIS) / clinical data repositories (CDRs) to allow users / physicians to effectively and efficiently transform the stored information into knowledge.
[0045] According to another aspect, a computer program element for controlling an apparatus and / or a system as described above is provided, wherein the computer program element is adapted to perform the steps of the method as described above when being run by a processing unit.
[0046] A computer-readable medium is also provided, wherein the computer-readable medium has stored therein the computer unit as described above.
[0047] Advantageously, the benefits provided by any of the above aspects also apply to all other aspects, and vice versa.
[0048] The above aspects and examples will become apparent from and will be elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Exemplary embodiments will be described below with reference to the accompanying drawings:
[0050] Figure 1 A schematic setup of an example of an apparatus for patient data availability analysis is shown;
[0051] Figure 2 shows a schematic setup of an example of a system for patient data availability analysis;
[0052] Figure 3A method for patient data availability analysis is shown;
[0053] Figure 4 shows a detailed high-level view of a system for patient data availability analysis related to an example of acute kidney injury (AKI); and
[0054] Figure 5 A detailed workflow related to a system for performing patient data availability analysis from multiple data sources including CDRs, laboratory information systems (LIS), and admission records (ANs) is shown. DETAILED DESCRIPTION
[0055] Figure 1 An example of an apparatus 10 for patient data availability analysis is shown. The apparatus 10 includes an input module 20, a data and model analysis module 30, and an optimization module 40. The input module 20 is configured to enable a user to specify a plurality of models. Each model provides an output according to a model variable. The input module 20 is also configured to enable a user to define at least some model variables for the plurality of models. The input module 20 is also configured to enable a user to specify a source variable. The model variable may be derived from one or more source variables. The input module 20 is configured to enable a user to specify at least one data source. The input module 20 is also configured to receive a plurality of data records from the at least one data source. Each data record includes at least one attribute. The data and model analysis module 30 is configured to determine a set of available data records for each model according to the plurality of data records, the determination for the model including utilizing the model variables for the model, the associated source variables, and at least one attribute for the plurality of data records. The data and model analysis module 30 is also configured to determine a plurality of availability metrics for the corresponding plurality of models. The determination of a plurality of availability measures for the corresponding plurality of models includes utilizing the determined set of available data records for each model. The optimization module 40 is configured to select a subset of models from the plurality of models as top models of data availability. The selection includes utilizing the plurality of availability measures.
[0056] In an example, the at least one attribute value includes one or more of the following: demographics of a particular patient, lifestyle information, medical information, care provider information, medical history and risk factor information, previous medical visits, procedure information, etc. Medical information includes basic health information, lesion information, device information, and follow-up information of the patient. Therefore, at least one attribute may include demographic data (age, gender, weight, race, etc.); presence / absence of chronic behavioral conditions (smoking, heavy drinking, consumption of various recreational drugs, etc.); presence / absence of various chronic clinical conditions (hypertension, diabetes, asthma, heart disease, etc.); presence / absence of various acute diseases (pneumonia or other acute respiratory diseases, various tumor conditions, etc.); characteristics associated therewith (e.g., cancer stage and grade); medical and care provider information, etc. Therefore, the at least one attribute value is a data field in a medical record, which may represent a medical problem, such as a symptom (e.g., bleeding), a serious medical outcome (e.g., death), or a medical state (e.g., normal), or other background information, such as a medical history, laboratory test results, or demographic data of the patient. The attribute value may be, for example, a value of contextual information or the likelihood of a medical problem.
[0057] In an example, the at least one data source includes one or more of a clinical data repository (CDR), a laboratory information system (LIS), and an admission record (AN).
[0058] In this way, for example, inherited noise including incomplete and multiple potentially conflicting data sources can be mitigated, providing an efficient and useful means to help users understand and make optimal decisions regarding assumptions (models) about the status (availability) of CDR data.
[0059] According to one example, the input module is configured to enable a user to provide constraints on desired data. The data and model analysis module is configured to determine, for each of the plurality of models, statistics of the variables specified in the constraints on the desired data provided by the user. Then, the selection of a subset of models includes utilizing the determined variable statistics for the plurality of models.
[0060] In an example, the statistical results of the variables specified in the user constraints include, for example, record count, gender, age in expected number of records, gender ratio, age range.
[0061] According to an example, the optimization module is configured to output information on whether any of the available data records for the top model are outside of said constraints on desired data input by said user.
[0062] According to an example, the data and model analysis module is configured to determine at least one model property of at least one model of the plurality of models.Then, the selection of the subset of models comprises utilizing the at least one model property.
[0063] According to an example, the optimization module is configured to determine a best model of the subset of models.
[0064] According to an example, the optimization module is configured to rank the subset of models.
[0065] According to an example, the data and model analysis module is configured to determine the number of clean data records of the available data records for each model. Each available data record may include multiple values, and the clean data records may be determined as those data records in the available data records that are missing less than a threshold number of values. Then, the availability metric for the model includes the number of clean data records for the model.
[0066] According to an example, the availability measure for the model comprises a number of clean data records for the model divided by a total number of clean data records.
[0067] According to an example, the threshold is a percentage of the number of values, which includes a zero percentage.
[0068] Thus, for example, clean data records may be those with no missing values, with less than 1% missing values, with less than 2% missing values, with less than 5% missing values, with less than 10% missing values, with less than 20% missing values.
[0069] According to an example, the availability measure for a model has a value weighted by the number of model variables for the model.
[0070] In an example, the at least one data source includes a CDR, a LIS, and an admission record (AN).
[0071] According to an example, the data and model analysis module is configured to derive at least one model variable for at least one of the plurality of models. The deriving includes matching at least some of the model variables input by the user with corresponding at least one source variable.
[0072] According to an example, the input module is configured to enable a user to input a search query.Then, the user specifying the plurality of models may include the input unit identifying at least one model according to the search query.
[0073] Figure 2 An example of a system 100 for patient data availability analysis is shown. The system 100 includes at least one data source 110 and a Figure 2The described apparatus 10 for patient data availability analysis. The system 100 further comprises an output unit 120. A plurality of patient records are provided to the input unit 20 from at least one data source 110. The output unit 120 is configured to output information related to a top model of data availability.
[0074] Figure 3 A method 200 for patient data availability analysis is shown in its basic steps. The method 200 includes:
[0075] In a specifying step 210, also referred to as step a), a plurality of models are specified, wherein each model provides an output based on a model variable;
[0076] In a definition step 220, also referred to as step b), at least some model variables are defined for a plurality of models;
[0077] In a specifying step 230 , also referred to as step c), source variables are specified, wherein the model variables may be derived from one or more source variables;
[0078] In a specifying step 240, also referred to as step d), at least one data source is specified;
[0079] In a receiving step 250, also referred to as step e), a plurality of data records are received from at least one data source, wherein each data record comprises at least one attribute;
[0080] In a determining step 260, also referred to as step f), a set of available data records for each model is determined from the plurality of data records, the determination for the model comprising utilizing a model variable for the model, an associated source variable, and at least one attribute for the plurality of data records;
[0081] In a determining step 270, also referred to as step g), a plurality of usability measures for a corresponding plurality of models are determined, said determining comprising utilizing a determined set of available data records for each model; and
[0082] In a selection step 280, also referred to as step h), a subset of models from the plurality of models is selected as top models of data availability, wherein the selection includes utilizing a plurality of availability measures.
[0083] In an example, wherein the method includes providing constraints on the expected data, and includes determining, for the top model, statistics of the variables specified in the constraints on the expected data. Then, selecting a subset of the models includes utilizing the statistics of the determined variables.
[0084] In an example, the method includes outputting information regarding whether any of the available data records of the model are outside (not satisfied) the constraints of the expected data.
[0085] In an example, the method includes determining at least one model property for at least one model of the plurality of models. Selecting a subset of models then includes utilizing the at least one model property.
[0086] In an example, the method includes determining a top model in the subset of models.
[0087] In an example, the method includes ranking a subset of models.
[0088] In an example, the method includes determining a number of clean data records in available data records for each model, wherein each available data record includes a plurality of values. The clean data records are then determined to be those data records in the available data records that are missing less than a threshold number of values. The usability metric for a model then includes the number of clean data records for the model.
[0089] In an example, determining the availability measure for the model includes calculating a number of clean data records for the model divided by a total number of clean data records.
[0090] In an example of the method, the threshold is a percentage of the number of values, which includes a zero percentage.
[0091] In an example of the method, the availability measure for a model has a value weighted by the number of model variables for the model.
[0092] In an example, the method includes deriving at least one model variable for at least one of a plurality of models, the deriving including matching at least some of the model variables to corresponding at least one source variable.
[0093] In an example, the method includes inputting a search query, and wherein specifying the plurality of models includes identifying at least one model based on the search query.
[0094] Now combine Figure 1 and 2 The apparatus, system and method for patient data availability analysis are described in more detail. 4-5.
[0095] Figure 4A detailed high-level view of a system for patient data availability analysis related to an example of acute kidney injury (AKI) is shown. As mentioned above, cardiovascular information systems (CVIS) have been used in the healthcare industry for many years. The main function of current CVIS is to store and evaluate patients' medical records - for example, see US6604115B1. Some advanced CVIS are also integrated with other systems in the hospital, such as electronic health records (EHR), laboratory information systems (LIS), etc. In this way, physicians will have an overview of the patient's health records. Typically, CVIS is used for reporting, planning and management purposes, while the clinical data repository (CDR) further records more in-depth information to perform advanced data analysis, such as modeling and predictive analysis. Here, CVIS and CDR are collectively referred to as CDR.
[0096] As in Figure 4 The exemplary system shown in the figure is specifically embodied and described below with reference to Figure 5 The detailed workflow shown is described in Figure 1-3 The described apparatus, method and system for patient record identification provide a module directly integrated in the CDR, allowing the user / physician to effectively transform the stored information into knowledge.
[0097] Typically, the focus of a (clinical) research topic is to validate a hypothesis (also referred to here as a model) or to pick out relevant hypotheses from the collected (or to be collected) data. Data availability is therefore crucial to determine whether a hypothesis can be validated before actual predictive analysis and risk modeling can be performed. In traditionally designed studies, data are collected in a very rigorous manner with commitment to quality, strict inclusion / exclusion criteria and labor-intensive follow-up. This way of working limits scalability and requires significant cost and labor to obtain representative conclusions for the entire population. On the other hand, with the widespread adoption of electronic records and health information systems and in-depth CDRs, hypotheses can be generated and validated using large amounts of existing data collected day after day. The challenge lies in the inherited noise, including incomplete and multiple potentially conflicting data sources. Data availability can help users understand and validate hypotheses (models) and make the best decisions regarding the status (availability) of CDRs, thereby facilitating the development of a new clinical research model. Figure 4-5 System data for the apparatus, systems, and methods embodied in the detailed examples shown in .
[0098] Figure 4A system is shown that utilizes CDR data and data from other systems to build an acute kidney injury (AKI) prediction model. Several existing AKI risk scores or hypotheses (models 1 to 5) are derived by logistic regression from patient data. In order to verify whether the existing risk scores are predictive of specific data and further build population-specific models with more informative variables (such as logistic regression), it is first necessary to clearly understand how much data can support the existing models and new models. Figure 4 As shown in , CDR and other data sources have missing values and may provide different availability for various variables of the model. The variables of the model are shown here as: age, sex; anemia; diabetes; CHF; use of IABP / balloon pump; hypertension; contrast volume and other model variables are possible.
[0099] Many challenges on multiple levels were solved:
[0100] 1. Different models have different variables, and data may support them with significantly different availability.
[0101] 2. The same variable can come from different data sources or be derived using multiple methods (e.g., combining multiple laboratory results), and the data can support them in significantly different ways.
[0102] 3. Introducing new variables to verify new hypotheses (to build new models) is also constrained by data availability (less clean data for more variables).
[0103] continue Figure 4 , taking the AKI data provided as an example, there are laboratory information system (LIS) data, admission records from EMR, CDR (coded as Cathpci) data, and other sources. As mentioned above, each data source may have considerable missing values for various reasons. Similarly, different models (see Models 1-5) result in different data availability (ranging from 279 records to 684 records). The determination of available data records enables the subsequent steps of prediction and modeling to be performed effectively.
[0104] To help understand the problem solved by the described apparatus, system, and method for patient data availability analysis, consider the following: Anemia (yes / no) can be obtained directly from the CDR, with considerable missing values, and can also be derived by checking the LIS if the baseline hemoglobin is <10g / dL (100g / L). However, the different combinations of ways to derive a single variable, multiplied by the various variables that need to be derived, will result in a large number of data availability combinations that the user cannot directly handle. For simplicity, assume that all models have the same variable requirements. Imagine that there are 5 models, each with 8 variables, 2 of the 8 variables have 4 different ways to derive each, and the other 3 have 2 possible sources. The remaining 3 variables come from the same source and do not need to be derived. Therefore, there are 5x(42x23x13)=640 combinations. Adding each additional variable from these choices to build a new model will result in another level of multiplication (*2 each time if the new variable has two alternative ways). Furthermore, for some clinically relevant studies, users (clinicians) will want to meet requirements such as a minimum number of patients, gender ratio, age range, etc. This further complicates feasibility and makes it unclear whether users can be confident that their research goals can be achieved: if the hypothesis can be rigorously tested, or if no conclusions can be drawn simply because the data are not available.
[0105] exist Figure 4-5 The apparatus, system and method embodied in the detailed example shown in solves this problem by analyzing data availability during the selection period between multiple hypotheses (models) before further prediction and modeling steps. Figure 4 As shown on the right side of , it is desirable to make data availability selections at the first level to determine the best availability between different variable sources and export selections to simplify decisions for subsequent analysis, otherwise each model will have a large number of combinations to choose from. And it is expected that ( Figure 4The system (not shown) provides the best available model to the user based on the data (optionally also with predictive capabilities that take into account the available data). In this way, a data processing system is provided that weighs data availability against the tradeoffs of selecting different hypotheses for noisy data (e.g., models for predictive analysis). In other words, as described herein, a data availability analysis system for a clinical data repository (CDR) can provide self-service for clinical research on noisy practice data. For a typical clinical research problem, several input variables (features) need to be extracted from the CDR, and there can be multiple ways to obtain these variables from different sources through different rules, but due to the presence of noise in the data, the availability of the data is different. Using the system, users can easily understand the tradeoff between the required input variables and the size of the data that supports the required set of variables. They can also identify potential data quality issues through variable availability analysis and check whether the data supports their research hypotheses (models) for further verification through subsequent data analysis.
[0106] Figure 5 A detailed workflow related to a system for patient data availability analysis is shown, the system includes the following major elements, as described below, and runs related functions in the workflow:
[0107] 1. Input module, which is used by users to specify the data sources, models, model variables and data expectations (constraints) collected as specifications.
[0108] • The system provides an interface for the user to input variables for the model, specify source variables for variable derivation, and provide expected data constraints.
[0109] It retrieves existing risk models from the knowledge base of the clinical field based on user searches or keywords, thereby simplifying the input of model variables and variable derivation rules. However, it is not necessary to retrieve risk models in this way.
[0110] 2. Data and hypothesis (model) analysis module, used to merge multiple data sources and standardize statistical results on data availability.
[0111] The system maintains a list of relevant data sources, such as CDR, LIS, and Admission Record (AN).
[0112] · It matches the hypothesized input variables, the corresponding source variables to derive the hypothesized variables, the target variables – thus, retrieving the data after specifying the model variables and the target variables.
[0113] For a specific model, its variables and source variables, and corresponding data attributes from multiple data sources, the system retrieves the available data and calculates the following:
[0114] i. Availability measure, an example of which may be the number of available records;
[0115] ii. Statistics of variables specified in user constraints (e.g., number of records, gender, age in expected number of records, gender ratio, age range); and
[0116] iii. If the hypothesis is a model that can be evaluated on, for example, the area under the curve (AUC) on a receiver operating characteristic curve (ROC), then calculate the model performance. Calculation of model performance can help determine the availability of patient data, but it is not required.
[0117] It relies on a connected self-service data cleansing system to unify aliases of equivalent attributes, merge conflicts from different data sources, and detect and remove noisy data. However, this functionality is not required.
[0118] 3. An optimization module for optimizing the availability metric taking into account data availability and user data expectations (constraints).
[0119] • The system works based on the derived statistics (usability metrics, variable statistics of user constraints, and optionally model performance) and optimizes the overall usability score from various choices of variables.
[0120] The system picks the top models based on a single overall score or a multi-objective pareto frontier and presents them to the user.
[0121] The system outputs information that compromises user expectations (violates constraints).
[0122] 4. The new attribute analysis module is used to accommodate data availability analysis of new attributes of interest to the user, and is therefore effectively an "add-on" to the above workflow.
[0123] The system also supports the introduction of new variables of interest based on user needs to collect more information for advanced analysis.
[0124] The system performs similar steps and gives new model data availability results.
[0125] Therefore, the new attribute analysis module addresses the case where the user wants to add more attributes (model variables) later.
[0126] Provides more details on some of the features listed above:
[0127] 1. The “availability metrics” described in point 2:
[0128] In a baseline example, the count of records with no missing values (clean set) for a particular variable alternative selection for a particular model can be used as the availability measure, optionally normalized by the total number of records with no missing values in the target variable.
[0129] In another example, the normalized counts described above can be further weighted by accepting a certain percentage of missing records, introducing a soft control when the available data cannot be completely cleaned.
[0130] - In another example, the measure can be further weighted by the number of variables, with a preference for more descriptive models (more variables) with sufficient available data.
[0131] 2. The "Optimization Module" described in point 3 optimizes all combinations of model and variable alternatives:
[0132] In one baseline example, all combinations may be fully evaluated against an optimization metric (as described above, the baseline may be an availability metric).
[0133] In another example, if the number of combinations is very large, a random search algorithm, such as a random search or a genetic algorithm, may be employed.
[0134] All user-specified data expectations (availability constraints) can be set as hard constraints, and any combination that does not meet the constraints will not be optimized.
[0135] In another example, user-specified availability constraints can be incorporated as part of the new availability metric, or added as an additional penalty term. The availability optimization results can then be such that they compromise (violate) certain user-specified expectations, such as a gender ratio of 1:3, while the expected ratio is 1:4. In terms of violating user-specified expectations, the differences can be presented to the user for selection and to help guide the decision.
[0136] In a further exemplary embodiment, a computer program or a computer program element is provided, characterized in that it is configured to execute the method steps of the method according to one of the preceding embodiments on a suitable system.
[0137] The computer program element can therefore be stored on a computing unit, which can also be a part of an embodiment. The computing unit can be configured to perform the steps of the above method or cause the execution of the steps of the above method. In addition, it can be configured to operate the parts of the above-mentioned device and / or system. The computing unit can be configured to automatically operate and / or execute the user's command. The computer program can be loaded into the working memory of a data processor. Therefore, a data processor can be equipped to perform a method according to one of the aforementioned embodiments.
[0138] This exemplary embodiment of the invention covers both a computer program that right from the start uses the invention and a computer program that by means of an update turns an existing program into a program that uses the invention.
[0139] Furthermore, the computer program element may be able to provide all necessary steps to complete the procedure of the exemplary embodiment of the method as described above.
[0140] According to another exemplary embodiment of the present invention, a computer-readable medium, such as a CD-ROM, a USB memory stick or the like, is proposed, wherein the computer-readable medium has a computer program element stored thereon, the computer program element being as described in the preceding section.
[0141] The computer program can be stored and / or distributed on suitable media, such as optical storage media or solid-state media provided together with other hardware or as part of other hardware, but the computer program can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.
[0142] However, the computer program may also be provided over a network like the World Wide Web and may be downloaded from such a network into a working memory of a data processor. According to a further exemplary embodiment of the invention, a medium for making a computer program element available for downloading is provided, the computer program element being arranged to perform one of the previously described embodiments of the invention.
[0143] It must be noted that embodiments of the present invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method claims, while other embodiments are described with reference to device claims. However, it can be concluded by a person skilled in the art from the above and following descriptions that, unless otherwise indicated, any combination of features relating to different subject matters, in addition to any combination of features belonging to the same type of subject matter, is also considered to be disclosed by the present application. However, all features can be combined to provide synergistic effects that exceed the simple sum of the features.
[0144] Although the present invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description should be considered illustrative or exemplary rather than restrictive. The present invention is not limited to the disclosed embodiments. Other variations of the disclosed embodiments can be understood and implemented by those skilled in the art in practicing the claimed invention by studying the drawings, the disclosure, and the dependent claims.
[0145] In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. Although certain measures are recited in mutually different dependent claims, this does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.
Claims
1. A device (10) for patient data availability analysis, comprising: Input module (20); Data and model analysis module (30); as well as Optimization module (40); wherein the input module is configured to enable a user to specify a plurality of models, wherein each model is used in clinical research and predicts disease outcomes, wherein each model provides outputs based on model variables, and the input module is configured to enable the user to define at least some model variables for the plurality of models, and the input module is configured to enable the user to specify source variables, wherein model variables can be derived from one or more source variables, and the input module is configured to enable the user to specify at least one data source; wherein the input module is configured to receive a plurality of data records from the at least one data source, wherein each data record comprises at least one attribute; wherein the data and model analysis module is configured to determine a set of available data records for each model based on the plurality of data records, the determination for a model comprising utilizing the model variables for the model, the associated source variables, and the at least one attribute for the plurality of data records; wherein the data and model analysis module is configured to determine a number of clean data records in available data records for each model, wherein each of the available data records includes a plurality of values, and wherein the clean data records are determined to be those data records in the available data records that are missing less than a threshold number of values; wherein the data and model analysis module is further configured to determine a plurality of usability metrics for a corresponding plurality of models, wherein the determining comprises utilizing the determined set of available data records for each model, and wherein the usability metrics for a model comprise a number of clean data records for the model; and Wherein the optimization module is configured to rank the plurality of models and select a subset of models from the plurality of models as top models for data availability, wherein the selecting includes utilizing the plurality of availability metrics.
2. The device according to claim 1, wherein: The input module is configured to enable a user to provide constraints on desired data, and wherein the data and model analysis module is configured to determine, for each of the plurality of models, statistics of the variables specified in the constraints on desired data provided by the user, and wherein the selection of the subset of models includes utilizing the statistics of the determined variables for the plurality of models.
3. The device according to claim 2, wherein: The optimization module is configured to output information regarding whether any of the available data records for the top model are outside the constraints on desired data input by the user.
4. The device according to any one of claims 1 to 3, wherein: The data and model analysis module is configured to determine at least one model property for at least one model of the plurality of models, and wherein the selection of the subset of models includes utilizing the at least one model property.
5. The device according to any one of claims 1 to 3, wherein: The optimization module is configured to determine a best model for the subset of models.
6. The device according to any one of claims 1 to 3, wherein: The optimization module is configured to rank the subset of models.
7. The device according to claim 1, wherein: The availability measure for the model comprises the number of clean data records for the model divided by the total number of clean data records.
8. The device according to claim 1 or 7, wherein: The threshold is a percentage of the number of values, which includes a zero percentage.
9. The device according to any one of claims 1 to 3, wherein: The availability measure for a model has a value weighted by the number of model variables for the model.
10. The device according to any one of claims 1 to 3, wherein: The data and model analysis module is configured to derive at least one model variable for at least one of the plurality of models, the deriving comprising matching at least some of the model variables input by the user with corresponding at least one source variable.
11. The device according to any one of claims 1 to 3, wherein: The input module is configured to enable the user to input a search query, and wherein the user specifying the plurality of models includes the input unit identifying at least one model based on the search query.
12. A system (100) for patient data usability analysis, comprising: at least one data source (110); The device (10) for patient data availability analysis according to any one of the preceding claims; as well as Output unit (120); wherein the plurality of patient records are provided to the input unit (20) from the at least one data source; and Wherein, the output unit is configured to output information related to the top model of data availability.
13. A method (200) for patient data availability analysis, comprising: a) specifying (210) a plurality of models, wherein each model is used in a clinical study and predicts a disease outcome, wherein each model provides an output based on a model variable; b) defining (220) at least some model variables for the plurality of models; c) specifying (230) source variables, wherein the model variables can be derived from one or more source variables; d) specifying (240) at least one data source; e) receiving (250) a plurality of data records from the at least one data source, wherein each data record comprises at least one attribute; f) determining (260) a set of available data records for each model based on the plurality of data records, the determining of the model comprising utilizing the model variables for the model, the associated source variables, and the at least one attribute of the plurality of data records, and determining a number of clean data records in the available data records for each model, wherein each of the available data records comprises a plurality of values, and wherein the clean data records are determined to be those data records in the available data records that are missing less than a threshold number of values; g) determining (270) a plurality of availability measures for a corresponding plurality of models, wherein the determining comprises utilizing the determined set of available data records for each model, and wherein the availability measure for a model comprises a number of clean data records for the model; and h) selecting (280) a subset of models from the plurality of models as top models for data availability and ranking the plurality of models, wherein the selecting includes utilizing a plurality of availability metrics.
14. A computer program element for controlling an apparatus according to one of claims 1 to 11 and / or a system according to claim 12, the computer program element being configured to perform the method according to claim 13 when run by a processor.
