Imputation Model Conviction Scores for Missing Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer-based reasoning systems face challenges with incomplete training data, particularly when older systems lack sensors and data collection mechanisms used in newer systems, leading to sparse data that cannot be effectively used for training models.
Innovation Solution
The techniques involve analyzing cases in a computer-based reasoning model to determine missing fields, calculating conviction scores, and using an imputation model to fill in missing data, which can be a supervised machine learning model, to create a more complete and updated model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If older oil pumps are retrofitted with modern sensors and data collection mechanisms, then data completeness for newer systems is improved, but data from the period before retrofitting remains incomplete and cannot be used for training
Solution Approach 1:
The patent introduces an imputation model as an intermediary that fills in missing data fields in historical cases. The imputation model uses supervised machine learning to predict and populate missing sensor data, enabling older data to be used for training while maintaining data completeness without requiring physical retrofitting of legacy equipment.
Solution Approach 2:
The patent changes the state of historical data from incomplete to complete by applying imputation techniques. The imputation model transforms missing or sparse data fields into populated values, effectively changing the data completeness parameter of historical cases to enable their use in training modern reasoning systems.
2Loss of information
If data imputation is performed on all missing fields, then data completeness is improved, but computational resources and processing time increase
Solution Approach 1:
The patent applies partial imputation by focusing on the most critical missing fields rather than attempting to fill all possible gaps. The system identifies and imputes only the most important missing data fields needed for effective training, rather than performing exhaustive imputation on every missing value, thus reducing computational overhead while maintaining sufficient data completeness.
3Loss of time
If imputation models are trained on limited data, then model development time is reduced, but imputation accuracy decreases
Solution Approach 1:
The patent performs preliminary data preparation and feature engineering before training the imputation model. By pre-processing the available data, selecting relevant features, and preparing the training dataset in advance, the system enables the imputation model to achieve higher accuracy even with limited training data, reducing the need for extensive data collection before model development.
Data Source
AI summary
Techniques are provided for imputation in computer-based reasoning systems. The techniques include performing the following until there are no more cases in a computer-based reasoning model with missing fields for which imputation is desired: determining which cases have fields to impute (e.g., missing fields) in the computer-based reasoning model and determining conviction scores for the cases that have fields to impute. The techniques proceed by determining for which cases to impute data based on conviction scores. For each of the determined one or more cases with missing fields to impute data is imputed for the missing field, and the case is modified with the imputed data. Control of a system is then caused using the updated computer-based reasoning model.