Imputation Model for Missing Data in Reasoning Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer-based reasoning systems face challenges with incomplete training data, known as sparse data, where older systems lack sensors and data collection mechanisms, leading to missing data fields that cannot be used for training models, even after retrofitting with modern sensors.
Innovation Solution
The techniques involve analyzing cases in the reasoning model to determine missing fields, calculating conviction scores for cases and features, and using an imputation model, such as a supervised machine learning model, to fill in the missing data, updating the model iteratively until no more data is missing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is collected from older systems without modern sensors, then historical data availability is improved, but data completeness deteriorates due to missing fields
Solution Approach 1:
The patent introduces an imputation model as an intermediary component that bridges the gap between incomplete historical data and the requirements of modern reasoning systems. This imputation model predicts and fills in missing data fields using available features and conviction scores, allowing older data to be effectively utilized without requiring complete information from the original sources
Solution Approach 2:
The patent transforms the quality parameters of historical data by calculating conviction scores for cases and features, then using these scores to guide the imputation process. This changes the data from a state of incomplete information to a state where missing values are statistically inferred, effectively altering the data completeness parameter while preserving the original historical record
2Loss of information
If retrofitting older systems with modern sensors is performed, then data completeness is improved, but system complexity and cost increase
Solution Approach 1:
The patent creates a virtual copy of the missing sensor data through the imputation model, which generates predicted values based on correlations with other available sensors and historical patterns. This copying approach eliminates the need for physical sensor installation while achieving comparable data completeness, avoiding the complexity and cost of hardware modifications
Solution Approach 2:
The imputation model serves as a computational intermediary that replaces the need for physical sensor intermediaries. Instead of installing additional hardware sensors to capture missing data, the system uses software-based imputation that processes existing sensor readings to infer missing values, thereby avoiding hardware complexity
3Loss of information
If imputation is performed on all missing data, then data completeness is improved, but computational resources and time increase
Solution Approach 1:
The patent applies imputation selectively rather than uniformly across all data. By calculating conviction scores for individual cases and features, the system identifies specific locations where imputation is most beneficial and applies processing only to those areas, leaving high-confidence data unchanged. This local quality approach optimizes the balance between completeness and processing efficiency
Solution Approach 2:
The patent implements partial imputation by using conviction scores to determine which missing fields should be imputed and which can be left as-is. This partial action approach processes only the necessary subset of missing data rather than attempting to fill every gap, reducing computational overhead while achieving sufficient data completeness for effective model training
Data Source
AI summary
Techniques are provided for imputation in computer-based reasoning systems. The techniques include performing the following until there are no more cases in a computer-based reasoning model with missing fields for which imputation is desired: determining which cases have fields to impute (e.g., missing fields) in the computer-based reasoning model and determining conviction scores for the cases that have fields to impute. The techniques proceed by determining for which cases to impute data based on the conviction scores. For each of the determined one or more cases with missing fields to impute data is imputed for the missing field, and the case is modified with the imputed data. Control of a system is then caused using the updated computer-based reasoning model.