Data Imputation Using Stratified Masking and Attention Mechanisms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data imputation methods struggle with handling unknown-unknown data values in datasets, leading to inaccuracies in machine learning model configurations and downstream processing.
Innovation Solution
The proposed solution involves a computer-implemented method that identifies a truth source dataset, generates an updated dataset using a stratified masking algorithm, and utilizes a trained model with an attention layer to create a probability dataset and threshold set for data imputation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional data imputation methods are used, then the process is simple and fast, but the accuracy is poor when handling unknown-unknown data values
Solution Approach 1:
The patent introduces an attention mechanism as an intermediary component between the input data and the imputation model. This attention layer processes the input features and generates attention weights that guide the imputation process, allowing the model to focus on relevant features when predicting unknown-unknown values. The attention mechanism acts as a mediator that enhances the model's ability to handle uncertain data without requiring complete redesign of the imputation system.
Solution Approach 2:
The imputation system is segmented into distinct functional components: an attention mechanism layer, a probability estimation module, and a threshold-based filtering system. The attention mechanism segments the feature processing to identify important patterns, while the probability module segments the prediction task into confidence assessments. This segmentation allows each component to specialize in specific aspects of the imputation problem, improving overall accuracy while maintaining manageable system complexity.
2Reliability
If probability thresholds are applied to filter imputed values, then the reliability of downstream processing improves, but the loss of potentially useful data increases
Solution Approach 1:
The patent dynamically adjusts the probability threshold parameter based on the specific characteristics of the data and the imputation context. Rather than using a fixed threshold, the system modifies the threshold parameter to balance between reliability and data retention. This allows the system to maintain high reliability in critical downstream applications while preserving more data points that could still be valuable, especially when the estimated probability of correctness is close to the threshold.
Solution Approach 2:
The system applies probability thresholding selectively rather than universally. For certain critical data fields or downstream applications with high reliability requirements, stricter thresholding is applied. For other fields where data volume is more important, more lenient thresholds are used. This partial application of thresholding allows the system to achieve reliability where needed without excessive data loss across the entire dataset.
Data Source
AI summary
Embodiments of the present disclosure provide for improved data imputation and use of imputed data in processing of downstream models. Some embodiments specially train a model that performs improved data imputation utilizing a specially-configured attention mechanism. Some embodiments train a model utilizing stratified masking. Some embodiments train a particular pre-processing layer of a downstream task-specific model to adaptively learn threshold values for imputing particular data. The pre-processing layer is usable to improve accuracy training and/or use of a downstream task-specific model based at least in part on the imputed data.