Tabular Data Imputation Using Noisy Training Data Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Filling in user interface forms is a time-consuming and error-prone task, leading to inefficiencies and inaccuracies in data recording and process execution, especially when dealing with missing values that require intelligent and resource-efficient data imputation.
Innovation Solution
A machine learning model trained on denoised noisy data using application-specific masking rules to predict and impute missing values in tabular data objects, leveraging denoising techniques for efficient and accurate data completion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual data entry is used for user interface forms, then data accuracy can be maintained, but time consumption increases and productivity decreases
Solution Approach 1:
The system enables self-service by having the machine learning model automatically impute missing values in form fields based on provided field values and learned patterns from historical data, eliminating the need for manual entry of all fields and allowing users to only provide essential information
Solution Approach 2:
The patent replaces manual mechanical data entry processes with an automated machine learning-based imputation system that uses neural networks to predict and fill in missing values, substituting human effort with intelligent automation
2Measurement precision
If traditional data imputation methods are used, then implementation is simple, but imputation accuracy is insufficient
Solution Approach 1:
The system changes the approach by transforming the data representation into a tabular format with engineered features that capture relationships between fields, and by adjusting model parameters and architecture to optimize imputation accuracy for specific domain patterns
Solution Approach 2:
The patent applies preliminary action through extensive pre-training on historical form data to learn patterns and relationships between fields, allowing the model to make accurate imputation predictions when deployed without requiring complex real-time computation during form filling
3Measurement precision
If complex machine learning models are used for data imputation, then imputation accuracy improves, but computational cost increases
Solution Approach 1:
The system performs preliminary training action offline on historical data to learn imputation patterns, which allows the trained model to make fast predictions during actual form filling operations without requiring expensive real-time computational resources
Solution Approach 2:
The patent uses copying by creating a trained model that replicates the imputation capabilities learned from historical data, allowing the model to make accurate predictions during deployment without requiring the same computational complexity as the training process
Data Source
AI summary
Methods, systems, and apparatus, including medium-encoded computer program products for training a model to perform tabular data imputation include: obtaining initial tabular training data for imputing data for a tabular data object defined for a user interface form of an application, wherein the initial tabular training data includes rows of data collected from entries for the user interface form; generating noisy tabular training data by invoking a second model trained over the initial tabular training data, wherein generating noisy tabular training data comprises up-sampling the initial tabular training data according to learned application-specific masking rules defined as part of the second model, the application-specific masking rules being generated for the user interface form of the application; and training a first model by inputting the generated noisy tabular training data as a predictor and by applying denoising techniques to output predicted field values for fields of the user interface form.