Data Table Remediation Rules for Automated Error Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data quality rules face challenges such as subjectivity, complexity, dynamic interdependencies, high costs, false positives, and incomplete information, which hinder effective correction of erroneous data entries in datasets.
Innovation Solution
Implementing a computer-implemented method that uses remediation rule definitions, prediction models, and automated remediation plans to identify and correct erroneous data entries in data tables, leveraging machine learning and mapping engines to optimize the remediation workflow.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual correction of erroneous data entries is performed, then accuracy of data correction can be controlled, but time consumption and labor cost increase significantly
Solution Approach 1:
The system enables self-service through automated remediation plans that autonomously identify and correct erroneous data entries without requiring manual intervention. The prediction model automatically selects appropriate remediation actions from predefined templates, allowing the system to correct its own errors independently.
Solution Approach 2:
The patent replaces the mechanical manual correction process with an automated computational system. The prediction model uses machine learning algorithms to substitute human judgment and decision-making with automated processes, thereby reducing time consumption while maintaining correction accuracy through validated remediation templates.
2Reliability
If complex remediation plans are created to handle multiple data errors, then completeness of error correction improves, but system complexity and difficulty of implementation increase
Solution Approach 1:
The remediation system is segmented into modular components: prediction model, remediation template library, mapping engine, and execution engine. Each component handles a specific aspect of the correction process independently, making the overall complex system manageable through functional decomposition.
Solution Approach 2:
The system achieves universality by creating a single platform that can handle multiple types of data errors through a unified prediction model and standardized remediation templates. The mapping engine universally applies to different data tables and structures, enabling the system to correct various error types without requiring separate specialized systems.
3Productivity
If automated remediation actions are applied without validation, then processing speed increases, but risk of introducing new errors increases
Solution Approach 1:
The system incorporates feedback mechanisms where the validation engine continuously monitors and verifies the results of automated remediation actions. This feedback loop allows the system to detect and correct potential new errors introduced during the automated process, maintaining reliability while preserving high processing speeds through efficient validation techniques.
Data Source
AI summary
Methods, systems, and computer-readable storage media for remediation management. Remediation rule definitions are received. The definitions include a replacement remediation action defining placeholder parameters and conditions for replacing erroneous entries of a plurality of data tables to remedy the erroneous entries of the plurality of data tables. The remediation rule definitions are mapped to the data tables. An identification of data tables to be verified is received. Data of the data tables is verified to identify erroneous entries. Remediation plans including applicable remediation rule definitions mapped to the one or more data tables are selected using a prediction model. A remediation plan includes remediation rule definitions to correct the erroneous entries in each of the data tables. The remediation plan is applied to replace the erroneous entries in the one or more data tables with corrected entries.