Data Quality Workflow Generation for Recurring Error Resolution

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems struggle to automatically identify and correct data quality issues in datasets without requiring extensive manual analysis, especially when dealing with recurring errors.

Innovation Solution

A data processing system that automatically identifies data quality issues, generates workflows to resolve them, and escalates or reassigns responsibility based on metadata and test results, using a feedback loop to update workflows for recurring problems.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual analysis is used to identify and correct data quality issues, then accuracy of correction is improved, but productivity and scalability deteriorate

Engineering Contradiction:

Improveaccuracy of data quality issue identificationVSAvoidproductivity of data quality issue correction

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system enables automatic self-correction of data quality issues through automated workflow generation and execution. The system identifies data quality issues, generates appropriate correction workflows, executes them, and updates the system state automatically without requiring manual intervention for each issue, thereby maintaining accuracy while dramatically improving productivity and scalability

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system changes the state parameters of data quality issues based on workflow execution outcomes. When issues are resolved, the system updates their state from open to closed; when reoccurring, it escalates them. This automated parameter management enables high-volume processing while maintaining precise tracking and correction of each issue

Inventive Principle:

Principle #35Parameter changes

2Productivity

If automated systems are used to correct data quality issues, then productivity is improved, but reliability deteriorates due to lack of contextual understanding

Engineering Contradiction:

Improveproductivity of data quality issue correctionVSAvoidreliability of data quality issue correction

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system implements feedback loops where workflow execution results are captured and used to update the state of data quality issues. The system monitors whether issues are resolved or reoccurring, and automatically adjusts its behavior accordingly - closing resolved issues or escalating reoccurring ones. This feedback mechanism ensures reliable, context-aware automated correction

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system performs preliminary analysis of data quality issues to determine their nature and generate appropriate correction workflows before execution. By pre-configuring correction strategies based on issue characteristics and maintaining state information, the system ensures that automated actions are reliable and context-appropriate

Inventive Principle:

Principle #10Preliminary action

3Ease of operation

If all data quality issues are treated uniformly, then ease of operation is improved, but adaptability deteriorates

Engineering Contradiction:

Improveease of managing data quality issuesVSAvoidadaptability to different data quality issue types

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The system applies local quality by treating different data quality issues according to their specific characteristics and states. Rather than uniform treatment, the system generates customized workflows based on issue type, and applies different state transitions (close vs. escalate) based on resolution outcomes. This enables versatile adaptation while maintaining simple automated operation

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system dynamically adapts its behavior based on the state of data quality issues. Issues transition between states (open, resolved, reoccurring) and the system adjusts its responses accordingly - generating different workflows for different issue types and applying different escalation rules based on recurrence patterns, enabling both ease of operation and adaptability

Inventive Principle:

Principle #15Dynamics

4Measurement precision

If manual tracking of data quality issues is used, then measurement precision is improved, but loss of time deteriorates

Engineering Contradiction:

Improveprecision of tracking data quality issue stateVSAvoidtime spent tracking and updating data quality issues

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system replaces manual mechanical tracking with automated electronic state management. The system automatically updates issue states, generates workflows, executes them, and records outcomes without human intervention. This substitution maintains precise tracking of all issue states while eliminating the time loss associated with manual tracking and updates

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12585529B2Generation and execution of processing workflows for correcting data quality issues in data sets

Publication Date: 2026.03.24 AB INITIO TECHNOLOGY LLC

AI summary

Systems and methods are for executing, by a data processing system, a workflow to process results data indicating an output of a data quality test on data records by generating, responsive to receiving the results data and metadata describing the results data, a data quality issue associated with a state and one or more processing steps of the workflow to resolve a data quality error associated with the data quality test. Operations include generating a workflow for processing results data based a state specified by a data quality issue. Generating the workflow includes: assigning, based on the results data and the state of the data quality issue, an entity responsible for resolving the data quality error; determining, based on the metadata, one or more actions for satisfying the data quality condition specified in the data quality test; and updating the state associated with the data quality issue.