Parallel Data Quality Assessment Engine
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data quality assessment systems are inefficient in processing high volumes of data records, often requiring serial processing and being prone to errors due to poor data quality, which can lead to computational issues and inaccurate analytics.
Innovation Solution
A quality assessment computing device with an assessment engine and data warehouse that applies predefined data quality criteria in parallel, using in-database processing to score data records and assign quality values, thereby enhancing speed and security while allowing non-programmers to define scoring schemes easily.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If serial processing is used to assess data quality, then system complexity is reduced, but processing speed and productivity deteriorate
Solution Approach 1:
The patent segments the data quality assessment process into multiple independent evaluation criteria (completeness, accuracy, consistency, timeliness, validity) that can be processed in parallel. Each criterion operates as an independent module assessing different aspects of data quality simultaneously, rather than sequentially
Solution Approach 2:
The patent introduces a multi-dimensional assessment framework where data quality is evaluated across multiple dimensions (completeness, accuracy, consistency, timeliness, validity) simultaneously. This dimensional approach enables parallel processing of different quality attributes without increasing system complexity
2Ease of manufacture
If serial processing is used to assess data quality, then implementation simplicity is improved, but processing time increases exponentially with data volume
Solution Approach 1:
The assessment process is divided into separate, independently executable criteria modules. Each module (completeness check, accuracy validation, consistency verification, etc.) can process data simultaneously without interfering with others, reducing overall processing time while maintaining implementation simplicity through modular design
Solution Approach 2:
The patent performs preliminary data validation and scoring against multiple criteria simultaneously before final quality determination. By pre-establishing assessment rules and executing multiple checks in parallel beforehand, the system reduces total processing time while keeping the implementation straightforward
3Device complexity
If traditional data quality assessment is used, then system simplicity is maintained, but measurement precision and reliability of quality scores deteriorate
Solution Approach 1:
The patent segments quality assessment into distinct measurable criteria (completeness, accuracy, consistency, timeliness, validity), each contributing to the overall quality score. This segmented approach improves measurement precision by evaluating multiple independent dimensions rather than using a single simplified metric
Solution Approach 2:
The assessment system is designed to evaluate multiple quality dimensions simultaneously using a unified framework. The same system structure handles completeness, accuracy, consistency, timeliness, and validity assessments in parallel, improving measurement precision while maintaining system simplicity through universal design
4Productivity
If parallel processing is implemented for data quality assessment, then productivity and processing speed improve, but device complexity increases
Solution Approach 1:
The patent segments the assessment into independent criteria modules that naturally lend themselves to parallel execution. This segmentation enables parallel processing without significant complexity increase because each module operates independently with well-defined interfaces
Solution Approach 2:
The system uses a universal assessment framework that can evaluate multiple quality dimensions simultaneously through the same structural components. This multi-functionality approach achieves parallel processing capability while avoiding the complexity of designing separate systems for each quality metric
Data Source
AI summary
A computer-implemented method for assessing data quality of one or more data records. The method is implemented using a quality assessment (QA) computing device. The method includes storing, into a data warehouse, predefined data quality assessment criteria to be applied to the one or more data records to assess the data quality of the one or more data records. The method further includes assessing, by the QA computing device, the one or more data fields of each data record in the one or more views using the assessment criteria, wherein a data quality value is assigned to the one or more data fields based on the data quality.