Temporal Data Quality Measurement Using Consistency Rates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data quality measurement techniques provide only instantaneous views of database quality without showing how quality changes over time, failing to reveal trends or causes of these changes.
Innovation Solution
A computer-implemented method and system that stores records with common attributes, reads values at different time periods, and generates quality change rates by processing consistency data reflecting the extent to which these values adhere to business rules, enabling the analysis of data quality changes and volatility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing data quality measurement techniques are used, then data quality can be measured at a given time, but only an instantaneous snapshot is provided without showing changes over time
Solution Approach 1:
The patent adds the time dimension to data quality measurement by storing records with timestamps and comparing data across multiple time periods. This transforms the measurement from a single-point snapshot to a multi-dimensional view that captures evolution over time, enabling trend analysis and identification of quality change causes.
Solution Approach 2:
The system performs preliminary actions by storing not only data values but also timestamps and business rules in advance. This preliminary storage of contextual information enables subsequent temporal comparisons and analysis of quality changes without requiring additional data collection efforts.
2Loss of information
If data is stored and accessed at different time periods, then changes in data quality over time can be measured, but the system complexity increases
Solution Approach 1:
The patent creates a multi-functional system that simultaneously performs data storage, temporal comparison, business rule evaluation, and trend analysis. By designing the system to handle multiple functions through a unified architecture that leverages existing database capabilities, the complexity increase is minimized while achieving comprehensive data quality monitoring over time.
3Loss of information
If consistency data is generated and processed to determine quality change rates, then trends and causes of quality changes can be understood, but the processing time and computational resources increase
Solution Approach 1:
The system performs preliminary actions by pre-storing business rules and data values with timestamps. This advance preparation allows for efficient batch processing of consistency data when temporal comparisons are needed, reducing the computational burden and processing time during actual quality analysis operations.
Data Source
AI summary
Systems, methods, and computer-readable media are disclosed for evaluating data quality. An exemplary embodiment includes storing a plurality of records, the records sharing a common attribute, and reading first values for the common attribute corresponding to a first time period and second values for the common attribute corresponding to a second time period. A business rule for evaluating the common attribute is accessed, and first and second consistency data are generated. The first consistency data may reflect the extent to which the first values of the common attribute are consistent with the business rule at the first time. The second consistency data may reflect the extent to which the second values of the common attribute are consistent with the business rule at the second time. The first consistency data and the second consistency data are processed to generate a quality change rate of the common attribute from the first time period to the second time period, based on the difference between the first consistency data and the second consistency data.