Consistency Checker for Data Repository Inconsistencies
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting data inconsistencies in large data repositories are time-consuming and resource-intensive, limiting the frequency of inconsistency analysis and requiring extensive data processing.
Innovation Solution
A consistency checker that utilizes pre-defined inconsistency patterns and analysis rules to efficiently identify inconsistencies by matching data attribute values with stored patterns, prioritizing rules based on previous occurrence frequencies and updating probability weights for faster and more accurate results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all data rows are analyzed by a data analyzing tool to detect inconsistencies, then the completeness of inconsistency detection is improved, but the processing time and computing resources increase significantly
Solution Approach 1:
The patent segments the large-scale data inconsistency detection problem into smaller, manageable units by dividing the data repository into partitions and processing them in batches. This allows the system to analyze completeness metrics for subsets of data rather than requiring all data to be processed simultaneously, thereby reducing processing time while maintaining detection completeness.
Solution Approach 2:
The patent implements partial action by using sampling techniques to analyze a representative portion of the data repository. By calculating completeness metrics from sampled data posts and their corresponding inconsistency patterns, the system can estimate overall data quality without processing every single record, significantly reducing computing resources while providing reliable completeness assessment.
2Measurement precision
If all data columns are verified in redundancy control between primary and secondary HLR, then the accuracy of inconsistency detection is improved, but the computing resources required increase dramatically
Solution Approach 1:
The patent extracts and prioritizes only the most critical data columns for verification in redundancy control between primary and secondary HLR systems. By identifying and focusing on key attributes that have the highest impact on data consistency, the system achieves accurate inconsistency detection without the need to process every single column, thereby reducing computing resource consumption while maintaining detection accuracy.
Solution Approach 2:
The patent applies local quality by using different verification strategies for different data columns based on their importance and characteristics. Critical columns undergo rigorous verification to ensure high accuracy, while less critical columns use simplified checking methods. This differentiated approach maintains overall detection accuracy while optimizing computing resource usage across the data verification process.
3Reliability
If multiple inconsistency criteria are checked to identify all possible inconsistencies, then the thoroughness of data analysis is improved, but the processing complexity and resource consumption increase
Solution Approach 1:
The patent performs preliminary action by pre-defining and storing inconsistency patterns and their associated rules in a database before actual data analysis. These pre-configured patterns capture common inconsistency scenarios across multiple criteria. During data processing, the system simply matches observed data against these pre-established patterns, which simplifies the processing complexity while maintaining thoroughness in detecting various types of inconsistencies.
Solution Approach 2:
The patent uses copying by creating standardized templates for different inconsistency patterns and criteria. Instead of implementing complex custom logic for each inconsistency type, the system replicates and adapts proven detection patterns. This template-based approach allows thorough multi-criteria analysis to be performed through reusable, pre-tested patterns, reducing processing complexity while maintaining analytical thoroughness.
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
A consistency checker and (200) a method performed by a consistency checker (200) for finding inconsistencies of data posts stored in a data repository (220) is provided. The consistency checker (200) has a storage unit (202) in which a number of inconsistency patterns are stored, each inconsistency pattern being associated with an inconsistency analysis rule for a value of a data attribute. The consistency checker further has a receiving unit (204) for receiving a data post from the data repository, and a logic unit (206) arranged for matching a value of at least one data attribute of the data post with a value of the corresponding data attribute of the stored inconsistency patterns, and, if there is a match, applying the inconsistency analysis rule that is connected with the matched inconsistency pattern to the data post to evaluate whether there is an inconsistency in the data post or not, wherein the number of inconsistency patterns each defines an inconsistency that has previously occurred for a value of a data attribute in the data repository.