Data Partitioning for Inconsistency Detection in Record Systems

Solution Overview

Problem

Existing solutions for identifying data inconsistencies between disparate data systems are inefficient in terms of resource utilization and time-consuming, struggling to keep pace with the rapid creation and change of data across multiple database systems.

Innovation Solution

The method involves data partitioning, where a partition parameter is used to divide data into subsets, allowing for a partition-level parity check to determine consistency without element-by-element comparison, and further partitioning until the data volume falls below a threshold for detailed inconsistency identification.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional bit-by-bit or block-by-block replication and synchronization schemes are used to keep data repositories consistent, then data consistency between systems is maintained, but resource utilization becomes inefficient and the process becomes extremely time-consuming

Engineering Contradiction:

Improvedata consistencyVSAvoidresource utilization efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent segments the data comparison process into multiple hierarchical levels: first partitioning data into subsets using a partition parameter, then further dividing subsets into groups, and finally into individual records. This segmentation allows the system to perform parity checks at coarser granularities first, significantly reducing the total number of comparisons needed while maintaining data consistency verification.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs partial actions by conducting parity checks at multiple levels of granularity rather than performing complete element-by-element comparison. The system performs subset-level parity checks, group-level parity checks, and only performs full record-level comparison when necessary, thus executing only the necessary portion of the comparison process.

Inventive Principle:

Principle #16Partial or excessive action

2Reliability

If traditional bit-by-bit or block-by-block replication and synchronization schemes are used to keep data repositories consistent, then data consistency between systems is maintained, but the process becomes extremely time-consuming and cannot keep up with the pace of data creation and change

Engineering Contradiction:

Improvedata consistencyVSAvoidtime consumption

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent segments the data comparison process into multiple hierarchical levels: first partitioning data into subsets using a partition parameter, then further dividing subsets into groups, and finally into individual records. This segmentation allows the system to perform parity checks at coarser granularities first, significantly reducing the total number of comparisons needed while maintaining data consistency verification.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs partial actions by conducting parity checks at multiple levels of granularity rather than performing complete element-by-element comparison. The system performs subset-level parity checks, group-level parity checks, and only performs full record-level comparison when necessary, thus executing only the necessary portion of the comparison process.

Inventive Principle:

Principle #16Partial or excessive action

3Measurement precision

If element-by-element data comparison is performed to identify inconsistencies between record systems, then accurate inconsistency detection is achieved, but computing resources and networking resources are excessively consumed

Engineering Contradiction:

Improveinconsistency detection accuracyVSAvoidcomputing and networking resource consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent segments the data comparison process into multiple hierarchical levels: first partitioning data into subsets using a partition parameter, then further dividing subsets into groups, and finally into individual records. This segmentation allows the system to perform parity checks at coarser granularities first, significantly reducing the total number of comparisons needed while maintaining data consistency verification.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs partial actions by conducting parity checks at multiple levels of granularity rather than performing complete element-by-element comparison. The system performs subset-level parity checks, group-level parity checks, and only performs full record-level comparison when necessary, thus executing only the necessary portion of the comparison process.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS10606829B1Methods and systems for identifying data inconsistencies between electronic record systems using data partitioning

Publication Date: 2020.03.31 INTUIT INC

AI summary

Certain aspects of the present disclosure provide methods and systems for efficiently identifying data inconsistencies between electronic record systems using data partitioning. In certain embodiments, data stored in each of two record systems may be partitioned into a number of groups. Corresponding groups of data from the two record systems may be compared for data inconsistency. Upon determining data inconsistencies between certain corresponding groups, additional partitioning and data comparison may be performed to identify the source of the data inconstancies.