Fingerprint Pattern Data Duplication Identification

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Manual and computerized methods for identifying duplicate data across numerous tables, databases, and systems are inefficient, requiring excessive time and resources as the number of data sets increases, leading to increased storage costs and maintenance challenges.

Innovation Solution

A method involving the identification of reference and target fields, transformation into fingerprint patterns, and comparison to efficiently locate duplicate data by reducing the search space using heuristics and knowledge of likely duplicated fields, thereby accelerating the detection process.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual review methods are used to identify duplicate data, then accuracy can be maintained, but the time required increases dramatically to days, months, or years

Engineering Contradiction:

Improveduplicate data identification accuracyVSAvoidtime required for duplicate data identification

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent introduces fingerprint patterns as an intermediary representation of data fields. Instead of directly comparing raw data values across tables and databases, the system transforms data into fingerprint patterns that capture essential characteristics while reducing complexity. This intermediary layer enables efficient comparison while maintaining identification accuracy.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces manual mechanical review processes with automated computerized fingerprint comparison. The system uses algorithms to generate and compare fingerprint patterns, substituting the slow manual inspection process with rapid automated computation that maintains accuracy while dramatically reducing time requirements.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If computerized search methodologies are used to locate duplicate data, then time can be reduced, but efficiency remains insufficient for large-scale data across hundreds of thousands of tables

Engineering Contradiction:

Improveduplicate data search efficiencyVSAvoidsearch time for large-scale data

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent segments the large-scale data search problem into manageable components by creating fingerprint patterns for individual fields and tables. This segmentation allows the system to process and compare data in discrete units rather than attempting to search all data simultaneously, dramatically improving efficiency for large-scale operations.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent transforms data parameters by converting raw field values into fingerprint patterns with specific characteristics (such as hash values or normalized representations). This parameter transformation enables more efficient comparison operations while preserving the ability to identify duplicates, making large-scale searches feasible.

Inventive Principle:

Principle #35Parameter changes

3Reliability

If all data in numerous tables, databases, and systems is searched for duplicates, then comprehensive identification can be achieved, but storage capacity is consumed and maintenance costs increase

Engineering Contradiction:

Improvecompleteness of duplicate data identificationVSAvoiddata storage capacity

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent creates fingerprint pattern copies of the original data fields rather than duplicating the actual data values. These fingerprint representations capture the essential information needed for duplicate identification while occupying minimal storage space, enabling comprehensive searching without proportionally increasing storage requirements.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS8429137B2Enterprise data duplication identification

Publication Date: 2013.04.23 FEDERAL EXPRESS CORP

AI summary

Systems, methods, and computer program products are provided for identifying duplicate data. In one exemplary embodiment, there is provided a method for identifying duplicate data. The method may include identifying one or more reference fields that include one or more data values. The method may include retrieving the one or more reference fields and one or more data values. The method may also include transforming the one or more reference fields into one or more reference fingerprint patterns. The method may also include identifying one or more target fields that include one or more target field values. The method may also include retrieving the one or more target fields. The method may also include transforming the one or more target field values into one or more target fingerprint patterns. The method may also include comparing the one or more reference fingerprint patterns with the one or more target fingerprint patterns. The method may further include determining an overlap between the one or more reference fingerprint patterns and the one or more target fingerprint patterns.