Redundant Dataset Detection for Lower-Cost Database Management

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Large organizations face high infrastructure costs due to duplicate datasets stored across multiple databases, leading to skewed data analysis and inaccurate conclusions during data processing.

Innovation Solution

A computing system performs data analysis on separate datasets to identify redundancies by comparing data values and interpreting similarities, determining if one dataset is a subset of another, and transmitting prompts to user devices to indicate potential redundancy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If duplicate datasets are stored to various databases and servers, then data availability for multiple teams is improved, but infrastructure costs increase

Engineering Contradiction:

Improvedata availabilityVSAvoidinfrastructure costs

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The system consolidates duplicate datasets into a single centralized location, merging multiple redundant copies into one master dataset that multiple teams can access, thereby reducing infrastructure costs while maintaining data availability

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

A single centralized dataset serves multiple teams and projects simultaneously, making the data universally accessible without requiring separate duplicate storage for each team, thus eliminating redundant infrastructure

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Ease of operation

If duplicate datasets are stored to various databases, then data access for different projects is improved, but data analysis accuracy deteriorates

Engineering Contradiction:

Improvedata accessVSAvoiddata analysis accuracy

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

The system implements a feedback mechanism that monitors and detects duplicate datasets across databases, providing information about redundancies to enable corrective actions that maintain data consistency and analysis accuracy

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system performs preliminary detection and identification of duplicate datasets before they can cause analysis errors, allowing for proactive consolidation and prevention of accuracy issues in data processing

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12450240B2Database management systems

Publication Date: 2025.10.21 TRUIST BANK

AI summary

Systems and methods perform data analysis on at least two separate datasets to identify any redundancies. The data analysis includes comparing first data values of a first dataset with second data values of a second dataset, the comparing including evaluating similarities of the first data values and second data values, identifying, from the comparing, that the first data values and the second data values include at least a portion of substantially similar data, and interpreting similarities of the portion of substantially similar data, the interpreting including determining that a dataset of the first dataset and second dataset is a subset of the other dataset. Further, control signal(s) are transmitted to a user device to initiate displaying, via a user interface of the user device, a prompt indicating that the at least one dataset is likely the subset of the other dataset.