Removing Invisible Data Packages in Data Warehouses

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Data warehouses face issues with 'invisible' data packages due to broken links, leading to zombie requests that cannot be deleted, causing operational disruptions and requiring labor-intensive manual resolution.

Innovation Solution

An algorithm that detects inconsistencies between records associated with identification tags, identifies and removes problematic data packages, and resolves zombie requests, thereby preventing operational disruptions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If manual identification and removal of invisible data packages is performed, then data integrity is improved, but labor intensity and time consumption increase

Engineering Contradiction:

Improvedata integrityVSAvoidtime consumption

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs self-diagnosis by automatically detecting broken links and identifying invisible data packages through consistency checks between data warehouse records and data provider records, eliminating the need for manual identification while maintaining data integrity

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The manual mechanical process of identifying and removing invisible data packages is replaced with an automated computer-based system that uses algorithms to detect link failures, identify inconsistent records, and remove problematic data packages automatically

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Reliability

If database administrators remove all data and reload to resolve zombie requests, then operational disruption is resolved, but productivity loss increases

Engineering Contradiction:

Improveoperational continuityVSAvoidproductivity

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system extracts and removes only the specific invisible data packages and zombie requests that are causing operational disruptions, rather than removing all data and reloading. This targeted approach resolves the operational issue while preserving the majority of functional data and maintaining productivity

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The system performs preliminary detection and removal of invisible data packages and zombie requests before they cause major operational disruptions. By identifying and removing these problematic elements proactively, the system prevents operational failures and maintains continuous productivity

Inventive Principle:

Principle #10Preliminary action

3Ease of operation

If automated detection of invisible data packages is implemented, then labor intensity is reduced, but system complexity increases

Engineering Contradiction:

Improvelabor intensityVSAvoidsystem complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The system implements automated feedback mechanisms by continuously monitoring data warehouse operations, detecting link failures, and triggering automatic identification and removal of invisible data packages. This feedback loop reduces labor intensity while managing system complexity through structured automated responses to detected anomalies

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS10235438B2Removal of invisible data packages in data warehouses

Publication Date: 2019.03.19 AT&T INTELLECTUAL PROPERTY I L P

AI summary

In accordance with one embodiment of the disclosed technology, inconsistencies are detected between various records relating to data that has been associated with an identification tag. Data packages associated with the inconsistencies may then be removed. In accordance with another aspect of the disclosed technology, requests relating to data packages associated with inconsistencies in the various stored records are identified and removed. The disclosed technology may be implemented in data warehouses.