Shared Accelerator for Consolidated Database Access

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data integration methods, such as data warehouses and federated database systems, face challenges like data replication delays, inconsistencies, high network traffic, and maintenance costs, and are inflexible when dealing with structural changes in source databases.

Innovation Solution

A method that uses a shared accelerator to replicate tables from multiple source databases, allowing a consolidated DBMS to access these replicated tables for query execution, thereby providing a centralized, consolidated view without additional data replication, and dynamically determines whether to execute queries on the source database or the accelerator for improved performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If data is replicated to a central data warehouse for consolidated access, then centralized data access is improved, but network traffic and data consistency delays increase

Engineering Contradiction:

Improvecentralized data accessVSAvoidnetwork traffic

Core Design Contradiction:

Ease of operationVSLoss of energy

Solution Approach 1:

The patent segments the centralized data warehouse into distributed data marts at each source system, eliminating the need for centralized data replication while maintaining consolidated access capabilities through virtualization and federated querying

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a federated database system as an intermediary layer that virtually integrates data from multiple distributed sources without physical replication, using query decomposition and result composition to provide centralized access functionality

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of operation

If data is replicated to a data warehouse, then consolidated access is improved, but data consistency and update delays worsen

Engineering Contradiction:

Improveconsolidated data accessVSAvoiddata consistency

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

Instead of copying data from sources to a central warehouse (traditional ETL), the system inverts the approach by having the federated system query and retrieve data on-demand from distributed sources, ensuring always-fresh data without replication delays

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

Each source system maintains its own data independently without relying on centralized replication, and the federated system automatically queries the most current data from each source when needed, ensuring data consistency without manual intervention

Inventive Principle:

Principle #25Self-service

3Loss of energy

If federated database systems are used for consolidated view, then data replication is reduced, but query execution performance and flexibility worsen

Engineering Contradiction:

Improvenetwork trafficVSAvoidquery execution performance

Core Design Contradiction:

Loss of energyVSProductivity

Solution Approach 1:

The system performs preliminary actions by pre-computing and caching query results, statistics, and metadata from distributed sources, and by optimizing query decomposition strategies in advance to improve execution performance without increasing network traffic

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically changes query execution parameters such as decomposition strategies, access paths, and result composition methods based on query characteristics and data distribution, improving flexibility and performance while maintaining low network traffic

Inventive Principle:

Principle #35Parameter changes

4Loss of energy

If federated database systems are used, then data replication is reduced, but adaptability to structural changes worsens

Engineering Contradiction:

Improvenetwork trafficVSAvoidflexibility to structural changes

Core Design Contradiction:

Loss of energyVSAdaptability or versatility

Solution Approach 1:

The system implements dynamic metadata catalogs and query decomposition strategies that automatically adapt to structural changes in distributed data sources, allowing the federated system to remain flexible and responsive to schema evolution without requiring manual reconfiguration

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11120043B2Accelerator based data integration

Publication Date: 2021.09.14 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A method, system and computer program product for providing consolidated access to data of a plurality of source databases. Tables of each of the source databases are replicated to a shared accelerator. The source DBMSs are configured to dispatch queries to the accelerator for accelerating query execution. The accelerator is configured such that the replicated tables can only be accessed by the source DBMS having provided said tables for executing a dispatched query. A user can select one of the source DBMSs to act as a consolidated DBMS—C-DBMS. The C-DBMS provides the consolidated access. The user is enabled to select tables managed by another one of the DBMSs. In response to receiving the selection of the tables, the accelerator is re-configuring such that the C-DBMS is granted access also to the copies of the selected tables in the accelerator.