Database Engine Reference Column External Data Integration

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for aggregating data from multiple sources are inefficient and cumbersome, requiring bulky intermediaries or thick clients to manage interactions with multiple data sources.

Innovation Solution

A database engine that incorporates a reference column into its schema, allowing it to communicate directly with external data sources to retrieve and populate data items on demand or through periodic caching, enhancing data management efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a federation engine is used to aggregate data from multiple data sources, then data aggregation capability is improved, but system complexity and resource consumption increase due to the bulky intermediary architecture

Engineering Contradiction:

Improvedata aggregation capabilityVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent extracts the data aggregation capability from the client application and integrates it directly into the database engine. The database engine now contains built-in functionality to query multiple data sources, parse their schemas, and aggregate results, eliminating the need for separate federation engine components while maintaining the same data aggregation capabilities.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent merges the data aggregation functionality with the database engine by adding reference columns that can point to external data sources. The database engine combines its existing query processing capabilities with new multi-source aggregation capabilities, allowing clients to access aggregated data from multiple sources through standard SQL queries without requiring separate aggregation infrastructure.

Inventive Principle:

Principle #5Merging (Combining)

2Adaptability or versatility

If thick clients are used to independently communicate with multiple data sources and aggregate data on the client-side, then data access flexibility is improved, but client resource consumption and processing burden increase

Engineering Contradiction:

Improvedata access flexibilityVSAvoidclient resource consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent extracts the data aggregation logic from the thick client and relocates it to the database engine. The database engine now handles the complex tasks of querying multiple data sources, managing connections, and aggregating results, while the client application only needs to send standard SQL queries and receive aggregated results, significantly reducing client resource consumption.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The database engine serves as an intermediary between the client and multiple external data sources. It manages all communications with external sources, handles schema differences, and performs data aggregation, shielding the client from the complexity of multi-source data access while maintaining flexibility in how data can be queried and aggregated.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If existing data aggregation methods are used, then multi-source data access is enabled, but efficiency is reduced due to bulky architectures and inefficient data retrieval processes

Engineering Contradiction:

Improvemulti-source data accessVSAvoiddata management efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent makes the database engine universal by enabling it to handle both traditional single-source queries and multi-source aggregation operations through the same SQL interface. The reference column mechanism allows the database engine to automatically determine whether to query local tables or external data sources, providing a unified approach to data access that improves efficiency across all query types.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent implements preliminary action by pre-defining reference columns in the database schema that specify external data sources and their relationships. This allows the database engine to prepare and optimize query execution plans in advance, caching schema information from external sources, and reducing the overhead of dynamic schema discovery during query execution, thereby improving overall data management efficiency.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10649964B2Incorporating external data into a database schema

Publication Date: 2020.05.12 RED HAT INC

AI summary

Various examples are directed to systems and methods for managing a database to include data from an external data source. A database engine may receive a request to add a reference column to a database described by a database schema. The request may comprise a location parameter describing a location of the external data source comprising data for populating the reference column and a data identifying parameter describing data at the external data source for populating the reference column. The database engine may modify the database schema to include the reference column, send a query to the external data source to obtain at least one data item for populating the reference column, and populate the reference column with the at least one data item.