Heterogeneous Database Metadata Binding for Cross-Source Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face inefficiencies in cross-source queries due to high usage thresholds and limited supported scenarios, leading to low utilization rates and inefficient data management across heterogeneous databases.
Innovation Solution
A data processing method that utilizes metadata to map association relationships between data tables in heterogeneous databases, enabling cross-source storage and query optimization by binding data tables, allowing for efficient cross-source queries through a query engine.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If federated query/cross-source query is used to resolve data island problems, then query capability across multiple databases is improved, but use threshold is high and utilization rate is low
Solution Approach 1:
The patent introduces a unified query interface as an intermediary layer between users and heterogeneous databases. This interface automatically handles complex cross-source query operations, translating user-friendly queries into optimized execution plans that span multiple databases. The intermediary absorbs the complexity of federated query management, allowing users to perform cross-database queries without needing to understand the underlying complexity, thus lowering the use threshold while maintaining versatile query capability.
2Adaptability or versatility
If materialized view is created using SQL or dedicated query tool, then cross-source query is enabled, but supported scenarios are limited and efficiency is reduced
Solution Approach 1:
The patent implements dynamic query optimization that adapts to different query scenarios and data characteristics. The system automatically analyzes query patterns, data distribution, and access frequencies to dynamically generate optimized execution plans. This dynamic approach allows the system to efficiently handle diverse query scenarios including ad-hoc queries, analytical queries, and transactional queries across heterogeneous databases, overcoming the limitations of static materialized views while maintaining high query efficiency.
Solution Approach 2:
The system employs parameter-based query optimization where execution plans are adjusted based on varying parameters such as query type, data volume, access patterns, and resource availability. By changing optimization parameters dynamically rather than relying on fixed materialized view structures, the system achieves high efficiency across multiple query scenarios without being constrained to specific use cases.
3Adaptability or versatility
If multiple database systems are selected to satisfy different scenario requirements, then scenario coverage is improved, but data management complexity increases
Solution Approach 1:
The patent creates a universal query management system that can handle multiple heterogeneous database types through a single unified interface. The system incorporates multi-functional capabilities including automatic schema mapping, cross-database transaction management, data type conversion, and optimized query routing. This universal approach allows the system to support diverse database systems (relational, NoSQL, time-series, etc.) and various query scenarios without requiring separate management mechanisms for each database type, thereby reducing overall data management complexity while maintaining broad scenario coverage.
Data Source
AI summary
A data processing method includes, obtaining metadata for a heterogeneous database including a first database and a second database, wherein the metadata maps an association relationship between a first data table of the first database and a second data table of the second database; updating the second data table by storing, in a cross-source operation, data from the first data table in the second data table based on the association relationship; receiving a query statement indicating the first data table; and executing the query statement on the updated second data table to obtain a query result indicating at least one piece of the data from the first data table.