Hybrid Multi Data Source Query Routing via Cost Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In service-oriented architectures, applications face challenges in efficiently querying hybrid collections of multiple data sources due to differences in query mechanisms and languages supported by each data source, making it impractical for them to make optimal use of available data sources, especially when sources change over time.
Innovation Solution
A hybrid multi data source system that includes a hint resolver, a rule evaluator, and a federated query engine to filter and select the most suitable data source for queries based on hints and rules, forming query plans, evaluating their costs, and executing the optimal plan, thereby providing a unified interface for accessing diverse data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If applications directly query multiple hybrid data sources with different query mechanisms and languages, then they can access diverse data, but the system complexity and difficulty of making optimal selections increase significantly
Solution Approach 1:
The patent introduces a hybrid multi-data source system that acts as an intermediary between applications and multiple data sources. This system receives queries from applications, automatically selects appropriate data sources based on query characteristics and data source capabilities, and manages the complexity of querying multiple heterogeneous data sources with different query mechanisms and languages.
2Productivity
If applications are made aware of each data source's query mechanisms and capabilities, then optimal query selection is possible, but the operational burden and complexity increase
Solution Approach 1:
The hybrid multi-data source system performs self-service by automatically analyzing query characteristics, evaluating data source capabilities, and selecting the most appropriate data source without requiring application developers to manually configure or be aware of each data source's specific query mechanisms. The system autonomously optimizes query routing based on inherent query properties.
3Adaptability or versatility
If the system maintains awareness of changing data sources and their capabilities, then optimal query routing is achieved, but the system complexity and maintenance burden increase
Solution Approach 1:
The system implements dynamic adaptation to changing data sources by automatically detecting and responding to changes in data source availability, capabilities, and performance characteristics. The hybrid multi-data source system dynamically adjusts query routing decisions based on current system state without requiring manual reconfiguration, enabling flexible adaptation to evolving data source landscapes.
Data Source
AI summary
A system and method of querying hybrid multi data sources includes receiving a query at a hybrid multi data source running on a computer server, creating a list of data sources, filtering the list of data sources based on data source hints included in the query, filtering the list of data sources based on data source rules, forming one or more query plans for processing the query, evaluating each of the query plans, selecting a first query plan from the query plans based on results of the evaluating, and performing the first query plan. Each query plan corresponds to a data source in the filtered list of data sources. In some examples, the first query plan has a lowest cost of each of the query plans. In some examples, the first query plan is based on probabilities associated with the cost of each of the query plans.