Federated Query Translation for Heterogeneous Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data aggregation methods, such as extract, load, and transform approaches, are complex, time-consuming, and not optimized for rapid connection to new databases or data sources, and they do not enable one-click federated searches of disparate data.
Innovation Solution
A method and system for on-demand delivery of data from heterogeneous external sources to a data analytics tool, which involves parsing no-code client queries, dynamically translating them into external queries, and reformulating responses in real-time to provide data subsets, allowing for immediate analysis and alerts without manual data gathering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If extract, load, and transform approaches are used for data aggregation, then data can be collected from multiple sources, but the process becomes complex and time-consuming
Solution Approach 1:
The patent introduces a federated search system as an intermediary layer between data sources and analytics tools. This mediator handles query translation, result aggregation, and data model mapping automatically, eliminating the need for manual extract-load-transform processes and reducing both complexity and time requirements.
Solution Approach 2:
The federated search system provides universal data access capabilities across heterogeneous data sources through a single interface. It supports multiple data models (relational, hierarchical, document-oriented) and performs various operations (search, aggregation, filtering) through one unified system, reducing the need for source-specific aggregation pipelines.
2Ease of operation
If manual data gathering methods are used, then data can be collected from heterogeneous sources, but it requires significant manual effort and time
Solution Approach 1:
The federated search system enables self-service data access where analysts can directly query multiple heterogeneous data sources using natural language or simple queries. The system automatically handles connection management, query translation, and result synthesis, eliminating manual data gathering efforts while providing intuitive ease of operation.
Solution Approach 2:
The system performs preliminary actions by pre-configuring connectors to various data sources, establishing data models and mappings in advance. When a query is executed, the pre-prepared infrastructure automatically handles the complex tasks of connecting to sources, translating queries, and aggregating results, saving significant time during actual data access operations.
3Adaptability or versatility
If traditional data aggregation methods are used, then data can be consolidated, but rapid connection to new databases is not enabled
Solution Approach 1:
The federated search system dynamically adapts to new data sources through configurable connectors that can be added or modified without system reconfiguration. The architecture supports dynamic registration of new data sources with automatic schema detection and adaptation, enabling rapid connection to heterogeneous databases while maintaining high productivity through automated query translation and result aggregation.
Data Source
AI summary
A method enables on-demand delivery of data from a plurality of heterogeneous external data sources to a data analytics tool. With a mapping of one or more identified data connectors, a no-code client query, as formulated in a first data model, is dynamically translated to one or more external queries formulated in one or more alternate data models of the heterogeneous external data sources. With the mappings of the one or more identified connectors, each response to the one or more external queries is reformulated from the one or more alternate data models to the first data model to yield one or more client query results objects. The client query results objects are sent to the data analytics tool.