Federated API Schema Mapping for Heterogeneous Data Sources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for accessing heterogenous data sources in cloud platforms face challenges in integrating data sources with different API formats and functionalities, requiring manual code modifications and leading to errors and inefficiencies.
Innovation Solution
A federated API is generated using an intermediate federation layer that maps different data source types to a common schema, automatically integrating diverse data sources through an intermediate metadata federation layer that translates queries into source-specific fetch operations, reducing manual intervention and errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If manual code modifications are used to integrate heterogenous data sources, then integration capability is improved, but error rate increases and efficiency decreases
Solution Approach 1:
The patent introduces an intermediate federation layer that acts as a mediator between heterogenous data sources and the unified API interface. This layer automatically maps and translates data from different sources without requiring manual code modifications, thereby maintaining integration capability while reducing errors introduced by manual coding.
Solution Approach 2:
The federation layer enables self-service integration by automatically discovering, mapping, and translating data from heterogenous sources based on predefined schemas. The system performs self-configuration and adaptation without human intervention, eliminating manual code modifications and their associated errors.
2Adaptability or versatility
If manual code modifications are used to access heterogenous data sources, then customization is improved, but productivity decreases
Solution Approach 1:
The federation layer serves as an intermediary that handles all translation and mapping operations automatically. This eliminates the time-consuming manual code modification process while maintaining full customization capability through configurable schemas and automatic adaptation to different data sources.
Solution Approach 2:
The system performs preliminary actions by pre-defining data schemas, mapping rules, and translation logic in the federation layer. This preparation work is done once and reused across multiple integrations, dramatically improving productivity while maintaining customization through the predefined configurable framework.
3Ease of operation
If unified interface is implemented for heterogenous data sources, then ease of operation is improved, but system complexity increases
Solution Approach 1:
The patent segments the system into distinct layers: the simplified unified API interface layer and the complex federation layer. This segmentation allows the ease of operation at the user interface level while containing system complexity in the intermediate layer, which is isolated from end users.
Solution Approach 2:
The federation layer acts as an intermediary that absorbs and manages system complexity related to heterogenous data sources. It handles all the complex mapping, translation, and integration logic internally, presenting a simple unified interface to users without exposing the underlying complexity.
Data Source
AI summary
Methods, systems, and devices for data processing are described. A federation management service may receive a user selection of a first data source and a second data source via a user interface of the federation management service. The first data source and the second data source may be associated with different data source types. The federation management service may extract schema information from the first data source and the second data source by using source extensions to parse the first data source and the second data source. The federation management service may generate a federated data model by using the schema information to translate the first data source and the second data source into intermediate data sources with a common feature set. Accordingly, the federation management service may generate a target data schema for a federated application programming interface (API) based on the federated data model.