Virtual Data Federation for Editable Cross-Server Datasets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data federation techniques fail to allow manipulation of combined datasets without impacting the original datasets, lack privacy in modifications, and are time-intensive, often requiring specialists and taking weeks to complete integration.
Innovation Solution
A data federation process that generates a separate, customizable federated dataset from disparate datasets hosted on separate servers, allowing users to modify the federated dataset without affecting the underlying data, and can be performed quickly by non-specialists.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional data federation integration is performed, then data from multiple sources can be combined, but the integration is permanent and any changes to combined data impact the original datasets
Solution Approach 1:
The patent segments the federated data into a separate virtual table that is distinct from the original source datasets. This allows the federated view to be manipulated independently while the original data remains intact and unchanged. The segmentation is achieved through virtual table generation that references source tables without physically copying or modifying them.
Solution Approach 2:
The patent creates a virtual copy of the federated data structure that does not physically replicate the actual data. Instead, it generates a virtual table that references and queries the original source datasets dynamically. This virtual copy can be modified without affecting the source data, as it is merely a view rather than a physical duplicate.
2Adaptability or versatility
If traditional data federation is performed, then datasets can be integrated, but the integration is not private and anyone with access to original datasets can access the federated data
Solution Approach 1:
The patent segments access control by creating a separate virtual table layer between the source datasets and the user interface. This virtual table can have its own access control mechanisms that are independent from the source dataset permissions, allowing fine-grained control over who can access federated data without exposing the underlying source data structure.
Solution Approach 2:
The virtual table acts as an intermediary layer that mediates between the source datasets and the users. It provides a controlled interface that can enforce privacy policies and access controls independently of the source data, preventing direct access to original datasets while still enabling federated data queries.
3Productivity
If traditional data federation processes are used, then datasets can be integrated, but the process is time intensive and requires specialists
Solution Approach 1:
The patent implements self-service through automated virtual table generation that requires minimal manual intervention. The system automatically discovers source datasets, generates appropriate virtual tables, and establishes relationships without requiring specialist data federation experts. Users can perform federation operations through simple interface interactions rather than complex manual processes.
Solution Approach 2:
The patent changes the fundamental parameters of data federation from physical data movement and manual integration to virtual table generation and automated query composition. This parameter change transforms the process from time-intensive manual work to rapid automated operations, significantly improving productivity while reducing the skill level required.
4Ease of operation
If data federation is performed to combine datasets, then a unified view can be created, but the process takes weeks to complete integration
Solution Approach 1:
The patent uses virtual copying to create federated views without physically copying data. The virtual tables are generated instantly by establishing references to source datasets, eliminating the time-consuming data movement and transformation processes that occur in traditional federation. This approach achieves unified data views in seconds rather than weeks.
Solution Approach 2:
The patent performs preliminary actions by pre-defining the virtual table structure and relationships before actual data queries are executed. The virtual tables are set up in advance with their metadata and relationships, allowing rapid query execution without needing to perform complex data integration operations at query time.
Data Source
AI summary
Systems and methods for federating datasets hosted on separate servers are provided herein. An example data federation process includes receiving a federation request that contains a user-defined data domain distributed across two or more datasets hosted on separate servers. The federation request includes a request for first federation data and second federation data. The data federation process includes sending the federation request to the first server, which determines that it hosts the first federation data and determines call information associated with the first federation data. The first server then determines that the second server hosts the second federation data. The first server generates a model query including a procedure call for the first federation data and the second federation data. Upon fetching the first and second federation data based on the model query, the first server combines the first and second federation data together to generate a federated dataset.


