Federated Dataset Views for Private Cross-Server Data Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data federation techniques fail to allow manipulation of federated data without impacting underlying datasets, lack privacy in data modifications, and are time-intensive, often requiring specialists and taking weeks to complete integration.
Innovation Solution
A data federation process that generates a federated dataset separate from underlying datasets, enabling user-directed modifications and rapid integration within minutes, allowing users to customize and update data without affecting original datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data federation integrates datasets from multiple servers, then data accessibility and unity are improved, but the ability to manipulate federated data without impacting original datasets is lost
Solution Approach 1:
The patent creates a virtual copy of the federated data view that allows manipulation without affecting original datasets. The system generates a federated dataset that is a representation of combined data from multiple sources, enabling users to modify this virtual copy while the original datasets remain unchanged and intact.
2Adaptability or versatility
If traditional data federation methods are used, then data integration is achieved, but the process is time-intensive requiring weeks to months and specialist involvement
Solution Approach 1:
The patent replaces manual, mechanical data federation processes with an automated system. Instead of requiring specialists to manually integrate datasets over weeks or months, the system automatically discovers data sources, establishes connections, and creates federated views through automated procedures, reducing integration time to minutes or hours.
Solution Approach 2:
The system enables self-service data federation by providing automated tools that allow users to create and manage federated datasets without requiring specialist knowledge. The automated discovery and integration processes handle the complex tasks independently, freeing users from needing deep technical expertise.
3Adaptability or versatility
If data federation combines datasets into a unified view, then data unity is improved, but privacy is compromised as anyone accessing original datasets can access federated data
Solution Approach 1:
The patent creates a virtual copy of the federated data that can be accessed and modified independently from the original datasets. This virtual representation maintains data unity and accessibility while preventing direct access to underlying sensitive data, thereby preserving privacy.
Solution Approach 2:
The system introduces an intermediary layer between users and the original datasets. This federated view acts as a mediator that provides unified data access while controlling and limiting access to the underlying source datasets, thus maintaining privacy boundaries.
Data Source
AI summary
Systems and methods for federating datasets hosted on separate servers are provided herein. An example data federation process includes receiving a federation request that contains a user-defined data domain distributed across two or more datasets hosted on separate servers. The federation request includes a request for first federation data and second federation data. The data federation process includes sending the federation request to the first server, which determines that it hosts the first federation data and determines call information associated with the first federation data. The first server then determines that the second server hosts the second federation data. The first server generates a model query including a procedure call for the first federation data and the second federation data. Upon fetching the first and second federation data based on the model query, the first server combines the first and second federation data together to generate a federated dataset.