Schema-Matched Federated Computing for Private Data Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for federated computing fail to effectively process and extract features from heterogeneous data sources, leading to inefficiencies and privacy concerns, making it unsuitable for environments like industrial plants and autonomous vehicles.
Innovation Solution
A client-server system that extracts input data schemas, registers them, and generates output data records using a process pipeline tailored to matching data schemas, ensuring compatibility and privacy, with encrypted and signed pipelines for secure transmission and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred between nodes in distributed computing, then computing power can be distributed, but large amounts of data require large bandwidth and increase transmission costs
Solution Approach 1:
The patent extracts only the essential computing instructions and model parameters from the complete data set, transmitting only these extracted elements between nodes rather than transferring entire data sets, thereby reducing transmission volume while maintaining distributed computing capability
Solution Approach 2:
The patent introduces a coordinator node that acts as an intermediary, managing data localization decisions and coordinating computations across nodes without requiring direct peer-to-peer data transfer, thus reducing overall data transmission requirements
2Adaptability or versatility
If data is shared across distributed nodes, then collaborative computing is enabled, but data privacy and security risks increase
Solution Approach 1:
The patent segments data into local portions retained at each node and shared portions managed by the coordinator, allowing collaborative computing on shared data while maintaining local data privacy through spatial segmentation of data access rights
Solution Approach 2:
The coordinator node serves as a trusted intermediary that manages data access and sharing policies, enabling collaborative computing tasks while protecting individual node data privacy through centralized access control and audit mechanisms
3Adaptability or versatility
If heterogeneous data sources are processed, then data diversity and application versatility improve, but processing complexity and feature extraction difficulty increase
Solution Approach 1:
The patent implements a universal schema-based framework that can handle multiple heterogeneous data sources through a common data model and unified feature extraction interface, allowing diverse data types to be processed through the same computational pipeline
Solution Approach 2:
The patent transforms heterogeneous data from different sources into a standardized parameter representation through schema mapping and feature normalization, converting diverse input formats into uniform parameter sets that can be processed by the same computational algorithms
4Measurement precision
If data is pre-processed for each application, then data quality and accuracy improve, but energy consumption and storage requirements increase
Solution Approach 1:
The patent performs preliminary schema extraction and data validation once during data ingestion, preparing data in advance for multiple potential applications without requiring repeated pre-processing, thereby reducing energy consumption while maintaining data quality
Solution Approach 2:
The patent creates a universal pre-processed representation of data that can serve multiple different applications and analytical tasks simultaneously, eliminating the need to duplicate pre-processing efforts for each application and reducing overall energy consumption and storage requirements
Data Source
AI summary
A client for a federated computing system is configured to store one or more input data records, extract input data schemas therefrom and transfer them to a server. Using the process pipeline received from the server, an output data record is generated from one or more of the input data records whose input data schema matches a process data schema of the process pipeline and transferred to the server. A server for a federated computing system is configured to receive and register input data schemas from one or more clients, obtain a process pipeline based on a process data schema from a user, and transfer it to one or more of the clients whose registered input data schemas match the process data schema. Output data records are then received from one or more clients, aggregated and provided to the user. Computer-implemented methods and computer program products are also provided.