Blind Multimodal Learning Neural Network Splitting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current federated learning and blind learning techniques face challenges in training neural networks using vertically partitioned datasets, where decentralized data consists of distributed features associated with the same entity, and data types across different clients are different.
Innovation Solution
The proposed solution involves a blind learning approach that splits a neural network into client-side and server-side portions, allowing for parallel and independent processing across clients. Additionally, a multi-modal artificial intelligence (MMAI) platform is used to handle different types of data from various clients by splitting the global model into parts and using a fusion layer to combine outputs from each client.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If federated learning or blind learning is used to train on decentralized data, then data privacy is maintained and computational costs are reduced, but the system cannot handle vertically partitioned datasets with different data types across clients
Solution Approach 1:
The neural network is divided into client-side portions and server-side portions, with each client training only the portion relevant to their data type. This segmentation allows clients with different data types (vertical partitioning) to train independently without requiring all clients to process all data types, resolving the contradiction between handling diverse data types and maintaining training effectiveness.
Solution Approach 2:
The server-side portions are designed to be universal and can process multiple data types (images, text, tabular data) that come from different clients. This multi-functionality at the server level allows the system to handle vertically partitioned datasets where each client has different data types, while maintaining the benefits of federated learning for privacy and computational efficiency.
2Quantity of substance
If a single global neural network is used for all clients, then model consistency is maintained, but computational costs increase and data privacy is compromised
Solution Approach 1:
Instead of training a single global network on all client data, the network is segmented into client-side and server-side portions. Each client trains only their local portion on their specific data type, significantly reducing computational resources required at each client while the server coordinates the overall training process and aggregates results.
Solution Approach 2:
The server acts as an intermediary that receives outputs from different client-side portions processing different data types, combines them through a fusion layer, and generates unified server-side portions. This intermediary approach allows the system to handle vertically partitioned data without requiring all clients to process all data types, reducing overall computational burden while maintaining model consistency.
3Reliability
If all clients process all data types, then comprehensive training is achieved, but data privacy is compromised and computational costs increase
Solution Approach 1:
The training process is segmented so that each client processes only their local data type (e.g., images at vision clients, text at language clients). This segmentation ensures that clients never access or process data types they don't possess, maintaining data privacy while achieving comprehensive training through the aggregation of specialized client results at the server.
Solution Approach 2:
The server serves as an intermediary that combines outputs from specialized client-side portions without exposing clients to each other's data types. The fusion layer at the server integrates information from different data types while keeping the training process private to each client, thus maintaining training completeness without compromising data privacy.
Data Source
AI summary
A system and method are disclosed for providing a private multi-modal artificial intelligence platform. The method includes splitting a neural network into a first client-side network, a second client-side network and a server-side network and sending the first client-side network to a first client. The first client-side network processes first data from the first client, the first data having a first type. The method includes sending the second client-side network to a second client. The second client-side network processes second data from the second client, the second data having a second type. The first type and the second type have a common association. Forward and back propagation occurs between the client side networks and disparate data types on the different client side networks and the server-side network to train the neural network.