Distributed Machine Learning Model Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning approaches require all models to have a common fundamental architecture for sharing model parameters, limiting heterogeneity and necessitating the sharing of full training data sets, which raises concerns about user privacy and bandwidth usage.
Innovation Solution
A method where client devices send partial data points and associated labels to a server system, allowing for the training of different machine learning models without sharing the full feature vector, enabling a model-agnostic distributed learning approach that accommodates diverse architectures and prioritizes privacy and bandwidth efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If federated learning approaches share model parameters, then models can be trained collaboratively, but all models must have the same fundamental architecture which limits heterogeneity
Solution Approach 1:
The patent segments the feature vector into multiple subsets, where each subset can be independently processed by different machine learning models. This allows models with different architectures to work on different feature subsets simultaneously, enabling heterogeneity while maintaining collaborative training through the shared label information.
Solution Approach 2:
Instead of requiring complete feature vectors and full model parameter sharing, the patent uses partial feature subsets for training. Each model receives only the feature subset it needs, reducing the complexity of architecture uniformity requirements while still achieving collaborative learning through shared labels.
2Productivity
If full training data sets are shared, then models can be trained collaboratively, but user privacy concerns arise
Solution Approach 1:
The patent segments both the feature vector and the training data into subsets that are distributed across different devices. Each device trains its model on local data subsets and only shares the learned label information, preventing exposure of raw user data while maintaining collaborative training effectiveness.
Solution Approach 2:
The patent extracts only the essential label information from the training process and transmits it between devices, leaving the sensitive raw feature data local to each device. This extraction approach enables collaborative learning while removing the privacy risk associated with sharing complete training datasets.
3Productivity
If full training data sets are shared, then models can be trained collaboratively, but bandwidth usage increases
Solution Approach 1:
The patent extracts and transmits only the compressed label information rather than complete feature vectors during collaborative training. This dramatically reduces the amount of data transmitted over the network, lowering bandwidth consumption while preserving the ability to train models collaboratively through shared label knowledge.
Solution Approach 2:
Instead of transmitting complete training datasets, the patent uses partial information (labels derived from feature subsets) for inter-device communication. This partial information approach maintains collaborative training effectiveness while minimizing bandwidth usage.
Data Source
AI summary
A method comprising, by first computer equipment: obtaining an input data point comprising a set of values, each being a value of a different element of an input feature vector; inputting the input data point to a first machine learning model on the first computer equipment to generate at least one associated output label based on the input data point; sending a partial data point to second computer equipment, the partial data point comprising the values of only part of the feature vector; and sending the associated label to the second computer equipment in association with the partial data point, thereby causing the second computer equipment to train a second machine learning model on the second computer equipment based on the sent part and the associated label.