Collaborative Non-Parametric ML Training via Parametric Combinators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Non-parametric Machine Learning (ML) models cannot be trained collaboratively due to the lack of globally optimizable parameters, which limits their efficiency when processing new data and hinders collaborative learning across decentralized systems while preserving data privacy.
Innovation Solution
The system employs a trainable parametric combinator, such as a Neural Network, to process and combine obfuscated non-parametric ML models from different nodes, generating a composite model that is then trained using Swarm learning to minimize local bias and enhance model efficiency, ensuring data privacy through obfuscation and decentralized processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If non-parametric ML models are used to process new data, then model adaptability is improved, but collaborative training capability deteriorates due to lack of globally optimizable parameters
Solution Approach 1:
The patent introduces a master node as an intermediary that coordinates collaborative training among worker nodes. The master node collects local training results, performs global parameter optimization, and distributes updated parameters back to worker nodes, enabling non-parametric models to participate in collaborative training through this mediating structure
Solution Approach 2:
The patent transforms non-parametric models into a collaborative training framework by introducing learnable parameter matrices (e.g., projection matrices, attention matrices) that are optimized globally at the master node. These parameters enable the models to be tuned collaboratively across distributed nodes while maintaining their non-parametric nature for actual inference
2Loss of information
If local data is used for training non-parametric ML models, then data privacy is improved, but model performance deteriorates due to local bias
Solution Approach 1:
The patent merges local training results from multiple worker nodes at the master node. Each node trains on its local data privately, then the master node aggregates these results and performs global parameter optimization, combining the benefits of local data privacy with global model performance through coordinated learning
Solution Approach 2:
The patent segments the training process into local training phases (performed privately at worker nodes on local data) and global coordination phases (performed at the master node). This segmentation allows each node to maintain data privacy while still contributing to and benefiting from a globally optimized model
Data Source
AI summary
A system and a method for training non-parametric Machine Learning (ML) model instances in a collaborative manner is disclosed. A non-parametric ML model instance is trained at each of a plurality of data processing nodes to obtain a plurality of non-parametric ML model instances. Each non-parametric ML model instance developed at each data processing node is shared with each of remaining data processing nodes of the plurality of data processing nodes. Each non-parametric ML model instance is processed through a trainable parametric combinator to generate a composite model at each of the plurality of data processing nodes. The composite model is trained at each of the plurality of data processing nodes, over the respective local dataset, using Swarm learning to obtain trained composite models.