Private AI Model Training with Fixed Public Model
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods of fine-tuning public and private AI models through transfer learning fail to preserve privacy, as they often require updating the public AI model with private data, leading to potential data breaches and increased costs due to energy consumption and inference overhead.
Innovation Solution
A method where a public AI model is kept fixed, and only the private AI model is trained with private samples, forming a combined AI model that maintains the privacy of the user's data while leveraging the public model's knowledge for improved accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the public AI model is updated with private data through transfer learning, then the accuracy of the private AI model is improved, but the privacy of the data is compromised and energy consumption increases
Solution Approach 1:
The patent segments the AI model training process into two independent components: a fixed public AI model that processes public data, and a trainable private AI model that processes private data. This segmentation allows the private model to learn from the public model's features without requiring access to or updates of the public model with private data, thereby preserving privacy while maintaining accuracy.
Solution Approach 2:
The patent introduces an intermediary mechanism where the private AI model learns from the public AI model through feature extraction and transfer without direct data exchange. The public model serves as an intermediary that provides learned representations to the private model, enabling accurate training while preventing private data from contaminating the public model.
2Measurement precision
If the public AI model is updated with private data through transfer learning, then the accuracy of the private AI model is improved, but the energy consumption and inference overhead increase
Solution Approach 1:
The patent segments the computational workload by keeping the large public AI model fixed and only training the smaller private AI model. This segmentation reduces the computational burden and energy consumption, as only the private model's parameters need to be updated during training, while the public model serves as a static feature extractor.
Solution Approach 2:
The patent applies partial training action by training only the private AI model rather than the entire combined system. This partial action approach reduces energy consumption and training time while still achieving the goal of improving private model accuracy through transfer learning from the public model.
3Device complexity
If the public and private AI models are merged into a single model, then the training process is simplified, but the privacy of private data is compromised
Solution Approach 1:
The patent maintains segmentation between the public and private AI models throughout the training process, with the public model remaining fixed and the private model being trained separately. This segmentation preserves privacy by preventing private data from being incorporated into the public model, while the models work together through feature extraction and transfer mechanisms.
Solution Approach 2:
The patent uses copying mechanisms where the private AI model creates copies of features and representations from the public model without accessing the public model's internal state or training data. This copying approach allows the private model to benefit from public model knowledge while maintaining privacy boundaries.
Data Source
AI summary
A computer-implemented method comprises linking a private AI model to a public AI model to thereby form a combined AI model comprising the private AI model and the public AI model; and training the combined AI model with private samples while keeping the public AI model fixed so that only the private AI model is trained with the private samples.