Entity Model Training with Frozen Parameters for Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning architectures face challenges in maintaining the privacy of closed data sets while generating accurate output predictions, as they often require sharing sensitive data across entities, leading to model leakage and reduced predictive performance.
Innovation Solution
The proposed method involves training a machine learning architecture using a two-stage technique, where an entity model is first trained on an open data set and then further trained on a closed data set specific to a subset of entities, with parameters associated with non-frozen entities remaining fixed, allowing for entity-specific predictions without sharing sensitive data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If closed data sets are shared across entities for training machine learning models, then model training can be performed, but data privacy is compromised and model leakage occurs
Solution Approach 1:
The patent segments the training process into two distinct stages: (1) training on open data sets that can be shared across entities, and (2) fine-tuning on closed data sets that remain private to each entity. This segmentation allows the system to benefit from shared data while preserving entity-specific data privacy, resolving the contradiction between model training capability and data privacy protection.
Solution Approach 2:
The patent introduces an intermediary mechanism where entity-specific parameters are frozen during the second training stage, acting as a mediator that prevents direct access to closed data sets while still allowing the model to learn from them indirectly through the frozen parameters. This enables training capability while maintaining data privacy through the intermediary frozen parameters.
2Measurement precision
If entity-specific models are created for each entity, then predictive accuracy is improved, but device complexity and resource requirements increase
Solution Approach 1:
The patent merges the benefits of entity-specific models with the efficiency of shared models by combining a shared base model (trained on open data) with entity-specific frozen parameters (from closed data). This merging achieves high predictive accuracy through entity personalization while avoiding the full complexity of completely separate models for each entity, as the shared base model handles common patterns.
Solution Approach 2:
The patent applies local quality by making only specific parameters entity-specific (frozen parameters related to closed data) while keeping the rest of the model shared across entities. This selective entity-specificity achieves the necessary predictive accuracy for each entity without the overhead of entirely separate models, reducing device complexity while maintaining local accuracy where needed.
3Measurement precision
If all parameters are retrained on closed data sets, then entity-specific performance is maximized, but training time and computational resources increase
Solution Approach 1:
The patent applies preliminary action by first training the model on open data sets to establish a solid base model before performing the second stage training on closed data sets with frozen parameters. This preliminary training reduces the computational burden and time required for the second stage, as the model already has learned general patterns and only needs entity-specific adjustments.
Solution Approach 2:
The patent uses partial action by retraining only the necessary entity-specific parameters (those associated with closed data) while keeping other parameters frozen, rather than retraining all parameters. This partial retraining achieves entity-specific performance optimization without the excessive computational cost and time of full model retraining.
Data Source
AI summary
Systems, methods and computer-readable storage media utilized to train a machine-learning architecture. One method includes receiving, by one or more processing circuits, a data set. The method further includes determining, by the one or more processing circuits, a first portion of the data set associated with a plurality of entities. The method further includes training, by the one or more processing circuits and utilizing the first portion of the data set, an entity model. The method further includes determining, by the one or more processing circuits, a second portion of the data set associated with a first subset of entities and determining a second subset of entities. The method further includes freezing, by the one or more processing circuits, one or more parameters associated with the second subset of entities and training, utilizing the second portion of the data set, the entity model.