On-Site Machine Learning Model Updates with Confidential User Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models deployed in the field often exhibit lower accuracy than expected due to user data differing from training data, and there is no practical way for providers to approximate or replicate user data to improve performance, as users are reluctant to share private data.
Innovation Solution
A method and system that allows on-site updating of machine learning models using user training data within a container environment, ensuring user data remains confidential and provider technology is secure, by employing a framework that includes a container with an ML model, training data, and a library of functions to build and compare accuracy of updated models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the ML model provider shares training data with users to improve model accuracy, then the model accuracy on user data improves, but user data confidentiality is compromised
Solution Approach 1:
The patent introduces a third-party service that acts as an intermediary between the user and the model training process. This service enables users to train custom models using their own data without the data leaving their controlled environment, thus improving model accuracy while preserving data confidentiality through the intermediary architecture.
Solution Approach 2:
Instead of sharing actual user data with the provider, the system creates and uses copies of the data locally on the user's premises. The user's data is copied to a local machine learning service, processed there, and only the model parameters (not the data itself) are shared or used for improvement, maintaining confidentiality while enabling accuracy improvement.
2Measurement precision
If the provider replicates user data to improve model performance, then model accuracy improves, but the complexity of data handling increases
Solution Approach 1:
The system enables users to perform self-service model training on their own data using a simplified interface. The user provides their data and desired model parameters, and the system automatically handles the complex data processing, feature extraction, and model training operations, reducing the complexity burden on the provider while improving accuracy.
Solution Approach 2:
The patent transforms the approach from replicating entire datasets to working with transformed data parameters and features. The system extracts relevant features and parameters from the user's data, trains models on these transformed parameters, and uses them for prediction, thereby improving accuracy while simplifying data handling complexity.
3Measurement precision
If users share private data with the provider, then the provider can improve model accuracy, but user trust and security are compromised
Solution Approach 1:
The third-party service acts as a trusted intermediary that processes data locally on the user's premises. This intermediary architecture ensures that user data never leaves their controlled environment, maintaining security and trust while still enabling the provider to improve models using the user's data patterns and characteristics.
Solution Approach 2:
The system extracts only the necessary information and patterns from the user's data for model training purposes, while leaving the actual data private and local. By taking out only the essential training information rather than sharing the complete data set, the system improves model accuracy while preserving user trust and data security.
Data Source
AI summary
Updating machine learning models with user data includes executing, by a data processing system, a container including a first machine learning (ML) model, training data for the first ML model, and a library of machine learning functions. The data processing system executes one or more of the machine learning functions of the library. The one or more of the machine learning functions are configured to build a second ML model trained, at least in part, on user training data and to compare accuracy of the first ML model with accuracy of the second ML model. An ML model also may be trained to predict compilation time for circuit designs using training data that includes circuit design features, hardware features of a data processing system, and runtime features from the data processing system.