On-Sensor ML Adaptation With Encrypted Feature Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models for artificial reality systems face accuracy issues due to inadequate training data and privacy concerns when raw images are used for training, leading to potential data breaches.
Innovation Solution
Implementing smart sensors with computing resources for local adaptation of machine learning models, using encrypted data transmission and federated learning to ensure privacy and security while allowing periodic updates and global improvements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If raw images are used for training machine learning models, then model accuracy is improved, but data privacy and security are compromised
Solution Approach 1:
The system segments the machine learning training process into local and remote components. Local smart sensors perform incremental training on raw images to capture user-specific patterns, while the server maintains a centralized model. This segmentation allows high accuracy through local data processing while preventing privacy breaches by keeping raw images local.
Solution Approach 2:
The system introduces feature data as an intermediary between raw images and model training. Instead of transmitting raw images to the server, the smart sensor extracts features locally and transmits only these encrypted features. This intermediary approach enables the server to train models using aggregated data while preserving user privacy.
2Measurement precision
If machine learning models are trained on server with aggregated data, then global model accuracy is improved, but local user customization is reduced
Solution Approach 1:
The system implements dynamic model adaptation where the base model received from the server is continuously updated by local incremental training. The local smart sensors dynamically adjust the model parameters based on user-specific data while maintaining alignment with global trends, enabling both accuracy and customization.
Solution Approach 2:
The system applies local quality by allowing each smart sensor to perform incremental training on user-specific data independently. This enables each user to have a customized model adapted to their preferences while the server maintains a unified base model. The local training process modifies only the relevant parameters for that user without affecting other users' models.
3Productivity
If feature data is transmitted to server for training, then model improvement is enabled, but data security is compromised
Solution Approach 1:
The system uses encrypted feature data as an intermediary that conveys training information without exposing raw images. The features are extracted locally, encrypted, and transmitted to the server, which processes them to update the base model. This intermediary approach enables efficient model improvement while maintaining data security through encryption.
4Adaptability or versatility
If local adaptation of models is performed, then user customization is improved, but computational resources are increased
Solution Approach 1:
The system applies partial action by performing incremental training only on the necessary parameters of the machine learning model rather than retraining the entire model. The smart sensors identify and train only the relevant parameters for local customization, significantly reducing computational resource consumption while maintaining effective user adaptation.
Data Source
AI summary
A wearable includes a sensor to generate sensor data and a processor. The sensor includes compute resources configured to update a machine learning model based on feature data extracted from the sensor data. The processor is configured to receive the encrypted data associated with the machine learning model from the sensor and transmit the encrypted data to a server. The parameters of a base machine learning model are received from the server and the parameters are transmitted to the sensor. The sensor is configured to locally update the received base machine learning model based on the feature data.