Secure On-Device ML Model Loading for Shared Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing on-device machine learning (ML) models face security concerns, computational redundancy, and frequent updates, leading to increased application size and execution overhead, especially when multiple applications require the same inferences.
Innovation Solution
A method and electronic device that load ML models into a secure storage based on detected contexts, using a smart agent to generate inferences for registered applications, eliminating the need for encryption and decryption, and reducing redundant computations by sharing inferences across multiple applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If ML models are deployed to on-device storage, then security of user data is improved, but application size increases and frequent updates are required
Solution Approach 1:
The system segments ML models into separate downloadable components rather than embedding them in the application package. The application installer only contains the framework, while individual ML models are downloaded separately based on context detection, reducing initial application size while maintaining security capabilities.
Solution Approach 2:
The system performs preliminary context detection to determine which ML models are needed before downloading them. This allows the application to prepare and download only necessary models in advance, avoiding unnecessary downloads and reducing application size while maintaining security readiness.
2Adaptability or versatility
If multiple applications use the same ML model independently, then each application can access the model, but computational redundancy increases
Solution Approach 1:
The system merges multiple application requests for the same ML model into a single inference execution. When multiple applications need the same model, the system detects this through context sharing and executes the inference once, then shares the result across all requesting applications, eliminating redundant computations while maintaining universal access.
Solution Approach 2:
The system creates a universal inference mechanism that serves multiple applications simultaneously. A single ML model inference can be shared across multiple applications that share the same context, making the inference engine multi-functional and eliminating the need for separate execution instances for each application.
3Measurement precision
If ML models are updated frequently, then model accuracy is improved, but application updates are required frequently
Solution Approach 1:
The system performs preliminary context detection to identify when an ML model update is actually needed based on whether the context has changed. This prevents unnecessary downloads and updates, allowing the system to maintain high inference accuracy by updating only when necessary while minimizing update frequency and time loss.
Solution Approach 2:
The system implements feedback mechanisms to monitor whether ML model updates are necessary based on context changes. By detecting whether the inference context has changed, the system can determine if an update is needed, providing feedback that prevents unnecessary updates and reduces update frequency while maintaining accuracy.
4Reliability
If encryption techniques are used to secure ML models, then security is improved, but execution time increases
Solution Approach 1:
The system performs preliminary context detection and ML model selection before inference execution. By determining which model to use in advance based on detected context, the system avoids runtime encryption/decryption overhead and directly loads the appropriate pre-identified model, maintaining security while reducing execution time.
Data Source
AI summary
A method for performing an inference includes: detecting a context among at least one context associated with at least one application; triggering a model execution command to a smart agent of an electronic device, based on the detected context; loading a machine learning (ML) model into a secure storage of the electronic device, based on the detected context and the triggered model execution command; generating, using the loaded ML model, an inference, based on data associated with the detected context; and sharing the generated inference with each application of the at least one application that is registered for the detected context.