On-Device ML Platform Sharing Models via System API
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning architectures face challenges in data security, network traffic, latency, and resource intensity due to the need for centralized model training and inference, which also complicate application development and user experience.
Innovation Solution
An on-device machine learning platform allows applications to share machine-learned models through a system-level API, enabling local model training and inference while maintaining data security and optimizing resource usage, with mechanisms for feedback and resource exchange.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If machine-learned models are stored and implemented within each application on the user device, then inference can be performed locally without network transmission, but the application data size and memory footprint increase significantly
Solution Approach 1:
The patent segments the machine learning infrastructure into separate components: the machine learning engine and models are separated from individual applications. The engine is implemented as a standalone system service that multiple applications can access, rather than embedding complete ML capabilities in each application. This segmentation reduces application data size while maintaining local inference capability.
Solution Approach 2:
The patent creates a universal machine learning engine that serves multiple applications simultaneously. This single engine can handle inference requests from any application that registers its models, eliminating the need for each application to maintain its own ML infrastructure. The engine provides multi-functional support for training, inference, and model management across the entire device.
2Quantity of substance
If machine-learned models are trained and stored at a centralized server location, then applications can access models without increasing local storage, but network transmission is required which increases latency and reduces communication speed
Solution Approach 1:
The patent introduces an intermediary machine learning engine that runs as a system service on the user device, positioned between the applications and any remote servers. This intermediary handles local inference requests without network transmission, reducing latency. It can also manage model updates from servers and distribute them to applications, acting as a buffer that minimizes network dependency for routine operations.
3Adaptability or versatility
If each application implements its own machine learning engine, then the application has full control over model training and inference, but the development complexity increases and developers must learn multiple ML engines
Solution Approach 1:
The patent implements a self-service architecture where the machine learning engine automatically manages model registration, training data collection, and inference operations. Applications simply register their models and training data through standardized interfaces, and the engine handles the complex ML operations autonomously. This reduces development complexity while maintaining flexibility, as the engine adapts to different application needs through configuration rather than requiring developers to understand underlying ML engine intricacies.
4Adaptability or versatility
If machine learning models are updated frequently within applications, then the models can incorporate latest advancements, but application updates require network usage and user downtime
Solution Approach 1:
The patent implements preliminary action by having the machine learning engine pre-load and prepare updated models in the background before they are needed. When new models or engine updates are available, the system downloads and validates them during idle periods, so that when applications need updated functionality, the updates are already ready to activate instantly. This eliminates user downtime and network interruptions during critical moments.
Data Source
AI summary
The present disclosure provides an on-device machine learning platform that enables sharing of machine-learned models between applications on a computing device. For example, a first application which has a machine-learned model for a specific task can expose the model to other applications through a system level application programming interface (API) for the other applications to use. Communications using the API can be handled by the on-device machine learning platform. In some implementations, some exchange of resources (e.g., computing resources) can be provided so that the first application is compensated for sharing the machine-learned model (e.g., on a per model invocation basis).