Dynamic ML Execution Optimization for Resource-Constrained Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models, particularly neural networks, face challenges in efficiently executing on devices with limited compute and memory resources, such as embedded devices, due to their resource-intensive nature, which hinders their performance and energy efficiency.
Innovation Solution
The solution involves optimizing machine learning models by generating execution data based on multiple performance criteria, allowing the processor to dynamically adjust the execution of the model based on different criteria, such as minimizing processor cycles or external memory bandwidth, through grouping layers and utilizing cache memory efficiently, and providing this execution information to the target hardware for optimized runtime performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If machine learning models are executed on devices with limited compute and memory resources, then energy efficiency is improved, but execution performance deteriorates
Solution Approach 1:
The patent implements dynamic execution optimization by generating multiple execution data sets corresponding to different performance criteria (e.g., speed-optimized, memory-optimized, energy-optimized). The system dynamically selects and switches between different execution strategies based on real-time conditions such as available hardware resources, power constraints, and performance requirements, allowing the ML model to adapt its execution profile to balance between performance and energy efficiency.
Solution Approach 2:
The system changes execution parameters by transforming the ML model into different execution representations with varying computational characteristics. Different execution data sets modify parameters such as data precision, computation granularity, memory access patterns, and operator selection to optimize for specific criteria like speed, memory usage, or energy consumption, thereby adjusting the trade-off between performance and energy efficiency.
2Productivity
If machine learning models are optimized for specific performance criteria, then execution efficiency is improved, but adaptability to different hardware conditions deteriorates
Solution Approach 1:
The patent creates a universal execution framework that generates multiple execution data sets covering different performance criteria and hardware conditions. This multi-functional approach allows a single ML model to be executed efficiently across diverse hardware platforms (embedded devices, mobile devices, cloud servers) by selecting the appropriate execution data set that matches the target hardware's capabilities and constraints.
Solution Approach 2:
The system dynamically adapts to different hardware conditions by monitoring available resources (compute power, memory capacity, energy budget) and selecting the most suitable execution strategy at runtime. This dynamic adaptation mechanism ensures the model maintains optimal execution efficiency while being versatile enough to handle varying hardware environments without requiring separate model versions.
3Adaptability or versatility
If multiple execution data sets are generated for different performance criteria, then adaptability to different criteria is improved, but device complexity increases
Solution Approach 1:
The patent segments the execution optimization process into distinct execution data sets, each corresponding to a specific performance criterion (speed, memory, energy). This segmentation allows the complex optimization problem to be divided into manageable components, where each execution data set can be independently generated, stored, and selected based on current needs, reducing the cognitive load on the execution management system.
Solution Approach 2:
The system incorporates feedback mechanisms that monitor hardware performance and resource availability to automatically select the most appropriate execution data set. By continuously feedback about actual execution conditions and outcomes, the system learns which execution strategies perform best under specific conditions, automating the selection process and reducing manual configuration complexity.
Data Source
AI summary
An electronic device, comprising one or more processors, wherein the one or more processors are configured to execute instructions causing the one or more processors to: receive a machine learning (ML) model and execution information associated with the ML model, wherein the execution information including first execution data indicating how to execute the ML model optimized based on a first performance criterion, and second execution data execution data indicating how to execute the ML model optimized based on a second performance criteria, the second performance criterion different from the first performance criteria; execute the ML model based on the first execution data; determine to execute the ML model based on the second execution data; and execute the ML model based on the second execution data.