ML Model Distribution Across Edge and Non-Edge Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in efficiently managing and distributing machine learning (ML) models across edge devices and non-edge devices, where edge devices are resource-constrained and typically less powerful than web service providers, making it difficult to execute ML models effectively while meeting throughput and power usage objectives.
Innovation Solution
A model location engine analyzes edge device and non-edge device characteristics, network conditions, and ML model specifics to generate an execution plan that optimally distributes the execution of ML models between edge devices and non-edge devices, adjusting based on conditions such as network congestion and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If ML models are executed on edge devices, then latency is reduced and real-time processing is improved, but device complexity and power consumption increase
Solution Approach 1:
The patent segments the ML model into multiple components or layers, distributing different segments across edge devices and non-edge devices. This allows the edge device to execute only the necessary segments locally, reducing latency for time-sensitive operations while offloading complex segments to non-edge devices, thereby balancing speed improvement with device complexity management.
Solution Approach 2:
The patent implements dynamic model distribution where the execution plan is not fixed but adapts based on changing conditions such as network availability, device workload, and power constraints. The system can dynamically shift model segments between edge and non-edge devices, optimizing the balance between latency reduction and device complexity in real-time.
2Speed
If ML models are executed on edge devices, then real-time processing is improved, but power usage increases
Solution Approach 1:
The patent segments the ML model to execute only critical, time-sensitive portions on the edge device while offloading less time-critical segments to non-edge devices. This selective segmentation reduces the computational burden and power consumption on the edge device while maintaining real-time processing capabilities for urgent operations.
Solution Approach 2:
The system dynamically adjusts the execution plan based on power availability and network conditions. When power is constrained or network is available, the system shifts more model execution to non-edge devices. When real-time processing is critical and power is available, it executes more segments locally, optimizing the trade-off between speed and power usage.
3Productivity
If ML models are distributed across multiple devices, then resource utilization is optimized, but system complexity increases
Solution Approach 1:
The patent introduces an intermediary component (such as a model location engine or orchestration service) that manages the distribution and coordination of ML model execution across multiple devices. This intermediary handles the complexity of model segmentation, device selection, and result aggregation, allowing individual edge devices to remain relatively simple while achieving optimized resource utilization through coordinated distributed execution.
4Power
If ML models are executed on non-edge devices, then computational power is sufficient, but latency increases
Solution Approach 1:
The patent segments the ML model to identify and extract time-critical components that must be executed locally on the edge device to maintain low latency, while non-critical components are offloaded to non-edge devices for their superior computational power. This segmentation ensures that latency-sensitive operations are handled quickly at the edge while leveraging remote computational resources for heavier processing.
Data Source
AI summary
Techniques for generating and executing an execution plan for a machine learning (ML) model using one of an edge device and a non-edge device are described. In some examples, a request for the generation of the execution plan includes at least one objective for the execution of the ML model and the execution plan is generated based at least in part on comparative execution information and network latency information.