Orchestrating ML Execution on Constrained Devices via Capability Discovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current ML orchestration on constrained devices faces challenges due to resource limitations, requiring a resource control component that is incompatible with the devices' processing, memory, and storage capabilities, and limited node capabilities, restricting the execution of ML models to only those operators available in the current firmware, which limits flexibility and is prone to firmware update risks.
Innovation Solution
An orchestration node that discovers and selects computing nodes with exposed capabilities to execute component computational operations, using a lightweight protocol to expose and negotiate computational resources as RESTful elements, allowing for distributed orchestration without a resource control component, and utilizing CoAP endpoints to facilitate the execution of complex computational operations like ML models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a resource control component is included to enable ML orchestration on constrained devices, then orchestration capability is improved, but device complexity and resource consumption exceed the processing, memory, and storage limitations of constrained devices
Solution Approach 1:
The resource control component functionality is extracted from the constrained device and relocated to an external orchestrator. The constrained device only needs to expose its computational capabilities as RESTful resources, while the orchestrator handles the complex orchestration logic, making the system adaptable without burdening the constrained device with excessive complexity.
Solution Approach 2:
The orchestrator is designed as a universal component that can manage multiple types of constrained devices with different capabilities. By standardizing the interface through RESTful resources and a common protocol, a single orchestrator can adapt to various device types without requiring device-specific resource control components on each constrained device.
2Adaptability or versatility
If firmware is updated to add new operators for ML execution, then model execution capability is improved, but device reliability decreases due to update risks and operational disruptions
Solution Approach 1:
Instead of static firmware updates to add operators, the system dynamically adapts by having the orchestrator discover and utilize available computational resources on constrained devices at runtime. The orchestrator can flexibly assign different ML models to devices based on their current capabilities without requiring firmware changes, thereby maintaining reliability while improving execution capability.
Solution Approach 2:
The orchestrator performs preliminary capability assessment by discovering what computational operators are already available on each constrained device before assigning ML models. This preliminary action avoids the need for firmware updates by utilizing existing resources, thus preventing update-related reliability issues while still enabling appropriate model execution.
3Extent of automation
If a strictly hierarchical orchestration process is implemented with pre-specified node roles, then orchestration control is improved, but onboarding complexity and preparation time increase significantly
Solution Approach 1:
Constrained devices automatically expose their computational capabilities as RESTful resources without requiring manual configuration or role assignment. The orchestrator autonomously discovers these resources and determines how to utilize them, eliminating the need for complex pre-onboarding procedures while maintaining effective orchestration control.
Solution Approach 2:
Instead of the orchestrator imposing a hierarchical structure with pre-specified roles on devices, the system inverts the approach by allowing devices to present their capabilities autonomously. The orchestrator then adapts to the devices' self-declared capabilities rather than forcing devices to conform to pre-defined roles, significantly reducing onboarding complexity.
Data Source
AI summary
A method (100) for orchestrating execution of a complex computational operation by at least one computing node is disclosed, wherein the complex computational operation can be decomposed into a plurality of component computational operations. The method, performed by an orchestration node, comprises discovering at least one computing node that has exposed, as a resource, a capability of the computing node to execute at least one component computational operation of the plurality of component operations (110). The method further comprises, for each component computational operation of the complex computational operation, selecting a discovered computing node for execution of the component computational operation (120), and sending a request message to each selected computing node requesting the selected computing node execute the component computational operation for which it has been selected (130). The method further comprises checking for a response to each sent request message (140).