Machine-Learned Model Blocks for Resource-Aware Edge Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
On-device machine-learned models face performance issues due to varying computational resources, leading to increased latency and suboptimal resource allocation, especially in resource-constrained environments.
Innovation Solution
A machine-learned model with adjustable computational demand is implemented, allowing for intelligent deactivation of model blocks based on resource allocation parameters, using residual connections to maintain model robustness and quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the full machine-learned model is executed on edge devices, then model accuracy and completeness are maintained, but computational resource consumption increases and processing time increases
Solution Approach 1:
The machine-learned model is segmented into multiple independently executable blocks or layers. The system can selectively execute different subsets of these blocks based on available computational resources, allowing the model to be divided into optional components that can be activated or deactivated dynamically.
Solution Approach 2:
The model execution is made dynamic by allowing the set of active blocks to change based on runtime conditions such as available computational resources. The system can adaptively adjust which model blocks are executed, transitioning between different model configurations to optimize the balance between accuracy and resource consumption.
2Productivity
If more computational resources are allocated to the machine-learned model, then processing speed and performance improve, but device resource availability decreases for other tasks
Solution Approach 1:
The model execution adapts dynamically to available device resources by adjusting the number and complexity of blocks executed. When resources are abundant, more blocks are activated for faster processing; when resources are constrained, fewer blocks are executed to preserve resources for other device tasks.
Solution Approach 2:
The system changes execution parameters such as the number of model blocks, depth of processing, or complexity of operations based on available resources. This allows the model to operate at different performance levels by modifying execution parameters rather than requiring separate models for different resource scenarios.
3Use of energy by moving object
If the machine-learned model is optimized for resource-constrained environments, then resource consumption decreases, but model performance and accuracy deteriorate
Solution Approach 1:
By segmenting the model into blocks with varying importance or contribution to accuracy, the system can selectively execute critical blocks that maintain performance while omitting less important blocks to reduce resource consumption. This ensures that core functionality is preserved even in constrained environments.
Solution Approach 2:
The system performs partial model execution by activating only the necessary subset of blocks required to achieve acceptable performance thresholds. This partial action approach allows the model to function effectively with reduced computational resources while maintaining sufficient accuracy for the given context.
4Stability of the object's composition
If a single machine-learned model is designed for all devices, then model consistency is maintained, but it cannot adapt to varying computational capabilities across devices
Solution Approach 1:
The model is segmented into standardized blocks that can be consistently defined across all devices. This segmentation allows the same model architecture to be deployed universally while enabling each device to execute only the blocks appropriate for its capabilities, maintaining consistency in model design while adapting execution to device variations.
Solution Approach 2:
The model architecture is designed with universal block structures that can serve multiple functions across different device types. The same set of model blocks can be executed in different combinations or configurations to accommodate varying device capabilities, allowing a single model design to serve diverse hardware platforms.
Data Source
AI summary
A computing device is disclosed that includes at least one processor and a machine-learned model. The machine-learned model can include a plurality of blocks and one or more residual connections between two or more of the plurality of blocks. The machine-learned model can be configured to receive a model input and, in response to receipt of the model input, output a model output. The machine-learned model can be configured to perform operations including determining a resource allocation parameter that corresponds to a desired allocation of system resources to the machine-learned model at an inference time: deactivating a subset of the plurality of blocks of the machine-learned model based on the resource allocation parameter: inputting the model input into the machine-learned model with the subset of the plurality of blocks deactivated; and receiving, as an output of the machine-learned model, the model output.