Deep Learning Operator Assignment for Heterogeneous IoT Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for large-scale deep learning model distribution across multiple IoT devices are limited in scalability and do not account for varying computing power, memory, and network conditions, leading to suboptimal end-to-end reasoning latency.
Innovation Solution
A deep learning image classification method that models the deep learning model as a directed acyclic graph, generates a new graph to represent communication tasks, sets parameters for computing and communication tasks, and optimally assigns operators to heterogeneous devices to minimize reasoning completion time, considering processing time, memory, and data transmission latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a large-scale deep learning model is distributed across multiple IoT devices, then the model can be executed on resource-constrained devices, but the end-to-end reasoning latency increases due to network transmission and coordination overhead
Solution Approach 1:
The patent segments the large-scale deep learning model into multiple sub-models and further divides operators within each sub-model into fine-grained tasks. This segmentation enables distribution across multiple IoT devices while maintaining optimization capability at each device level, balancing adaptability and latency through hierarchical decomposition.
Solution Approach 2:
The patent introduces dynamic operator assignment that adapts to real-time network conditions and device states. The system dynamically adjusts which operators are assigned to which devices based on current latency conditions, computing power availability, and network transmission status, thereby optimizing the trade-off between model adaptability and reasoning latency.
2Productivity
If operator assignment is optimized for minimum latency, then end-to-end reasoning speed improves, but the complexity of assignment calculation increases significantly
Solution Approach 1:
The patent segments the complex operator assignment problem into hierarchical levels: first assigning entire operators to devices, then further segmenting operators into fine-grained tasks within each operator. This multi-level segmentation reduces the computational complexity of finding optimal assignments while still achieving high reasoning speed through coordinated optimization at each level.
Solution Approach 2:
The patent applies local optimization by considering device-specific characteristics such as computing power, memory capacity, and network bandwidth when assigning operators. Each device receives operators tailored to its local capabilities, and the assignment algorithm optimizes locally at each device while coordinating globally to minimize overall latency, thereby managing complexity through localized decision-making.
3Ease of manufacture
If computing tasks are assigned without considering device heterogeneity, then assignment simplicity is maintained, but computing efficiency decreases due to mismatched task-device pairing
Solution Approach 1:
The patent explicitly addresses device heterogeneity by introducing device-specific parameters including computing power, memory size, and network bandwidth. The assignment algorithm uses these local qualities to match operators with appropriate devices, ensuring that each device executes tasks suited to its capabilities. This maintains relative simplicity through automated matching while significantly improving computing efficiency through heterogeneous optimization.
Solution Approach 2:
The patent changes the assignment approach from uniform treatment of all devices to parameter-based differentiation. By incorporating device-specific parameters (computing power, memory, network bandwidth) into the assignment decision process, the system automatically adapts task allocation to device characteristics, thereby improving computing efficiency without requiring manual configuration complexity.
4Device complexity
If more operators are assigned to fewer devices, then device utilization is simplified, but network transmission overhead increases due to more inter-device communication
Solution Approach 1:
The patent segments operators into fine-grained tasks that can be distributed across multiple devices with minimal inter-device communication. By breaking down operators into smaller executable units, the system can assign related tasks to the same device, reducing the need for network transmission while maintaining simplified device management through the hierarchical task structure.
Solution Approach 2:
The patent merges related fine-grained tasks onto the same computing device when possible, thereby reducing the number of inter-device data transmissions. By combining tasks that require frequent communication into single-device execution units, the system minimizes network overhead while maintaining manageable device complexity through automated merging decisions.
Data Source
AI summary
The present disclosure relates to deep learning image classification oriented to heterogeneous computing devices. According to embodiments of the present disclosure, the deep learning model can be modeled as an original directed acyclic graph, with nodes representing operators of the deep learning model and directed edges representing data transmission between the operators. Then, a new directed acyclic graph is generated by replacing the directed edges in the original directed acyclic graph with new nodes and adding two directed edges to maintain a topological structure.