Neural Network Compiler Logical to Physical Address Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing large and complex deep neural networks on single accelerator devices is challenging due to computational and bandwidth limitations, and existing compilers struggle to optimize neural networks for different topologies of physical resources, leading to inefficiencies in resource utilization.
Innovation Solution
A system that includes a compiler generating neural networks with logical identifiers, a scheduler mapping these identifiers to physical addresses based on latency information and topology selection, and a loader dynamically converting logical identifiers to physical addresses for optimal resource allocation and execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a deep neural network is implemented on a single accelerator device, then the computational complexity is reduced, but the bandwidth limitations and computational capacity constraints prevent deployment of large and complex networks
Solution Approach 1:
The patent divides the deep neural network into multiple segments or layers that can be distributed across multiple accelerator devices. Each accelerator device handles a portion of the computational workload, allowing large and complex networks to be deployed without overwhelming the bandwidth and computational capacity of a single device.
Solution Approach 2:
The patent transitions from a single-device architecture to a multi-device distributed architecture, adding the dimension of spatial distribution. This allows the system to scale computational capacity by adding more accelerator devices rather than being constrained by the limits of a single device.
2Manufacturing precision
If a compiler is configured with specific physical resource topology information, then optimization for that topology is improved, but adaptability to different topologies deteriorates
Solution Approach 1:
The patent creates a universal compiler configuration that can adapt to multiple physical resource topologies without requiring specific topology information. The compiler uses abstract resource models and parameter-based configurations that work across different accelerator device arrangements, maintaining optimization capability while achieving broad topology adaptability.
Solution Approach 2:
The patent employs parameter-based configuration where the compiler accepts topology-independent parameters and automatically adapts to different physical resource arrangements. By changing parameters rather than hardcoding topology-specific configurations, the system maintains optimization precision across diverse topologies.
3Quantity of substance
If neural networks are deployed across multiple accelerator devices, then computational capacity and bandwidth are increased, but resource utilization efficiency deteriorates due to lack of optimization
Solution Approach 1:
The patent implements dynamic resource allocation and load balancing across multiple accelerator devices. The system can dynamically adjust the distribution of computational tasks based on current device utilization, workload characteristics, and performance metrics, thereby maintaining high resource utilization efficiency while scaling computational capacity across multiple devices.
Data Source
AI summary
In some examples, a system generates a neural network comprising logical identifiers of compute resources. For executing the neural network, the system maps the logical identifiers to physical addresses of physical resources, and loads instructions of the neural network onto the physical resources, wherein the loading comprises converting the logical identifiers in the neural network to the physical addresses.