Uni-Directional Ring NoC for Neural Network Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Network-on-Chip (NoC) architectures for neural network accelerators are inefficient in handling diverse workloads, leading to suboptimal throughput, increased area, and power consumption, as well as lack of flexibility to work with different modules in Chip Multi-Processors (CMPs) or System-on-Chip (SoC).
Innovation Solution
A uni-directional link-based NoC architecture is proposed, featuring a matrix arrangement of routers and cores with uni-directional ring-shape data links, and a routing control circuit that manages routing logic by converting physical addresses to logical addresses and determining routing ports based on address masks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional bus or mesh-based NoC architectures are used, then data migration flexibility is improved, but area and power consumption increase significantly
Solution Approach 1:
The NoC is segmented into multiple uni-directional ring-shaped data links organized in a matrix pattern, where each row and column forms a separate ring. This segmentation allows data to be routed along specific directional paths, reducing the need for complex cross-point connections while maintaining flexibility for various data migration patterns including one-to-one, one-to-many, many-to-one, and many-to-many communications.
Solution Approach 2:
The patent introduces a two-dimensional ring structure where data can be transmitted along row rings and column rings independently. This dimensional organization allows data to reach any core by combining horizontal and vertical ring traversals, providing flexible routing without requiring a complete mesh of bidirectional connections, thus reducing area while maintaining adaptability.
2Adaptability or versatility
If conventional bus or mesh-based NoC architectures are used, then data migration flexibility is improved, but power consumption increases significantly
Solution Approach 1:
The NoC is segmented into multiple uni-directional ring-shaped data links organized in a matrix pattern, where each row and column forms a separate ring. This segmentation allows data to be routed along specific directional paths, reducing the need for complex cross-point connections while maintaining flexibility for various data migration patterns including one-to-one, one-to-many, many-to-one, and many-to-many communications.
Solution Approach 2:
The patent introduces a two-dimensional ring structure where data can be transmitted along row rings and column rings independently. This dimensional organization allows data to reach any core by combining horizontal and vertical ring traversals, providing flexible routing without requiring a complete mesh of bidirectional connections, thus reducing area while maintaining adaptability.
3Productivity
If customized NoC for neural network workload is used, then computation efficiency is improved, but flexibility to work with different modules in CMPs/SoCs deteriorates
Solution Approach 1:
The uni-directional ring-based NoC architecture serves multiple functions: it optimizes neural network workload processing through efficient data migration patterns while simultaneously providing universal compatibility with different CMP and SoC configurations. The matrix organization of row and column rings allows the same structure to support various processing topologies and communication patterns, making it adaptable to different computational workloads and system architectures.
Solution Approach 2:
The routing control circuit dynamically determines the optimal path through the ring structure based on the communication requirements. For neural network workloads, data can be efficiently migrated along rings in parallel, while for general-purpose CMP/SoC applications, the routing can be dynamically adjusted to accommodate different module interconnections, thus providing both specialized performance and general flexibility.
4Area of stationary object
If traditional NoC architecture is used, then area is reduced, but fault tolerance capability deteriorates
Solution Approach 1:
The ring-based architecture provides inherent fault tolerance by creating redundant communication paths before failures occur. If a core or link fails, data can be rerouted along the ring in the opposite direction or through alternative row/column rings, ensuring continued system operation. This built-in redundancy is achieved without significant area overhead due to the efficient ring structure.
Solution Approach 2:
The routing control circuit can dynamically change routing parameters to bypass defective cores or links. When a fault is detected, the system alters the routing path parameters to use alternative routes through the ring structure, maintaining communication functionality despite component failures. This adaptive parameter adjustment enables fault tolerance without requiring additional hardware redundancy.
Data Source
AI summary
This application describes a network-on-chip system on a hardware accelerator for accelerating neural network computations. An example NoC system in the NN accelerator may include interconnected routers with routing control circuits and cores respectively coupled to the routers. The cores are arranged into a matrix. Each row of cores are connected with a first uni-directional ring-shape data link and every two adjacent data links are in opposite directions. Each column of cores are connected with a second uni-directional ring-shape data link and every two adjacent data links are in opposite directions. In a given router of the plurality of routers, the routing control circuit is configured to: receive a data package; convert physical addresses of the given router and the target router into logical addresses; determine a routing port of the given router based on the logical addresses; and output the data package through the routing port.