2D-Grid Interconnection Network for Deterministic Communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed computing hardware for massively parallel applications, such as deep neural networks, faces limitations in scalability due to interconnection network architectures that require large buffers and result in out-of-order data receipt and frequent blocking, leading to complex software requirements and poor performance.
Innovation Solution
A scalable multi-stage hypercube-based interconnection network with deterministic communication between processing elements, arranged in a 2D-grid with vertical and horizontal buses, allowing for concurrent broadcast and multicast operations without queuing, using switches capable of sending and receiving packets and comprising processors and memory for efficient data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional interconnection network architecture is used for distributed computing hardware, then broadcast and multicast operations can be performed, but large buffers are required and out-of-order data receipt occurs
Solution Approach 1:
The interconnection network is segmented into multiple stages with dedicated routing paths. Each stage handles specific routing decisions, allowing data tokens to be forwarded through predetermined paths without requiring large buffers at any single node. This segmentation enables orderly data flow while maintaining broadcast and multicast capabilities.
Solution Approach 2:
Routing paths are predetermined and established before data transmission begins. The system performs preliminary routing setup that defines the exact path each data token will follow through the network stages. This preliminary action eliminates the need for dynamic buffer management and ensures in-order data receipt without requiring large buffers.
2Ease of operation
If conventional interconnection network architecture is used, then data communication between processing elements is enabled, but frequent blocking occurs requiring complex software
Solution Approach 1:
Routing paths are predetermined and hardcoded into the interconnection network architecture before operation. This preliminary configuration of routing paths eliminates the need for complex software-based routing decisions during runtime. Data tokens simply follow their predetermined paths through the network stages, enabling easy operation without frequent blocking.
Solution Approach 2:
The interconnection network performs routing automatically through its predetermined path structure without requiring external software intervention. Each processing element and network stage self-manages data flow according to the predetermined routing configuration, eliminating the need for complex software control and reducing blocking events.
3Adaptability or versatility
If scalable interconnection network is implemented for massively parallel applications, then concurrent broadcast and multicast are required, but out-of-order receipt and blocking occur
Solution Approach 1:
The network is divided into multiple stages with dedicated routing paths for different data flows. This segmentation allows concurrent broadcast and multicast operations to proceed through different stages simultaneously without interfering with each other. Each stage maintains deterministic forwarding behavior that ensures in-order data receipt even during concurrent operations.
Solution Approach 2:
Routing paths for broadcast and multicast operations are predetermined before execution. This preliminary setup defines the exact sequence and path each data token will follow, ensuring that data is received in the correct order regardless of concurrent operations. The predetermined paths eliminate race conditions and ensure deterministic delivery.
4Productivity
If interconnection network with large buffers is used, then broadcast capability is achieved, but silicon area increases
Solution Approach 1:
The broadcast function is segmented across multiple network stages rather than requiring large buffers at a single location. Each stage performs a portion of the broadcast operation with minimal local buffering, achieving full broadcast capability while keeping the silicon area of individual components small. The distributed segmentation approach replaces concentrated buffer storage with distributed path-based forwarding.
Data Source
AI summary
A scalable multi-stage hypercube-based interconnection network with deterministic communication between two or more processing elements (“PEs”) or processing cores (“PCs”) arranged in a 2D-grid using vertical and horizontal buses (i.e., each bus is one or more wires) is disclosed. In one embodiment the buses are connected in pyramid network configuration. At each PE, the interconnection network comprises one or more switches (“interconnect”) with each switch concurrently capable to send and receive packets from one PE to another PE through the bus connected between them. Each packet comprises data token, routing information such as source and destination addresses of PEs and other information.Each PE, in addition to interconnect, comprises a processor and/or memory. In one embodiment the processor is a Central Processing Unit (“CPU”) comprises functional units that perform such as additions, multiplications, or logical operations, for executing computer programs. In another embodiment the processor comprises a domain specific architecture (“DSA”) based Deep Neural Network (“DNN”) processor comprising one or more multiply accumulate (“MAC”) units for matrix multiply operations. In one embodiment at each PE processor, memory and interconnect are directly connected to each other. The 2D-grid of PEs is of size α×b where α≥1, b≥1, a+b>2, and both α and b are integers is disclosed.Methods for all the PEs of the 2D-grid of PEs concurrently broadcasting packets to all the other PEs in the 2D-grid in a non-blocking, collision-free and without requiring to queue in a deterministic number of time steps, in a fixed predetermined path between each two PEs are disclosed. Methods for all the PEs of the 2D-grid of PEs concurrently arbitrary fan-out multicasting and unicasting packets to the other PEs in the 2D-grid in a non-blocking, collision-free and without requiring to queue in a deterministic number of time steps, in a fixed predetermined path between each two PEs are also disclosed.