Configurable Switch Network for In-Fabric Parallel Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Software-based solutions for parallel operations in applications like deep neural network training and graph analysis are limited by memory bandwidth, resulting in performance that is only about 25-30% of the peak rate supported by processors due to multiple memory accesses.
Innovation Solution
The use of integrated circuits with a network of configurable switches that perform operations in a pipelined manner, minimizing data movement and memory access by mapping operations to a collective tree structure, allowing memory access only once in the forward path and broadcasting results in the reverse path.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If software techniques with multiple threads are used to perform parallel computations, then parallel operations can be executed, but performance is limited to 25-30% of peak rate due to multiple memory accesses
Solution Approach 1:
The system segments the parallel computation task into multiple independent operation threads that can be executed simultaneously. Each thread handles a specific computation operation, allowing the system to perform multiple operations in parallel without sequential memory access bottlenecks. The segmentation of computation into independent threads enables each thread to operate on its own data paths, reducing overall memory access time.
Solution Approach 2:
The patent transitions from traditional software-based parallel processing to a hardware-dimension solution by implementing parallel computation capabilities directly in the network switch fabric. This dimensional shift from software to hardware enables simultaneous execution of multiple computation operations at the network layer, bypassing the memory bandwidth limitations that constrain software-based approaches.
2Productivity
If multiple reads and writes to memory are performed for parallel computations, then computations can be executed, but memory bandwidth is exceeded resulting in less than desirable performance
Solution Approach 1:
The system performs preliminary actions by pre-configuring the network switch fabric with computation operations before data arrives. Operation threads are set up in advance to execute specific computations, and data is routed through the switch fabric along predetermined paths. This preliminary configuration eliminates the need for multiple sequential memory reads and writes, as computations are performed in-flight within the network fabric itself.
Solution Approach 2:
The network switch fabric acts as an intermediary between data sources and destinations, performing computation operations within the fabric rather than requiring data to be read from and written to memory multiple times. This intermediary computation approach reduces memory bandwidth consumption by eliminating redundant memory access operations while maintaining high computation throughput.
3Productivity
If data is moved extensively for parallel operations, then computations can be performed, but energy consumption increases
Solution Approach 1:
The patent merges computation operations with data forwarding operations in the network switch fabric. Instead of separating computation into distinct memory read-process-write cycles that require extensive data movement, the system combines these functions into a single integrated operation where data flows through the switch fabric and computations are performed in-line. This merging eliminates redundant data movement and reduces energy consumption.
Solution Approach 2:
The network switch fabric provides self-service computation capabilities, performing operations on data as it passes through the fabric without requiring external memory access. The switch fabric serves its own computation needs by utilizing its existing data paths and resources, eliminating the need for additional energy-intensive data movement to and from external memory systems.
Data Source
AI summary
Embodiments herein may present an integrated circuit including a switch, where the switch together with other switches forms a network of switches to perform a sequence of operations according to a structure of a collective tree. The switch includes a first number of input ports, a second number of output ports, a configurable crossbar to selectively couple the first number of input ports to the second number of output ports, and a computation engine coupled to the first number of input ports, the second number of output ports, and the crossbar. The computation engine of the switch performs an operation corresponding to an operation represented by a node of the collective tree. The switch further includes one or more registers to selectively configure the first number of input ports and the configurable crossbar. Other embodiments may be described and/or claimed.