SIMD Data Routing Module for Low-Latency Neural Network Shuffling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing SIMD architecture computers face challenges in efficiently managing data movement and rearrangement for deep neural networks, leading to degraded performance due to high latency and energy consumption, as they lack a scalable and modular data routing mechanism.
Innovation Solution
An automatic data routing module is integrated into the hardware accelerator, comprising input and output buffers, selectors, assemblers, and transfer modules, controlled by a global controller to perform data rearrangement efficiently and adaptively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If data rearrangement is performed without dedicated hardware, then device complexity is reduced, but execution time increases and performance degrades
Solution Approach 1:
The patent introduces a dedicated data routing module as an intermediary component between memory and processing units. This module includes input buffers, output buffers, selectors, and assemblers that specialize in data rearrangement operations. By offloading this specific function to dedicated hardware, the system achieves fast data shuffling without requiring complex software routines, thus resolving the contradiction between hardware complexity and execution time.
Solution Approach 2:
The data routing module operates autonomously to perform data rearrangement operations. The selectors and assemblers automatically route data from input buffers to output buffers based on computation requirements, without requiring external intervention or complex control logic. This self-service capability enables the system to maintain low execution time while avoiding the need for overly complex hardware structures.
2Productivity
If buffer memories are added for data rearrangement, then data movement capability is improved, but memory usage increases and scalability decreases
Solution Approach 1:
The data routing module is segmented into distinct functional components: input buffers for receiving data, output buffers for transmitting data, selectors for choosing data items, and assemblers for consolidating data. This segmentation allows each component to be independently optimized and configured based on specific application requirements, improving data movement capability while maintaining scalability through modular design.
Solution Approach 2:
The routing module employs dynamic selectors and assemblers that can be reconfigured based on computation requirements. The selectors can dynamically choose different data items from input buffers, and assemblers can dynamically consolidate data from different selectors. This dynamic behavior enables the system to adapt to various data distribution patterns and computation types without requiring additional buffer memories, thus maintaining scalability.
3Speed
If data is quickly transferred to minimize latency, then execution speed is improved, but energy consumption increases
Solution Approach 1:
The data routing module enables continuous data transfer operations by maintaining input and output buffers that operate in parallel. Data can be continuously loaded into input buffers while simultaneously being routed to output buffers and processed. This continuous operation eliminates idle time and reduces the need for repeated data movement operations, achieving high data transfer speed with optimized energy consumption.
Data Source
AI summary
An automatic data routing module for a “single instruction, multiple data” architecture computer includes a plurality of elementary processors each associated with a local memory, the routing module including: an input interface including a plurality of input buffers, each intended to receive data read from a respective local memory; an output interface including a plurality of output buffers, each intended to transmit data to be written to a respective local memory; a selector, for each input buffer, configured to select one or more data items contained in the input buffer; at least one assembler configured to consolidate the data selected by at least two selectors into an assembly buffer; a transfer module for each assembler, configured to transfer the data from the assembly buffer of said assembler to at least one output buffer for writing said data to at least one local memory.