SIMD Data Routing Module for Low-Latency Memory Rearrangement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing SIMD computing architectures face challenges in efficiently and reliably handling massive volumes of data for deep neural networks due to inadequate data routing mechanisms, leading to degraded performance and high energy consumption, especially when rearranging data in memory locations.
Innovation Solution
An automatic data routing module is integrated into the hardware accelerator, featuring input and output buffers, selectors, assemblers, and transfer modules, controlled by configuration signals to adaptively rearrange data in local memories, ensuring scalability and modularity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If additional buffers are used for data rearrangement in prior art solutions, then data reordering capability is improved, but memory usage increases and scalability decreases
Solution Approach 1:
The system segments data into blocks and uses selectors to choose specific data items from input buffers, assembling them in output buffers. This segmentation allows flexible data rearrangement without requiring additional large buffers, as only small fixed-size buffers are needed for each selector-output buffer connection.
Solution Approach 2:
The routing module is designed to handle multiple data rearrangement operations simultaneously using the same infrastructure of selectors, assemblers, and transfer modules. The configuration signals enable the system to adapt to different data distribution requirements without adding physical hardware components.
2Adaptability or versatility
If manual data manipulation is performed by host processor, then data placement flexibility is improved, but processing latency and energy consumption increase
Solution Approach 1:
The hardware accelerator performs data rearrangement operations autonomously using dedicated routing modules with selectors and assemblers. The system self-manages data placement without requiring continuous intervention from the host processor, reducing latency and energy consumption while maintaining flexibility through configuration signals.
Solution Approach 2:
The patent replaces the mechanical process of manual data manipulation by the host processor with an automated hardware-based routing system. The routing module uses electrical signals and hardware logic to perform data rearrangement, eliminating the need for software-based data movement operations.
3Adaptability or versatility
If data is transferred to level 2 memory for rearrangement, then data placement adaptability is improved, but parallelism is broken and latency increases
Solution Approach 1:
The routing module acts as an intermediary between the host processor and the hardware accelerator, managing data placement locally without requiring transfers to level 2 memory. The selectors and assemblers within the routing module enable flexible data rearrangement in-situ, maintaining parallelism and reducing latency.
Data Source
Figure 1a
Figure 1b
Figure 2
AI summary
Automatic Data Routing Module (ARA) for a computer with a "single instruction, multiple data" architecture comprising several elementary processors, each associated with a local memory, the routing module comprising: - An input interface comprising several input buffers, each intended to receive data read from a respective local memory, - An output interface comprising several output buffers, each intended to transmit data to be written to a respective local memory, - For each input buffer, a selector (SEL) configured to select one or more data items contained in the input buffer, - At least one assembler (ASB) configured to group the data selected by at least two selectors (SELs) into an assembly buffer, - a transfer module (MTR) for each assembler.configured to transfer data from the assembler's assembly buffer to at least one output buffer for writing to at least one local memory location.