Hardware Accelerator Tensor Stride Reshape Transpose
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI hardware accelerators face challenges in efficiently performing tensor operations such as stride, reshape, and transpose, which are crucial for AI neural network computations, leading to suboptimal performance and increased CPU load due to frequent switching between hardware accelerators and CPUs.
Innovation Solution
A hardware accelerator is designed with a memory system that includes a source address generation unit, data collection and dispersion units, and address queues to perform tensor stride, reshape, and transpose operations, enabling efficient tensorflow depth-to-space and space-to-depth permutations, thereby reducing CPU load and enhancing AI computing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If tensor operations (stride, reshape, transpose) are performed using conventional AI hardware accelerators, then computing tasks can be executed, but frequent switching between hardware accelerator and CPU occurs, increasing CPU load and reducing overall efficiency
Solution Approach 1:
The hardware accelerator performs tensor operations (stride, reshape, transpose) independently without requiring frequent CPU intervention. The device collects data from first memory, performs all necessary tensor transformations, and writes results to second memory autonomously, enabling the system to serve itself and reduce CPU burden.
Solution Approach 2:
The patent divides the computing task into distinct functional units: data collection unit, tensor operation units (stride, reshape, transpose), and data writing unit. This segmentation allows each unit to operate independently and efficiently, improving overall productivity while reducing the need for centralized CPU control.
2Speed
If tensor operations are performed with frequent CPU-hardware accelerator switching, then flexibility is maintained, but computing speed and overall performance deteriorate
Solution Approach 1:
The hardware accelerator is pre-configured with multiple tensor operation units (stride operation unit, reshape operation unit, transpose operation unit) that can execute operations without real-time CPU intervention. This preliminary preparation enables high-speed execution while maintaining operational flexibility through programmable control interfaces.
Data Source
AI summary
A hardware accelerator is provided. The hardware accelerator includes a first memory; a source address generation unit coupled to the first memory; a data collection unit coupled to the first memory; a first data queue coupled to the data collection unit; a data dispersion unit coupled to the first data queue; a destination address generation unit coupled to the data dispersion unit; an address queue coupled to the destination address generation unit; a second data queue coupled to the data dispersion unit; and a second memory coupled to the second data queue. The hardware accelerator can perform anyone or any combination of tensor stride, tensor reshape and tensor transpose to achieve tensorflow depth-to-space permutation or tensorflow space-to-depth permutation.