Vector Processor Tile Array With DMA Streaming for Low-Latency Parallelism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vector processors suffer from memory management inefficiencies, leading to latency overhead and limited parallelizability due to frequent CPU communication and buffer loading/unloading of data, which reduces processing efficiency.
Innovation Solution
A computing system with a hardware accelerator featuring a vector processor tile array that utilizes direct memory access (DMA) and a first-in-first-out (FIFO) interface to stream vector elements and results, allowing parallel computation and reduced latency by minimizing CPU interaction and buffer usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is loaded into input buffers and unloaded from output buffers in conventional vector processors, then data can be processed, but memory management inefficiencies and latency overhead occur due to frequent CPU communication
Solution Approach 1:
The patent extracts the buffer management function from the CPU and assigns it to a dedicated buffer manager component within the vector processor. This separation allows the CPU to focus on computation while the buffer manager handles data loading and unloading, reducing communication overhead and latency.
Solution Approach 2:
The buffer manager acts as an intermediary between the CPU and the vector processor tiles, managing the input and output buffers. This intermediary component coordinates data flow and minimizes direct CPU involvement in buffer operations, thereby reducing latency overhead.
2Productivity
If frequent CPU communication is used for memory management, then data can be accessed, but parallelizability is limited due to communication overhead
Solution Approach 1:
The vector processor is segmented into multiple independent tiles that can operate in parallel. Each tile has its own computation units and can process different data elements simultaneously. The buffer manager coordinates these tiles, enabling parallel execution while minimizing communication overhead through efficient memory access patterns.
3Productivity
If buffer loading and unloading operations are performed, then data can be transferred, but processing efficiency is reduced due to memory management inefficiencies
Solution Approach 1:
The buffer manager is integrated within the vector processor, allowing it to autonomously manage buffer operations without requiring continuous CPU intervention. This self-service capability enables the buffer manager to efficiently handle data loading and unloading, reducing the time spent on memory management and improving overall processing efficiency.
Data Source
AI summary
A computing system including a hardware accelerator configured to receive vector processing instructions from a processor. The vector processing instructions include an initial read address, an input increment size, and a vector processing operation. During vector processing iterations performed at vector processor tiles included in a vector processor tile array, the hardware accelerator reads vector elements into respective vector processor tiles in an input stream. The vector elements are read into the vector processor tiles from locations in memory that start at the initial read address and advance by the input increment size at successive vector processing iterations. At each of the vector processor tiles, the hardware accelerator computes a vector processing result at least in part by performing the vector processing operation on the vector element read into the vector processor tile. The hardware accelerator outputs the vector processing results from the vector processor tiles in an output stream.