Hardware Sequencer DMA for VPU Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional vector processing units (VPUs) face inefficiencies in detecting and analyzing minimum and maximum values, leading to increased processing cycles and latency due to the need for additional memory operations and software pipelining. Additionally, existing SIMD architectures restrict operand sharing among data lanes, leading to inefficient data manipulation and memory access patterns.
Innovation Solution
The proposed solution enhances VPUs with a min/max hardware collector that stores minimum and maximum values in the data path to memory, reducing latency. It also introduces automatic predication to optimize iteration execution, allows operand sharing among data lanes through a SIMD architecture with defined slices, and uses transposed load and store operations with a stride parameter to improve memory access efficiency. Furthermore, a decoupled lookup table accelerator and a hardware sequencer are used to optimize table lookups and DMA operations, respectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional VPUs write all values to memory for min/max detection, then completeness of data analysis is improved, but processing cycles and latency increase
Solution Approach 1:
The patent extracts only the necessary min/max values from the complete data stream using hardware collectors (min/max collectors) that operate in parallel with the VPU. This extraction occurs during the data movement phase without requiring complete data to be written to memory first, thereby reducing latency while maintaining detection accuracy.
Solution Approach 2:
The min/max collectors perform preliminary detection of minimum and maximum values during the data transfer process itself, before the data needs to be fully processed or stored. This preliminary action eliminates the need for separate post-processing steps to find min/max values, reducing overall processing time.
2Productivity
If SIMD units operate independently in separate data lanes, then processing parallelism is improved, but data sharing capability deteriorates
Solution Approach 1:
The patent segments the SIMD data lanes into adjacent groups (e.g., lanes 0-3, 4-7, 8-11, 12-15) where sharing is enabled within each segment. This segmentation allows independent operation across segments while enabling coordinated data sharing within segments, resolving the contradiction between parallelism and sharing capability.
Solution Approach 2:
The patent applies different operational characteristics to different parts of the SIMD unit: adjacent lanes within segments can share operands locally, while segments operate independently from each other. This local quality approach enables selective sharing where needed while maintaining overall parallel processing efficiency.
3Speed
If loop unrolling is used to achieve high throughput, then processing speed is improved, but code size and latency increase due to remainder loops
Solution Approach 1:
The patent implements dynamic loop unrolling where the VPU automatically determines the appropriate unrolling factor based on the iteration count. When the iteration count is not evenly divisible by the unrolling factor, the system dynamically adjusts the number of unrolled iterations, eliminating the need for separate remainder loops and reducing code complexity.
4Stability of the object's composition
If memory reads use standard unit width, then memory access consistency is improved, but data access efficiency deteriorates
Solution Approach 1:
The patent enables partial memory reads where the VPU can read only the specific number of elements needed from memory, rather than always reading full standard-width units. This partial action approach improves data access efficiency by avoiding unnecessary reads while maintaining memory access consistency through controlled read operations.
Data Source
AI summary
In various examples, a VPU and associated components may be optimized to improve VPU performance and throughput. For example, the VPU may include a min/max collector, automatic store predication functionality, a SIMD data path organization that allows for inter-lane sharing, a transposed load/store with stride parameter functionality, a load with permute and zero insertion functionality, hardware, logic, and memory layout functionality to allow for two point and two by two point lookups, and per memory bank load caching capabilities. In addition, decoupled accelerators may be used to offload VPU processing tasks to increase throughput and performance, and a hardware sequencer may be included in a DMA system to reduce programming complexity of the VPU and the DMA system. The DMA and VPU may execute a VPU configuration mode that allows the VPU and DMA to operate without a processing controller for performing dynamic region based data movement operations.