VPU Point Table Lookup Optimization via Segmented Memory Banks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional vector processing units (VPUs) face inefficiencies in detecting minimum and maximum values, operand sharing among data lanes, memory access, data manipulation, table lookups, and fault detection, leading to increased latency and processing complexity in applications like computer vision and deep learning.
Innovation Solution
The proposed solution includes a min/max hardware collector, automatic predication, SIMD architecture with lane communication, transposed load/store functionality, load with permute instruction, and a decoupled DMA system with a hardware sequencer to optimize data movement and processing, reducing latency and improving throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If table replication is performed for each value to enable parallel lookups, then lookup speed is improved, but memory usage increases
Solution Approach 1:
The patent divides the lookup table into multiple segments or banks, allowing parallel access to different segments simultaneously. This segmentation enables multiple lookups to occur in parallel without requiring complete table replication, thus improving lookup speed while controlling memory usage through efficient partitioning of the table data across segments.
2Productivity
If additional read ports are added to memory banks to allow multiple values to be read in a single cycle, then throughput is improved, but chip space requirements increase
Solution Approach 1:
The patent designs read ports with multi-functional capability, allowing a single read port to serve multiple purposes and handle different data access patterns. This universal design enables the read ports to be reused across different operations and time cycles, achieving high throughput without requiring additional dedicated read ports for each function, thus improving productivity while minimizing chip space consumption.
3Measurement precision
If data is written to memory for subsequent min/max analysis, then analysis accuracy is improved, but processing latency increases
Solution Approach 1:
The patent performs preliminary min/max analysis directly on data during the write operation or immediately upon arrival, before the data is fully committed to memory. This preliminary action allows the system to identify and flag out-of-range values early in the processing pipeline, maintaining analysis accuracy while significantly reducing the latency associated with waiting for complete memory writes and subsequent separate analysis passes.
4Ease of operation
If standard memory read operations are used with fixed width, then memory access simplicity is improved, but data access efficiency deteriorates when reading partial ranges
Solution Approach 1:
The patent implements dynamic memory read operations that can adaptively adjust the read width and range based on the specific data access requirements. This dynamic capability allows the system to read only the necessary portion of memory (e.g., elements 4 to 67) without being constrained by fixed-width read operations, thereby improving data access efficiency by eliminating unnecessary data transfers while maintaining operational simplicity through automated range detection and adjustment.
Data Source
AI summary
In various examples, a VPU and associated components may be optimized to improve VPU performance and throughput. For example, the VPU may include a min/max collector, automatic store predication functionality, a SIMD data path organization that allows for inter-lane sharing, a transposed load/store with stride parameter functionality, a load with permute and zero insertion functionality, hardware, logic, and memory layout functionality to allow for two point and two by two point lookups, and per memory bank load caching capabilities. In addition, decoupled accelerators may be used to offload VPU processing tasks to increase throughput and performance, and a hardware sequencer may be included in a DMA system to reduce programming complexity of the VPU and the DMA system. The DMA and VPU may execute a VPU configuration mode that allows the VPU and DMA to operate without a processing controller for performing dynamic region based data movement operations.