Decoupled VPU Accelerators for Low-Latency SoC Task Offloading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional vector processing units (VPUs) face inefficiencies in handling dynamic range detection, memory access, data manipulation, and programming complexity, leading to increased latency and processing overhead, particularly in safety-critical applications like autonomous vehicles.
Innovation Solution
Incorporating a min/max hardware collector, automatic predication, and decoupled accelerators within the VPU architecture, along with optimized memory banks and a DMA system, to reduce latency and improve parallel processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all computed values are written to memory for dynamic range detection, then anomaly detection capability is improved, but processing latency increases due to additional memory access cycles
Solution Approach 1:
The patent extracts only the minimum and maximum values from the computed data stream using dedicated hardware collectors, rather than writing all values to memory. This selective extraction maintains anomaly detection capability while eliminating the need to store and process unnecessary data, thereby reducing memory access latency significantly.
Solution Approach 2:
The min/max hardware collectors operate in parallel with the VPU computation, performing range detection准备工作 before the main processing completes. This preliminary action allows the system to have anomaly detection results ready without adding post-processing delays.
2Productivity
If loop unrolling is used to achieve high throughput, then processing speed is improved, but code size and latency increase due to remainder loops
Solution Approach 1:
The VPU automatically handles iteration counting and remainder loop management through hardware-supported loop control mechanisms. This self-service capability eliminates the need for software-managed remainder loops, reducing both code size and latency while maintaining high throughput through efficient unrolled loop execution.
3Ease of operation
If memory reads use standard vector processing width for alignment, then memory access simplicity is improved, but read efficiency deteriorates when data range does not align with memory bank width
Solution Approach 1:
The patent segments the memory access operation into multiple aligned reads when the required data range exceeds a single memory bank width. Instead of forcing a non-aligned read that would require reading unnecessary data, the system performs multiple precise aligned reads to exact data ranges, improving memory utilization and reducing wasted bandwidth.
Solution Approach 2:
The system introduces an additional dimension of control by allowing variable-length memory reads within the vector processing framework. This enables memory accesses to be precisely matched to the actual data requirements rather than being constrained to fixed vector widths, improving read efficiency without sacrificing alignment benefits.
4Speed
If table lookup data is replicated for each value or additional read ports are added, then lookup speed is improved, but memory usage and chip space increase
Solution Approach 1:
The patent implements continuous table lookup operations where the VPU can sequentially access table entries without stalling. By maintaining continuous access to the same table structure through efficient pointer management and incremental addressing, the system achieves high lookup speeds without requiring replicated table copies for each processing lane.
Solution Approach 2:
A single table memory structure serves multiple processing lanes simultaneously through the VPU's ability to perform broadcast operations and multi-lane concurrent access. This universal table implementation allows all lanes to share the same table data, eliminating the need for replication while maintaining parallel lookup capability.
5Productivity
If VPU executes enhanced instructions for commonly occurring operations, then operation performance is improved, but processor availability decreases as the processor becomes busy during execution
Solution Approach 1:
The patent introduces decoupled lookup table accelerators that act as intermediaries between the VPU and memory systems. These accelerators handle table lookup operations independently, allowing the VPU to execute enhanced instructions at high performance while the accelerators simultaneously manage data retrieval, maintaining overall system availability and versatility.
Solution Approach 2:
The system segments processing functions by separating data retrieval operations (handled by decoupled accelerators) from computation operations (handled by VPU). This functional segmentation allows the VPU to focus on high-performance computation while accelerators handle data access, improving both operation performance and processor availability.
Data Source
AI summary
In various examples, a VPU and associated components may be optimized to improve VPU performance and throughput. For example, the VPU may include a min/max collector, automatic store predication functionality, a SIMD data path organization that allows for inter-lane sharing, a transposed load/store with stride parameter functionality, a load with permute and zero insertion functionality, hardware, logic, and memory layout functionality to allow for two point and two by two point lookups, and per memory bank load caching capabilities. In addition, decoupled accelerators may be used to offload VPU processing tasks to increase throughput and performance, and a hardware sequencer may be included in a DMA system to reduce programming complexity of the VPU and the DMA system. The DMA and VPU may execute a VPU configuration mode that allows the VPU and DMA to operate without a processing controller for performing dynamic region based data movement operations.