Convolution Engine Look-Up Tables for Address Calculation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolution operations in data flow architectures face challenges in generating memory addresses efficiently, particularly for high-dimensional matrices and convolutions with fractional strides, due to the computational complexity of calculating addresses for kernel, input, and output elements.
Innovation Solution
The implementation of a convolution calculation engine with specialized hardware circuits in memory units to generate addresses, utilizing kernel element counters, accumulator counters, and look-up tables to optimize address calculations, allowing for efficient convolution operations across various dimensions and hyperparameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional address calculation methods are used for convolution operations, then the implementation is simple, but the computational complexity increases and throughput decreases for high-dimensional matrices and fractional strides
Solution Approach 1:
The patent pre-calculates and stores address offsets in look-up tables (LUTs) during system initialization or configuration phase. These pre-computed offsets account for kernel sizes, strides, and padding parameters. During actual convolution operations, the address generation unit simply retrieves these pre-computed offsets from LUTs and combines them with current position indices, avoiding complex real-time calculations and enabling high-speed address generation for both integer and fractional strides.
Solution Approach 2:
The patent replaces complex mechanical/divmod-based address calculation mechanisms with a streamlined system using look-up tables and simple arithmetic operations. Instead of performing expensive integer division and modulo operations at runtime to compute addresses for high-dimensional tensors and fractional strides, the system substitutes these mechanical calculations with memory lookups and basic addition operations, dramatically reducing computational complexity and increasing throughput.
2Measurement precision
If divmod function is used to generate addresses for multidimensional matrices, then the address calculation is accurate, but the computational cost increases significantly
Solution Approach 1:
The patent pre-calculates address offsets for all possible kernel positions and dimensions during system configuration, storing these values in look-up tables. During convolution operations, the system retrieves these pre-computed addresses or address components from LUTs using simple indexing operations, eliminating the need for time-consuming divmod calculations while maintaining precise address generation for elements in high-dimensional input tensors, kernels, and output feature maps.
Solution Approach 2:
The patent creates pre-computed copies of address offset patterns in look-up tables that can be rapidly retrieved and reused. Instead of recalculating addresses using divmod operations for each kernel position, the system copies relevant address components from the LUTs based on current convolution parameters, significantly reducing calculation time while preserving address accuracy for multidimensional matrix operations.
3Productivity
If high bandwidth data provision is implemented to sustain high compute throughput, then the compute unit utilization improves, but the address generation complexity increases
Solution Approach 1:
The patent pre-generates and stores address sequences and offsets in look-up tables during system configuration, enabling the address generation unit to rapidly produce high-bandwidth address streams during convolution operations. By having address patterns pre-computed and stored, the system can sustain high compute throughput and maximize compute unit utilization without requiring complex real-time address calculation hardware, as the LUTs provide ready-to-use address components for all possible kernel positions and dimensions.
Data Source
AI summary
A convolution calculation engine includes a kernel element counter for a convolution operation between a kernel and an input tensor. The kernel element counter wraps back to an initial kernel count value after reaching a maximum kernel count value. The convolution calculation engine also includes an offset look-up table (LUT) that provides a relative input offset into the input tensor based on an output of the kernel element counter and input location calculation logic that provides an input location within an input tensor for the convolution operation based on the relative input offset provided by the offset LUT.