Programmable Multi-Level Data Access Address Generator for CNN Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face high computational complexity due to the need for simultaneous access and manipulation of large, multi-dimensional data arrays, which is inefficient and energy-intensive, especially in deep learning applications where high-dimensional convolutions and extensive data operations are required.
Innovation Solution
A memory address generator system that uses a combination of iteration variables, convolutional strides, and MUL-ADD units to efficiently generate addresses for accessing input, coefficient, and output data in memory, allowing for pre-fetching and instant calculation, thereby optimizing memory access and reducing computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional 'for' statements are used to generate iteration variables for multi-dimensional data access, then the code is simple and easy to write, but the computational complexity and data retrieval operations become excessively high
Solution Approach 1:
The patent replaces software-based iteration variable generation (mechanical programming approach) with hardware-based address generation units that automatically compute memory addresses using multiplication and addition operations. This substitution eliminates the need for complex nested for-loops while reducing computational complexity through dedicated hardware circuits that handle address calculation in parallel.
Solution Approach 2:
The patent segments the address generation process into separate functional units: iteration variable generators for each dimension, multiplication units for calculating stride contributions, and addition units for combining offset values. This segmentation allows each component to handle a specific part of the complex address calculation independently, reducing overall computational burden while maintaining ease of use.
2Productivity
If multi-dimensional data arrays are accessed using traditional iteration methods, then data retrieval operations are straightforward, but the amount of data retrieval and mathematical operations becomes excessively high
Solution Approach 1:
The patent implements preliminary action by pre-calculating and storing stride values and dimension sizes in lookup tables or registers before the main data retrieval process begins. The address generation unit uses these pre-stored values to compute memory addresses directly without performing complex calculations during data retrieval, thereby reducing the quantity of mathematical operations and improving data access efficiency.
Solution Approach 2:
The address generation unit is designed as a universal component that can handle multiple data types and array configurations through a single unified architecture. It universally computes addresses for different dimensions and data structures using the same multiplication and addition operations, eliminating the need for separate specialized routines for each data access pattern and reducing overall operational quantity.
3Reliability
If convolutional neural networks perform high-dimensional convolutions with multiple filters and channels, then accuracy and classification ability improve, but computational complexity and energy consumption increase
Solution Approach 1:
The patent addresses the dimensionality challenge by introducing a separate address generation dimension that operates independently from the data processing dimensions. Instead of nesting multiple loops for each dimension (increasing computational complexity), the system uses a dedicated address generation unit that computes all dimension combinations simultaneously through parallel multiplication and addition operations, thereby reducing energy consumption while maintaining accuracy.
Solution Approach 2:
The patent optimizes energy consumption by dynamically adjusting address generation parameters such as stride values and dimension sizes based on the specific convolutional operation requirements. The system can modify these parameters to match the actual data access patterns needed for high-dimensional convolutions, reducing unnecessary computations and energy waste while preserving classification accuracy through precise address calculation.
Data Source
AI summary
A programmable address generator has an iteration variable generator for generation of an ordered set of iteration variables, which are re-ordered by an iteration variable selection fabric, which delivers the re-ordered iteration variables to one or more address generators. A configurator receives an instruction containing fields which provide configuration constants to the address generator, iteration variable selection fabric, and address generators. After configuration, the address generators provide addresses coupled to a memory. In one example of the invention, the address generators generate an input address, a coefficient address, and an output address for performing convolutional neural network inferences.