Sparse CNN Feature Map Index Processing Circuitry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolutional neural networks (CNNs) face challenges in reducing memory access and simplifying index manipulation and arithmetic operations, particularly due to the complexity of handling sparse tensors.
Innovation Solution
The proposed system includes index processing circuitry that generates data path processor commands from data value indexes and weight indexes, and data path circuitry with a scheduler, multiplication circuit, accumulator, and register bank to efficiently process convolution operations in CNNs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional CNN processing is used, then comprehensive data processing is achieved, but memory access amount increases and operation complexity increases
Solution Approach 1:
The patent segments the feature map into multiple blocks and processes them in parallel using multiple accumulators. Each block is handled by dedicated circuitry, reducing the total memory access required compared to sequential processing of the entire feature map. This segmentation allows the system to process sparse data more efficiently by only accessing non-zero elements within each block.
Solution Approach 2:
The patent introduces a new dimensional organization of data by arranging feature map blocks and accumulators in a multi-dimensional array structure. This dimensional reorganization allows for more efficient memory access patterns and enables parallel processing across multiple dimensions, reducing the overall memory access burden while maintaining comprehensive data processing.
2Ease of operation
If traditional index manipulation is used, then data positioning is achieved, but operation complexity increases
Solution Approach 1:
The patent introduces an intermediary indexing mechanism that translates complex memory addresses into simplified block indices. Instead of directly manipulating detailed memory addresses for each element, the system uses block-level indices as intermediaries, which reduces the complexity of index manipulation operations while still enabling precise data positioning within the segmented feature map blocks.
3Productivity
If sparse tensor processing is implemented, then computational efficiency is improved, but device complexity increases
Solution Approach 1:
The patent merges multiple processing functions into unified circuitry blocks. Each block contains integrated elements for data retrieval, multiplication, and accumulation operations, reducing overall device complexity compared to having separate dedicated circuits for each function. This merging approach maintains computational efficiency for sparse tensors while simplifying the hardware architecture through functional integration.
Data Source
AI summary
A convolutional neural network (CNN) system is provided that includes a flexible accelerator configured to convert an input feature map into a set of input sub-feature maps, each having a similar amount of sparsity. The system allows each of the sub-feature maps to be processed independently while taking advantage of the sparsity. In some aspects, the CNN system is configured with an index processor that receives data value indexes and weight indexes and generates data path processor commands for processing by a separate data path processor. In other aspects, unroll circuitry is configured to unroll feature maps to provide index-value compression. The unroll/compression scheme allows an input feature map to be read sequentially (tile-by-tile) so that an accumulate buffer can be implemented with a single read-only path and single write-only path. This can simplify memory control design, eliminating requirements for expensive cache-like structures while also reducing power.