CNN Feature Map Compression via Automatic Lossless Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face significant challenges in terms of memory efficiency and power consumption due to the large amounts of data processed, particularly in handling weight matrices and feature maps, which require substantial memory access and bandwidth, leading to increased power usage.
Innovation Solution
Implementing automatic and lossless data compression for feature map memory operations in a CNN processing system, where feature map writes are automatically compressed and reads are automatically decompressed, preserving memory bus bandwidth and reducing power requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If CNN data is processed using standard memory operations, then processing accuracy is maintained, but power consumption increases due to large amounts of data transfer
Solution Approach 1:
The patent applies parameter changes by transforming the data representation format from standard floating-point to a custom compressed format that encodes multiple values per word. This changes the fundamental parameters of data storage and transmission, enabling reduced bandwidth usage while maintaining the ability to reconstruct original values for accurate processing.
Solution Approach 2:
The patent implements segmentation by dividing feature map data into blocks that can be independently compressed and processed. Each block is handled separately through the compression pipeline, allowing parallel processing and reducing the overall data transfer burden while maintaining processing accuracy through systematic reconstruction of segmented data.
2Quantity of substance
If feature map data is stored in main memory, then large data volumes are accommodated, but memory bandwidth consumption increases leading to higher power usage
Solution Approach 1:
The patent changes the data representation parameters by encoding multiple feature map values into single memory words using a compressed format. This parameter transformation reduces the total number of memory words that must be transferred, directly addressing the bandwidth consumption issue while preserving the ability to represent the full data volume when decompressed.
Solution Approach 2:
The patent introduces a new dimension of data organization by using index-based referencing and block-based compression. Instead of storing each value individually in linear memory, the system organizes data in compressed blocks with index references, effectively adding a structural dimension that reduces the memory footprint and transfer requirements.
3Productivity
If data compression is applied to CNN feature maps, then memory bandwidth is reduced and power consumption decreases, but processing complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing compression parameters, block boundaries, and reconstruction information during the forward pass. This preliminary organization of compressed data structures enables efficient decompression during the backward pass without requiring complex real-time computations, thus reducing processing complexity despite the compression overhead.
Solution Approach 2:
The patent introduces intermediary data structures including index arrays, block metadata, and reconstruction buffers that mediate between the compressed storage format and the processing requirements. These intermediaries simplify the complexity by providing a structured translation layer that handles the conversion between compressed and uncompressed formats in a systematic manner.
Data Source
AI summary
Embodiments described herein provide a processing apparatus comprising compute logic to generate output feature map data for a convolutional neural network (CNN) and write the feature map data to a memory buffer; a direct memory access (DMA) controller including a feature map encoder, the DMA controller to read the feature map data from the memory buffer, encode the feature map data using one of multiple encode algorithms, and write encoded feature map data to memory coupled with the processing apparatus; and wherein the compute logic is to read the encoded feature map data from the memory in an encoded format and decode the encoded feature map data while reading the encoded feature map data.