Neural Network Processor On-Chip Kernel Storage Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks face challenges with computational limits and data limits, particularly in managing large amounts of kernel data required for convolutional neural networks, which leads to inefficiencies in processing and power consumption.
Innovation Solution
The approach involves flattening kernel data into vectors to identify and exploit underlying patterns, enabling vector compression techniques that allow for on-chip storage of neural weights, using a codebook and arithmetic circuitry for efficient dot product calculations, and employing scalar and product quantization for further compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If kernel data is stored in external memory, then storage capacity is sufficient, but data transfer time and power consumption increase significantly
Solution Approach 1:
The patent embeds kernel data storage directly within the neural network processor chip, nesting the storage function inside the processing unit. This eliminates the need for external memory access during inference, reducing data transfer time while maintaining sufficient storage capacity for all kernel data needed for classification operations
Solution Approach 2:
The patent divides kernel data into multiple segments stored in separate on-chip memory banks, allowing parallel access to different kernel portions during processing. This segmentation enables efficient utilization of limited on-chip storage space while maintaining fast access to all required kernel data
2Quantity of substance
If kernel data is stored in external memory, then storage capacity is sufficient, but power consumption increases significantly
Solution Approach 1:
The patent embeds kernel data storage directly within the neural network processor chip, nesting the storage function inside the processing unit. This eliminates the need for continuous data transfer between external memory and processor, dramatically reducing power consumption while maintaining sufficient storage capacity for all kernel data needed for classification operations
Solution Approach 2:
The patent loads all required kernel data into on-chip memory before processing begins, performing the storage action in advance. This preliminary loading eliminates subsequent repeated access cycles to external memory, reducing both power consumption and transfer time during the actual classification operations
3Quantity of substance
If vector compression techniques are applied to kernel data, then on-chip storage becomes feasible, but data organization complexity increases
Solution Approach 1:
The patent applies vector quantization that transforms continuous kernel weight values into discrete codebook indices, fundamentally changing the data representation parameters. This compression technique reduces the data size to fit on-chip memory while the codebook structure provides an organized framework that manages the complexity of the transformed data
Data Source
AI summary
A neural network processor architecture provides decompression circuitry that can exploit patterns of data in kernel weights of a convolutional neural net as flattened to a vector, the compression allowing reduced kernel data storage costs including on-chip-storage.