Neural Network Residual Coding for Intermediate Data Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network systems face high bandwidth and power consumption issues due to large memory access requirements for processing raw data across intermediate layers, leading to complex processing demands and increased power usage.
Innovation Solution
Implementing a neural network device or hardware component with a neural processing unit (NPU) that performs bandwidth compression using techniques such as lossless data compression, applying prediction schemes to intermediate data, and coding residual values using methods like variable-length coding (VLC) or arithmetic coding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large amounts of raw data are accessed for processing by intermediate nodes of the neural network, then processing accuracy is maintained, but bandwidth consumption and power usage increase significantly
Solution Approach 1:
The patent extracts and removes redundant information from neural network data by applying prediction schemes that identify and eliminate predictable patterns. Only the residual (unpredictable) information is retained and transmitted, significantly reducing bandwidth requirements while preserving essential processing information.
Solution Approach 2:
The patent transforms data representation parameters by converting full-precision neural network activations into compressed residual representations. This parameter transformation maintains computational accuracy for essential features while reducing data volume and associated power consumption.
2Reliability
If large amounts of raw data are accessed for processing by intermediate nodes of the neural network, then processing completeness is maintained, but memory access bandwidth requirements increase
Solution Approach 1:
The patent extracts and removes redundant information from neural network data by applying prediction schemes that identify and eliminate predictable patterns. Only the residual (unpredictable) information is retained and transmitted, significantly reducing bandwidth requirements while preserving essential processing information.
Solution Approach 2:
The patent performs prediction and compression operations preliminarily before data is stored or transmitted through the neural network pipeline. This preliminary compression reduces the volume of data that needs to be managed in memory, decreasing bandwidth requirements for subsequent access operations.
3Measurement precision
If 8 bit or 16 bit fixed or floating point operations are performed for intermediate layers, then numerical precision is maintained, but processing complexity and power consumption increase
Solution Approach 1:
The patent extracts and removes redundant information from neural network data by applying prediction schemes that identify and eliminate predictable patterns. Only the residual (unpredictable) information is retained and transmitted, significantly reducing bandwidth requirements while preserving essential processing information.
Solution Approach 2:
The patent transforms data representation parameters by converting full-precision neural network activations into compressed residual representations. This parameter transformation maintains computational accuracy for essential features while reducing data volume and associated power consumption.
Data Source
AI summary
Techniques and systems are provided for compressing data in a neural network. For example, output data can be obtained from a node of the neural network. Re-arranged output data having a re-arranged scanning pattern can be generated. The re-arranged output data can be generated by re-arranging the output data into the re-arranged scanning pattern. One or more residual values can be determined for the re-arranged output data by applying a prediction mode to the re-arranged output data. The one or more residual values can then be compressed using a coding mode.