Neural Network Data Compression for On-Chip Transfer Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network performance is hindered by the time-consuming data transfer between on-chip and off-chip storage due to limited bandwidth, where the transfer time exceeds calculation time.
Innovation Solution
A method that compresses data sub-blocks in neural networks by splicing them into continuous data streams, determining compression gains based on the length of the original and compressed data sub-blocks, and outputting the compressed data only if a gain is achieved, thereby improving data transfer efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred between on-chip and off-chip storage without compression, then data transfer bandwidth is充分利用, but data transfer time exceeds calculation time, resulting in poor neural network performance
Solution Approach 1:
The patent extracts only the essential data elements by compressing data sub-blocks into continuous data streams, removing redundant information while preserving necessary data for neural network operations. This reduces the volume of data transferred between on-chip and off-chip storage, directly addressing the time loss issue.
Solution Approach 2:
The patent changes the physical state of data by applying compression algorithms that transform raw data sub-blocks into compressed continuous streams. This parameter change in data representation reduces the amount of data that needs to be transferred, thereby reducing transfer time while maintaining data integrity for neural network calculations.
2Productivity
If data compression is applied to all data sub-blocks, then data transfer efficiency is improved, but unnecessary compression operations increase processing overhead when compression gain is minimal
Solution Approach 1:
The patent applies compression selectively rather than universally - it compresses data sub-blocks only when compression gain is determined to be positive. This partial action approach avoids the overhead of compressing data that would not benefit from compression, while still achieving improved transfer efficiency for suitable data blocks.
Solution Approach 2:
The patent implements a feedback mechanism by calculating compression gain before deciding whether to compress a data sub-block. This feedback loop evaluates the potential benefit of compression and uses that information to make an informed decision, preventing unnecessary compression operations and optimizing the balance between transfer efficiency and processing overhead.
3Loss of time
If continuous data streams are compressed, then data volume is reduced for faster transfer, but determining whether compression is beneficial requires additional calculations comparing original and compressed lengths
Solution Approach 1:
The patent performs preliminary compression gain calculations before committing to compression operations. By预先 calculating the compression gain based on the comparison between original and compressed data lengths, the system determines whether compression will actually reduce transfer time, avoiding wasted effort on unsuccessful compression attempts.
Solution Approach 2:
The compression system serves itself by automatically evaluating its own effectiveness through compression gain calculations. The system uses its own output (compressed data length) to determine whether to continue with compression, creating a self-regulating mechanism that optimizes the balance between transfer time reduction and computational overhead.
Data Source
AI summary
A data output method, a data acquisition method, a device, and an electronic apparatus are provided, and a specific technical solution is: reading a first data sub-block, and splicing the first data sub-block into a continuous data stream, wherein the first data sub-block is a data sub-block in transferred data in a neural network; compressing the continuous data stream to acquire a second data sub-block; determining, according to a length of the first data sub-block and a length of the second data sub-block, whether there is a gain in compression of the continuous data stream; outputting the second data sub-block if there is the gain in the compression of the continuous data stream.