Neural Network Data Stream Compression for Transfer Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network performance is hindered by the time-consuming data transfer between on-chip and off-chip storage due to limited bandwidth, where the transfer time exceeds calculation time.
Innovation Solution
A method that reads and splices data sub-blocks into continuous streams, compresses them, and determines if there's a compression gain by calculating the number of beats required for transfer before and after compression, outputting the compressed data sub-blocks if a gain is achieved, thereby improving data transfer efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred between on-chip and off-chip storage without compression, then data transfer bandwidth is limited, but transfer time exceeds calculation time resulting in poor neural network performance
Solution Approach 1:
The patent extracts only the essential data elements by compressing data sub-blocks before transfer. By identifying and removing redundant information in the data stream, the system reduces the volume of data that needs to be transferred between on-chip and off-chip storage, directly addressing the bandwidth limitation and reducing transfer time.
Solution Approach 2:
The patent changes the parameter of data representation by applying compression algorithms to transform the data from its original form to a compressed form. This parameter change reduces the data size while preserving the essential information needed for neural network calculations, thereby improving transfer efficiency.
2Productivity
If compression is applied to all data sub-blocks, then data transfer efficiency may improve, but unnecessary compression overhead is introduced when compression gain is minimal
Solution Approach 1:
The patent applies partial compression by selectively compressing only those data sub-blocks that demonstrate sufficient compression gain. Rather than compressing all data uniformly, the system evaluates each sub-block and applies compression only where beneficial, avoiding unnecessary overhead while still improving overall transfer efficiency.
Solution Approach 2:
The patent introduces a feedback mechanism that evaluates the compression gain of each data sub-block before deciding whether to compress it. By calculating the compression ratio and comparing it against a threshold, the system dynamically adjusts its compression strategy based on the actual characteristics of the data being transferred.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A data output method, a data acquisition method, a device, and an electronic apparatus are provided, which relate to a field of neural network technology, and a specific technical solution is: reading a first data sub-block, and splicing the first data sub-block into a continuous data stream, wherein the first data sub-block is a data sub-block in transferred data in a neural network; compressing the continuous data stream to acquire a second data sub-block; determining, according to a length of the first data sub-block and a length of the second data sub-block, whether there is a gain in compression of the continuous data stream; outputting the second data sub-block if there is the gain in the compression of the continuous data stream. This application achieves a technical effect of improving performance of a neural network.