Fixed-Length Compressed Blocks for Parallel Decompression Sync
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression methods in hardware chips for machine learning algorithms face inefficiencies due to varying decompression rates of raw data blocks, leading to idle computation capacity and reduced algorithm speed, especially when I/O bandwidth is insufficient.
Innovation Solution
A method to generate standard compressed data blocks of fixed length, allowing multiple decompression engines to process synchronously, thereby improving decompression efficiency and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multi-engine parallel decompression is used to improve decompression rate, then decompression capacity increases, but decompression efficiency decreases due to varying decompression times of different compressed data blocks
Solution Approach 1:
The patent changes the parameter of compressed data block length from variable to fixed. By padding shorter compressed data blocks with dummy data to match the length of the longest block, all decompression engines process data blocks of identical length, ensuring synchronous completion and eliminating idle waiting time.
Solution Approach 2:
The patent segments the compressed data into multiple fixed-length data blocks, each assigned to a separate decompression engine. This segmentation allows parallel processing while maintaining synchronization through uniform block lengths, resolving the contradiction between parallel processing capacity and processing efficiency.
2Productivity
If variable-length compressed data blocks are processed in parallel, then I/O bandwidth utilization improves, but computation resources are wasted due to idle waiting
Solution Approach 1:
The patent standardizes the length parameter of compressed data blocks to a fixed value L2. This allows all decompression engines to operate at full capacity simultaneously without idle waiting, as each engine processes a data block of the same predetermined length, eliminating computation resource waste.
3Productivity
If standard fixed-length compressed data blocks are generated, then decompression synchronization improves, but compression process complexity increases
Solution Approach 1:
The patent applies preliminary padding action during the compression phase. By adding dummy data to shorter compressed blocks to reach the standard length L2, the system prepares all data blocks in advance for synchronous parallel processing, simplifying the decompression phase despite adding a step to compression.
Data Source
AI summary
In the data compression method, a raw data block in raw data is processed based on a compression algorithm to obtain a standard compressed data block that has a length of L2 and that corresponds to the raw data block, and the raw data is further compressed into one or more standard compressed data blocks that each have the length of L2 and that are to be decompressed in parallel by a decompression apparatus, where the decompression apparatus includes a plurality of decompression engines, and each decompression engine is capable of decompressing one standard compressed data block within one processing cycle. According to the data compression method, a standard compressed data block with a fixed length can be obtained through compression.