Neural Entropy Compression with Parallel Data Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression technologies are inefficient due to prolonged processing times, leading to increased computing costs and affected other processing operations, especially in the context of growing data volumes from IoT and hybrid clouds.
Innovation Solution
A data compression system that divides data into partial units and processes them in parallel using a neural network-based probability calculation unit and entropy coding unit, improving processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If compression processing is performed on data, then data volume is reduced, but processing time increases
Solution Approach 1:
The compression target data is divided into multiple pieces of partial data, and multiple compression processing units process different partial data in parallel. This segmentation approach maintains data volume reduction while significantly reducing overall processing time by distributing the computational load across multiple units.
Solution Approach 2:
The patent introduces parallel processing as an additional dimension of computation. By organizing multiple compression processing units to operate simultaneously on different data segments, the system transforms sequential processing into parallel processing, thereby reducing processing time while maintaining effective data compression.
2Quantity of substance
If compression processing is performed on data, then data volume is reduced, but computing resource cost increases
Solution Approach 1:
By segmenting data into partial pieces and processing them in parallel, the system optimizes computing resource utilization. Each compression processing unit handles a specific segment independently, reducing overall computing resource cost through efficient parallel processing while achieving data volume reduction.
3Quantity of substance
If compression processing is performed on data, then data volume is reduced, but other processing is affected
Solution Approach 1:
The patent segments compression processing into independent parallel units, allowing compression operations to proceed without blocking other processing tasks. This segmentation enables other processing to continue concurrently, maintaining overall system productivity while achieving data volume reduction.
Solution Approach 2:
By implementing parallel compression processing units that operate continuously and independently, the system ensures that compression processing does not interrupt or affect other processing operations. The continuous parallel execution of multiple units maintains system productivity while achieving effective data compression.
Data Source
AI summary
In a computer that compresses data, a compressor that divides the compression target data into a plurality of pieces of partial data, and performs compression processing on each piece of the partial data in parallel, the compressor including a probability calculation unit that includes a neural network and calculates an appearance probability for each predetermined data unit of the partial data, and an entropy coding unit that outputs a coded bit string that is an entropy-coded bit string based on the data unit and the appearance probability for each data unit.