Neural Data Compression With DCT Subbands and Multi-Pass Bitstreams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression technologies face challenges in efficiently compressing and decompressing data while maintaining quality, especially in large-scale data centers and mobile devices, where storage and bandwidth are limited.
Innovation Solution
The use of a multi-pass neural network system for data compression and decompression, which includes preprocessing, discrete cosine transform (DCT), latent feature encoding, and arithmetic coding, to generate efficient compressed bitstreams.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional data compression techniques are used, then storage space is reduced, but compression ratio and quality maintenance are insufficient
Solution Approach 1:
The patent replaces traditional mechanical/mathematical compression algorithms with a neural network-based system. The neural network learns optimal compression representations through training, substituting conventional deterministic algorithms with a learned probabilistic model that achieves superior compression ratios while maintaining quality.
Solution Approach 2:
The system changes the parameter representation by transforming data into a latent space encoding. Instead of using traditional compression parameters, the neural network learns optimal latent representations that capture essential data characteristics, enabling better compression quality trade-offs through learned parameter transformations.
2Quantity of substance
If data is compressed to reduce file size, then storage and bandwidth requirements are reduced, but transmission speed and quality may be compromised
Solution Approach 1:
The system performs preliminary training of the neural network compression model offline before actual data compression. This preliminary action prepares the model to quickly compress data in real-time applications, separating the computationally intensive learning phase from the efficient inference phase during actual compression operations.
3Quantity of substance
If complex compression algorithms are used to improve compression ratio, then data size is reduced more effectively, but computational complexity and processing time increase
Solution Approach 1:
The compression system is segmented into distinct components: preprocessing module, neural network compression module, and post-processing module. This segmentation allows each component to be optimized independently and enables parallel processing of different data streams, reducing overall computational complexity while maintaining high compression ratios.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables efficient data compression by identifying patterns in data, reducing storage and bandwidth requirements, and improving data transmission speeds, making it suitable for various computer applications and emerging technologies.
Implementation Method 1
a discrete cosine transform (DCT) subsystem comprising a second plurality of programming instructions that, when operating on the processor, cause the computing device to: perform a DCT operation on the uncompressed dataset, and create a plurality of subbands for the uncompressed dataset, wherein each subband represents a frequency range
Data Source
AI summary
For compressing data, preprocessing operations are performed on raw input data. A discrete cosine transform is performed on the preprocessed data, and multiple subbands are created, where each subband represents a particular range of frequencies. The subbands are organized into multiple groups, where the multiple groups comprise a first low frequency group, a second low frequency group, and a high frequency group. A latent space representation is generated corresponding to each of the multiple groups of subbands. A first bitstream is created based on the latent space representation, and an alternate representation of the latent space is used for creating a second bitstream, enabling multiple-pass techniques for data compression. The multiple bitstreams may be multiplexed to form a combined bitstream for storage and/or transmission purposes.