Autoencoder Compression with Batch Normalization and Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression techniques, particularly using neural networks, face challenges in achieving high compressibility with minimal distortion, especially for high-definition moving images and voice data, where existing methods struggle to optimize the trade-off between distortion and code length effectively.
Innovation Solution
The implementation of an autoencoder system that utilizes batch normalization and multivalued quantization, combined with arithmetic coding based on conditional independence assumptions, to optimize the compression process, allowing for efficient lossy and lossless data compression by separating global and local feature information and using pseudo quantization functions for improved backpropagation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data compression is performed using conventional neural network methods, then compression ratio is improved, but distortion increases
Solution Approach 1:
The patent segments the compression process into two distinct stages: lossy compression using an autoencoder to reduce data dimensionality, followed by lossless compression using arithmetic coding to eliminate redundancy. This segmentation allows each stage to optimize for its specific purpose, achieving high overall compression ratio while controlling distortion through the coordinated operation of both compression methods
Solution Approach 2:
The patent applies batch normalization to standardize the output distributions of neural network layers, and uses multivalued quantization to convert continuous values into discrete representations. These parameter changes enable more efficient encoding while preserving essential information, thereby improving compression ratio without excessive distortion
2Productivity
If compression ratio is increased to reduce data size, then data transfer efficiency is improved, but distortion in lossy compression increases
Solution Approach 1:
The patent transforms the output of the autoencoder through batch normalization to achieve standardized distributions, then applies multivalued quantization to create discrete symbolic representations. This parameter transformation enables the compressed data to be encoded more efficiently with fewer bits while preserving reconstructability, thus improving data transfer efficiency without excessive distortion
Solution Approach 2:
The patent introduces an intermediate representation layer that converts continuous neural network outputs into discrete multivalued symbols. This intermediary step acts as a bridge between the continuous feature space and the discrete compressed domain, enabling efficient lossless compression of the quantized symbols while maintaining the quality of the lossy compression stage
3Device complexity
If conventional compression methods are used, then implementation is simpler, but ability to handle high-definition data is insufficient
Solution Approach 1:
The patent divides the compression system into modular components: an autoencoder for lossy compression, a batch normalization layer for distribution standardization, a multivalued quantization layer for discretization, and an arithmetic coding module for lossless compression. This segmentation allows each component to be optimized independently for handling high-definition data while maintaining overall system manageability
Solution Approach 2:
The patent combines multiple compression techniques (neural network-based lossy compression, batch normalization, multivalued quantization, and arithmetic coding) into a composite compression system. This composite approach leverages the strengths of each method to effectively handle high-definition data with large bandwidth requirements, achieving reliable compression that conventional single-method approaches cannot provide
Data Source
AI summary
An autoencoder includes memory configured to store data including an encode network and a decode network, and processing circuitry coupled to the memory. The processing circuitry is configured to cause the encode network to convert inputted data to a plurality of values and output the plurality of values, batch-normalize values indicated by at least two or more layers of the encode network, out of the output plurality of values, the batch-normalized values having a predetermined average value and a predetermined variance value, quantize each of the batch-normalized values, and cause the decode network to decode each of the quantized values.