Stop-code tolerant image compression neural networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image compression techniques using convolutional neural networks face challenges in maintaining image quality at various compression rates, often resulting in blocking artifacts and blurry reconstructions due to the spatial dependence on omitted symbols and difficulties in stop-code trimming.
Innovation Solution
The implementation of a stop-code tolerant (SCT) approach with a two-pass training method for recurrent convolutional neural networks, which adaptively varies the number of symbols transmitted based on image content and uses masking logic to determine symbol assignments that satisfy stop-code structure and represent the input image accurately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If conventional image compression techniques using convolutional neural networks are used, then compression is achieved, but blocking artifacts and blurry reconstructions occur due to spatial dependence on omitted symbols
Solution Approach 1:
The image is divided into multiple tiles, and the encoding process is segmented into multiple iterations. In each iteration, only a subset of tiles is encoded with binary codes, while other tiles are masked off. This segmentation allows the network to focus computational resources on important regions and avoid the spatial dependence problems that cause blocking artifacts and blurriness in conventional approaches.
Solution Approach 2:
The encoder neural network is trained in advance using a two-pass training method. In the first pass, the network learns to generate binary codes for all tiles. In the second pass, the network learns to generate codes while masking off certain tiles, preparing it to handle the stop-code tolerant encoding process. This preliminary training action enables the network to adapt to the iterative encoding scheme and maintain image quality.
2Productivity
If the number of binary codes per tile is reduced to achieve lower bit-rates, then compression efficiency improves, but reconstruction quality deteriorates
Solution Approach 1:
The number of binary codes transmitted for each tile is made dynamic rather than fixed. The encoder adaptively determines how many codes to generate for each tile based on its importance and the current iteration number. This dynamic approach allows the system to achieve lower bit-rates by using fewer codes for less important tiles while maintaining reconstruction quality for important tiles, resolving the contradiction between compression efficiency and reconstruction quality.
3Quantity of substance
If stop codes are used to terminate encoding for certain tiles, then transmission data is reduced, but the network struggles to handle omitted symbols due to spatial dependence
Solution Approach 1:
The network is trained in advance using a two-pass training method that specifically prepares it to handle stop codes and omitted symbols. In the first pass, the network learns standard encoding. In the second pass, it learns encoding with masking, where certain tiles are deliberately omitted. This preliminary training action reduces the complexity of handling omitted symbols during actual encoding by pre-adapting the network's spatial dependencies.
Solution Approach 2:
The encoder neural network automatically adapts to the masking pattern and generates appropriate binary codes for the remaining tiles without requiring external intervention to handle the omitted symbols. The network self-adjusts its internal representations to account for the missing tiles, reducing the overall system complexity despite the presence of stop codes.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for image compression and reconstruction. A request to generate an encoded representation of an input image is received. The encoded representation of the input image is then generated. The encoded representation includes a respective set of binary codes at each iteration. Generating the set of binary codes for the iteration from an initial set of binary includes: for any tiles that have already been masked off during any previous iteration, masking off the tile. For any tiles that have not yet been masked off during any of the previous iterations, a determination is made as to whether a reconstruction error of the tile when reconstructed from binary codes at the previous iterations satisfies an error threshold. When the reconstruction quality satisfies the error threshold, the tile is masked off.