Invertible Flow Encoding with Multiply-Divide Compression Layers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional flow-based models for lossless data compression suffer from poor model representation capability and low compression efficiency due to limitations in operations, primarily relying on integer addition and subtraction, which restricts probability distribution estimation.
Innovation Solution
Employ an invertible flow-based model that incorporates multiplication and division operations, constrained by model parameters, to enhance the representation capability and improve compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If integer addition and subtraction operations are used in flow-based models, then invertibility is ensured, but model representation capability deteriorates
Solution Approach 1:
The patent introduces a parameter α (alpha) that controls the mixing ratio between original features and transformed features in the MRFM layer. By adjusting this parameter, the model can balance between maintaining invertibility and improving representation capability. The forward transform uses y = α ⊗ x + (1-α) ⊗ σ(Wx+b), where α allows flexible control over the transformation strength while preserving the ability to reconstruct original data.
Solution Approach 2:
The patent introduces auxiliary variables (s1, s2, s3, s4) as intermediaries that store intermediate computation results during the forward transform. These auxiliary variables enable the division operation to be inverted by storing the necessary information for reconstruction, thus maintaining invertibility while enabling more powerful multiplication and division operations.
2Measurement precision
If multiplication and division operations are introduced, then model probability distribution estimation improves, but device complexity increases
Solution Approach 1:
The patent divides the complex transform operation into multiple simpler stages: linear transformation (Wx+b), activation function (σ), and mixing operation (α ⊗ x + (1-α) ⊗ σ(Wx+b)). Each stage can be independently computed and stored, reducing the complexity of any single operation while achieving the overall effect of improved probability distribution estimation.
Solution Approach 2:
The auxiliary variables s1, s2, s3, s4 act as intermediaries that break down the complex multiplication and division operations into manageable steps. By storing intermediate results, the system can perform accurate probability distribution estimation without requiring all operations to be computed simultaneously, thus reducing peak computational complexity.
3Device complexity
If conventional flow-based models are used, then device complexity is reduced, but compression efficiency deteriorates
Solution Approach 1:
The patent merges the advantages of traditional normalizing flows (invertibility) with the representational power of neural network transformations. The MRFM layer combines linear transformations, non-linear activations, and parameter-controlled mixing in a single unified operation, achieving both structural elegance and high compression efficiency.
Solution Approach 2:
The model uses a composite transformation approach, combining different types of operations (linear, non-linear, element-wise multiplication) within the MRFM layer. This composite structure leverages the strengths of each operation type to achieve superior compression efficiency while maintaining a relatively compact model architecture.
Data Source
AI summary
This application relates to the field of artificial intelligence, and discloses a data encoding method, a data decoding method, and data processing apparatuses. Both the data encoding method and the data decoding method relate to an invertible flow-based model. The invertible flow-based model includes a target invertible flow layer, a model parameter of the target invertible flow layer is used to constrain an auxiliary variable generated in an inverse transform processing process, an operation corresponding to the target invertible flow layer includes a multiplication operation and a division operation that are determined based on the model parameter, and the auxiliary variable is an increment of a product of the multiplication operation or a remainder generated through the division operation.