Neural Tensor-Train Decomposition for Accurate Lossy Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing tensor compression methods, such as tensor-train decomposition (TTD), suffer from low accuracy due to limited expressiveness and require structural assumptions that many real-world tensor data do not meet, leading to inefficient storage and reconstruction of large tensors.
Innovation Solution
A neural tensor-train decomposition (NTTD) model using a recurrent neural network is employed to improve expressiveness by allowing TT cores to depend on all previous mode indices, incorporating non-linear operations, and reordering mode indices to reduce the number of parameters and compression size, while maintaining low reconstruction error.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If tensor-train decomposition (TTD) is used for compression, then compression size is reduced, but reconstruction accuracy deteriorates due to limited expressiveness
Solution Approach 1:
The patent replaces the traditional mechanical/mathematical TTD system with a neural network-based system. The NTTD model uses neural networks to learn optimal decomposition patterns, substituting the rigid algebraic operations of conventional TTD with adaptive neural computations that can capture complex tensor structures more effectively, thereby improving reconstruction accuracy while maintaining compression efficiency
Solution Approach 2:
The patent changes the parameters of the decomposition system by introducing learnable parameters in the neural network. Instead of fixed decomposition rules, the NTTD model learns optimal parameters through training data, allowing it to adapt to different tensor types and structures. This parameter adaptation enables the system to maintain low reconstruction error across diverse real-world tensor data while preserving compression benefits
2Quantity of substance
If conventional TTD is applied to high-dimensional tensors, then storage cost is reduced, but reconstruction error increases due to structural assumptions
Solution Approach 1:
The patent introduces dynamics into the previously static TTD framework. The neural network in NTTD dynamically adjusts decomposition parameters based on the input tensor characteristics. This dynamic adaptation allows the system to handle high-dimensional tensors without relying on restrictive structural assumptions, maintaining both compression efficiency and reconstruction reliability across varying data structures
Solution Approach 2:
The NTTD model performs self-adjustment through the neural network's learning capability. The system automatically adapts to the specific structure of each tensor it processes, eliminating the need for manual structural assumptions. This self-service mechanism enables reliable reconstruction of high-dimensional tensors by learning their inherent patterns directly from the data
3Loss of information
If the number of tensor dimensions increases, then information capacity increases, but storage cost increases proportionally
Solution Approach 1:
The patent applies segmentation by decomposing high-dimensional tensors into multiple lower-dimensional components through the neural network. The NTTD model breaks down the complex high-dimensional structure into manageable segments that can be stored and processed more efficiently, preserving information capacity while reducing the storage cost proportional to the original tensor size
Data Source
AI summary
Disclosed is a lossy tensor compression method using neural tensor-train decomposition (NTTD). A lossy tensor compression method performed by a computer system may include inputting, to a neural tensor-train decomposition (NTTD) model, mode indices of a target entry to be reconstructed, and obtaining tensor-train (TT) cores from the mode indices of the target entry to be reconstructed through the NTTD model.