DNN Tensor Decomposition Decoding for Shape Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current compression methods for Deep Neural Networks (DNNs) face challenges in efficiently compressing and decompressing large tensors, leading to high inference complexity and resource requirements, particularly in environments with limited hardware and software resources, and lack a solution for reconstructing original tensor shapes from decomposed tensors during decoding.
Innovation Solution
The method involves tensor decomposition into smaller tensors using Low Rank (LR) or Low Displacement Rank (LDR) techniques, followed by quantization and entropy coding, with a proposed syntax and mechanism for reconstructing the original tensor shape using a Decoded Tensor Buffer (DTB) to store and reuse decoded tensors for accurate reconstruction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If tensor decomposition is applied to compress DNN parameters, then compression efficiency is improved, but device complexity increases due to the need for reconstruction mechanisms
Solution Approach 1:
The patent applies segmentation by decomposing large tensors into multiple smaller tensors (e.g., factor matrices in low-rank decomposition). This segmentation enables efficient compression by encoding smaller tensors separately, reducing the overall bandwidth requirements while maintaining a systematic reconstruction approach through the Decoded Tensor Buffer that manages the segmented components.
Solution Approach 2:
The Decoded Tensor Buffer (DTB) acts as an intermediary mechanism that stores decoded tensors and facilitates their reconstruction. This intermediary structure manages the complexity of reconstruction by providing a organized storage and retrieval system for tensor components, enabling accurate reassembly without requiring complex real-time processing.
2Quantity of substance
If tensors are decomposed into smaller tensors for encoding, then bandwidth requirements are reduced, but reconstruction accuracy may be compromised
Solution Approach 1:
The patent employs preliminary action by pre-allocating and organizing the Decoded Tensor Buffer before reconstruction occurs. The buffer is prepared to store specific tensor components with appropriate sizing and structure, ensuring that when decomposition is reversed, the components can be accurately reassembled. This preliminary preparation maintains reconstruction accuracy by preventing data loss or misalignment that could occur without proper buffer management.
Solution Approach 2:
The patent utilizes parameter changes by dynamically adjusting the buffer size and configuration based on the specific tensor being reconstructed. The buffer adapts its parameters (size, allocation) to match the requirements of the decoded tensors, ensuring optimal storage and retrieval conditions that preserve reconstruction accuracy while accommodating varying bandwidth constraints.
3Productivity
If separate encoding of decomposed tensors is implemented, then compression efficiency is enhanced, but encoding complexity increases
Solution Approach 1:
The encoding process is segmented into separate operations for each decomposed tensor component. Instead of encoding a single large tensor, the system encodes smaller tensors (such as factor matrices) independently through the compression pipeline. This segmentation improves compression efficiency by allowing optimized processing of smaller data units while the overall complexity is managed through systematic buffer management for reassembly.
Data Source
AI summary
The disclosure relates to a method comprising, responsive to a determination that at least one first tensor of at least one layer of at least one Deep Neural Network is decomposed into a second tensor and a third tensor whose parameters are encoded in a bitstream, decoding from the bitstream a size of at least one of the second tensor and the third tensor, and decoding the at least one of the second tensor and the third tensor from the bitstream based on the decoded size. Corresponding apparatus, encoding method, signal; bitstream, storage media and encoder and/or decoder devices are also provided.