3D Block Partitioning for Neural Network Model Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large neural networks require significant storage and computational resources, making them challenging to deploy on devices with limited processing power and memory, such as mobile devices, due to their large size and complexity.
Innovation Solution
The development of techniques for neural network model compression and decompression, including coding tree unit block partitioning, 3D pyramid structure-based coding, and unification-based coding, which allow for efficient encoding and decoding of neural network models, reducing their size and computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If neural network models are deployed on resource-constrained devices, then performance improvement is achieved, but storage and computational resource requirements become prohibitive
Solution Approach 1:
The patent applies segmentation by dividing the neural network model into multiple blocks, where each block contains a subset of weights and biases. This allows selective compression of different blocks using different techniques (such as quantization, pruning, or low-rank factorization) based on their importance and characteristics, thereby reducing overall model size while preserving critical functionality for device deployment
Solution Approach 2:
The patent employs parameter changes by transforming model parameters through quantization (reducing precision from 32-bit float to lower precision formats), sparsity induction (creating sparse weight matrices), and low-rank decomposition. These parameter transformations significantly reduce storage requirements while maintaining approximate performance on resource-constrained devices
2Reliability
If neural network models are deployed on resource-constrained devices, then performance improvement is achieved, but computational complexity becomes prohibitive
Solution Approach 1:
The patent segments the computational workload by dividing the neural network into multiple blocks that can be processed independently or in parallel. This segmentation reduces the computational complexity of individual operations on resource-constrained devices while maintaining overall model performance through distributed computation across available hardware resources
Solution Approach 2:
The patent introduces dynamic computation strategies where the level of compression and computation varies based on device capabilities, input characteristics, and performance requirements. This allows the system to adaptively adjust computational complexity to match the constraints of resource-constrained devices while preserving performance when possible
3Quantity of substance
If model compression techniques are applied, then storage requirements are reduced, but model performance may deteriorate
Solution Approach 1:
The patent applies local quality by treating different blocks of the neural network differently based on their local characteristics and importance. Critical blocks that heavily influence performance are compressed less or with higher-fidelity techniques, while less critical blocks undergo more aggressive compression. This selective approach reduces overall model size while preserving performance in the most important regions of the network
Data Source
AI summary
An apparatus of neural network model decompression includes processing circuitry. The processing circuitry can be configured to receive, from a bitstream of a compressed neural network representation, one or more first syntax elements associated with a 3-dimensional coding unit (CU3D) partitioned from a 3-dimensional coding tree unit (CTU3D). The first CTU3D can be partitioned from a tensor in a neural network. The one or more first syntax elements can indicate that the CU3D is partitioned based on a 3D pyramid structure that includes multiple depths. Each depth corresponds to one or more nodes. Each node has a node value. Second syntax elements corresponding to the node values of the nodes in the 3D pyramid structure can be received from the bitstream in a breadth-first scan order for scanning the nodes in the 3D pyramid structure. Model parameters of the tensor can be reconstructed based on the received second syntax elements.