Composite Binary Decomposition Network for Neural Model Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks face significant challenges due to large memory and computing power requirements, which hinder their deployment on low-end or embedded processors and limit the expansion of new applications into smaller and lower-cost computing devices.
Innovation Solution
The composite binary decomposition network (CBDNet) transforms pre-trained floating-point neural network models into multi-bit binary models through a training-free process, using binary operations to achieve efficient model compression and inference speedup by expanding and decomposing weight tensors into lower-rank binary matrices, thereby reducing parameter size and computational cost.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural networks use floating-point models to maintain high accuracy, then model accuracy is preserved, but model size and computing requirements increase significantly
Solution Approach 1:
The patent segments the weight tensor into multiple binary matrices through composite binary decomposition, where each binary matrix represents a bit plane of the original floating-point weights. This segmentation allows the model to process information in distributed binary form, achieving compression while preserving accuracy through the collective contribution of multiple binary components.
Solution Approach 2:
The patent employs composite binary decomposition that combines multiple binary matrices to reconstruct floating-point-like functionality. By compositing several binary matrices with different bit-plane weights, the system creates a composite representation that approximates continuous floating-point values, thereby maintaining model accuracy while using discrete binary storage.
2Ease of manufacture
If deep neural networks are compressed to reduce model size, then deployment on low-end devices becomes feasible, but inference accuracy may deteriorate
Solution Approach 1:
The patent transitions from single-matrix floating-point representation to multi-matrix binary representation, adding a dimensional aspect through bit-plane decomposition. Each binary matrix operates in a discrete binary dimension, but their weighted combination reconstructs the continuous value space, enabling accurate inference in a compressed multi-dimensional binary structure.
Solution Approach 2:
The patent performs preliminary composite binary decomposition on pre-trained floating-point models before deployment. By pre-processing the weight tensors into binary matrices through decomposition and quantization, the system prepares the model for efficient binary inference while preserving accuracy, eliminating the need for retraining on resource-constrained devices.
3Productivity
If binary neural networks are used to reduce computing requirements, then inference speed improves on binary hardware, but conversion from floating-point models becomes complex
Solution Approach 1:
The patent replaces complex floating-point arithmetic operations with simpler binary operations. By decomposing floating-point weight tensors into binary matrices, the system substitutes multi-precision floating-point multiplication and addition with binary matrix multiplication and element-wise operations, which can be efficiently executed on binary neural network hardware.
Solution Approach 2:
The patent changes the parameter representation from continuous floating-point values to discrete binary matrices. Through composite binary decomposition, the weight parameters are transformed from high-precision floating-point numbers to multi-bit binary representations, fundamentally altering the numerical domain while maintaining functional equivalence through the aggregation of binary components.
Data Source
AI summary
Embodiments are directed to a composite binary decomposition network. An embodiment of a computer-readable storage medium includes executable computer program instructions for transforming a pre-trained first neural network into a binary neural network by processing layers of the first neural network in a composite binary decomposition process, where the first neural network having floating point values representing weights of various layers of the first neural network. The composite binary decomposition process includes a composite operation to expand real matrices or tensors into a plurality of binary matrices or tensors, and a decompose operation to decompose one or more binary matrices or tensors of the plurality of binary matrices or tensors into multiple lower rank binary matrices or tensors.