Binary Neural Network Channel Computation via Minimum Spanning Tree
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep binary neural networks face challenges in computational complexity and resource efficiency, even with quantization methods like binary neural networks (BNNs), which struggle to balance depth, width, and accuracy while managing computational burdens.
Innovation Solution
A computation method for binary neural networks that involves generating a fully connected graph based on output channels of a convolutional layer, extracting a minimum spanning tree, and re-arranging computation orders to optimize processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If the binary neural network uses conventional computation methods for convolutional layers, then the network can be implemented with binary weights and activations, but the computational complexity remains high due to the need to process all output channels independently
Solution Approach 1:
The patent merges the computation of multiple output channels by identifying a reference output channel and computing other channels relative to it. Instead of independently computing each output channel, the method combines computations across channels by sharing common operations, thereby reducing overall computational complexity while maintaining accuracy.
Solution Approach 2:
The patent segments the set of output channels into a reference channel and multiple target channels. This segmentation allows the computation to be organized hierarchically, where the reference channel is computed first and then used as a basis for computing other channels, reducing redundant calculations and improving computational efficiency.
2Adaptability or versatility
If the binary neural network increases depth and width to meet practical requirements, then the network capacity increases, but the computational burden increases even further
Solution Approach 1:
The patent applies merging principles to reduce computational burden in deeper and wider networks by combining redundant operations across output channels. By computing relative differences between channels rather than absolute values for each channel, the method significantly reduces the number of operations required, enabling larger networks to run with lower energy consumption.
3Ease of manufacture
If the binary neural network processes output channels in conventional order, then the implementation is simple, but the computational cost is high due to redundant calculations between channels
Solution Approach 1:
The patent performs preliminary computation of a reference output channel before computing other channels. By pre-computing the reference channel and storing its results, subsequent channel computations can reuse these results, avoiding redundant calculations. This preliminary action reduces computational cost while adding minimal implementation complexity.
Data Source
AI summary
A computation method for the binary neural network is performed on a computing device that includes one or more processors and a memory storing one or more programs executed by the one or more processors, and includes generating a fully connected graph based on output channels of a convolutional layer of a binary neural network, extracting a minimum spanning tree from the fully connected graph, and re-arranging an order of computations between respective output channels based on the minimum spanning tree.