Binarized Convolution Layer for Fast Neural Network Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks, particularly convolutional neural networks, experience slow performance due to time-consuming analysis processes, leading to user frustration.
Innovation Solution
Binarization of convolutional neural network filters and input data, allowing for faster computations using binary operations and optional use of look-up tables to pre-compute results, thereby increasing the speed of analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If floating-point operations are used in convolutional neural networks, then calculation accuracy is maintained, but processing speed becomes slow
Solution Approach 1:
The patent transforms the data representation format from floating-point to binary values, fundamentally changing the parameter type. This allows the system to use bitwise operations instead of floating-point arithmetic, achieving speedup while maintaining acceptable accuracy through careful binarization strategies and lookup table implementations.
Solution Approach 2:
The patent creates lookup tables that pre-compute and store results of complex floating-point operations. During inference, these pre-computed binary results are copied and returned directly, avoiding repeated floating-point calculations while maintaining accuracy for common operation patterns.
2Loss of time
If standard convolution operations are performed, then accurate results are obtained, but computation time increases significantly
Solution Approach 1:
The patent performs preliminary binarization of input data and pre-computes convolution results using lookup tables before actual inference. By preparing binary representations and pre-calculating common convolution patterns in advance, the system reduces real-time computation time while maintaining result reliability.
Solution Approach 2:
The patent introduces binary representations and lookup tables as intermediary structures between the input data and final results. These intermediaries enable faster binary arithmetic operations while preserving the mathematical relationships needed for accurate convolution results.
3Productivity
If binary values are used for filters and input data, then processing speed increases, but computational complexity changes
Solution Approach 1:
The patent replaces traditional floating-point arithmetic mechanisms with binary bitwise operations. This substitution uses simpler logical operations (AND, OR, XOR, NOT) instead of complex floating-point multiplication and addition, reducing the mechanical complexity of each computational step while increasing overall productivity.
Data Source
AI summary
A convolutional neural network is trained to analyze input data in various different manners. The convolutional neural network includes multiple layers, one of which is a convolution layer that performs a convolution, for each of one or more filters in the convolution layer, of the filter over the input data. The convolution includes generation of an inner product based on the filter and the input data. Both the filter of the convolution layer and the input data are binarized, allowing the inner product to be computed using particular operations that are typically faster than multiplication of floating point values. The possible results for the convolution layer can optionally be pre-computed and stored in a look-up table. Thus, during operation of the convolutional neural network, rather than performing the convolution on the input data, the pre-computed result can be obtained from the look-up table.