Binary Neural Network Inference Using Stochastic Rounding Ensembles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network training methods face challenges in balancing accuracy and efficiency, particularly when transitioning from high-precision weights to binary weights, which can lead to performance deterioration and increased memory and energy consumption.
Innovation Solution
The method involves generating two neural networks with discrete weights through stochastic rounding of high-precision weights and calculating their outputs, which are then added to achieve high-performance outputs with reduced calculation steps and energy consumption, utilizing a hardware circuit as a pseudo-random number generator for efficient implementation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-precision edge weights are used in neural networks, then classification accuracy is improved, but memory footprint and power consumption increase significantly
Solution Approach 1:
The patent creates multiple binary copies of the neural network (first neural network and second neural network) with discrete edge weights instead of storing one high-precision network. Each binary network requires only 1 bit per edge weight compared to 32 bits for floating-point, achieving 32x memory reduction while maintaining accuracy through ensemble computation of multiple binary copies
Solution Approach 2:
The patent transforms the precision parameter of edge weights from high-precision floating-point format to binary format (discrete values). This parameter change reduces memory requirements from 32 bits to 1 bit per weight, and the patent compensates for precision loss by using stochastic rounding and ensemble methods with multiple binary networks
2Measurement precision
If high-precision edge weights are used in neural networks, then classification accuracy is improved, but power consumption increases
Solution Approach 1:
The patent replaces one high-precision neural network computation with multiple binary neural network computations. Although multiple networks are used, each binary computation is significantly less energy-intensive due to simpler operations (additions instead of multiplications) and reduced data movement, resulting in overall lower power consumption while maintaining accuracy through ensemble averaging
Solution Approach 2:
The patent substitutes complex floating-point arithmetic operations with simpler binary arithmetic operations. The computation mechanism changes from high-precision multiplications and additions to binary additions and XOR operations, which require less computational energy and can be implemented more efficiently in hardware
3Productivity
If edge weights are projected onto binary weighting factors to reduce complexity, then computation speed is improved, but classification accuracy deteriorates
Solution Approach 1:
The patent segments the single binary network computation into multiple independent binary network computations (first neural network and second neural network). Each network is trained separately with stochastic rounding, and their outputs are combined through addition. This segmentation allows the system to benefit from binary computation speed while mitigating accuracy loss through ensemble diversity
Solution Approach 2:
The patent uses more binary networks (two or more) than the minimum single binary network, performing slightly excessive computation by combining multiple binary inferences. This excessive action compensates for the precision loss of individual binary networks, achieving accuracy comparable to or exceeding high-precision networks while maintaining binary computation efficiency
4Quantity of substance
If stochastic rounding is used to binarize edge weights, then memory requirements are reduced, but computational complexity increases
Solution Approach 1:
The patent creates multiple binary copies of the neural network through stochastic rounding, where each copy is a simplified binary version. The computational complexity of generating these copies is offset by the simplicity of binary operations during inference, and the memory savings from binary representation far exceed the training overhead
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
The invention relates to a method for calculating an output of a neuronal network (100), comprising the steps of generating a first neuronal network (100-1) having discrete edge weights (W b ) from a neuronal network (100) having precise edge weights (W prec ) by stochastic rounding; generating a second neuronal network (100-2) having discrete edge weights (W b ) from the neuronal network (100) having precise edge weights (W prec ) by stochastic rounding; and calculating an output by adding the output of the first neuronal network (100-1) and of the second neuronal network (100-2).