Neural Network Quantization with Batch Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deepened and widened convolutional artificial neural networks (ANNs) face increased model size and operation time due to higher performance demands, necessitating methods to reduce model size and operation efficiency.
Innovation Solution
The proposed solution involves a neural network model driven by processors performing batch normalization, quantization, convolution operations, and activation functions, with basic blocks including batch normalization, quantization, convolution, and activation layers, and transition blocks incorporating pooling and channel upscaling, to reduce model size and enhance operation efficiency without using scale factors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If convolutional ANN is deepened and widened to be trained with more data, then performance is improved, but model size and operation time increase
Solution Approach 1:
The patent applies quantization to change the precision parameter of weight values from high precision (e.g., 32-bit floating point) to low precision (e.g., 8-bit integer or binary). This parameter change reduces the computational complexity and memory requirements while maintaining acceptable performance, thereby reducing operation time without sacrificing too much accuracy
Solution Approach 2:
The patent uses lightweight network architectures with reduced model size that can be quickly processed and discarded or retrained. By pruning unnecessary branches and using quantized weights, the system creates a more efficient model that requires less computational resources and time to operate
2Reliability
If convolutional ANN is deepened and widened to be trained with more data, then performance is improved, but model size increases
Solution Approach 1:
The patent changes the precision parameter of weight values from high precision to low precision through quantization. This reduces the memory required to store the model parameters significantly, thereby reducing model size while maintaining acceptable performance levels
Solution Approach 2:
The patent prunes unnecessary branches from the neural network architecture, extracting and removing redundant components. This reduces the overall model size by eliminating parts that do not contribute significantly to performance, creating a more compact and efficient network structure
Solution Approach 3:
The patent divides the neural network into modular components (basic blocks and transition blocks) that can be independently optimized and quantized. This segmentation allows for more efficient memory management and reduces the overall model size by enabling targeted optimization of different network portions
3Volume of stationary object
If quantization is applied to reduce model size, then model size decreases, but information loss increases
Solution Approach 1:
The patent applies batch normalization layers before quantization operations to stabilize the distribution of input data. This beforehand processing cushions against the information loss that would otherwise occur during quantization by ensuring that the quantized values more accurately represent the underlying data distribution
Solution Approach 2:
The patent incorporates residual connections that provide feedback paths allowing original information to be preserved and combined with quantized outputs. This feedback mechanism compensates for information loss by allowing the network to learn and correct quantization errors during training
4Productivity
If batch normalization and quantization are applied, then operation efficiency is improved, but device complexity increases
Solution Approach 1:
The patent merges batch normalization and quantization operations into unified computational blocks that can be executed more efficiently. By combining these operations and optimizing their implementation together, the patent reduces the overall computational overhead and simplifies the device requirements while maintaining operation efficiency
Data Source
AI summary
A device includes: one or more processors configured to perform a first operation for driving one or more basic blocks of a neural network model and a second operation for driving one or more transition blocks of the neural network model to drive the neural network model, wherein, for the performing of the first operation, the one or more processors are configured to: perform first batch normalization on input data; quantize the first batch normalized input data; perform a convolution operation based on the quantized input data; determine output data by applying an activation function to a result of the convolution operation; and perform the first operation by performing second batch normalization on the output data.