Fixed-Point Neural Network Arithmetic Device Bit Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional arithmetic processing devices for multi-layer convolutional neural networks face challenges in achieving high-speed operation and reducing production costs due to the need for large memory capacity when using fixed-point number processing, as they often require different bit lengths for various layers, limiting parallel processing freedom and increasing production costs.
Innovation Solution
The proposed arithmetic processing device performs a trial recognition process to determine the maximum absolute values of intermediate results across layers, allowing for the adjustment of weight and bias values by multiplying them with a positive constant, enabling fixed-point number processing with a consistent number of bits across all layers without compromising accuracy, thereby reducing memory requirements and enhancing parallel processing capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If fixed-point number processing is performed with high precision to maintain accuracy in layers with large intermediate values, then recognition accuracy is improved, but memory capacity requirements increase extremely
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the number of bits allocated for fixed-point number processing in each layer based on the actual magnitude of intermediate values. Instead of using a uniform high precision across all layers, the system adapts the precision parameter to match the data characteristics of each specific layer, thereby reducing overall memory requirements while preserving necessary accuracy.
Solution Approach 2:
The patent implements local quality by allowing different bit allocations for different layers of the neural network. Each layer receives a customized precision level matched to its specific computational needs, rather than applying a uniform precision standard throughout the entire network. This localized optimization reduces total memory capacity requirements.
2Device complexity
If a uniform number of bits is used for fixed-point number processing across all layers, then processing complexity is reduced, but layers with small intermediate values waste memory capacity
Solution Approach 1:
The patent changes the precision parameter dynamically across layers based on intermediate value magnitudes. By calculating the actual bit requirements for each layer and allocating accordingly, the system avoids the memory waste of uniform high-precision allocation while maintaining manageable processing complexity through automated bit determination.
3Measurement precision
If more bits are allocated for fixed-point number processing, then recognition accuracy is maintained, but freedom of parallel processing decreases and processing time increases
Solution Approach 1:
The patent optimizes the precision parameter for each layer to match actual computational needs. By allocating the minimum necessary bits for each layer rather than using a uniform high bit allocation, the system enables greater parallel processing freedom and reduces processing time while maintaining sufficient recognition accuracy.
Data Source
AI summary
An arithmetic processing device to realize a multi-layer convolutional neural network circuit to perform a process with fixed-point number format, according to an embodiment comprising: a processing circuitry and a memory, the processing circuitry conducting: a learning process to perform weight learning or bias learning using learning data stored the memory to calculate initial weight values and initial bias values of the multi-layer convolutional neural network circuit; a trial recognition process to perform a recognition process to part of the learning data or of input data using the initial weight values and the initial bias values; a processing treatment process to multiply the initial weight values and the initial bias values by a positive constant to calculate processed weight values or processed bias values; and a recognition process to perform a recognition process using the processed weight values and the processed bias values.