CNN Input Binarization with Accuracy-Compliant Signal Lookup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural networks using binarized input signals experience accuracy degradation, particularly in the first layer of a multilayer neural network.
Innovation Solution
A neural network processing apparatus that includes a memory to store a table associating input values with binary signals, a binary signal acquisition unit to retrieve binary signals from this table, and a neural network operation unit to perform operations using these binary signals, ensuring accuracy compliance with predetermined neural network standards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If binarized input signals are used in the first layer of a multilayer neural network, then processing speed increases and memory usage decreases, but accuracy of the network degrades
Solution Approach 1:
The patent segments the input signal processing into two distinct parts: binary signal processing for computational operations and precision signal processing for accuracy-critical operations. The input layer processes signals in binary form for speed, while the first layer performs precision operations on the original high-precision input signals, separating the computational path from the accuracy-critical path.
Solution Approach 2:
The patent performs preliminary processing by preparing both binary signals and high-precision signals in advance. The input layer prepares binary versions of input signals for rapid processing, while simultaneously preserving the original high-precision signals for the first layer's accurate computations, ensuring both speed and accuracy requirements are met before the main processing begins.
2Measurement precision
If conventional arithmetic processing with multiple digits is performed, then accuracy is maintained, but processing speed decreases due to limited calculation resources
Solution Approach 1:
The patent applies different signal precision qualities to different layers based on their specific requirements. The input layer uses binary signals (low precision, high speed) for operations that tolerate quantization, while the first layer uses high-precision signals (high precision, lower speed) for operations where accuracy is critical, optimizing the trade-off locally for each layer's function.
Solution Approach 2:
The patent dynamically changes the precision parameter of input signals based on the operational requirements of different layers. Binary signals (1-bit precision) are used for the input layer and subsequent convolutional layers, while high-precision signals (multiple-bit precision) are used for the first layer, allowing the system to adapt signal parameters to match computational needs and resource constraints.
Data Source
AI summary
A CNN processing apparatus (1) includes an input buffer (10) configured to store an input signal given to a CNN, a storage unit (12) configured to store a table (120) in which the input signal given to the CNN and a binary signal corresponding to the value of the input signal are associated with each other, a binary signal acquisition unit (13) configured to acquire the binary signal corresponding to the value of the input signal stored in the input buffer (10) by referring to the table (120), and a convolutional operation unit (14) configured to perform an operation of the CNN based on the binary signal acquired by the binary signal acquisition unit (13). The binary signal has an accuracy complying with a predetermined operation accuracy of the CNN.