Neural Network Circuit Quantization for Embedded Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolutional neural networks (CNNs) face challenges in being implemented in embedded devices like IoT devices due to the difficulty in embedding large-scale dedicated circuits and the limited hardware resources such as CPU or memory, which hinder sufficient operational performance.
Innovation Solution
A neural network circuit is designed with a first memory unit for storing input data, a convolution operation circuit for performing convolution operations, a second memory unit for storing output data, and a quantization operation circuit for quantizing the output data. The circuit performs convolution operations on quantized data, reducing computational load and enabling efficient operation in resource-constrained devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If large-scale dedicated circuits are used for convolutional neural networks, then operational performance is improved, but device complexity and resource requirements increase
Solution Approach 1:
The patent applies quantization to change the precision parameter of neural network weights and activations from high precision (e.g., 32-bit floating point) to low precision (e.g., 8-bit integers or lower). This parameter change reduces the computational complexity and memory requirements while maintaining acceptable operational performance, enabling deployment on embedded devices with limited resources.
Solution Approach 2:
The patent segments the neural network computation into discrete quantization-aware operations that can be efficiently implemented on resource-constrained hardware. By breaking down the computational graph into quantizable operations and using separate memory units for different data types, the system reduces the complexity of the dedicated circuit while preserving functionality.
2Measurement precision
If high precision data is used in convolution operations, then computational accuracy is improved, but memory usage and computational load increase
Solution Approach 1:
The patent changes the precision parameter of data storage from high precision to low precision through quantization. By representing neural network weights and activations with fewer bits (e.g., 8-bit integers instead of 32-bit floating point), the memory usage is significantly reduced while the quantization-aware training process ensures that computational accuracy is maintained within acceptable limits.
Solution Approach 2:
The patent uses low-precision data representations that require less memory resources. By accepting a controlled reduction in precision, the system can use cheaper, more efficient memory structures and reduce the overall quantity of data that needs to be stored and processed, making the system suitable for embedded devices with limited memory.
3Productivity
If quantization operation is performed between convolution operations, then computational load is reduced, but operational complexity increases
Solution Approach 1:
The patent merges the quantization operation with the convolution operation by designing a unified quantization-aware convolution circuit. Instead of implementing quantization as a separate preprocessing step, the quantization logic is integrated into the convolution computation unit, allowing both operations to be performed in a single hardware module. This reduces the overall circuit complexity while maintaining computational efficiency.
Solution Approach 2:
The patent introduces quantization as an intermediary operation between convolution layers. By inserting quantization modules at strategic points in the computational graph, the system transforms high-precision intermediate results into low-precision representations that are easier to process subsequently. This intermediary quantization step reduces the computational load of subsequent operations while the first memory unit stores the quantized results for reuse.
Data Source
AI summary
A neural network circuit that can be embedded in an embedded device such as an IoT device, and that provides high performance. The neural network circuit includes a first memory unit that stores input data; a convolution operation circuit that performs a convolution operation on a weight and the input data stored in the first memory unit; a second memory unit that stores convolution operation output data from the convolution operation circuit; and a quantization operation circuit that performs a quantization operation on the convolution operation output data stored in the second memory unit; wherein the first memory unit stores a quantization operation output data from the quantization operation circuit; and the convolution operation circuit performs the convolution operation on the quantization operation output data stored in the first memory unit as the input data.