Approximate Computing and Data Reuse for AI Edge Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) are computationally and memory-intensive, making them challenging to implement on resource-constrained devices like IoT and mobile devices due to high memory overhead and computation requirements, which limits their widespread use on edge devices.
Innovation Solution
The use of approximation techniques such as linear and quadratic approximations during training to reduce the number of parameters and computations, combined with data and computational reuse methods to minimize memory storage and energy consumption, allowing for efficient inference on edge devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If DNNs are implemented with full precision parameters and computations, then accuracy is maintained, but memory storage and computation requirements become prohibitively high for resource-constrained devices
Solution Approach 1:
The patent applies parameter changes by transitioning from full-precision floating-point representations to low-precision integer representations (e.g., 8-bit, 4-bit, or 1-bit quantization). This fundamentally changes the parameter format and precision level, enabling significant memory reduction (up to 32x or more) while maintaining acceptable inference accuracy through careful quantization strategies and calibration processes.
Solution Approach 2:
The patent implements partial precision computation by applying different precision levels to different parts of the DNN computation graph. Critical layers maintain higher precision while less sensitive layers use lower precision, achieving a balance between accuracy and resource consumption. This selective precision approach allows the system to use only the necessary computational precision for each specific operation.
2Measurement precision
If DNNs are implemented with full precision parameters and computations, then accuracy is maintained, but computation requirements become prohibitively high for resource-constrained devices
Solution Approach 1:
The patent changes computational parameters from high-precision floating-point operations to low-precision integer operations. This parameter transformation reduces the computational complexity and resource requirements significantly, as integer arithmetic requires fewer transistors and less energy while maintaining sufficient precision for inference tasks through quantization-aware training and calibration.
Solution Approach 2:
The patent substitutes complex floating-point arithmetic mechanisms with simpler integer arithmetic mechanisms. This replacement fundamentally changes the computational substrate from high-precision but resource-intensive floating-point units to low-precision but efficient integer units, enabling DNN inference on resource-constrained edge devices with limited compute capabilities.
3Quantity of substance
If approximation techniques are used to reduce parameters and computations, then resource consumption is reduced, but training time increases due to additional approximation computations
Solution Approach 1:
The patent applies preliminary action by performing quantization and approximation transformations during the training phase rather than during inference. By pre-computing quantized weights and activations and storing them in low-precision format, the system incurs the computational overhead only once during training, while benefiting from reduced model size and faster inference at deployment. This shifts the time cost from the operational phase to the preparation phase.
Solution Approach 2:
The patent uses copying by creating and storing pre-computed quantized versions of weights and activations. Instead of performing full-precision computations and then converting results, the system copies data into low-precision representations during training and uses these copied quantized values throughout the training and inference process, reducing both memory usage and computational requirements.
4Use of energy by stationary object
If data and computational reuse methods are applied, then memory storage and energy consumption are reduced, but system complexity increases
Solution Approach 1:
The patent applies merging by combining multiple operations into single computational units. For example, it merges weight and activation quantization into unified quantization schemes, combines multiple small matrices into larger batched operations, and integrates data reuse patterns directly into the computational graph. This consolidation reduces the number of separate operations and memory accesses, lowering energy consumption while managing complexity through systematic integration.
Solution Approach 2:
The patent implements universality by designing computational kernels and data structures that serve multiple functions simultaneously. For instance, the same quantized weight representation is used for both memory storage efficiency and computational efficiency, and the same low-precision arithmetic unit handles both inference and training operations. This multi-functionality reduces the need for specialized hardware components, managing system complexity while achieving energy savings.
Data Source
AI summary
A method can be used to reduce the memory storage and energy used by deep neural networks. The method can include determining the weights associated with the deep neural network. An input feature map can be received and used with the weights to generate approximated weights. Using the approximated weights and the input feature map a convolution inference can be performed.