Neural Network MAC Bit Truncation for Lower-Area AI Arithmetic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing demand for accurate machine learning operations in artificial intelligence systems leads to a growing number of multiply-accumulate (MAC) operators, resulting in larger area occupation and higher power consumption, while traditional floating-point number systems also face limitations in efficiency.
Innovation Solution
An electronic device and method that truncates adaptive bits for input and weight data, reducing the size of modules performing multiplication operations and converting results into a floating-point form with a first bit as a unit scale, thereby decreasing the word size of buffers and optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If more MAC operators are accumulated to enhance machine learning accuracy, then the accuracy of the machine learning algorithm is improved, but the area occupied by logic for performing MAC operations becomes larger
Solution Approach 1:
The patent changes the numerical representation parameter from traditional floating-point (FP32, FP64) to a custom fixed-point number system with optimized bit allocation. This parameter change allows the same computational accuracy to be achieved with fewer MAC operators, thereby reducing the logic area while maintaining machine learning algorithm accuracy
Solution Approach 2:
The patent applies different bit allocations to different parts of the number system (integer part vs. fractional part) based on the specific requirements of neural network operations. By optimizing the local quality of each bit position's contribution to accuracy, the system achieves high precision with reduced overall operator count and smaller logic area
2Measurement precision
If traditional floating-point number systems (FP64, FP32) are utilized to enhance accuracy, then the accuracy of operations is improved, but the area occupied by operators and power consumption become bigger
Solution Approach 1:
The patent replaces expensive traditional floating-point operators with simpler fixed-point arithmetic operations. The custom number system uses basic integer arithmetic that can be implemented with simpler, lower-power logic circuits, reducing power consumption while maintaining sufficient accuracy for neural network operations
Solution Approach 2:
The patent changes the numerical representation from high-precision floating-point to an optimized fixed-point format with custom bit allocation. This parameter change reduces the complexity of arithmetic operations, allowing implementation with lower-power operators that consume less energy while achieving the required operational accuracy
3Measurement precision
If traditional floating-point number systems are utilized to enhance accuracy, then the accuracy of operations is improved, but the area occupied by operators becomes bigger
Solution Approach 1:
The patent replaces area-intensive floating-point operators with compact fixed-point arithmetic logic. The custom number system enables implementation using simple integer addition, subtraction, and bit-shifting operations that occupy significantly less silicon area while maintaining the accuracy needed for neural network computations
Solution Approach 2:
The patent changes the numerical representation parameter from traditional floating-point to a custom fixed-point format optimized for neural network operations. This parameter change allows the use of compact arithmetic logic units with reduced operator area while achieving the required operational accuracy through optimized bit allocation
Data Source
AI summary
An electronic device and a method for controlling are provided. The electronic device may include a memory storing first input data and first weight data used in operations of a neural network model and a processor configured to input the first input data and the first weight data into a first module, and acquire second input data and second weight data, where a part of the first input data is truncated, and where a part of the first weight data is truncated, input the second input data and the second weight data into a second module that performs multiplication operations, and acquire first output data, and based on scaling factors of the first input data and first weight data identified through the first module, convert the acquired first output data into a floating point form expressing a first bit as a unit scale and acquire second output data.