Convolution Operator Sub-Multipliers for Mixed-Precision Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network technologies face inefficiencies in processing large amounts of input data in real-time, particularly in maintaining accuracy with varying bit lengths during operations, which affects the performance of convolution operations.
Innovation Solution
The method involves determining first-bit and second-bit feature map operands and weight operands from input feature maps and kernels, respectively, and dispatching these operands to decomposed sub-multipliers for parallel processing, optimizing the use of hardware resources by employing different bit precisions and clock-gating zero operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single multiplier is used for convolution operations, then device complexity is reduced, but productivity decreases due to inability to process different bit precisions in parallel
Solution Approach 1:
The patent divides a single multiplier into multiple sub-multipliers with different bit precisions (e.g., first sub-multiplier for 8-bit operations, second sub-multiplier for 16-bit operations). This segmentation allows parallel processing of convolution operations with different precision requirements, significantly improving productivity while keeping each sub-multiplier relatively simple in structure.
2Manufacturing precision
If full precision multipliers are used, then manufacturing precision is maintained, but use of energy increases due to unnecessary computational operations
Solution Approach 1:
The patent applies different bit precisions to different sub-multipliers based on the specific requirements of the convolution operation. For example, less critical computations use lower precision (8-bit) to reduce energy consumption, while critical computations use higher precision (16-bit) to maintain accuracy. This local optimization of precision levels balances manufacturing precision with energy efficiency.
3Adaptability or versatility
If varying bit lengths are used during operations, then adaptability improves, but device complexity increases due to need for multiple precision handlers
Solution Approach 1:
The patent designs the convolution operator to universally handle multiple bit precisions through dedicated sub-multipliers. Each sub-multiplier is optimized for a specific precision level, but the overall operator can process mixed-precision operations in parallel. This universal design provides adaptability for different neural network layers and operations without requiring complex dynamic precision management logic.
Data Source
AI summary
Provided are a method and apparatus for processing a convolution operation in a neural network, the method includes determining a precision of feature map operands and a precision of weight operands, respectively, on which the convolution operation is to be performed in parallel, decomposing a multiplier included in a convolution operator into sub-multipliers based on the precision of the feature map operands and the precision of the weight operands, performing the convolution operation between the feature map operands and the weight operands by using the decomposed sub-multipliers, each operand being processed in a sub-multiplier corresponding to a precision of the operand, and obtaining output feature maps corresponding to results of the convolution operation.