Mixed-Precision AI Processor for Precision-Speed Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
AI processors face challenges in balancing calculation precision and speed, as they typically use a single data format (Int8, BF16, or FP32) that either prioritizes precision or speed but not both effectively.
Innovation Solution
A mixed-precision AI processor with multiple calculation modules (Int8, BF16, and TF32) and a control module that dynamically switches between modes based on a calculation strategy to select the appropriate format for each calculation, allowing for a combination of formats in certain scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single data format (FP32, BF16, or Int8) is used for AI calculations, then calculation precision can be maintained at a certain level, but calculation speed decreases significantly
Solution Approach 1:
The AI processor is divided into multiple calculation modules, each dedicated to a specific data format (FP32 module, BF16 module, Int8 module). This segmentation allows different parts of the system to handle different precision requirements independently, enabling high-speed Int8 calculations for less critical operations while maintaining FP32 precision for operations requiring accuracy.
Solution Approach 2:
The control module dynamically selects and switches between different calculation modules based on the specific calculation requirements of each operation. This dynamic allocation allows the system to adaptively choose the appropriate precision level for each calculation, optimizing the balance between speed and precision in real-time without sacrificing either.
2Productivity
If a single data format (Int8) is used for AI calculations, then calculation speed is maximized, but calculation precision deteriorates
Solution Approach 1:
The AI processor is designed as a universal computing system that can handle multiple data formats (FP32, BF16, Int8) within a single architecture. The control module intelligently routes different calculation tasks to appropriate modules, making the system capable of performing both high-speed Int8 calculations and high-precision FP32 calculations as needed, rather than being limited to a single format.
Solution Approach 2:
The system changes the precision parameter dynamically based on calculation requirements. For operations where high speed is critical and precision requirements are lower, Int8 format is used. For operations requiring higher accuracy, the system switches to BF16 or FP32 formats, thus adapting the precision parameter to match the specific needs of each calculation task.
3Measurement precision
If high-precision data format (FP32) is used for all calculations, then calculation precision is maintained, but calculation speed and computing power decrease
Solution Approach 1:
Different parts of the calculation pipeline are assigned different precision levels based on their specific requirements. Critical calculations that directly impact model accuracy use FP32 or BF16 formats, while less critical operations such as activation functions or normalization use Int8 format. This local quality approach ensures high precision where needed while maintaining high computing power overall.
Solution Approach 2:
The control module dynamically adjusts the precision level for each calculation operation based on the specific requirements of the neural network layer being processed. This dynamic adaptation allows the system to maintain high computing power by using lower precision formats for most operations while selectively applying high-precision formats only when necessary for maintaining model accuracy.
Data Source
AI summary
A mixed-precision artificial intelligence (AI) processor and an operating method thereof are provided. The AI processor includes a first calculation module, a second calculation module and a control module. The first calculation module is configured to perform calculation based on the data with a first format. The second calculation module is configured to perform calculation based on the data with a second format different from the first format. The control module is coupled to the first calculation module and the second calculation module to select one of the first calculation module or the second calculation module to perform calculation based on an input data according to a calculation strategy.