Neural Processing Precision Conversion Using Intermediary Buffers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural processing units face inefficiencies in data processing due to the need for precision conversion, which consumes significant hardware resources and timing resources, and often results in increased power consumption.
Innovation Solution
The neural processing device employs a buffer memory to convert precision during data transmission, minimizing the need for precision conversion logic in processing units and reducing power consumption by utilizing converting buffers and compute units to handle different precision formats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If precision conversion is performed within calculation modules, then data precision requirements are met, but power consumption increases and processing efficiency decreases
Solution Approach 1:
The patent extracts the precision conversion function from the calculation modules and relocates it to dedicated converting buffers. This separation allows calculation modules to focus on computational tasks while precision conversion is handled by specialized buffer units, thereby reducing power consumption and improving overall processing efficiency.
Solution Approach 2:
The patent introduces converting buffers as intermediary components between calculation modules of different precisions. These buffers act as mediators that perform precision conversion on data during transmission, eliminating the need for calculation modules to perform conversion operations themselves.
2Measurement precision
If precision conversion is performed within calculation modules, then data precision requirements are met, but processing efficiency and productivity decrease
Solution Approach 1:
The precision conversion function is extracted from calculation modules and assigned to dedicated converting buffers. This functional separation enables calculation modules to operate at full efficiency without the overhead of precision conversion operations, while conversion is performed by specialized buffer units optimized for this specific task.
Solution Approach 2:
The patent performs precision conversion in advance during data transmission between calculation modules, before the data reaches the destination compute unit. This preliminary conversion ensures that data is ready in the required precision format when needed, eliminating delays and improving processing throughput.
3Adaptability or versatility
If precision conversion logic is included in processing units, then conversion functionality is available, but device complexity and resource concentration increase
Solution Approach 1:
The precision conversion logic is extracted from processing units and consolidated into dedicated converting buffers. This extraction reduces the complexity of individual processing units while maintaining the necessary conversion functionality through specialized buffer components.
Solution Approach 2:
The patent segments the precision conversion functionality into separate converting buffer components that are distributed between different precision domains. This segmentation prevents resource concentration in single processing units and distributes conversion capabilities across the data flow path.
Data Source
AI summary
A neural processing device and a method for converting data thereof are provided. The neural processing device comprises a first compute unit configured to receive first input data in first precision and generate first output data in the first precision by performing calculations, a second compute unit configured to receive second input data in second precision which is different from the first precision and generate second output data in the second precision by performing calculation, and a first converting buffer configured to receive and store the first output data, generate the second input data by converting the first output data into the second precision, and transmit the second input data to the second compute unit.