Accelerator Data Format Conversion Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing environments face performance limitations due to the need for frequent data format conversions between processors and AI accelerators, which increases processing time and complexity, especially in data-intensive tasks like deep learning.
Innovation Solution
A method is introduced to reduce data format conversion processing by determining which operations can be performed on the accelerator, blocking unnecessary conversions, and using convert_input and convert_output parameters to optimize data format handling, thereby enhancing processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data format conversion is performed for every operation between processor and accelerator, then data compatibility is ensured, but processing time increases and throughput decreases
Solution Approach 1:
The patent extracts and removes redundant data format conversion operations from the processing pipeline. By analyzing the computation graph, it identifies consecutive operations that both require the same data format, and eliminates the unnecessary conversion steps between them, thereby reducing overhead while maintaining data compatibility where actually needed.
Solution Approach 2:
The patent changes the parameter of data format conversion from a mandatory operation to a conditional operation. It introduces format propagation mechanisms that track data format requirements through the computation graph, allowing the system to dynamically adjust conversion behavior based on the actual needs of adjacent operations, thus optimizing throughput without sacrificing compatibility.
2Reliability
If data format conversion is performed for every operation between processor and accelerator, then data format compatibility is maintained, but processing overhead increases
Solution Approach 1:
The patent extracts and removes redundant data format conversion operations from the processing pipeline. By analyzing the computation graph, it identifies consecutive operations that both require the same data format, and eliminates the unnecessary conversion steps between them, thereby reducing overhead while maintaining data compatibility where actually needed.
Solution Approach 2:
The patent performs preliminary analysis of the computation graph to pre-identify operations that can share data formats. By propagating format information forward through the graph and marking operations that don't require conversion, the system prepares the optimization in advance, avoiding the overhead of runtime conversion decisions while ensuring compatibility is maintained where necessary.
3Productivity
If all operations are offloaded to the accelerator, then processing speed improves, but data format conversion complexity increases
Solution Approach 1:
The patent changes the parameter of data format conversion from a mandatory operation to a conditional operation. It introduces format propagation mechanisms that track data format requirements through the computation graph, allowing the system to dynamically adjust conversion behavior based on the actual needs of adjacent operations, thus optimizing throughput without sacrificing compatibility.
Solution Approach 2:
The patent uses copying of format metadata through the computation graph to avoid actual data format conversions. By propagating format information as copyable metadata alongside the data flow, the system enables the accelerator to understand format requirements without performing expensive conversion operations, reducing complexity while maintaining speed.
Data Source
AI summary
Data format conversion processing of an accelerator accessed by a processor of a computing environment is reduced. The processor and accelerator use different data formats, and the accelerator is configured to perform an input conversion to convert data from a processor data format to an accelerator data format prior to performing an operation using the data, and an output conversion to convert resultant data from accelerator data format back to processor data format after performing the operation. The reducing includes determining that adjoining operations of a process to run on the processor and accelerator are to be performed by the accelerator, where the adjoining operations include a source operation and destination operation. Further, the reducing includes blocking an output data format conversion of the source operation and an input data format conversion of the input data for the destination operation.