Tensor Inference Compiler Tuning for Input Model Fit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing compilers for tensor data inference are not optimized, leading to low efficiency in fitting the specified input model.
Innovation Solution
A method and apparatus for optimizing a compiler by obtaining design structure information, setting target parameter tuning learning parameters, determining a compiler object, and performing parameter tuning optimization through an automatic parameter tuning strategy to fit the specified input model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the compiler is not optimized in time, then the compiler can operate without additional processing steps, but the compiler will not fit the input model, resulting in low efficiency of tensor data inference
Solution Approach 1:
The system performs preliminary analysis of the input model's computational graph to identify optimization opportunities before compilation. By pre-processing the model to detect tensor operations, data flow patterns, and computational bottlenecks, the compiler can apply targeted optimizations in advance, ensuring the compiled code fits the specific input model and achieves high inference efficiency without adding runtime complexity
Solution Approach 2:
The optimization process dynamically adjusts compiler parameters such as parallelization degree, memory allocation strategies, and computation offloading decisions based on the analyzed computational graph characteristics. By changing these parameters according to the specific model structure and tensor operations detected, the compiler adapts to different input models and achieves optimal inference performance without requiring a complete redesign of the compilation pipeline
2Productivity
If the compiler is optimized to fit the input model, then the tensor data inference efficiency is improved, but the optimization process requires additional time and resources
Solution Approach 1:
The system applies partial optimization by focusing only on the most impactful sections of the computational graph rather than optimizing the entire model uniformly. It identifies critical tensor operations and data flow paths that have the greatest influence on inference performance and applies optimization techniques selectively to these regions, achieving significant efficiency improvements with minimal optimization time and resource consumption
Solution Approach 2:
The optimization process applies different optimization strategies to different parts of the computational graph based on local characteristics. For example, it may apply parallelization to independent computation nodes, apply memory optimization to data-intensive operations, and apply operator fusion to sequential operations with compatible data types. This localized approach ensures that optimization time is spent where it provides the most benefit while maintaining overall compilation efficiency
Data Source
AI summary
Disclosed are a method, an apparatus and a device for optimizing a compiler based on tensor data calculation inference. The method includes: obtaining design structure information of a compiler to be optimized, and obtaining a target intermediate conversion layer and an execution operation of the target intermediate conversion layer according to the design structure information; setting a target parameter tuning learning parameter according to the target intermediate conversion layer; determining a compiler object to be optimized according to the execution operation of the target intermediate conversion layer; and performing parameter tuning optimization on the compiler object to be optimized according to the target parameter tuning learning parameter through a target automatic parameter tuning strategy.