Code Analysis Result Cache for Neural Network Compilation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Compiling complex neural networks with millions of parameters and operators is a time-consuming process due to the repetitive analysis of loop-nest constructs, which can take days to complete, as each execution statement needs to be analyzed and optimized independently.
Innovation Solution
Implementing a code analysis result cache that stores and retrieves analysis results for execution statements with similar execution conditions, utilizing a least recently used (LRU) cache policy to speed up the compilation process by avoiding repeated analysis and optimizing loop-nest constructs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If each execution statement in loop-nest constructs is analyzed and optimized independently, then the compilation produces optimized machine code, but the compilation time becomes excessively long (taking days to complete)
Solution Approach 1:
The patent performs code analysis on loop-nest constructs beforehand to identify statements with similar execution conditions. By pre-categorizing statements into equivalence classes based on their control flow characteristics, the system prepares optimization opportunities in advance, avoiding redundant analysis during the main compilation process.
Solution Approach 2:
The patent creates equivalence classes of execution statements that share the same execution conditions. A single analysis result can be applied universally to all statements within an equivalence class, allowing one analysis to serve multiple statements and significantly reducing total analysis time while maintaining optimization quality.
2Reliability
If repetitive analysis of loop-nest constructs is performed for each execution statement, then thorough optimization is achieved, but the same analysis is repeated multiple times increasing compilation time
Solution Approach 1:
The patent copies analysis results from representative statements and applies them to all equivalent statements within the same equivalence class. This copying mechanism ensures that thorough analysis is performed only once per class while still achieving comprehensive optimization coverage across all statements with similar execution conditions.
Solution Approach 2:
The patent merges statements with identical execution conditions into equivalence classes. By combining multiple statements that require the same analysis into a single group, the system performs the analysis once for the entire group rather than individually for each statement, eliminating redundant work while maintaining optimization thoroughness.
3Measurement precision
If millions of operators in neural networks are processed through complete compilation analysis, then accurate optimized code is generated, but the compilation process takes days to complete
Solution Approach 1:
The patent segments the set of millions of operators into equivalence classes based on their execution conditions within loop-nest constructs. This segmentation divides the vast compilation task into manageable groups where statements within each segment share common characteristics, allowing efficient batch processing while maintaining analysis accuracy for each segment.
Data Source
AI summary
Techniques to speed up code compilation may include caching code analysis results such that the analysis of subsequent code having a similar structured can be omitted. For example, a loop-nest construct in the code can be parsed, and an execution statement in the loop-nest construct can be analyzed by a compiler to generate an analysis result indicating a set of execution conditions for the execution statement. A lookup key can be generated from the control statements bounding the execution statement, and the analysis result can be stored with the lookup key in a cache entry of the cache. The execution statement is then modified according to the analysis result for optimization. Instead of having to analyze a subsequent execution statement bounded by the same control statements, the analysis result of the subsequent execution statement can be retrieved from the cache and be used to modify the subsequent execution statement.