Compiler-Based Resource Determination for AI Code Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current utilization of dedicated processing resources in AI applications is inflexible, leading to decreased resource utilization efficiency and waste, as existing methods cannot accurately determine the required resources for executing code segments without running the applications, which is time and computation resource-consuming.
Innovation Solution
A method and device that compile a code segment using a compiler to determine the resources needed for execution by generating intermediate results, allowing for the identification of memory and thread requirements without actual execution, using a dummy DL compiler to collect resource information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If dedicated processing resources are allocated to AI applications without accurate resource determination, then applications can run, but resource utilization efficiency decreases and resources are wasted
Solution Approach 1:
The patent applies preliminary action by performing static analysis on the code segment during the compilation phase to determine resource requirements before the AI application actually runs. The system analyzes the code to identify operations that will require dedicated processing resources (such as GPU operations, memory allocations, and parallel computing tasks) and calculates the total resource requirements in advance. This allows the resource allocation system to allocate exactly the needed resources without over-provisioning, thereby improving resource utilization efficiency and preventing resource waste.
2Measurement precision
If actual execution is performed to determine resource requirements, then accurate resource information is obtained, but time and computation resources are consumed
Solution Approach 1:
The patent applies the extraction principle by separating the resource determination process from the actual execution process. Instead of requiring full execution to measure resource usage, the system extracts resource requirement information through static code analysis during compilation. The system identifies and extracts information about dedicated processing operations, memory requirements, and computational intensity from the code structure itself, obtaining accurate resource requirements without the time and computational overhead of actual execution.
Solution Approach 2:
The system performs resource determination as a preliminary action during the compilation phase, before execution. By analyzing the code segment statically and determining resource requirements in advance, the system obtains accurate resource information without needing to actually execute the code, thereby avoiding the time and computational resource consumption that would result from running the application just to measure its resource usage.
3Productivity
If flexible resource allocation is implemented, then resource utilization efficiency improves, but system complexity increases
Solution Approach 1:
The patent applies the self-service principle by enabling the code segment itself to provide information about its resource requirements through static analysis. The compilation process automatically identifies operations requiring dedicated processing resources and calculates resource needs without requiring external profiling tools or manual configuration. This self-describing approach allows the resource allocation system to implement flexible allocation based on actual needs while keeping the system relatively simple, as the complexity is shifted to the compilation phase rather than the runtime allocation phase.
Data Source
AI summary
A method comprises: compiling the code segment with a compiler; and determining, based on an intermediate result of the compiling, a resource associated with a dedicated processing unit and for executing the code segment. As such, the resource required for executing a code segment may be determined quickly without actually executing the code segment and allocating or releasing the resource, which helps subsequent resource allocation and further brings about a better user experience.