AI Cluster Configuration Selection Using Application Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for obtaining cluster configuration information result in low cluster resource utilization and poor AI application running performance.
Innovation Solution
A method to obtain M pieces of cluster configuration information, each with corresponding application feature information, and select configuration information based on running latencies to improve resource utilization and performance by constructing clusters that meet specific conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual selection of key instruction segments and expert estimation are used to obtain cluster configuration information, then the process is simple to implement, but the cluster resource utilization is low and AI application running performance is poor
Solution Approach 1:
The patent replaces the manual expert estimation process with an automated emulator-based system. The emulator automatically compiles AI applications, executes them on virtual devices, and measures running latencies without requiring human intervention. This substitution of manual mechanical processes with automated computational processes resolves the contradiction by maintaining ease of operation while dramatically improving measurement precision and subsequent productivity.
Solution Approach 2:
The system performs self-service by automatically obtaining cluster configuration information through emulator-based measurement. The emulator independently compiles the AI application, executes it on multiple virtual devices with different configurations, measures latencies, and determines optimal configurations without external expert input. This self-service mechanism eliminates dependency on manual expertise while improving accuracy and performance.
2Measurement precision
If emulator-based measurement of running latency is performed, then the measurement precision is improved, but the device complexity increases
Solution Approach 1:
The patent uses virtual devices (copies) instead of physical hardware for measurement. The emulator creates virtual representations of computing devices that replicate the behavior and performance characteristics of real hardware. By measuring on these virtual copies rather than physical devices, the system achieves high measurement precision while avoiding the complexity of managing actual hardware testbeds. The virtual devices can be rapidly instantiated and configured without physical constraints.
Solution Approach 2:
The emulator serves as an intermediary layer between the AI application and the measurement process. Rather than directly measuring on physical devices, the system uses the emulator as a mediator that translates application execution into measurable latency data. This intermediary simplifies the measurement system by providing a standardized, software-based interface that abstracts away hardware complexity while maintaining measurement accuracy.
Data Source
AI summary
A method for obtaining cluster configuration information includes: obtaining M pieces of first cluster configuration information, where M is an integer greater than 1; obtaining, based on an artificial intelligence (AI) application and the M pieces of first cluster configuration information, M pieces of first application feature information respectively corresponding to the M pieces of first cluster configuration information, where each piece of first application feature information includes description information of a plurality of operators in the AI application and a dependency relationship between the plurality of operators; obtaining, based on the M pieces of first application feature information, running latencies of running the AI application by M clusters, where the M clusters are in one-to-one correspondence with the M pieces of first cluster configuration information; and selecting, from the M pieces of first cluster configuration information, corresponding first cluster configuration information whose running latency satisfies a first condition.