An online automatic tuning and compilation system and method for large model training

Abstract

The present invention discloses an online automatic tuning and compilation system and method for large-scale model training, which belongs to the field of computing framework programming and compilers. The system includes an automatic tuning client, an automatic tuning server, a configuration manager, a Triton operator register, a communication manager, an automatic tuning scheduler, and a data collector. The present invention improves the computing performance of the fusion operator by searching online for the optimal hyperparameter configuration of the fusion operator during large-scale model training, thereby improving the efficiency of utilizing GPU hardware resources during large-scale model training. The online automatic tuning and compilation system of the present invention provides a search for a larger hyperparameter space, and quickly finds the optimal hyperparameter configuration of the fusion operator for end-to-end performance during large-scale model training.

Description

Technical Field

[0001] The present invention belongs to the field of computing framework programming and compilers for large-scale artificial intelligence model training, and in particular relates to an online automatic tuning and compilation system and method for large-scale model training. Background Art

[0002] In large-scale model training, the number of operators and hyperparameters involved in training increases dramatically as the model scale increases. Traditional hyperparameter tuning methods often require extensive manual intervention and repeated experimentation, resulting in low efficiency and lack of universal applicability. Existing automatic tuning methods often focus on tuning a single operator and struggle to adapt to the dynamic demands of multi-operator fusion and distributed computing environments. Summary of the Invention

[0003] The purpose of the present invention is to address the deficiencies of the existing technology and provide an online automatic tuning and compilation system and method for large model training.

[0004] The object of the present invention is achieved through the following technical solutions: an online automatic tuning and compilation system for large model training, comprising an automatic tuning client, an automatic tuning server, a configuration manager, a Triton operator register, a communication manager, an automatic tuning scheduler, and a data collector;

[0005] The data collector is used to collect performance indicator data during the large model training process as an optimization target;

[0006] The automatic tuning client is used to call the communication manager to send the performance indicator data collected by the data collector to the automatic tuning server after the large model training is started;

[0007] The automatic tuning server is used to initialize the hyperparameter space according to different tuning strategies after receiving the automatic tuning request from the automatic tuning client, and adjust the hyperparameter configuration required for the next round of iterative training as the updated hyperparameter configuration based on the performance indicator data fed back by the automatic tuning client, and call the communication manager to send it to the automatic tuning client;

[0008] The configuration manager is used to provide the user with a switch for starting online automatic tuning, and configure the updated hyperparameter configuration received by the automatic tuning client into the Triton fusion operator in the large model framework;

[0009] The Triton operator register is used to register the Triton fusion operator to the large model training framework and receive updated hyperparameter configurations;

[0010] The communication manager is responsible for implementing data transmission between the automatic tuning client and the automatic tuning server using a communication protocol;

[0011] The automatic tuning scheduler is used to pass hyperparameter configurations to multiple GPU servers in the cluster that participate in large model training.

[0012] Furthermore, the data collector includes a performance indicator collection module, a data preprocessing module and an indicator aggregation module;

[0013] The performance indicator collection module is used to collect performance indicator data of each training node in the local training process in real time, and the performance indicator data includes but is not limited to computing throughput, memory usage, GPU utilization and communication overhead;

[0014] The data preprocessing module is used to clean and standardize the collected performance indicator data to obtain processed performance indicator data;

[0015] The indicator aggregation module is used to aggregate and compile the processed performance indicator data of different training nodes and send it to the automatic tuning client.

[0016] Furthermore, the automatic tuning client includes a data acquisition interface module, an indicator preprocessing module, a configuration update module and a communication interface module;

[0017] The data acquisition interface module is used to receive performance indicator data from the data collector;

[0018] The indicator preprocessing module is used to detect outliers on the performance indicator data from the data collector, remove noise data, and then standardize the data of different dimensions to obtain standardized performance indicator data; finally, perform time series aggregation on the standardized performance indicator data to obtain time series aggregated performance indicator data;

[0019] The configuration update module is responsible for receiving the updated hyperparameter configuration sent by the automatic tuning server and applying the updated hyperparameter configuration to the local training process;

[0020] The communication interface module is used to send the performance indicator data after time series aggregation to the automatic tuning server through the communication manager.

[0021] Furthermore, the automatic tuning server includes a hyperparameter space initialization module, an indicator analysis module, an optimization strategy selection module and a configuration optimization module;

[0022] The hyperparameter space initialization module is used to initialize the hyperparameter space according to different tuning strategies after receiving the automatic tuning request from the automatic tuning client;

[0023] The indicator analysis module is used to analyze the performance indicator data from the automatic tuning client and build a performance evaluation model; the performance evaluation model comprehensively considers computing efficiency, memory efficiency, and communication efficiency to generate a comprehensive performance score for the current configuration;

[0024] The optimization strategy selection module is used to dynamically select an appropriate optimization strategy based on the characteristics of the training scenario and the optimization goal and the generated comprehensive performance score during the local training process of the large model; the optimization strategy includes a Bayesian optimization strategy, a reinforcement learning strategy, or a genetic algorithm strategy;

[0025] The configuration optimization module adjusts the hyperparameter configuration required for the next round of iterative training as the updated hyperparameter configuration by selecting a suitable optimization strategy and calls the communication manager to send it to the automatic tuning client.

[0026] Furthermore, the configuration manager includes a configuration storage module, a configuration update module, a version control layer and a monitoring layer;

[0027] The configuration storage module is used to use a distributed database, which not only stores the optimal solution exceeding the configuration in the previous round of iterative training, but also stores the corresponding relationship between historical configuration records and performance data;

[0028] The configuration update module is used to configure the updated hyperparameter configuration received by the automatic tuning client into the Triton fusion operator in the large model framework after the user turns on the switch to start online automatic tuning;

[0029] The version control layer is used to manage the version evolution process of the configuration, and implements a rapid rollback mechanism for the configuration, so that it can be restored to a stable version in a timely manner when performance anomalies are found;

[0030] The monitoring layer is used to continuously monitor the effectiveness and performance impact of the configuration, collect feedback information from each node, evaluate the effect of the configuration update in real time, and notify the system administrator in a timely manner when a configuration anomaly is found.

[0031] Furthermore, the communication manager uses HTTP or gRPC protocol to implement data transmission.

[0032] Furthermore, the Triton operator registrar includes a compilation module, a registration module, an optimization module and a verification module;

[0033] The compilation module is used to compile the Triton fusion operator into a binary file;

[0034] The registration module is used to register the compiled Triton fusion operator into the large model training framework;

[0035] The optimization module is used to optimize the compiled Triton fusion operator;

[0036] The verification module is used to verify the correctness of the compiled Triton fusion operator registered to the large model training framework.

[0037] Furthermore, the automatic tuning scheduler includes a load balancing module, a configuration synchronization module and a status monitoring module;

[0038] The load balancing module is used to dynamically allocate tuning tasks according to the load status of each GPU node;

[0039] The configuration synchronization module is used to ensure the consistency of hyperparameter configurations of all nodes in the cluster;

[0040] The status monitoring module is used to monitor the operating status and optimization progress of each node in real time.

[0041] Furthermore, the automatic tuning client also includes an anomaly detection module and a log recording module;

[0042] The anomaly detection module is used to monitor anomalies during the large model training process and notify the user when an anomaly is detected;

[0043] The logging module is used to record key operations and performance changes during the tuning process to facilitate user auditing and system maintenance.

[0044] The present invention also provides an online automatic tuning and compilation method for large model training. The method is implemented based on the above-mentioned online automatic tuning and compilation system for large model training. The method specifically includes:

[0045] The data collector collects performance indicator data during the large model training process as an optimization target;

[0046] After the large model training is started, the automatic tuning client is responsible for sending the performance indicator data collected by the data collector to the automatic tuning server by calling the communication manager;

[0047] After receiving the automatic tuning request from the automatic tuning client, the automatic tuning server initializes the hyperparameter space according to different tuning strategies, and adjusts the hyperparameter configuration required for the next round of iterative training as the updated hyperparameter configuration based on the performance indicator data fed back by the automatic tuning client, and calls the communication manager to send it to the automatic tuning client;

[0048] Through the configuration manager, a switch for starting online automatic tuning is provided to the user, and the updated hyperparameter configuration received by the automatic tuning client is configured into the Triton fusion operator in the large model framework;

[0049] Register the Triton fusion operator to the large model training framework through the Triton operator register and receive the updated hyperparameter configuration;

[0050] The communication manager is responsible for implementing data transmission between the automatic tuning client and the automatic tuning server using a communication protocol;

[0051] Through the automatic tuning scheduler, the hyperparameter configuration is passed to multiple GPU servers in the cluster that participate in large model training.

[0052] The beneficial effects of the present invention are as follows: the online automatic tuning and compilation system provided by the present invention can automatically search and adjust hyperparameter configurations during large-scale model training, significantly improving the computational performance of fusion operators. Compared with traditional methods, the present invention can not only handle large-scale hyperparameter spaces, but also adapt to heterogeneous hardware environments and maximize the utilization of GPU resources. In addition, the system can reduce manual intervention and shorten the tuning cycle through automated tuning, thereby improving the efficiency and accuracy of large-scale model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a structural diagram of an online automatic tuning and compilation system for large model training;

[0054] Figure 2 This is the structural diagram of the data collector;

[0055] Figure 3 This is a diagram of the structure of the automatic tuning client;

[0056] Figure 4 This is a diagram of the structure of the automatic tuning server;

[0057] Figure 5 This is the structure diagram of the configuration manager;

[0058] Figure 6 This is the structural diagram of the Triton operator register;

[0059] Figure 7 This is the structural diagram of the automatic tuning scheduler. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to illustrate the present invention, rather than to represent all embodiments. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0061] Technical term explanation:

[0062] Triton, a language proposed by OpenAI for writing efficient custom deep learning primitives.

[0063] HTTP (Hypertext Transfer Protocol) is an Internet protocol for distributed, collaborative, hypermedia information systems. It serves as the data communication foundation for the World Wide Web, allowing hypertext data to be transferred from one system to another across the Internet. HTTP defines the format of requests and responses between clients and servers, as well as the possible request methods a client can send, such as GET, POST, PUT, and DELETE.

[0064] gRPC, short for gRPC Remote Procedure Call, is a high-performance, open-source, and general-purpose RPC (Remote Procedure Call) framework developed by Google. It allows transparent communication between client and server applications and supports multiple programming languages.

[0065] 1. Overall Architecture of an Online Automatic Tuning and Compilation System for Large Model Training

[0066] like Figure 1 As shown in the figure, the online automatic tuning and compilation system for large-scale model training provided by this invention adopts a distributed architecture design and mainly includes seven core functional modules: automatic tuning client, automatic tuning server, configuration manager, Triton operator register, communication manager, automatic tuning scheduler, and data collector. Each module communicates with each other through standardized interfaces, forming a complete closed-loop optimization system.

[0067] The data collector is used to collect performance indicator data during the large model training process as an optimization target.

[0068] The automatic tuning client is deployed on each training node. After the large model training is started, it is responsible for sending the performance indicator data collected by the data collector during the large model training process to the automatic tuning server by calling the communication manager.

[0069] The automatic tuning server first initializes the hyperparameter space according to different tuning strategies after receiving the automatic tuning request from the automatic tuning client, and uses multiple optimization strategies to adjust the hyperparameter configuration required for the next round of iterative training based on the performance indicator data fed back by the automatic tuning client as the updated hyperparameter configuration and calls the communication manager to send it to the automatic tuning client.

[0070] The configuration manager saves the optimal solution in the previous round of iterative training and provides the user with a switch to start online automatic tuning. After the user turns on the switch to start online automatic tuning, the updated hyperparameter configuration received by the automatic tuning client is configured into the Triton fusion operator in the large model framework.

[0071] The Triton operator register registers the Triton fusion operator to the large model training framework and receives updated hyperparameter configuration.

[0072] The communication manager is responsible for using a reliable communication protocol to realize data transmission between the automatic tuning client and the automatic tuning server; the communication manager uses HTTP or gRPC protocol to realize data transmission.

[0073] The automatic tuning scheduler passes the hyperparameter configuration to multiple GPU servers in the cluster that participate in large model training.

[0074] 2. Detailed design of core modules

[0075] 2.1 Data Collector

[0076] like Figure 2 As shown, the data collector includes a performance indicator collection module, a data preprocessing module and an indicator aggregation module.

[0077] The performance indicator collection module is used to collect performance indicator data of each training node in the local training process in real time. The performance indicator data includes but is not limited to computing throughput, memory usage, GPU utilization and communication overhead.

[0078] The data preprocessing module is used to clean and standardize the collected performance indicator data to obtain processed performance indicator data.

[0079] The indicator aggregation module is used to aggregate and compile the processed performance indicator data of different training nodes and send it to the automatic tuning client.

[0080] 2.2 Automatically Tuning the Client

[0081] like Figure 3 As shown, the automatic tuning client includes a data acquisition interface module, an indicator preprocessing module, a configuration update module and a communication interface module.

[0082] The data acquisition interface module is used to receive performance indicator data from the data collector; through standardized interface definitions, compatibility with different training frameworks is ensured.

[0083] The indicator preprocessing module is used to detect outliers on the performance indicator data from the data collector, remove noise data, and then standardize the data of different dimensions to obtain standardized performance indicator data, so that the standardized performance indicator data are comparable; finally, time series aggregation is performed on the standardized performance indicator data to obtain time series aggregated performance indicator data, and the time series aggregated performance indicator data is used as a statistical indicator reflecting performance trends.

[0084] The configuration update module is responsible for receiving updated hyperparameter configurations from the automatic tuning server and applying them to the local training process. This module implements atomic configuration updates, ensuring consistency during the configuration update process. It also maintains a local configuration cache, enabling rapid rollback to the last stable configuration in the event of a network anomaly.

[0085] The communication interface module is used to send the performance indicator data after time series aggregation to the automatic tuning server through the communication manager. The communication interface module implements functions such as data compression, encrypted transmission, and breakpoint resume, ensuring data transmission reliability in complex network environments.

[0086] The automatic tuning client also includes an anomaly detection module and a logging module. The anomaly detection module monitors anomalies during large model training and notifies the user when an anomaly is detected. The logging module records key operations and performance changes during the tuning process for user auditing and system maintenance.

[0087] The automatic tuning client also includes an anomaly detection module and a logging module. The anomaly detection module monitors anomalies during large model training and notifies the user when an anomaly is detected. The logging module records key operations and performance changes during the tuning process for user auditing and system maintenance.

[0088] 2.3 Automatically Tuning the Server

[0089] like Figure 4 As shown in the figure, the automatic tuning server is the core decision-making module of the online automatic tuning compilation system for large model training, including four main components: hyperparameter space initialization module, indicator analysis module, optimization strategy selection module and configuration optimization module.

[0090] The hyper-parameter space initialization module is used to initialize the hyper-parameter space according to different tuning strategies after receiving the automatic tuning request from the automatic tuning client;

[0091] The indicator analysis module analyzes performance indicator data from the automatic tuning client and constructs a performance evaluation model. This performance evaluation model comprehensively considers computational efficiency, memory efficiency, and communication efficiency to generate a comprehensive performance score for the current configuration. The performance analyzer also implements trend analysis, using the comprehensive performance score to predict performance trends under the current configuration, providing a basis for optimization decisions.

[0092] The optimization strategy selection module is used to dynamically select a suitable optimization strategy based on the characteristics of the training scenario and the optimization objectives and the comprehensive performance score generated during the local training process of the large model; the optimization strategy includes a Bayesian optimization strategy, a reinforcement learning strategy or a genetic algorithm strategy; the optimization strategy selection module also maintains a strategy evaluation mechanism to evaluate and adjust the effects of different optimization strategies based on historical optimization results.

[0093] The configuration optimization module selects an appropriate optimization strategy to adjust the hyperparameter configuration required for the next round of iterative training as the updated hyperparameter configuration and calls the communication manager to send it to the automatic tuning client. During the adjustment process, the configuration optimization module first verifies whether the new hyperparameter configuration meets the hardware constraints and safety boundaries. It then encodes the hyperparameter configuration that meets the hardware constraints and safety boundaries and generates a standardized configuration description as the updated hyperparameter configuration. The configuration optimization module also maintains a configuration version control system to support configuration rollback and tracking.

[0094] 2.4 Configuration Manager

[0095] like Figure 5 As shown, the configuration manager includes a configuration storage module, a configuration update module, a version control layer and a monitoring layer.

[0096] The configuration storage module is used to use a distributed database, which not only stores the optimal solution exceeding the configuration in the previous round of iterative training, but also stores the corresponding relationship between historical configuration records and performance data.

[0097] The configuration update module is used to configure the updated hyperparameter configuration received by the automatic tuning client into the Triton fusion operator in the large model framework after the user turns on the switch to start online automatic tuning.

[0098] The version control layer is used to manage the version evolution process of the configuration, and implements a rapid rollback mechanism for the configuration, so that it can be restored to a stable version in a timely manner when performance anomalies are found.

[0099] The monitoring layer is used to continuously monitor the effectiveness and performance impact of the configuration, collect feedback information from each node, evaluate the effect of the configuration update in real time, and notify the system administrator in a timely manner when a configuration anomaly is found.

[0100] 2.5 Triton Operator Registry

[0101] like Figure 6 As shown, the Triton operator register includes a compilation module, a registration module, an optimization module and a verification module.

[0102] The compilation module is used to compile the Triton fusion operator into a binary file. The compilation process is divided into multiple stages: first, syntax analysis and semantic checking, then intermediate code generation and optimization, and finally, the target platform's machine code. The compilation module supports multiple backends and can generate optimized code for different hardware platforms.

[0103] The registration module is used to register the compiled Triton fusion operator with the large model training framework. The registration process includes steps such as metadata registration, symbol table update, and memory allocation. The registration module implements a hot-swappable mechanism to support dynamic updates of operator implementations during training.

[0104] The optimization module is used to optimize compiled Triton fusion operators. It implements multi-level code optimization. At the IR level, it performs basic optimizations such as constant propagation and dead code elimination. At the platform level, it implements memory access optimization and instruction scheduling optimization. The optimizer also includes an adaptive optimization mechanism that dynamically adjusts optimization strategies based on runtime information.

[0105] The verification module verifies the correctness of the compiled Triton fusion operator registered to the large model training framework. The verification process includes interface consistency checks, numerical accuracy verification, and performance benchmarking. The verifier also maintains a test case library for regression testing and performance comparison.

[0106] 2.6 Automatic Tuning Scheduler

[0107] like Figure 7 As shown, the automatic tuning scheduler also includes a load balancing module, a configuration synchronization module, and a status monitoring module. The load balancing module is used to dynamically allocate tuning tasks based on the load status of each GPU node. The configuration synchronization module is used to ensure the consistency of hyperparameter configurations across all nodes in the cluster. The status monitoring module is used to monitor the operating status and tuning progress of each node in real time.

[0108] 3. System Workflow

[0109] The workflow of an online automatic tuning and compilation system for large model training can be divided into the following main stages:

[0110] Initialization phase: When the online automatic tuning compilation system for large model training is started, each functional module completes the initialization configuration; establishes a communication connection between the automatic tuning client and the automatic tuning server; loads the initial hyperparameter configuration and optimization strategy; and initializes the performance monitoring and data collection modules.

[0111] Data collection phase: The data collector continuously monitors and collects performance indicators during the local training process; cleans and standardizes the collected raw performance indicator data; packages the processed performance indicator data and sends it to the automatic tuning server; and maintains the local performance indicator cache.

[0112] Optimization decision phase: The automatic tuning server receives and aggregates performance indicator data from each node; the performance analyzer evaluates the optimization effect of the current configuration; the optimization strategy engine selects the optimization direction based on the analysis results; and generates new hyperparameter configuration recommendations.

[0113] Configuration update phase: The configuration manager verifies the validity of the new configuration; distributes configuration change notifications to each node; each node completes the configuration update and returns confirmation information; and monitors the system status after the configuration update.

[0114] Feedback and adjustment phase: Collect performance feedback after configuration updates; evaluate optimization results and update optimization strategies; record optimization history and update the experience database; and adjust the direction and intensity of the next round of optimization.

[0115] 4. Implementation of Key Technologies

[0116] 4.1 Adaptive Optimization Strategy

[0117] The online automatic tuning and compilation system for large-scale model training implements an adaptive optimization strategy selection mechanism. First, a strategy evaluation model is established that considers factors such as historical optimization results, computing resource consumption, and convergence speed. Then, the most appropriate optimization strategy is dynamically selected based on the current training stage and performance bottlenecks.

[0118] During the optimization process, the online automatic tuning and compilation system for large model training continuously evaluates the effectiveness of the strategy and adjusts strategy parameters or switches to other strategies based on the evaluation results. For example, in the early stages of training, a more exploratory genetic algorithm may be used, while in the later stages, a more refined Bayesian optimization may be used.

[0119] 4.2 Distributed Collaboration Mechanism

[0120] To address collaborative optimization issues in a cluster environment, the online automatic tuning and compilation system for large model training implements a distributed collaborative mechanism. This mechanism includes:

[0121] Global state synchronization: maintains the global optimization state within the cluster;

[0122] Configuration consistency guarantee: Distributed transactions are used to ensure the consistency of configuration updates;

[0123] Load balancing: dynamically adjust task allocation based on node performance;

[0124] Fault tolerance: Implement automatic recovery mechanism when node fails.

[0125] 4.3 Real-time Performance Modeling

[0126] The online automatic tuning and compilation system for large model training builds a real-time performance evaluation model that can:

[0127] Quickly respond to performance changes: Real-time performance evaluation is achieved through streaming data processing;

[0128] Predict performance trends: predict performance change trends based on historical data;

[0129] Identify performance bottlenecks: Automatically locate key factors affecting performance;

[0130] Generate optimization suggestions: Output optimization directions based on the performance model.

[0131] On the other hand, the present invention also provides an online automatic tuning and compilation method for large model training, which is implemented by the above-mentioned online automatic tuning and compilation system for large model training. The method specifically includes:

[0132] The data collector collects performance indicator data during the large model training process as an optimization target;

[0133] After the large model training is started, the automatic tuning client is responsible for sending the performance indicator data collected by the data collector to the automatic tuning server by calling the communication manager;

[0134] After receiving the automatic tuning request from the automatic tuning client, the automatic tuning server initializes the hyperparameter space according to different tuning strategies, and adjusts the hyperparameter configuration required for the next round of iterative training as the updated hyperparameter configuration based on the performance indicator data fed back by the automatic tuning client, and calls the communication manager to send it to the automatic tuning client;

[0135] Through the configuration manager, a switch for starting online automatic tuning is provided to the user, and the updated hyperparameter configuration received by the automatic tuning client is configured into the Triton fusion operator in the large model framework;

[0136] Register the Triton fusion operator to the large model training framework through the Triton operator register and receive the updated hyperparameter configuration;

[0137] The communication manager is responsible for implementing data transmission between the automatic tuning client and the automatic tuning server using a communication protocol;

[0138] Through the automatic tuning scheduler, the hyperparameter configuration is passed to multiple GPU servers in the cluster that participate in large model training.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An online automatic tuning and compilation system for large model training, characterized by: Includes automatic tuning client, automatic tuning server, configuration manager, Triton operator registrar, communication manager, automatic tuning scheduler and data collector; The data collector is used to collect performance indicator data during the large model training process as an optimization target; The automatic tuning client is used to call the communication manager to send the performance indicator data collected by the data collector to the automatic tuning server after the large model training is started; The automatic tuning server is used to initialize the hyperparameter space according to different tuning strategies after receiving the automatic tuning request from the automatic tuning client, and adjust the hyperparameter configuration required for the next round of iterative training as the updated hyperparameter configuration based on the performance indicator data fed back by the automatic tuning client, and call the communication manager to send it to the automatic tuning client; The configuration manager is used to provide the user with a switch for starting online automatic tuning, and configure the updated hyperparameter configuration received by the automatic tuning client into the Triton fusion operator in the large model framework; The Triton operator register is used to register the Triton fusion operator to the large model training framework and receive updated hyperparameter configurations; The communication manager is responsible for implementing data transmission between the automatic tuning client and the automatic tuning server using a communication protocol; The automatic tuning scheduler is used to pass hyperparameter configurations to multiple GPU servers in the cluster that participate in large model training.

2. The online automatic tuning and compilation system for large model training according to claim 1 is characterized in that: The data collector includes a performance index collection module, a data preprocessing module and an index aggregation module; The performance indicator collection module is used to collect performance indicator data of each training node in the local training process in real time, and the performance indicator data includes but is not limited to computing throughput, memory usage, GPU utilization and communication overhead; The data preprocessing module is used to clean and standardize the collected performance indicator data to obtain processed performance indicator data; The indicator aggregation module is used to aggregate and compile the processed performance indicator data of different training nodes and send it to the automatic tuning client.

3. The online automatic tuning and compilation system for large model training according to claim 1 is characterized in that: The automatic tuning client includes a data acquisition interface module, an indicator preprocessing module, a configuration update module and a communication interface module; The data acquisition interface module is used to receive performance indicator data from the data collector; The indicator preprocessing module is used to detect outliers on the performance indicator data from the data collector, remove noise data, and then standardize the data of different dimensions to obtain standardized performance indicator data; finally, perform time series aggregation on the standardized performance indicator data to obtain time series aggregated performance indicator data; The configuration update module is responsible for receiving the updated hyperparameter configuration sent by the automatic tuning server and applying the updated hyperparameter configuration to the local training process; The communication interface module is used to send the performance indicator data after time series aggregation to the automatic tuning server through the communication manager.

4. The online automatic tuning and compilation system for large model training according to claim 1 is characterized in that: The automatic tuning server includes a hyperparameter space initialization module, an indicator analysis module, an optimization strategy selection module and a configuration optimization module; The hyperparameter space initialization module is used to initialize the hyperparameter space according to different tuning strategies after receiving the automatic tuning request from the automatic tuning client; The indicator analysis module is used to analyze the performance indicator data from the automatic tuning client and build a performance evaluation model; the performance evaluation model comprehensively considers computing efficiency, memory efficiency, and communication efficiency to generate a comprehensive performance score for the current configuration; The optimization strategy selection module is used to dynamically select an appropriate optimization strategy based on the characteristics of the training scenario and the optimization goal and the generated comprehensive performance score during the local training process of the large model; the optimization strategy includes a Bayesian optimization strategy, a reinforcement learning strategy, or a genetic algorithm strategy; The configuration optimization module adjusts the hyperparameter configuration required for the next round of iterative training as the updated hyperparameter configuration by selecting a suitable optimization strategy and calls the communication manager to send it to the automatic tuning client.

5. The online automatic tuning and compilation system for large model training according to claim 1 is characterized in that: The configuration manager includes a configuration storage module, a configuration update module, a version control layer and a monitoring layer; The configuration storage module is used to use a distributed database, which not only stores the optimal solution exceeding the configuration in the previous round of iterative training, but also stores the corresponding relationship between historical configuration records and performance data; The configuration update module is used to configure the updated hyperparameter configuration received by the automatic tuning client into the Triton fusion operator in the large model framework after the user turns on the switch to start online automatic tuning; The version control layer is used to manage the version evolution process of the configuration, and implements a rapid rollback mechanism for the configuration, so that it can be restored to a stable version in a timely manner when performance anomalies are found; The monitoring layer is used to continuously monitor the effectiveness and performance impact of the configuration, collect feedback information from each node, evaluate the effect of the configuration update in real time, and notify the system administrator in a timely manner when a configuration anomaly is found.

6. The online automatic tuning and compilation system for large model training according to claim 1 is characterized in that: The communication manager uses HTTP or gRPC protocol to realize data transmission.

7. The online automatic tuning and compilation system for large model training according to claim 1, characterized in that: The Triton operator register includes a compilation module, a registration module, an optimization module and a verification module; The compilation module is used to compile the Triton fusion operator into a binary file; The registration module is used to register the compiled Triton fusion operator into the large model training framework; The optimization module is used to optimize the compiled Triton fusion operator; The verification module is used to verify the correctness of the compiled Triton fusion operator registered to the large model training framework.

8. The online automatic tuning and compilation system for large model training according to claim 1 is characterized in that: The automatic tuning scheduler includes a load balancing module, a configuration synchronization module and a status monitoring module; The load balancing module is used to dynamically allocate tuning tasks according to the load status of each GPU node; The configuration synchronization module is used to ensure the consistency of hyperparameter configurations of all nodes in the cluster; The status monitoring module is used to monitor the operating status and optimization progress of each node in real time.

9. The online automatic tuning and compilation system for large model training according to claim 3 is characterized in that: The automatic tuning client also includes an anomaly detection module and a log recording module; The anomaly detection module is used to monitor anomalies during the large model training process and notify the user when an anomaly is detected; The logging module is used to record key operations and performance changes during the tuning process to facilitate user auditing and system maintenance.

10. An online automatic tuning and compilation method for large model training, characterized in that: The method is implemented based on the online automatic tuning and compilation system for large model training according to any one of claims 1 to 9, and specifically includes: The data collector collects performance indicator data during the large model training process as an optimization target; After the large model training is started, the automatic tuning client is responsible for sending the performance indicator data collected by the data collector to the automatic tuning server by calling the communication manager; After receiving the automatic tuning request from the automatic tuning client, the automatic tuning server initializes the hyperparameter space according to different tuning strategies, and adjusts the hyperparameter configuration required for the next round of iterative training as the updated hyperparameter configuration based on the performance indicator data fed back by the automatic tuning client, and calls the communication manager to send it to the automatic tuning client; Through the configuration manager, a switch for starting online automatic tuning is provided to the user, and the updated hyperparameter configuration received by the automatic tuning client is configured into the Triton fusion operator in the large model framework; Register the Triton fusion operator to the large model training framework through the Triton operator register and receive the updated hyperparameter configuration; The communication manager is responsible for implementing data transmission between the automatic tuning client and the automatic tuning server using a communication protocol; Through the automatic tuning scheduler, the hyperparameter configuration is passed to multiple GPU servers in the cluster that participate in large model training.

Images