Parallel-strategy search method and apparatus

Abstract

A parallel-strategy search method and apparatus, which are used for improving the flexibility of parallel-strategy search. The method comprises: a parallel-strategy search system acquiring a plurality of parallel strategies, each of the plurality of parallel strategies being used for indicating a parallel manner for training an AI model; providing an input interface, the input interface being used for acquiring input content, the input content being used for indicating at least one strategy screening rule, and the at least one strategy screening rule being used for screening the plurality of parallel strategies; on the basis of at least one strategy screening rule, screening the plurality of parallel strategies, so as to determine a plurality of candidate parallel strategies among the plurality of parallel strategies; and evaluating the plurality of candidate parallel strategies to determine a target parallel strategy, the target parallel strategy being a parallel strategy to be used for training the AI model.

Description

A search method and apparatus for a parallel strategy

[0001] This application claims priority to Chinese Patent Application No. 202411604631.4, filed on November 11, 2024, entitled “A Search Method and Apparatus for a Parallel Strategy,” the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of cloud computing, and more particularly to a method and apparatus for searching parallel strategies. Background Technology

[0003] In recent years, with the development of artificial intelligence technology, large models have easily exceeded trillions of parameters. The traditional single-machine single-card mode can no longer meet the requirements for training ultra-large models. Therefore, it is necessary to use single-machine multi-card or even multi-machine multi-card distributed model training to improve the training performance of large models.

[0004] Currently, to better utilize the computing power and memory space of computing devices for distributed training of large models, it is necessary to adopt appropriate parallel strategies, such as data parallelism, tensor parallelism, pipelined parallelism, and recomputation. In current cloud-based large model training solutions, training platforms often use automatic parallelism to train large models. Automatic parallelism means that given a model and training cluster devices, the system can automatically search for an optimal or relatively optimal parallel strategy for the user to train the large model.

[0005] In current automatic parallel solutions, the system needs to search a large number of parallel strategy options to select the globally optimal parallel strategy. However, when such automatic parallel solutions are provided to users as cloud services, the search system often has to rely on some hard-coded rules built into the cloud service to search for parallel strategies. Therefore, once the structural parameters of the model or the type and number of device clusters change, searching for parallel strategies based on these fixed hard-coded rules results in poor flexibility in searching for parallel strategies and makes it impossible to find the globally optimal parallel strategy. Summary of the Invention

[0006] This application provides a method for searching parallel strategies. The parallel strategy search system in this method provides an input interface for users to input strategy filtering rules, thereby enabling flexible adjustment of the strategy filtering rules and searching for parallel strategies based on these rules, thus improving the flexibility of parallel strategy search. This application also provides a parallel strategy search device, computing device, computing device cluster, computer-readable storage medium, and computer program product corresponding to this parallel strategy search method.

[0007] In a first aspect, embodiments of this application provide a method for searching parallel strategies. This method can be executed by a computing device, a component of the computing device (such as a processor, chip, or chip system), or a logic module or software capable of implementing all or part of the functions of the computing device. The method provided in the first aspect includes: a parallel strategy search system acquiring multiple parallel strategies, each of which indicates a parallel method for training an AI model. The parallel strategy search system provides an input interface for acquiring input content indicating at least one strategy filtering rule for filtering the multiple parallel strategies. The parallel strategy search system filters the multiple parallel strategies based on the at least one strategy filtering rule to determine multiple candidate parallel strategies. The parallel strategy search system evaluates the multiple candidate parallel strategies to determine a target parallel strategy, which is the parallel strategy used to train the AI ​​model.

[0008] In this embodiment, the user can flexibly set policy filtering rules based on the input interface provided by the parallel policy search system, and filter the parallel policies for training large models based on the policy filtering rules, and further evaluate and determine the target parallel policy. Compared with the existing solutions where the search system searches based on fixed hard-coded rules, the parallel policy search method provided in this embodiment can flexibly input policy filtering rules to search for parallel policies, thereby improving the search flexibility of the parallel policy search method and further improving the search performance of the target parallel policy.

[0009] In one possible implementation, at least one strategy selection rule includes one or more of the following: parallelization setting rules, parallel strategy relationship setting rules, and model splitting rules. The parallelization setting rules indicate the setting rules for a single parallel mode in the parallel strategy; the parallel strategy relationship setting rules indicate the setting rules for the relationships between multiple parallel modes in the parallel strategy; and the model splitting rules indicate the setting rules for model splitting points.

[0010] In the embodiments of this application, the strategy selection rules include multiple types of rules. These multiple types of strategy selection rules can indicate the parallelism and model segmentation related settings in the parallel strategy. By pre-screening the parallel strategy based on the strategy selection rules, the number of candidate parallel strategies can be reduced, thereby improving the search performance of the target parallel strategy and further improving the training efficiency of large model training.

[0011] In one possible implementation, during the process of the parallel strategy search system filtering multiple parallel strategies based on at least one strategy filtering rule, if the first parallel strategy among the multiple parallel strategies meets the constraints in the at least one strategy filtering rule, then the first parallel strategy is determined as a candidate parallel strategy; or, if the second parallel strategy among the multiple parallel strategies does not meet the constraints in the at least one strategy filtering rule, then the second parallel strategy is skipped.

[0012] The parallel strategy search system in this application embodiment can determine parallel strategies that meet the rules as candidate parallel strategies based on the strategy filtering rules, and skip parallel strategies that do not meet the rules, thereby improving the feasibility of filtering multiple parallel strategies based on the strategy filtering rules.

[0013] In one possible implementation, the input content includes one or more of the following: configuration files, rule codes, and graphical front-end configuration information. The configuration files, rule codes, and graphical front-end configuration information can all be input content from the input interface, and all of them can be input content generated based on expert knowledge.

[0014] The input content in this application embodiment can be of various types, including configuration files, rule codes, and graphical front-end configuration information, thereby improving the feasibility of the input content.

[0015] In one possible implementation, the input includes graphical front-end configuration information. The parallel strategy search system provides a graphical user interface (GUI) to display the configuration information, which includes one or more of the following: rule identifier ID, rule description, and rule content. For example, a user can input the configuration information through the GUI, which is used to display the configuration information. In this embodiment, the parallel strategy search system provides a graphical user interface, allowing users to input user-defined content, thereby providing the parallel strategy search system with user-defined content corresponding to expert knowledge. The user interacts with the parallel strategy search system through the graphical user interface, enhancing the flexibility of generating strategy filtering rules.

[0016] In one possible implementation, the parallel strategy search system receives rule processing requests, including processing requests triggered by user modifications to input content in the GUI. Based on the rule modification request, the parallel strategy search system performs one or more of the following operations on at least one strategy filtering rule: adding a strategy filtering rule, modifying a strategy filtering rule, and deleting a strategy filtering rule. In this embodiment, the user can input graphical front-end configuration information through the graphical user interface (GUI), which can also be converted into configuration files or rule codes. These configuration files or rule codes can also be converted into graphical front-end configuration information and displayed in the graphical user interface; no specific limitation is made.

[0017] In this embodiment, users can add, modify, and delete user input content through the graphical user interface provided by the parallel strategy search system, thereby improving the flexibility of generating strategy filtering rules.

[0018] In one possible implementation, after the parallel strategy search system obtains user input, during the process of generating strategy filtering rules based on the user input, the parallel strategy search system parses and preprocesses the user input and outputs strategy filtering rules. The parsing process of the parallel strategy search system includes parsing rules for parallel configuration based on configuration files, and the preprocessing process of the parallel strategy search system includes cleaning, merging, and improving the parsed rules.

[0019] The parallel strategy search system in this embodiment parses and preprocesses user input, thereby cleaning up redundant strategy filtering rules, merging duplicate strategy filtering rules, and improving strategy filtering rules, thus enhancing the accuracy of strategy filtering rules.

[0020] In one possible implementation, after the parallel strategy search system determines candidate parallel strategies and searches for them, the parallel strategy search system searches for candidate parallel strategies based on a cost model to determine a target parallel strategy. The cost model is used to predict the search performance of the candidate parallel strategies, and the performance of the target parallel strategy is higher than the performance of other candidate parallel strategies among multiple candidate parallel strategies excluding the target parallel strategy.

[0021] The parallel strategy search system in this embodiment can search for candidate parallel strategies based on the cost model to determine the target parallel strategy. Since the cost model can predict the search performance of candidate parallel strategies, the parallel strategy search system improves the training performance of large model parallel training by searching for candidate parallel strategies based on the cost model.

[0022] In one possible implementation, the model parameters of the cost model are determined based on one or more of the following information: system information, task information, and historical information. System information is determined based on hardware detection, task information is determined based on task code, and historical information is determined based on the execution records of historical parallel strategies. System information includes machine ID, processor model, computing power, memory, GPU memory, and bandwidth, etc.; task information includes model information, hyperparameters, and cluster state information related to the large model; and historical information includes historical system information and historical task information collected during the training process.

[0023] The parallel strategy search system in this embodiment can determine the model parameters of the cost model based on a variety of information. Since the collected information includes system information, task information and historical information, the performance of the parallel strategy is predicted based on the cost model, which improves the prediction accuracy of the cost model in predicting the performance of the parallel strategy.

[0024] In one possible implementation, before the parallel strategy search system filters parallel strategies based on strategy filtering rules, the parallel strategy search system generates an initial parallel strategy based on the information collected by the information acquisition module. The information collected includes one or more of the following: system information, task information, and historical information.

[0025] The parallel strategy search system in this application embodiment can generate an initial parallel strategy based on collected information before filtering parallel strategies based on strategy filtering rules, thereby improving the feasibility of the solution.

[0026] In one possible implementation, after the parallel policy search system generates candidate parallel policies, the parallel policy search system further determines the target parallel policy based on the search algorithm and the cost model. The search algorithm can find the optimal or near-optimal solution in the parallel policy space, and the cost model is used to evaluate the performance and cost of these solutions. The search algorithms include dynamic programming (DP), integer linear programming (ILP), mixed integer linear programming (MILP), and machine learning.

[0027] After generating candidate parallel strategies, the parallel strategy search system in this embodiment can further search for target parallel strategies based on the search algorithm, thereby improving the performance of the parallel strategies.

[0028] In one possible implementation, after the parallel strategy search system determines the target parallel strategy, during the execution of the target parallel strategy based on the computing device cluster, the parallel strategy search system distributes the training tasks to different computing units according to the target parallel strategy distribution, and then executes the training tasks. During the execution of the training tasks, the parallel strategy search system uses an analyzer to collect system information and task information of the large model training, thereby generating historical information.

[0029] The parallel strategy search system in this application embodiment executes the target parallel strategy based on a cluster of computing devices, thereby improving the training efficiency of large model training.

[0030] Secondly, embodiments of this application provide a parallel strategy search apparatus, comprising an acquisition unit and a processing unit. The acquisition unit acquires multiple parallel strategies, each of which indicates a parallel training method for an AI model. The acquisition unit also provides an input interface for acquiring input content indicating at least one strategy filtering rule for filtering the multiple parallel strategies. The processing unit filters the multiple parallel strategies based on the at least one strategy filtering rule to determine multiple candidate parallel strategies. The processing unit further evaluates the multiple candidate parallel strategies to determine a target parallel strategy, which is the parallel strategy used to train the AI ​​model.

[0031] In one possible implementation, at least one strategy filtering rule includes one or more of the following: parallelization setting rules, parallel strategy relationship setting rules, and model splitting rules. The parallelization setting rules are used to indicate the setting rules for a single parallel mode in the parallel strategy. The parallel strategy relationship setting rules are used to indicate the setting rules for the relationship between multiple parallel modes in the parallel strategy. The model splitting rules are used to indicate the setting rules for model splitting points.

[0032] In one possible implementation, the processing unit is further configured to determine the first parallel strategy as a candidate parallel strategy if the first parallel strategy among the plurality of parallel strategies meets the constraints in at least one strategy filtering rule. Alternatively, the second parallel strategy among the plurality of parallel strategies is skipped if it does not meet the constraints in at least one strategy filtering rule.

[0033] In one possible implementation, the input includes one or more of the following: configuration files, rule codes, and graphical front-end configuration information.

[0034] In one possible implementation, the input content includes graphical front-end configuration information, and the acquisition unit is also used to provide a graphical user interface (GUI) for displaying the graphical front-end configuration information. The graphical front-end configuration information includes one or more of the following: rule identifier ID, rule description, and rule content.

[0035] In one possible implementation, the acquisition unit is further configured to receive rule processing requests, including processing requests triggered by user modifications to input content in the GUI. The processing unit is further configured to perform one or more of the following operations on at least one policy filtering rule based on the rule modification request: adding a policy filtering rule, modifying a policy filtering rule, and deleting a policy filtering rule.

[0036] In one possible implementation, the processing unit is specifically used to evaluate candidate parallel strategies based on a cost model and determine a target parallel strategy. The cost model is used to predict the performance of the candidate parallel strategies, and the performance of the target parallel strategy is higher than the performance of other candidate parallel strategies among a plurality of candidate parallel strategies.

[0037] In one possible implementation, the model parameters of the cost model are determined based on one or more of the following information: system information, task information, and historical information.

[0038] Thirdly, embodiments of this application provide a computing device including a processor coupled to a memory. The processor stores instructions, which, when executed by the processor, cause the computing device to perform the method described in the first aspect or any possible implementation thereof.

[0039] Fourthly, embodiments of this application provide a computing device cluster, which includes one or more computing devices. Each computing device includes a processor coupled to a memory. The processor is used to store instructions, which, when executed by the processor, cause the computing device cluster to perform the method described in the first aspect or any possible implementation thereof.

[0040] Fifthly, embodiments of this application provide a computer-readable storage medium having instructions stored thereon, which, when executed, cause a computer to perform the method described in the first aspect or any possible implementation thereof.

[0041] Sixthly, embodiments of this application provide a computer program product including instructions that, when executed, cause a computer to implement the method described in the first aspect or any possible implementation thereof.

[0042] It is understood that the beneficial effects that can be achieved by any of the parallel strategies provided above, such as search devices, computing devices, computing device clusters, computer-readable media, or computer program products, can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0043] Figure 1a is a schematic diagram of the system architecture of a parallel strategy search system provided in an embodiment of this application;

[0044] Figure 1b is a schematic diagram of the system architecture of a cloud platform provided in an embodiment of this application;

[0045] Figure 2 is a flowchart illustrating a parallel strategy search method provided in an embodiment of this application;

[0046] Figure 3 is a schematic diagram of another parallel strategy search method provided in an embodiment of this application;

[0047] Figure 4 is a schematic diagram of a configuration file provided in an embodiment of this application;

[0048] Figure 5 is a schematic diagram of a rule code provided in an embodiment of this application;

[0049] Figure 6 is a schematic diagram of a graphical front-end configuration information provided in an embodiment of this application;

[0050] Figure 7 is a schematic diagram of another parallel strategy search method provided in an embodiment of this application;

[0051] Figure 8 is a schematic diagram of another parallel strategy search method provided in an embodiment of this application;

[0052] Figure 9 is a schematic diagram of a parallel strategy search device provided in an embodiment of this application;

[0053] Figure 10 is a schematic diagram of a computing device provided in an embodiment of this application;

[0054] Figure 11 is a schematic diagram of a computing device cluster provided in an embodiment of this application;

[0055] Figure 12 is a schematic diagram of another computing device cluster provided in an embodiment of this application. Detailed Implementation

[0056] This application provides a method and apparatus for searching parallel strategies, which improves the search flexibility of parallel strategies.

[0057] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0058] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0059] First, some of the terms used in the embodiments of this application are introduced to facilitate understanding of the technical solutions by those skilled in the art.

[0060] Automatic parallelism refers to the system automatically searching for parallel strategies based on given model information and given cluster device information to achieve maximum throughput for training large models. The given model information includes model parameters, training parameters, etc., and the given cluster device information includes chip specifications, quantity, and network topology, etc.

[0061] The cost model is used to measure the computation, memory, and communication overhead of different parallel strategies under given constraints, that is, the running cost of different parallel strategies.

[0062] Data parallelism refers to dividing a large model's dataset into multiple smaller subsets, each of which is independently assigned to a different computational unit, such as a graphics processing unit (GPU). Each computational unit holds a complete copy of the model and independently computes the gradients of its assigned subset of data.

[0063] Tensor parallelism (TP) refers to splitting tensors along a specific dimension and distributing them across different computational units, such as graphics processing units, for independent computation. The computational results are then aggregated to achieve parallel acceleration. Tensor parallelism mainly solves the problem of parallel computation within the model layer.

[0064] Pipeline parallelism (PP) is a parallel strategy that distributes different parts of a model (such as layers or stages) across different computational units, processing different batches of data simultaneously. In pipeline parallelism, the model is divided into multiple stages, each of which independently processes the forward and backward propagation of data on different computational units.

[0065] Sequence parallelism (SP) involves dividing a long input sequence into multiple shorter subsequences along the sequence dimension, and then distributing these subsequences across different computing units for parallel computation. Sequence parallelism can effectively reduce the computational burden on a single computing unit, improve overall training efficiency, and overcome the limitations of a single computing unit in terms of memory and computing power.

[0066] To make the technical solution of this application clearer and easier to understand, the system architecture of this application will be described below with reference to the accompanying drawings.

[0067] Please refer to Figure 1a, which is a schematic diagram of the system architecture of a parallel strategy search system provided in an embodiment of this application. In the example shown in Figure 1a, the parallel strategy search system 10 includes an information acquisition module 101, an expert knowledge processing module 102, a parallel strategy search module 103, and a strategy execution module 104. Since the parallel strategy search system 10 can automatically adjust the parallel strategy during the training of a large model, it can also be called a parallel training system or an automatic parallel system. The specific functions of each part of the parallel strategy search system 10 are described below.

[0068] The information acquisition module 101 is used to collect information from the user-defined large model, execution environment, and task records, and output the system information, task information, and historical information of the large model. The information acquisition module 101 includes a hardware detection submodule 1011, a model detection submodule 1012, and a historical information acquisition submodule 1013.

[0069] The hardware detection submodule 1011 collects system information based on hardware detection, including the number of nodes, processors, computing power, and memory in the large model training environment. The model detection submodule 1012 can collect task information based on the large model's task code. This task information includes model architecture, hyperparameters, and cluster status. The historical information collection submodule 1013 can collect historical information based on the large model's training records. This historical information includes information collected during historical training, such as historical system information and historical task information, including historical training settings, performance, and hardware requirements.

[0070] The expert knowledge processing module 102 receives user-submitted input, including constraint rules based on expert knowledge. The module also generates search rules for available parallel strategies based on the input. Expert knowledge guides the selection of parallel strategies based on experiential knowledge such as model characteristics, dataset size, and computational resources. User input can specifically be rule codes, such as domain-specific languages ​​(DSLs), configuration files, and graphical configuration information.

[0071] The expert knowledge processing module 102 includes a parsing submodule 1021 and a preprocessing submodule 1022. The parsing submodule 1011 parses the input content, including structured organization, syntactic analysis, and semantic analysis of the expert knowledge involved in the input content. Structured organization includes, for example, classification labeling and indexing. The preprocessing submodule 1022 preprocesses the parsed expert knowledge, including cleaning, merging, and improving the parsed expert knowledge.

[0072] The parallel policy search module 103 searches for parallel policies based on a search algorithm and a cost model, outputs a target parallel policy, and sends the target parallel policy to the policy execution module 104. The parallel policy search module 103 also filters parallel policies according to the policy filtering rules output by the expert knowledge processing module 102, thereby reducing the search space for parallel policies.

[0073] The parallel strategy search module 103 includes a filtering submodule 1031, a search submodule 1032, and a cost model submodule 1033. The filtering submodule 1031 filters parallel strategies based on strategy filtering rules, reducing the search space for parallel strategies. The search submodule 1032 evaluates, ranks, and filters parallel strategies according to a search algorithm, and selects a target parallel strategy based on the prediction results of the cost model. The cost model submodule 1033 predicts the performance of parallel strategies based on the cost model, thereby generating performance prediction results for the parallel strategies.

[0074] The strategy execution module 104 executes the target parallel strategy output by the parallel strategy search module 103 on the device cluster, such as a processor cluster or an accelerator cluster, and accelerators such as central processing units (CPUs), graphics processing units (GPUs), and neural network processing units (NPUs). The strategy execution module 104 is also used to collect and save performance data during the execution of the target parallel strategy, such as throughput, floating-point operations per second (TFLOPS), model FLOPs utilization (MFU), and video memory usage.

[0075] It is understood that the parallel strategy search system 10 in this application embodiment can be provided to users as a framework software package for deployment, or it can be provided to users as a cloud service for training large models, without any specific limitation.

[0076] Please refer to Figure 1b, which is a schematic diagram of an application scenario for a parallel strategy search system provided in an embodiment of this application. In the example shown in Figure 1b, the parallel strategy search system can serve as a cloud service provided to users as an artificial intelligence (AI) basic development platform. The AI ​​basic development platform is a Platform as a Service (PaaS) within the cloud platform. Based on the abundant basic resources and software capabilities possessed by public cloud service providers, the AI ​​basic development platform provides users with a software platform for building, training, and deploying auxiliary AI models.

[0077] In the example shown in Figure 1b, the interaction between the user and the AI ​​basic development platform mainly includes: the user logs into the cloud platform through a client webpage, selects and purchases cloud services of the AI ​​basic development platform in the cloud platform, and after purchase, the user can carry out the entire process of AI development based on the functions provided by the AI ​​basic development platform. For example, the user can purchase the automatic parallel cloud service to obtain the search function of parallel strategies during the AI ​​model training process.

[0078] In the example shown in Figure 1b, when a user develops and trains their own AI model on an AI infrastructure development platform, they can train the model based on the basic resources in the cloud service provider's data center. These basic resources include computing and storage resources provided by one or more servers. Therefore, when purchasing and using an AI infrastructure development platform, users primarily pay for the resources they use.

[0079] When the parallel strategy search system 10 is used as an automatic parallel cloud service, it can rely on expert knowledge to help the automatic parallel service quickly and stably search for the optimal parallel strategy under various model structures and sizes, and homogeneous or heterogeneous training clusters. At the same time, users and cloud service operators can more easily modify and optimize the parallel strategy search system through the interface of the AI ​​basic development platform.

[0080] Based on the search system 10 using the parallel strategy shown in Figure 1a, this application also provides a search method for the parallel strategy. The search method for the parallel strategy provided in this application will be described below with reference to embodiments.

[0081] Please refer to Figure 2, which is a flowchart illustrating a parallel strategy search method provided in an embodiment of this application. In the example shown in Figure 2, the method includes the following steps:

[0082] 201. Obtain multiple parallel strategies, each of which indicates how to train the AI ​​model in parallel.

[0083] The parallel strategy search system 10 acquires multiple parallel strategies, each of which indicates the parallel method for training the AI ​​model. These parallel strategies can include various different types of parallel methods, such as data parallelism, tensor parallelism (TP), pipelined parallelism (PP), and sequence parallelism (SP).

[0084] Specifically, during the process of acquiring multiple parallel strategies, the parallel strategy search system 10 generates an initial parallel strategy based on the information acquired by the information acquisition module 101. The acquired information includes one or more of the following: system information, task information, and historical information. The system information, task information, and historical information in this embodiment are described below.

[0085] System information includes information obtained by the information acquisition module 101 based on hardware detection, including machine ID, processor model, computing power, memory, video memory, and bandwidth. Hardware detection also includes performance detection; the information acquisition module 101 collects performance information based on performance detection. Performance metrics include P2P bandwidth under different settings, bandwidth of the AllReduce operation, computational communication overlap coefficient, and a small model training performance metric, such as iteration time, CPU computation time, GPU computation time, and video memory usage.

[0086] Please refer to Table 1, which is a schematic table of system information collected by an information collection module 101 according to an embodiment of this application. In the example shown in Table 1, the system information collected by the information collection module 101 includes machine ID, processor model, computing power, memory, video memory, and bandwidth. For example, the information collection module 101 collects the machine ID of "Rank 0" as 32, model as A100, computing power as 272.51 TFLOPs, memory as 1.7T, video memory as 80G, memory bandwidth as 23.52GB / s, and the bandwidth between "Rank 0" and "Rank 1" as 458.39GB / s.

[0087] Table 1

[0088] Please refer to Table 2, which is a schematic table illustrating the performance information collected by the information acquisition module 101 according to an embodiment of this application. In the example shown in Table 2, in order to measure the performance information of the hardware, the information acquisition module 101 executes various applications for measuring hardware performance on a real computing unit. In addition, in order to collect specific performance information, the information acquisition module 101 can deploy user-defined tasks or smaller variations on real hardware, pre-run several rounds of training iterations, and collect performance data.

[0089] In the example shown in Table 2, the performance information collected by the information acquisition module 101 includes P2P bandwidth, AllReduce bandwidth, overlap coefficient, and training performance metrics of the small model under different settings. The training performance metrics of the small model include iteration time and GPU memory usage. For example, the performance value corresponding to P2P bandwidth (4 nodes) is 7812.14, the performance value corresponding to AllReduce bandwidth (4 nodes) is 52.29, and the overlap coefficient is 2.05.

[0090] Table 2

[0091] Task information includes model information, hyperparameters, and cluster status information related to large models. Model information includes model parameters such as model architecture, number of model layers, and hidden layer size. Hyperparameters include learning rate (LR) and batch size. Cluster status information includes the number of nodes and the number of graphics processing units (GPUs).

[0092] Historical information includes historical system information and historical task information collected during the training process. Historical system information includes hardware requirements such as machine configuration and performance, including throughput, TFLOPs, MFU, and GPU memory usage. Historical task information includes historical model information, historical hyperparameters, and historical cluster state information.

[0093] 202. Provide an input interface for obtaining input content, which indicates at least one policy filtering rule for filtering multiple parallel policies.

[0094] The parallel strategy search system 10 provides an input interface for obtaining user input, which instructs the expert knowledge processing module 102 to generate the expert knowledge required for strategy filtering rules. The input includes one or more of the following: configuration files, rule codes, and graphical front-end configuration information.

[0095] In the training scenario of large models, the cloud platform where the parallel policy search system 10 is located can provide users with a parallel policy search interface. Users can send input content through this interface. The input content contains expert knowledge required for parallel policy search. This expert knowledge can be in various forms, such as the configuration file mentioned above.

[0096] Please refer to Figure 3, which is a schematic diagram of another parallel strategy search method provided in an embodiment of this application. In the example shown in Figure 3, the expert knowledge processing module 102 receives input content provided by the user. This input content includes expert knowledge related to parallel strategy search. The parallel strategy search module 103 generates strategy filtering rules based on this expert knowledge. It can filter parallel strategies for training large models under given hardware constraints. The given hardware constraints include, for example, a given memory budget and a given number of computing units. The parallel strategies include, for example, tensor parallelism (TP) at different granularities, pipelined parallelism (PP), data parallelism, zero-redundancy optimization (ZeRO), and recomputation.

[0097] In this embodiment, the parallel strategy search system 10 generates strategy filtering rules based on user input, including one or more of the following: parallelization setting rules, parallel strategy relationship setting rules, and model segmentation rules. The parallelization setting rules indicate the setting rules for a single parallel method within the parallel strategy; the parallel strategy relationship setting rules indicate the setting rules for the relationship between multiple parallel methods within the parallel strategy; and the model segmentation rules indicate the setting rules for model splitting points. The parallelization setting rules, parallel strategy relationship setting rules, and model segmentation rules are described below:

[0098] The parallelization setting rules are used to indicate the setting rules for a single parallel mode among the different parallelization methods involved in the parallelization strategy. The parallelization setting rules include one or more of the following: parallel mode selection rules and data transmission setting rules related to the parallel mode.

[0099] Parallelization configuration rules include rules for selecting parallelism methods within a parallelization strategy. These parallelism methods include one or more of the following: data parallelism, tensor parallelism (TP), pipelined parallelism (PP), and sequence parallelism (SP). For example, a parallelization configuration rule could be: Avoid tensor parallelism between computational units. It is understood that avoiding tensor parallelism between computational units can reduce the imbalance of computational load across different computational units and reduce communication overhead between them.

[0100] Parallelization settings rules also include data transmission settings rules related to parallelization methods in the parallelization strategy. For example, one parallelization setting rule is: the bandwidth between processes in the same tensor parallel TP group is greater than or equal to a threshold. Here, the same tensor parallel TP group refers to a group of processing units responsible for processing different parts of the same model layer or the same computational task. It can be understood that setting the bandwidth between processes in the same tensor parallel TP group to be greater than or equal to a threshold ensures that the data transmission latency between tensor parallel computational units meets the requirements, thereby guaranteeing the overall training speed.

[0101] The parallel strategy relationship setting rules include the rules for setting the relationships between different parallel methods in the parallel strategy. For example, an example of a parallel strategy relationship setting rule is: Tensor Parallelism (TP) includes Sequence Parallelism. It is understandable that setting Tensor Parallelism (TP) to include Sequence Parallelism can improve the training efficiency of large models in some scenarios. For example, when training models with very long sequences, using only Tensor Parallelism (TP) may not effectively solve the memory and computational limitations, while using only Sequence Parallelism (SP) may lead to low computational efficiency due to excessively fine sequence segmentation. Setting Tensor Parallelism (TP) to include Sequence Parallelism can fully utilize the advantages of both to achieve more efficient training.

[0102] Model partitioning rules include rules related to the selection of model partition points. For example, one model partitioning rule might be: select layers or modules that can independently perform forward and backward propagation as partition points. Another example is: select partition points that reduce data redundancy and redundant computation based on the data flow path within the model. Yet another example is: merge computationally low-cost layers with adjacent computationally high-cost layers; that is, do not set model partition points between computationally low-cost and computationally high-cost layers. It is understandable that merging computationally low-cost layers with adjacent computationally high-cost layers can balance the load between different computational units while reducing communication overhead.

[0103] The input content in this embodiment includes one or more of the following: configuration files, rule codes, and graphical front-end configuration information. That is, the parallel strategy search system 10 in this embodiment can receive various types of expert knowledge-related user input content through user interaction. These types of input content are described below.

[0104] In one possible implementation, the user input can be configured into a configuration file. The configuration file is used to define various parameters and configuration information during the training process. The configuration file can be a text file provided by the user. For example, the configuration file can be in the format of JavaScript object notation (JSON), YAML, initialization (INI), etc.

[0105] Please refer to Figure 4, which is a schematic diagram of a configuration file provided in an embodiment of this application. In the example shown in Figure 4, the configuration file is a JSON file. This configuration file can define rules for the range of values ​​of parallel strategy parameters. For example, a rule named "PP-value-range" can be set in this configuration file. This rule can restrict the value range for pipeline parallelism, described as "Restrict the value range for pipeline parallelism". This rule defines the value of the "pipeline_model_parallel_size" parameter as a specific set of values ​​"[1,2,4,8,16,32,64,128,256]".

[0106] In one possible implementation, user input can also be rule code. Rule code refers to code that directly defines strategy filtering rules in the source code. For example, users can directly define parallelization setting rules, parallel strategy relationship setting rules, and model splitting rules through rule code. The defined rules can be commented out in the rule code. For example, a rule can be commented as a hard rule or a soft rule. A hard rule is a rule with mandatory constraints, while a soft rule is a preferred or recommended rule.

[0107] Please refer to Figure 5, which is a schematic diagram of a rule code provided in an embodiment of this application. In the example shown in Figure 5, the user defines three rules through the rule code. For example, the first rule is "sp:-tp(k), X>1." This rule code indicates that when the tensor parallel processing of the system is activated or used, the system is forced to use sequential parallelism to perform certain operations or calculations, thereby improving efficiency and reducing resource conflicts. In addition, the user can comment out this rule as a hard rule in the rule code using "%!enforce".

[0108] In the example shown in Figure 5, the second rule is "same_tp_group(X,Y):-node_id(X,NX),node id(Y,NY),NX=NY." This rule code indicates that when partitioned data is allocated to the same compute node, it is placed into the same tensor parallel group, thereby improving the utilization of computing resources. Furthermore, users can comment this rule as a soft rule using "%!recommend".

[0109] In the example shown in Figure 5, the third rule is “flops_threshold(R):-flops_delta(D),flops_total(T),num_layers(L),R is(1+D)*T / L.…”. This rule code means that operators with smaller computational cost will always be merged with the previous operation, thereby improving the utilization of computing resources.

[0110] In one possible implementation, the user input can also be rule code graphical front-end configuration information. In this embodiment, the parallel strategy search system provides a graphical user interface (GUI) to the user. The user can input graphical front-end configuration information through the GUI, which includes one or more rules corresponding to expert knowledge.

[0111] In one possible implementation, users can also add, modify, and delete policy filtering rules through a graphical user interface (GUI). Specifically, the parallel policy search system 10 receives rule processing requests, including processing requests triggered by user modifications to input content in the GUI. Based on the rule modification request, the parallel policy search system 10 performs one or more of the following operations on at least one policy filtering rule: adding a policy filtering rule, modifying a policy filtering rule, and deleting a policy filtering rule.

[0112] It is understandable that users can input graphical front-end configuration information through the graphical user interface (GUI), which can be converted into the above configuration file or rule code. The above configuration file or rule code can also be converted into graphical front-end configuration information and displayed in the graphical user interface. No specific limitation is made.

[0113] Please refer to Figure 6, which is a schematic diagram of a graphical front-end configuration information provided in an embodiment of this application. In the example shown in Figure 6, the graphical user interface (GUI) displays one or more rules corresponding to expert knowledge, and the displayed content includes rule ID, rule description, rule code, and rule permissions. For example, the rule corresponding to rule ID "0" has the rule description "TP entails SP", the rule code "if TP>1then AP=true", and the rule permission "always".

[0114] In the example shown in Figure 6, users can also add rules through the graphical user interface (GUI). For example, a user can create a new rule with the rule ID "11" and the rule description "Avoid activation checkpointing, if not absolutely necessary" and add it to the rule list through the GUI.

[0115] It should be noted that the user input content in this embodiment can also be a combination of the above-mentioned configuration files, rule codes and graphical front-end configuration information, and there is no specific limitation.

[0116] After the expert knowledge processing module 102 of the parallel strategy search system 10 obtains the input content, it generates strategy filtering rules based on the user input content. Specifically, the expert knowledge processing module 102 parses and preprocesses the user input content and outputs strategy filtering rules. The output strategy filtering rules are a data structure available to the parallel strategy search module 103.

[0117] The process of parsing user input by the expert knowledge processing module 102 includes parsing rules for parallel configuration based on the configuration file. The preprocessing process of the expert knowledge processing module 102 includes cleaning, merging and improving the parsed rules. For example, the expert knowledge processing module 102 cleans up redundant rules and merges duplicate rules. For another example, the expert knowledge processing module 102 infers new rules from the rule dataset to improve the rules.

[0118] Please refer to Figure 3. In the example shown in Figure 3, after receiving user input, the expert knowledge processing module 102 of the parallel strategy search system 10 parses the user input to obtain strategy filtering rules. For example, the expert knowledge processing module 102 parses one or more strategy filtering rules based on the user-input configuration file. The expert knowledge processing module 102 further preprocesses the parsed strategy filtering rules, such as cleaning up redundant rules, merging duplicate rules, and improving the rules based on the current rules.

[0119] 203. Based on at least one strategy screening rule, screen multiple parallel strategies to determine candidate parallel strategies among the multiple parallel strategies.

[0120] After the parallel strategy search system 10 generates strategy filtering rules, it filters parallel strategies based on the strategy filtering rules to determine candidate parallel strategies. The strategy filtering rules are used to evaluate parallel strategies based on expert knowledge, and the parallel strategies are used to indicate the parallel mode and parallel parameter settings of the large model being trained.

[0121] In one possible implementation, the parallel strategy search system 10 filters parallel strategies based on strategy filtering rules. If a first parallel strategy among multiple parallel strategies meets the constraints in at least one strategy filtering rule, then the first parallel strategy is determined as a candidate parallel strategy; or, if a second parallel strategy among multiple parallel strategies does not meet the constraints in at least one strategy filtering rule, then the second parallel strategy is skipped. That is, the parallel strategy search system 10 filters all parallel strategies based on strategy filtering rules, takes parallel strategies that meet the strategy filtering rules as candidate parallel strategies, and skips parallel strategies that do not meet the strategy filtering rules.

[0122] Please refer to Figure 7, which is a schematic diagram of another parallel strategy search method provided in an embodiment of this application. In the example shown in Figure 7, during the search for parallel strategies, the parallel strategy search system 10 filters the initial parallel strategies based on strategy filtering rules to select candidate parallel strategies, wherein the number of candidate parallel strategies is less than the number of initial parallel strategies. For example, the initial parallel strategies before the parallel strategy search system 10 performs filtering are strategy 1, strategy 2, ..., strategy n, and the candidate strategies after the parallel strategy search system 10 filters based on the strategy filtering rules are strategy 61, strategy 152, ..., strategy m.

[0123] Please refer to Figure 8, which is a schematic diagram of another parallel strategy search method provided in an embodiment of this application. In the example shown in Figure 8, during the process of the parallel strategy search system 10 filtering parallel strategies based on strategy filtering rules, the parallel strategy search system 10 traverses the parallel strategies. If a parallel strategy meets the strategy filtering rules, the parallel strategy search system 10 will take the parallel strategy that meets the strategy filtering rules as a candidate parallel strategy. If a parallel strategy does not meet the strategy filtering rules, the parallel strategy search system 10 will skip the parallel strategy that does not meet the strategy filtering rules.

[0124] 204. Evaluate multiple candidate parallel strategies and determine the target parallel strategy, which is the parallel strategy used to train the AI ​​model.

[0125] After the parallel strategy search system 10 screens and determines candidate search strategies, it further evaluates the candidate parallel strategies based on the cost model to determine the target parallel strategy. The cost model is used to predict the performance of the candidate parallel strategies. The performance of the candidate parallel strategies includes execution time, resource utilization, and communication resource overhead. The lower the execution time and communication resource overhead, the higher the performance of the parallel strategy. The higher the resource utilization, the better the performance of the parallel strategy.

[0126] In the embodiments of this application, the cost model can be of different types. Different types of cost models can predict different performance indicators. The performance of the target parallel strategy determined by the parallel strategy search system 10 is higher than the performance of other candidate parallel strategies among multiple candidate parallel strategies other than the target parallel strategy.

[0127] It should be noted that during the process of evaluating candidate parallel strategies based on the cost model, the parallel strategy search system 10 can search for candidate parallel strategies under given conditions based on the search algorithm, and at the same time predict the performance of each search result based on the cost model. Finally, the parallel strategy search system 10 determines the candidate parallel strategy with the best predicted performance among the search results as the target parallel strategy.

[0128] In the above search process, the search algorithm can find multiple solutions in the search space of candidate parallel strategies, and the cost model is used to evaluate the performance and cost of these solutions. The optimal or near-optimal solution is the target parallel strategy.

[0129] Understandably, since the parallel strategy search system 10 has already screened the parallel strategies based on the strategy screening rules before searching for candidate parallel strategies based on the search algorithm, the search space of parallel strategies is reduced and the search efficiency based on the search algorithm is improved.

[0130] The search algorithms in this application include dynamic programming (DP), integer linear programming (ILP), mixed-integer linear programming (MILP), and machine learning, including deep learning (DL) and reinforcement learning (RL).

[0131] Please refer to Figure 7. In the example shown in Figure 7, after the parallel strategy search system 10 selects candidate parallel strategies based on the strategy selection rules, it further searches for the candidate parallel strategies based on the search algorithm and cost model to obtain the target parallel strategy. For example, during the process of searching for candidate parallel strategies based on the search algorithm, the parallel strategy search system 10 needs to clarify the objective and constraints of parallel training. The objective may be, for example, minimizing training time, maximizing resource utilization, and reducing communication overhead. The constraints may involve limitations on hardware resources such as the amount of computation, memory limits, and network bandwidth.

[0132] In the example shown in Figure 7, after defining the objective and constraints of parallel training, the parallel policy search system 10 iteratively searches for the optimal solution in the parallel policy search space with the search algorithm. In each iteration, the search algorithm generates a new set of parallel policies as candidate solutions and evaluates these candidate solutions using a cost model. The evaluation results guide the next iteration of the search algorithm to move closer to a better solution until the parallel policy search system 10 outputs the target parallel policy.

[0133] Please refer to Figure 8. In the example shown in Figure 8, the parallel strategy search system 10 further evaluates the candidate parallel strategies based on the cost model. Specifically, the parallel strategy search system 10 calculates the performance of the candidate parallel strategies based on the cost model and sorts the candidate parallel strategies according to their performance. The parallel strategy search system 10 selects the target parallel strategy according to the sorting result. The performance of the target parallel strategy is higher than the performance of other candidate parallel strategies among the multiple candidate parallel strategies.

[0134] In one possible implementation, before the parallel strategy search system 10 searches for candidate parallel strategies based on the cost model, the parallel strategy search system 10 needs to construct a cost model, wherein the model parameters of the cost model are determined based on one or more of the following information: system information, task information, and historical information, wherein the system information is determined based on hardware detection, the task information is determined based on task code, and the historical information is determined based on the execution records of historical parallel strategies.

[0135] In one possible implementation, after determining the target parallel strategy, the parallel strategy search system 10 executes the target parallel strategy based on the computing device cluster. Specifically, the strategy execution module 104 of the parallel strategy search system 10 distributes the training tasks to different computing units according to the parallel strategy distribution, and then executes the training tasks. During the execution of the training tasks, the parallel strategy search system 10 uses a profiler to collect various system and task information to obtain historical information.

[0136] As can be seen from the above embodiments, the parallel strategy search system in this application embodiment filters the parallel strategies for AI model training based on the strategy filtering rules corresponding to the user input content, thereby searching for candidate parallel strategies that meet the strategy filtering rules, thus reducing the number of parallel strategies to be searched. Since the parallel strategy search system provided in this application embodiment can flexibly provide input content, it improves the search flexibility and search efficiency of the parallel strategy search method, and further improves the performance of the target parallel strategy and the efficiency of large model parallel training.

[0137] Based on the above method embodiments, this application also provides a parallel strategy search device, which is described in detail below.

[0138] Please refer to Figure 9, which is a schematic diagram of the structure of a parallel strategy search device provided in an embodiment of this application. In the example shown in Figure 9, the parallel strategy search device 900 is used to implement the various steps performed by the parallel strategy search system in the above embodiments. The parallel strategy search device 900 includes an acquisition unit 901 and a processing unit 902.

[0139] The acquisition unit 901 acquires multiple parallel strategies, each indicating a parallel training method for the AI ​​model. The acquisition unit 901 also provides an input interface for acquiring input content indicating at least one strategy selection rule for filtering the multiple parallel strategies. The processing unit 902 filters the multiple parallel strategies based on the at least one strategy selection rule to determine multiple candidate parallel strategies. The processing unit 902 further evaluates the multiple candidate parallel strategies to determine a target parallel strategy, which is the parallel strategy used to train the AI ​​model.

[0140] In one possible implementation, at least one strategy filtering rule includes one or more of the following: parallelization setting rules, parallel strategy relationship setting rules, and model splitting rules. The parallelization setting rules are used to indicate the setting rules for a single parallel mode in the parallel strategy. The parallel strategy relationship setting rules are used to indicate the setting rules for the relationship between multiple parallel modes in the parallel strategy. The model splitting rules are used to indicate the setting rules for model splitting points.

[0141] In one possible implementation, the processing unit 902 is further configured to determine the first parallel strategy as a candidate parallel strategy if the first parallel strategy among the plurality of parallel strategies meets the constraints in at least one strategy filtering rule. Alternatively, the second parallel strategy is skipped if the second parallel strategy among the plurality of parallel strategies does not meet the constraints in at least one strategy filtering rule.

[0142] In one possible implementation, the input includes one or more of the following: configuration files, rule codes, and graphical front-end configuration information.

[0143] In one possible implementation, the input content includes graphical front-end configuration information. The acquisition unit 901 is also used to provide a graphical user interface (GUI) for displaying the graphical front-end configuration information. The graphical front-end configuration information includes one or more of the following: rule identifier ID, rule description, and rule content.

[0144] In one possible implementation, the acquisition unit 901 is further configured to receive rule processing requests, including processing requests triggered by user modification of input content in the GUI. The processing unit 902 is further configured to perform one or more of the following operations on at least one policy filtering rule based on the rule modification request: adding a policy filtering rule, modifying a policy filtering rule, and deleting a policy filtering rule.

[0145] In one possible implementation, the processing unit 902 is specifically used to evaluate candidate parallel strategies based on a cost model and determine a target parallel strategy. The cost model is used to predict the performance of the candidate parallel strategies, and the performance of the target parallel strategy is higher than the performance of other candidate parallel strategies among a plurality of candidate parallel strategies.

[0146] In one possible implementation, the model parameters of the cost model are determined based on one or more of the following information: system information, task information, and historical information.

[0147] It is understandable that the acquisition unit 901 and processing unit 902 in the parallel strategy search device 900 can be mapped as functional modules to the various modules in the parallel strategy search system 10 in Figure 1a, thereby realizing the functions of the various modules in the parallel strategy search system 10.

[0148] It should be understood that the division of units in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, all units in the device can be implemented entirely through software calls from processing elements; all units can be implemented entirely in hardware; or some units can be implemented through software calls from processing elements, and others in hardware. For example, each unit can be a separate processing element, or it can be integrated into a chip within the device. Alternatively, it can be stored as a program in memory, called and executed by a processing element of the device. Moreover, these units can be fully or partially integrated together, or implemented independently. The processing element mentioned here can also be called a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above units can be implemented through integrated logic circuits in the processor element or through software calls from processing elements.

[0149] It is worth noting that, for the sake of simplicity, the above method embodiments are described as a series of actions. However, those skilled in the art should know that this application is not limited to the order of the described actions. Furthermore, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by this application.

[0150] Other reasonable combinations of steps that can be conceived by those skilled in the art based on the above description also fall within the scope of protection of this application. Furthermore, those skilled in the art should also be aware that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0151] Please refer to Figure 10, which is a schematic diagram of the structure of a computing device provided in an embodiment of this application. As shown in Figure 10, the computing device 1000 includes: a processor 1001, a memory 1002, a communication interface 1003, and a bus 1004. The processor 1001, the memory 1002, and the communication interface 1003 are coupled through the bus (not labeled in the figure). The memory 1002 stores instructions. When the execution instructions in the memory 1002 are executed, the computing device 1000 executes the method performed by the search system of the parallel strategy in the above method embodiment.

[0152] The computing device 1000 may be one or more integrated circuits configured to implement the methods described above, such as: one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these forms of integrated circuits. Furthermore, when the units in the device can be implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. Alternatively, these units may be integrated together and implemented as a system-on-a-chip (SOC).

[0153] The processor 1001 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0154] The memory 1002 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0155] The memory 1002 stores executable program code, and the processor 1001 executes the executable program code to implement the functions of the aforementioned units or modules, thereby implementing the search method of the parallel strategy described above. That is, the memory 1002 stores instructions for executing the search method of the parallel strategy described above.

[0156] The communication interface 1003 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1000 and other devices or communication networks.

[0157] In addition to the data bus, the 1004 bus can also include a power bus, a control bus, and a status signal bus. The bus can be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. The bus can be divided into address bus, data bus, and control bus.

[0158] Please refer to Figure 11, which is a schematic diagram of a computing device cluster provided in an embodiment of this application. As shown in Figure 11, the computing device cluster 1100 includes at least one computing device 1000.

[0159] As shown in Figure 11, the computing device cluster 1100 includes at least one computing device 1000. The memory 1002 of one or more computing devices 1000 in the computing device cluster 1100 may store the same instructions for executing the search method of the above-described parallel strategy.

[0160] In some possible implementations, the memory 1002 of one or more computing devices 1000 in the computing device cluster 1100 may also store partial instructions for executing the search method of the above-described parallel strategy. In other words, a combination of one or more computing devices 1000 can jointly execute the instructions for executing the search method of the above-described parallel strategy.

[0161] It should be noted that the memories 1002 in the different computing devices 1000 within the computing device cluster 1100 can store different instructions, each used to execute a portion of the functions of the search device in the aforementioned parallel strategy. That is, the instructions stored in the memories 1002 of the different computing devices 1000 can implement the functions of one or more modules in the processing unit and the acquisition unit.

[0162] In some possible implementations, one or more computing devices 1000 in the computing device cluster 1100 can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc.

[0163] Please refer to Figure 12, which is a schematic diagram of computer devices in a computer cluster connected via a network according to an embodiment of this application. As shown in Figure 12, two computing devices 1000A and 1000B are connected via a network. Specifically, they are connected to the network through the communication interfaces in each computing device.

[0164] In one possible implementation, the memory in computing device 1000A stores instructions for performing the functions of the fetch unit. Meanwhile, the memory in computing device 1000B stores instructions for performing the functions of the processing unit.

[0165] It should be understood that the functions of computing device 1000A shown in Figure 12 can also be performed by multiple computing devices. Similarly, the functions of computing device 1000B can also be performed by multiple computing devices.

[0166] In another embodiment of this application, a computer-readable storage medium is also provided, which stores computer-executable instructions. When the processor of the device executes the computer-executable instructions, the device executes the method performed by the search system of the parallel strategy in the above method embodiment.

[0167] In another embodiment of this application, a computer program product is also provided, which includes computer-executable instructions stored in a computer-readable storage medium. When the processor of the device executes the computer-executable instructions, the device performs the method executed by the search system of the parallel strategy in the above method embodiments.

[0168] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0170] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0171] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0172] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A search method using a parallel strategy, characterized in that, Applied to a cloud platform for managing infrastructure running cloud computing services, the infrastructure including at least one cloud data center, the method includes: Obtain multiple parallel strategies, each of which is used to indicate the parallel method for training the AI ​​model; Provide an input interface for obtaining input content, which indicates at least one policy filtering rule; Based on the at least one strategy filtering rule, the plurality of parallel strategies are filtered to determine a plurality of candidate parallel strategies among the plurality of parallel strategies; The multiple candidate parallel strategies are evaluated to determine the target parallel strategy, which is the parallel strategy used to train the AI ​​model.

2. The method according to claim 1, characterized in that, The at least one strategy filtering rule includes one or more of the following: parallelization setting rules, parallel strategy relationship setting rules, and model splitting rules. The parallelization setting rules are used to indicate the setting rules for a single parallel mode in the parallel strategy. The parallel strategy relationship setting rules are used to indicate the setting rules for the relationship between multiple parallel modes in the parallel strategy. The model splitting rules are used to indicate the setting rules for model splitting points.

3. The method according to claim 1 or 2, characterized in that, The filtering of the multiple parallel strategies based on the at least one strategy filtering rule includes: If the first parallel strategy among the plurality of parallel strategies meets the constraints in the at least one strategy selection rule, then the first parallel strategy is determined as a candidate parallel strategy; or... If the second parallel strategy among the plurality of parallel strategies does not meet the constraints in the at least one strategy filtering rule, then the second parallel strategy is skipped.

4. The method according to any one of claims 1 to 3, characterized in that, The input content includes one or more of the following: configuration files, rule codes, and graphical front-end configuration information.

5. The method according to claim 4, characterized in that, The input content includes the graphical front-end configuration information, and the method further includes: A graphical user interface (GUI) is provided, which is used to display the graphical front-end configuration information, including one or more of the following: rule identifier ID, rule description, and rule content.

6. The method according to claim 5, characterized in that, The method further includes: Receive rule processing requests, including processing requests triggered by the user modifying the input content in the GUI; Based on the rule modification request, perform one or more of the following operations on the at least one policy filtering rule: add a policy filtering rule, modify a policy filtering rule, and delete a policy filtering rule.

7. The method according to any one of claims 1 to 6, characterized in that, The step of evaluating the multiple candidate parallel strategies and determining the target parallel strategy includes: The candidate parallel strategies are evaluated based on the cost model to determine the target parallel strategy. The cost model is used to predict the performance of the candidate parallel strategies. The performance of the target parallel strategy is higher than that of the other candidate parallel strategies among the plurality of candidate parallel strategies.

8. The method according to claim 7, characterized in that, The model parameters of the cost model are determined based on one or more of the following information: system information, task information, and historical information.

9. A search device using a parallel strategy, characterized in that, include: An acquisition unit is used to acquire multiple parallel strategies, each of which is used to indicate the parallel method for training the AI ​​model; The acquisition unit is also configured to provide an input interface, the input interface being configured to acquire input content, the input content being configured to indicate at least one policy filtering rule; A processing unit is configured to filter the plurality of parallel strategies based on the at least one strategy filtering rule, and determine a plurality of candidate parallel strategies among the plurality of parallel strategies; The processing unit is further configured to evaluate the plurality of candidate parallel strategies and determine a target parallel strategy, which is the parallel strategy used to train the AI ​​model.

10. The apparatus according to claim 9, characterized in that, The at least one strategy filtering rule includes one or more of the following: parallelization setting rules, parallel strategy relationship setting rules, and model splitting rules. The parallelization setting rules are used to indicate the setting rules for a single parallel mode in the parallel strategy. The parallel strategy relationship setting rules are used to indicate the setting rules for the relationship between multiple parallel modes in the parallel strategy. The model splitting rules are used to indicate the setting rules for model splitting points.

11. The apparatus according to claim 9 or 10, characterized in that, The processing unit is also used for: If the first parallel strategy among the plurality of parallel strategies meets the constraints in the at least one strategy screening rule, then the first parallel strategy is determined as a candidate parallel strategy. or, If the second parallel strategy among the plurality of parallel strategies does not meet the constraints in the at least one strategy filtering rule, then the second parallel strategy is skipped.

12. The apparatus according to any one of claims 9 to 11, characterized in that, The input content includes one or more of the following: configuration files, rule codes, and graphical front-end configuration information.

13. The apparatus according to claim 12, characterized in that, The input content includes the graphical front-end configuration information, and the acquisition unit is further used for: A graphical user interface (GUI) is provided, which is used to display the graphical front-end configuration information, including one or more of the following: rule identifier ID, rule description, and rule content.

14. The apparatus according to claim 13, characterized in that, The acquisition unit is also used for: Receive rule processing requests, including processing requests triggered by the user modifying the input content in the GUI; The processing unit is also configured to perform one or more of the following operations on the at least one policy filtering rule based on the rule modification request: add a policy filtering rule, modify a policy filtering rule, and delete a policy filtering rule.

15. The apparatus according to any one of claims 9 to 14, characterized in that, The processing unit is specifically used for: The candidate parallel strategies are evaluated based on the cost model to determine the target parallel strategy. The cost model is used to predict the performance of the candidate parallel strategies. The performance of the target parallel strategy is higher than that of the other candidate parallel strategies among the plurality of candidate parallel strategies.

16. The apparatus according to claim 15, characterized in that, The model parameters of the cost model are determined based on one or more of the following information: system information, task information, and historical information.

17. A computing device, characterized in that, The device includes a processor coupled to a memory, the processor storing instructions which, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 8.

18. A computing device cluster, characterized in that, The system includes at least one computing device, the computing device including a processor coupled to a memory, the processor being used to store instructions that, when executed by the processor, cause the cluster of computing devices to perform the method of any one of claims 1 to 8.

19. A computer-readable storage medium having instructions stored thereon, characterized in that, When the instructions are executed, they cause the computer to perform the method of any one of claims 1 to 8.

20. A computer program product, the computer program product comprising instructions, characterized in that, When the instructions are executed, they cause the computer to implement the method of any one of claims 1 to 8.

Images