Resource configuration method, device, equipment, storage medium and program product for large model cluster

Abstract

The present invention discloses a resource configuration method, apparatus, device, storage medium and program product for a large model cluster. The method comprises: determining configuration parameters based on a configuration task selected by a user; determining at least one of computing power requirements, bandwidth requirements and video memory requirements based on the configuration task and the configuration parameters; constructing constraints and an objective function based on at least one of the computing power requirements, bandwidth requirements and video memory requirements; iteratively optimizing the objective function based on the constraints using a preset planning algorithm to solve the unknown configuration parameters and obtain target configuration parameters. The resource configuration method for a large model cluster provided in an embodiment of the present invention can determine the optimal large model learning parameters or hardware resource configuration, thereby improving the efficiency and reliability of resource configuration of the large model cluster.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a resource configuration method, apparatus, device, storage medium, and program product for a large model cluster. Background Art

[0002] The field of artificial intelligence has made significant progress in recent years, with large models attracting widespread attention due to their powerful learning capabilities and wide range of application scenarios. However, large models are more complex than deep learning models. Determining the hardware configuration for large model training or inference clusters typically requires consideration of various factors, including parallel configurations involving data parallelism, model parallelism, pipeline parallelism, and memory optimization techniques; various communication methods between nodes and GPUs; varying model sizes requiring different graphics memory requirements; and the balance between model acceleration and performance. These factors often interact with each other. Common methods for configuring hardware and software for large models include: 1. Searching for selection guides online: For example, referencing professional hardware selection guides, which typically provide recommendations and configurations regarding GPU selection, memory requirements, and storage requirements. 2. Manual configuration: This typically involves selecting hardware configurations based on historical data and expert knowledge. 3. Performance testing and evaluation: Conducting real-world performance testing to evaluate the impact of different hardware configurations on large model training and inference tasks. This may include benchmarking different GPU models, memory sizes, and storage solutions to determine the optimal hardware combination. 4. Formula calculation: Estimating the required hardware resources using mathematical formulas.

[0003] Existing methods have the following flaws: 1. Searching for selection guides on the Internet: Guides may provide general recommendations, but may not be suitable for all specific scenarios; hardware is developing rapidly, and guides may not be updated in a timely manner to reflect the latest technologies; guides may be based on specific suppliers or technology preferences and are highly targeted. 2. Manual experience configuration: highly dependent on the experience of individuals or teams, which may not be friendly to novices; configuration selection may be subjective, making it difficult to guarantee objectivity and universal applicability; technology is developing rapidly, and personal experience may not keep up with the latest hardware developments and best practices. 3. Performance testing and evaluation: Conducting detailed performance testing can be very time-consuming, increasing project cycles, and may require multiple trials and errors to find the optimal configuration, which is time-consuming and resource-intensive; test results may not fully predict performance in actual applications. 4. Formula calculation: Accurate input data is required, and inaccurate data will lead to biased results; it is easy to ignore physical limitations and compatibility issues in actual hardware configurations; it is overly theoretical and may sometimes be out of touch with actual application scenarios. Summary of the Invention

[0004] The embodiments of the present invention provide a resource configuration method, apparatus, device, storage medium and program product for a large model cluster, which can determine the optimal large model learning parameters or hardware resource configuration, thereby improving the efficiency and reliability of the large model cluster resource configuration.

[0005] In a first aspect, an embodiment of the present invention provides a resource configuration method for a large model cluster, comprising:

[0006] Determine configuration parameters based on the configuration task selected by the user; wherein the configuration parameters include: one or more of: hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, number of model parameters, data parallelism, pipeline parallelism, tensor parallelism, and zero-redundancy parallelism mode; and some of the configuration parameters are known configuration parameters and others are unknown configuration parameters;

[0007] Determine at least one of computing power requirements, bandwidth requirements, and video memory requirements based on the configuration task and the configuration parameters;

[0008] Constructing a constraint condition and an objective function based on at least one of the computing power requirement, bandwidth requirement, and video memory requirement; wherein the constraint condition includes at least one of a computing power constraint condition, a bandwidth constraint condition, a storage constraint condition, a cost constraint condition, and a training or inference speed equation; and the objective function includes a minimum time objective function and a maximum cost ratio objective function;

[0009] Based on the constraint conditions, the objective function is iteratively optimized using a preset planning algorithm to solve the unknown configuration parameters and obtain target configuration parameters.

[0010] In a second aspect, an embodiment of the present invention further provides a resource configuration device for a large model cluster, comprising:

[0011] A configuration parameter determination module is used to determine configuration parameters based on the configuration task selected by the user; wherein the configuration parameters include: one or more of: hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism, and zero-redundancy parallel mode; and some of the configuration parameters are known configuration parameters and others are unknown configuration parameters;

[0012] a demand determination module, configured to determine at least one of a computing power requirement, a bandwidth requirement, and a video memory requirement based on the configuration task and the configuration parameters;

[0013] a constraint condition and objective function construction module, configured to construct constraint conditions and objective functions based on at least one of the computing power requirement, bandwidth requirement, and video memory requirement; wherein the constraint conditions include at least one of a computing power constraint, a bandwidth constraint, a storage constraint, a cost constraint, and a training or inference speed equation; and the objective function includes a shortest time objective function and a highest cost ratio objective function;

[0014] The target configuration parameter acquisition module is used to iteratively optimize the objective function based on the constraint conditions using a preset planning algorithm to solve the unknown configuration parameters and obtain the target configuration parameters.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to execute the resource configuration method for the large model cluster described in the embodiment of the present invention.

[0019] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the resource configuration method for the large model cluster described in the embodiment of the present invention when executed.

[0020] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the resource configuration method for a large model cluster as described in an embodiment of the present invention.

[0021] The embodiment of the present invention discloses a resource configuration method, apparatus, device, storage medium and program product for a large model cluster. The method comprises: determining configuration parameters based on a configuration task selected by a user; wherein the configuration parameters include: one or more of hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism and zero-redundancy parallel mode; and wherein some of the configuration parameters are known configuration parameters and others are unknown configuration parameters; determining at least one of computing power requirements, bandwidth requirements and video memory requirements based on the configuration task and the configuration parameters; constructing constraints and an objective function based on at least one of computing power requirements, bandwidth requirements and video memory requirements; wherein the constraints include at least one of computing power constraints, bandwidth constraints, storage constraints, cost constraints and a training or inference speed equation; and the objective function includes a shortest time objective function and a highest cost ratio objective function; and iteratively optimizing the objective function using a preset planning algorithm based on the constraints to solve the unknown configuration parameters and obtain target configuration parameters. The resource configuration method for a large model cluster provided by an embodiment of the present invention can determine the optimal large model learning parameters or hardware resource configuration, thereby improving the efficiency and reliability of the resource configuration of the large model cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of a resource configuration method for a large model cluster in the first embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of the structure of a resource configuration system for a large model cluster in the first embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of a resource configuration device for a large model cluster in the second embodiment of the present invention;

[0025] Figure 4 It is a structural diagram of an electronic device in embodiment 3 of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0027] Example 1

[0028] Figure 1A flowchart of a resource configuration method for a large model cluster provided in Example 1 of the present invention is provided. This embodiment is applicable to situations where the resource configuration of a large model cluster is determined. The method can be executed by a resource configuration device for a large model cluster, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, PC or server, etc.

[0029] Figure 2 This is a schematic diagram of the structure of a resource configuration system for a large model cluster provided by an embodiment of the present invention. Figure 2 The system includes: a large model cluster optimal configuration task management module, a scale calculation module, a configuration optimization module, a calculation strategy learning module and a large model cluster management module; wherein the configuration optimization module includes a parameter management submodule, a configuration optimization submodule and a calculation strategy submodule.

[0030] The Large Model Cluster Optimal Configuration Task Management module manages configuration tasks. Users select configuration tasks and enter relevant parameters through this module, which then sends the selected tasks to the Configuration Optimization module. The Scale Calculation module consists of multiple scale calculation functions, which calculate scale parameters. Its inputs are the large model parameters and training corpus, and its outputs are scale parameters. Scale parameters include model parameter scale, forward and backward computing power requirements, and corpus size. The Configuration Optimization module consists of three submodules: the Parameter Management submodule, the Configuration Optimization submodule, and the Computation Strategy submodule. It takes the configuration task and its corresponding scale parameters as input and outputs the optimal configuration solution to the Large Model Cluster Management module. It then receives cluster configuration information (computing power and video memory) from the Large Model Cluster Management module and the new computing strategy from the Computation Strategy Learning module. Finally, it outputs the optimized optimal configuration to the Large Model Cluster Optimal Configuration Task Management module, which is responsible for model parameter configuration and optimization computation. The Parameter Management submodule comprises model network configuration, hyperparameter configuration, batch size, and parallel configuration. The Configuration Optimization module comprises optimization modules such as linear programming, nonlinear programming, hybrid programming, and dynamic programming. The computing strategy module consists of a computing power requirement, bandwidth requirement, and memory requirement structural equation management sub-module, a computing, storage, bandwidth and other constraint management sub-module, and a computing strategy management sub-module.

[0031] The computing strategy learning module consists of a learning algorithm and a strategy update mechanism. Its input is real-time status data, and its output is the updated computing strategy. The module is responsible for learning and updating the computing strategy based on historical data and real-time feedback.

[0032] The large model cluster management module consists of homogeneous or heterogeneous hardware clusters. The input is the optimal configuration, and the output is cluster configuration information (computing power, video memory). The module is responsible for calling cluster resources to test the large model parameter configuration.

[0033] Efficient operation is achieved through real-time data transmission and collaboration between modules. For example, the large-model cluster optimal configuration task management module and the scale calculation module collaborate to complete the calculation preparation for the optimal configuration, and under specific configuration tasks, jointly complete the input of the configuration optimization module.

[0034] The configuration optimization module, large model cluster management module, and computing strategy learning module collaborate to achieve and optimize the optimal configuration. The configuration optimization module calculates the optimal configuration based on the input and passes it to the large model cluster management module. The large model cluster management module tests the optimal configuration of the large model cluster under the corresponding hardware and software environment, feeds back cluster configuration information (computing power, video memory) to the configuration optimization module, and outputs real-time status data to the computing strategy learning module. The computing strategy learning module calculates a new computing strategy function based on the real-time status data and learning mechanism and passes it to the configuration optimization module. The configuration optimization module recalculates the optimal configuration based on the feedback and updates. This process is iterated, and a convergence standard (such as performance improvement less than a certain threshold, or reaching the expected goal) is set to determine whether to stop iteration. If the performance no longer improves significantly after multiple iterations, it is considered that the optimal configuration has been reached and the iteration can be stopped.

[0035] The configuration optimization module's submodules, the parameter management module, the configuration optimization module, and the calculation strategy module, collaborate to achieve the optimal configuration. The parameter management module determines the parameters to be configured based on the required task. The calculation strategy management module determines the calculation strategy based on the required task or updates the calculation strategy based on the calculation strategy learning module. The configuration optimization module uses the required parameters provided by the parameter management module. The calculation strategy module provides the structural equations, constraint equations, and objective function to determine the planning method and calculate the optimal configuration.

[0036] Specifically, such as Figure 1 As shown, the method includes the following steps:

[0037] S110: Determine configuration parameters based on the configuration task selected by the user.

[0038] Among them, the configuration parameters include: hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimensions, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism and zero redundancy parallelism (ZeRO) mode; and some of the configuration parameters are known configuration parameters, and the other part is unknown configuration parameters. Hyperparameters include the number of iterations and the number of training cycles. Micro-batch size and full batch size can be collectively referred to as batch size; the number of Transformer layers and hidden layer dimensions can be collectively referred to as model structure parameters; data parallelism, pipeline parallelism, tensor parallelism and zero redundancy parallelism mode can be collectively referred to as parallel configuration parameters.

[0039] In this embodiment, the configuration tasks are set according to different goals and scenarios, including but not limited to the following types: configuration tasks for determining model parameters based on known software and hardware information, configuration tasks for determining model scale based on known software and hardware, configuration tasks for determining software and hardware information based on cost, and configuration tasks for determining software and hardware information based on time consumption. Among them, the goal of the configuration task for determining model parameters based on known software and hardware information is to adjust the configuration parameters of the model so that the model takes the shortest time when training or reasoning under the existing software and hardware configuration. The goal of the configuration task for determining model scale based on known software and hardware is to determine the maximum model scale that can be supported under the current software and hardware conditions. The goal of the configuration task for determining software and hardware information based on cost is to compare the cost and performance of different software and hardware configurations while maintaining budget constraints, and select the most cost-effective software and hardware configuration suitable for large model training or reasoning. The goal of the configuration task for determining software and hardware information based on time consumption is to select the software and hardware combination that can minimize the running time by evaluating the computing speed and response time of various software and hardware configurations while maintaining budget constraints.

[0040] Specifically, when the user selects a configuration task that determines model parameters based on known hardware and software information, the determined configuration parameters include: hyperparameters (number of iterations, number of training epochs); batch size (micro-batch size, full batch size); and parallel configuration parameters (data parallelism, pipeline parallelism, tensor parallelism, and zero-redundancy parallelism). When the user selects a configuration task that determines model scale based on known hardware and software information, the determined configuration parameters include: model parameter count; vocabulary length; context length; model structure parameters (number of Transformer layers, hidden layer dimensions); and parallel configuration parameters (data parallelism, pipeline parallelism, tensor parallelism, and zero-redundancy parallelism). When the user selects a configuration task that determines hardware and software information based on cost, the determined configuration parameters include: hyperparameters (number of iterations, number of training epochs); batch size (micro-batch size, full batch size); and parallel configuration parameters (data parallelism, pipeline parallelism, tensor parallelism, and zero-redundancy parallelism). When the configuration task selected by the user is a configuration task that determines the software and hardware information based on time consumption, the determined configuration parameters include: parameters: number of iterations, number of training cycles; batch size: micro-batch size, full batch size; parallel configuration parameters: data parallelism, pipeline parallelism, tensor parallelism and zero-redundancy parallel mode.

[0041] S120: Determine at least one of computing power requirements, bandwidth requirements, and video memory requirements based on the configuration task and the configuration parameters.

[0042] Compute requirements indicate the computing power required for large models during training or inference. Bandwidth requirements indicate the bandwidth required for large models during training or inference. Memory requirements indicate the storage resources required for large models during training or inference.

[0043] In this embodiment, the computing power requirement, bandwidth requirement, and video memory requirement are affected by the configuration attributes of the configuration task. Different configuration attributes result in different configuration parameters and calculation methods required to determine the computing power requirement, bandwidth requirement, and video memory requirement.

[0044] The configuration task includes the following configuration attributes: activation state, large model processing phase, and parallel mode. The activation state includes whether activation recalculation is enabled or disabled; enabling activation recalculation includes full activation recalculation and selective activation recalculation; large model processing phase includes training and inference; and parallel modes include data parallelism (DP), pipeline parallelism (PP), tensor parallelism (TP), zero-redundancy parallelism, 3D parallelism, sequential parallelism, and no parallelism.

[0045] Optionally, the process of determining at least one of computing power requirements, bandwidth requirements and video memory requirements based on configuration tasks and configuration parameters may be: determining computing power requirements based on activation value status, large model processing stage and configuration parameters; or, determining bandwidth requirements based on parallel mode, large model processing stage and configuration parameters; or, determining video memory requirements based on activation value status, large model processing stage, parallel mode and configuration parameters.

[0046] In this embodiment, in order to facilitate the determination of computing power requirements, bandwidth requirements and video memory requirements, each configuration parameter is represented by a letter, namely: micro-batch size is represented as b, full batch size is represented as B, vocabulary length is represented as v, context length is represented as s, number of Transformer layers is represented as l, hidden layer dimension is represented as h, number of model parameters is represented as P, data parallelism is represented as d, pipeline parallelism is represented as p, and tensor parallelism is represented as t.

[0047] Optionally, the method of determining computing power requirements based on activation value status, large model processing stage and configuration parameters can be: determining forward propagation computing power requirements based on full batch size, context length, number of Transformer layers, hidden layer dimension and vocabulary length; determining computing power requirements based on forward propagation computing power requirements, activation value status and large model processing stage.

[0048] The calculation formula for determining the forward propagation computing power requirement based on the full batch size, context length, number of Transformer layers, hidden layer dimension, and vocabulary length can be expressed as:

[0049] The process of determining the computing power requirement based on the forward propagation computing power requirement, activation value status, and large model processing stage can be: when the large model processing stage is the training stage, if the activation value status is not enabled for activation value recalculation, the computing power requirement is three times the forward propagation computing power requirement, and the formula can be expressed as: C1 = 3C 前 If the activation value state is to enable activation value recalculation, the computing power required is four times the forward propagation computing power required. The formula can be expressed as: C2 = 4C 前 When the large model processing stage is the inference stage, the computing power requirement is the forward propagation computing power requirement, and the formula can be expressed as: C3 = C 前 .

[0050] Optionally, the method of determining the bandwidth requirement based on the parallel mode, large model processing stage and configuration parameters can be: when the large model processing stage is the training stage, if the parallel mode is tensor parallel, the bandwidth requirement is determined according to the micro-batch size, context length, hidden layer dimension, number of Transformer layers, full batch size, pipeline parallelism and data parallelism; if the parallel mode is pipeline parallel, the bandwidth requirement is determined according to the micro-batch size, context length, hidden layer dimension, tensor parallelism, full batch size and data parallelism; if the parallel mode is data parallel, the bandwidth requirement is determined according to the number of Transformer layers, hidden layer dimension, pipeline parallelism and tensor parallelism; if the parallel mode is zero-redundancy parallel mode, the bandwidth requirement is determined according to the number of model parameters; when the large model processing stage is the inference stage, if the parallel mode is tensor parallel, the bandwidth requirement is determined according to the context length, number of Transformer layers, hidden layer dimension and pipeline parallelism; if the parallel mode is pipeline parallel, the bandwidth requirement is determined according to the context length, hidden layer dimension and tensor parallelism.

[0051] The number of model parameters is determined by the number of Transformer layers, hidden layer dimensions, vocabulary length, and context length. Zero-redundancy parallel modes include Zero-redundancy parallel mode 1 (ZeRO-1), Zero-redundancy parallel mode 2 (ZeRO-2), and Zero-redundancy parallel mode 3 (ZeRO-3).

[0052] When the large model processing stage is the training stage, if the parallel mode is tensor parallel, the calculation formula for the bandwidth requirement can be expressed as: T1 = bsh*((4lB)) / (pdb); if the parallel mode is pipeline parallel, the calculation formula for the bandwidth requirement can be expressed as: T2 = (bsh / t)*(2B) / db; if the parallel mode is data parallel, the calculation formula for the bandwidth requirement can be expressed as: T3 = (12lh 2 ) / pt; if the parallel mode is zero-redundancy parallel mode 1, the calculation formula for the bandwidth requirement can be expressed as: T4 = 2P; if the parallel mode is zero-redundancy parallel mode 2, the calculation formula for the bandwidth requirement can be expressed as: T5 = 2P; if the parallel mode is zero-redundancy parallel mode 3, the calculation formula for the bandwidth requirement can be expressed as: T6 = 3P. Among them, the calculation formula for the model parameter P can be expressed as: P = 12lh 2 +13lh+vh+sh. When the large model processing stage is the inference stage, if the parallelization method is tensor parallelism, the bandwidth requirement calculation formula can be expressed as: T7 = sh * (2l / p); if the parallelization method is pipeline parallelism, the bandwidth requirement calculation formula can be expressed as: T8 = sh / t.

[0053] Optionally, the method of determining the video memory requirement based on the activation value state, large model processing stage, parallel mode and configuration parameters can be: when the large model processing stage is the training stage, determine the combination of the activation value state and the parallel mode, determine the required configuration parameters based on the combination, and determine the video memory requirement based on the configuration parameters; when the large model processing stage is the inference stage, determine the video memory requirement based on the model parameter quantity, data size, micro-batch size, number of attention heads, hidden layer dimension, context length, data parallelism, and number of intermediate variables.

[0054] Among them, the combinations of activation value states and parallel modes include the following: zero-redundancy basic parallel mode (baseline), zero-redundancy parallel mode 1, zero-redundancy parallel mode 2, zero-redundancy parallel mode 3, no parallel, 3D parallel, 3D parallel + sequence parallel, 3D parallel + selective activation value recalculation, 3D parallel + sequence parallel + selective activation value recalculation, 3D parallel + full activation value recalculation.

[0055] Specifically, when the combination mode is zero-redundancy basic parallel mode, the determined configuration parameters include model parameter quantity P, pipeline parallelism p and tensor parallelism t, and the calculation formula of video memory requirement can be expressed as: M1=20P / (pt); when the combination mode is zero-redundancy parallel mode 1, the determined configuration parameters include: data parallelism d, model parameter quantity P, pipeline parallelism p and tensor parallelism t, and the calculation formula of video memory requirement can be expressed as: M2=(4+16d)P / (pt); when the combination mode is zero-redundancy parallel mode 2, the determined configuration parameters include: data parallelism d, model parameter quantity P, pipeline parallelism p and tensor parallelism t, and the calculation formula of video memory requirement can be expressed as: It is shown as: M3 = (2 + 18d) P / (pt); When the combination mode is zero-redundancy parallel mode 3, the determined configuration parameters include: data parallelism d, model parameter quantity P, pipeline parallelism p and tensor parallelism t, and the calculation formula of video memory requirement can be expressed as: M4 = 20P / (dpt); When the combination mode is no parallel, the determined configuration parameters include: micro-batch size b, context length s, hidden layer dimension h, number of Transformer layers l, number of attention heads a, and the calculation formula of video memory requirement can be expressed as: M5 = bshl (34 + 5sa / h); When the combination mode is 3D parallel, the determined configuration parameters include: micro-batch size b, context length s, The hidden layer dimension h, the number of Transformer layers l, the tensor parallelism t, the number of attention heads a, the calculation formula for the video memory requirement can be expressed as: M6 = bshl (10 + 24 / t + 5sa / (ht)); when the combination mode is 3D parallel + sequence parallel, the determined configuration parameters include: micro-batch size b, context length s, hidden layer dimension h, the number of Transformer layers l, the tensor parallelism t, the number of attention heads a, the calculation formula for the video memory requirement can be expressed as: M7 = bshl (34 / t + 5sa / (ht)); when the combination mode is 3D parallel + selective activation value recalculation, the determined configuration parameters include: micro-batch size b, context length s The calculation formula of video memory requirement can be expressed as: M8 = bshl (10 + 24 / t); when the combination mode is 3D parallelism + sequence parallelism + selective activation value recalculation, the determined configuration parameters include: micro-batch size b, context length s, hidden layer dimension h, number of Transformer layers l, tensor parallelism t, the calculation formula of video memory requirement can be expressed as: M9 = bshl (34 / t); when the combination mode is 3D parallelism + full activation value recalculation, the determined configuration parameters include: micro-batch size b, context length s, hidden layer dimension h, number of Transformer layers l, tensor parallelism t, the calculation formula of video memory requirement can be expressed as: M9 = bshl (34 / t); when the combination mode is 3D parallelism + full activation value recalculation, the determined configuration parameters include: micro-batch size b, context length s, hidden layer dimension h, the calculation formula of video memory requirement is: M 10 =2bsh.

[0056] Specifically, when the large model processing stage is the inference stage, the calculation formula for the video memory requirement can be expressed as: M 11 =P*n+b×(2×1×a×h×s×n)+b×(s×d×c×n), where n represents the data size (number of bytes) and c represents the number of intermediate variables.

[0057] Optionally, you can also average the number of contexts (tokens) on each sample in one iteration S average , the total number of samples in one iteration n total and the number of iterations n epoch Determine the corpus size S, the calculation formula can be expressed as: S = S average ×n total ×n epoch .

[0058] S130: Constructing constraint conditions and objective functions based on at least one of computing power requirements, bandwidth requirements, and video memory requirements.

[0059] Among them, the constraints include at least one of computing power constraints, bandwidth constraints, storage constraints, cost constraints and training or inference speed constraints; the objective function includes the shortest time objective function and the highest cost ratio objective function.

[0060] Among them, the computing power constraint can be understood as the total computing power required by the large model during training or inference cannot exceed the available computing power of the system; the bandwidth constraint can be understood as the total bandwidth demand required by the large model during training or inference cannot exceed the bandwidth capacity of the system; the storage constraint can be understood as the total storage demand required by the large model during training or inference cannot exceed the storage capacity of the system; the cost constraint can be understood as the hardware cost selected by the large model during training or inference cannot exceed the budget; the training or inference speed constraint can be understood as the training or inference speed of the hardware configuration of the large model during training or inference must meet the time requirements of the task.

[0061] Specifically, the method of constructing constraints based on at least one of computing power requirements, bandwidth requirements and video memory requirements can be: determining computing power constraints based on the relationship between computing power requirements and target computing power; determining bandwidth constraints based on the relationship between bandwidth requirements and target bandwidth; determining storage constraints based on the relationship between video memory requirements and target video memory; determining cost constraints based on the relationship between cost requirements and target cost; and determining training or inference speed constraints based on the relationship between training or inference speed and target speed.

[0062] The target computing power can be understood as the system's available computing power; the target bandwidth can be understood as the system's bandwidth capacity; the target video memory can be understood as the system's storage capacity; the target cost can be understood as the budget; and the target speed can be understood as the estimated inference or training speed. In this embodiment, the computing power requirement and the target computing power are related in that the computing power requirement is less than or equal to the target computing power. That is, the computing power constraint can be expressed as: computing power requirement ≤ target computing power. The bandwidth requirement and the target bandwidth are related in that the bandwidth requirement is less than or equal to the target bandwidth. That is, the bandwidth constraint can be expressed as: bandwidth requirement ≤ target bandwidth. The memory requirement and the target video memory are related in that the memory requirement is less than or equal to the target video memory. That is, the storage constraint can be expressed as: memory requirement ≤ target video memory. The cost requirement and the target cost are related in that the cost requirement is less than or equal to the target cost. That is, the cost constraint can be expressed as: cost requirement ≤ target cost. The training or inference speed and the target speed are related in that the training or inference speed is greater than or equal to the target speed. That is, the training or inference constraint can be expressed as: training or inference speed ≥ target speed.

[0063] In this embodiment, the objective function for minimizing training time can be expressed as: training time = 8TP / nX, where T is the total number of context tokens, P is the number of large model parameters, n is the total number of GPUs, X is the throughput, and X is the number of observations. The cost ratio can be understood as the ratio between hardware performance and cost. The objective function for achieving the highest cost ratio can be expressed as: cost ratio = hardware performance / hardware cost.

[0064] S140 , iteratively optimizing the objective function using a preset planning algorithm based on the constraint conditions to solve unknown configuration parameters and obtain target configuration parameters.

[0065] The planning algorithm may be linear programming, nonlinear programming, hybrid programming, or dynamic programming. In this embodiment, expressions for computing power requirements, bandwidth requirements, and video memory requirements are selected based on the configuration tasks and configuration parameters, and then constraints are determined based on the selected expressions for computing power requirements, bandwidth requirements, and video memory requirements. The process of iterative optimization of the objective function may be as follows: initial values ​​are set for unknown configuration parameters, and then the objective function is iteratively solved using the selected planning algorithm based on the constraints, and finally the optimal solution for the unknown configuration parameters is obtained, and the optimal solution and the known configuration parameters constitute the target configuration parameters.

[0066] For example, for any of the configuration tasks of determining model parameters based on known hardware and software information, determining model scale based on known hardware and software, determining hardware and software information based on cost, and determining hardware and software information based on time consumption, computing power requirements are selected from C1-C3, bandwidth requirements are selected from T1-T8, and bandwidth requirements are selected from M1-M according to the configuration attributes and configuration parameters of the configuration task.11 Select the video memory requirements, and build constraints based on the selected computing power requirements, bandwidth requirements, and video memory requirements. Then select the objective function with the shortest time. The planning algorithm is nonlinear programming (such as gradient descent method, interior point method). Finally, based on the constraints, use the nonlinear programming algorithm to iterate the objective function with the shortest time to obtain the optimal solution.

[0067] Optionally, after obtaining the target configuration parameters, i.e., the optimal configuration parameters, if the hardware and software environment is input by the Large Model Cluster Optimal Configuration Task Management Module, testing is performed directly in the specified environment. The required operating system, drivers, and other necessary software components are configured and loaded. If the hardware and software environment is configured by the Configuration Optimization Module, multiple different hardware and software combinations are selected for testing based on the optimal configuration. These combinations include different computing resources (such as different GPU and CPU models) and storage structures (such as SSDs and HDDs). Independent testing is performed for each environment to evaluate the versatility and adaptability of the configuration. The large model is launched in the cluster environment and load tested based on the configuration to evaluate the model's performance in different environments. Performance data is collected across multiple dimensions, including throughput, latency, energy consumption, and video memory usage. Cluster health (such as temperature and power consumption) is also monitored to prevent hardware overload or failure. All data generated during testing (including performance metrics, system logs, and error reports) is stored in a central database for subsequent analysis and policy adjustments. A detailed test report is generated for each test environment, including configuration parameters, test results, and a description of any anomalies or issues.

[0068] Real-time status data is transmitted to the calculation policy learning module for dynamic policy adjustment. After receiving the real-time status data, the calculation policy learning module generates a new policy function F(x) based on this data. F(x) is a performance prediction function that comprehensively considers different configurations in the current environment and predicts their impact on system performance. Specifically, F(x) can be expressed as follows: F(x) = α * f(x) + β. Here, f(x) is the performance function based on the current configuration, while α and β are coefficients used to adjust and modify the prediction function to better match the actual situation. Through continuous learning and adjustment, this policy function can be gradually optimized, thereby improving the overall performance of the system. After receiving the updated policy function F(x), the configuration optimization module uses it to update the existing optimization policy. The new optimization policy adjusts the system configuration based on the latest prediction function to achieve optimal performance. This optimization process is dynamic and can adapt to changes in the environment and real-time updates to the system status, ensuring that the system always operates in optimal conditions.

[0069] Using the calculation strategy described above, the resource configuration method for the large model cluster in this embodiment is used to reconfigure and generate new target configuration parameters. The large model is then redeployed in the cluster and tested to evaluate the effectiveness of the new configuration. The above process is repeated, performing multiple iterative calculations. The results of each iteration are used to further optimize the configuration and strategy. Through multiple iterations, the optimal solution is gradually approached, ensuring that the final configuration performs optimally across all tested hardware and software environments. A convergence criterion (such as performance improvement less than a certain threshold or reaching the expected target) is set to determine whether to stop iteration. If performance no longer significantly improves after multiple iterations, the optimal configuration is considered reached and iteration can be stopped. The final configuration results (including model parameters, resource allocation, strategy, etc.) after multiple iterations are transmitted back to the large model cluster optimal configuration task management module. A final configuration report is generated, containing a detailed record of all tests and iterations, for administrator review and decision-making. The administrator reviews the final configuration report to confirm that the configuration is reasonable and that performance meets expectations. If confirmed, the final configuration is deployed to the production environment. If there are any questions or further optimization is needed, the previous steps can be returned for adjustment.

[0070] The technical solution of this embodiment determines configuration parameters based on the configuration task selected by the user; wherein the configuration parameters include: one or more of hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism and zero-redundancy parallel mode; and some of the configuration parameters are known configuration parameters, and the other part is unknown configuration parameters; based on the configuration task and configuration parameters, at least one of computing power requirements, bandwidth requirements and video memory requirements is determined; according to at least one of computing power requirements, bandwidth requirements and video memory requirements, constraints and objective functions are constructed; wherein the constraints include computing power constraints, bandwidth constraints, storage constraints, cost constraints and at least one of training or inference speed equations; the objective function includes the shortest time objective function and the highest cost ratio objective function; based on the constraints, the objective function is iteratively optimized using a preset planning algorithm to solve the unknown configuration parameters and obtain the target configuration parameters. The resource configuration method for a large model cluster provided by the embodiment of the present invention can determine the optimal large model learning parameters or hardware resource configuration, thereby improving the efficiency and reliability of large model cluster resource configuration.

[0071] Example 2

[0072] Figure 3 This is a schematic diagram of the structure of a resource configuration device for a large model cluster provided by the second embodiment of the invention. Figure 3 As shown, the device includes:

[0073] Configuration parameter determination module 310 is used to determine configuration parameters based on the configuration task selected by the user; wherein the configuration parameters include: one or more of: hyperparameters, mini-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism, and zero-redundancy parallel mode; and some of the configuration parameters are known configuration parameters and others are unknown configuration parameters;

[0074] A demand determination module 320, configured to determine at least one of a computing power requirement, a bandwidth requirement, and a video memory requirement based on the configuration task and the configuration parameters;

[0075] The constraint and objective function construction module 330 is configured to construct constraint conditions and objective functions based on at least one of computing power requirements, bandwidth requirements, and video memory requirements; wherein the constraint conditions include at least one of computing power constraints, bandwidth constraints, storage constraints, cost constraints, and training or inference speed equations; and the objective functions include a shortest time objective function and a highest cost ratio objective function;

[0076] The target configuration parameter acquisition module 340 is used to iteratively optimize the objective function using a preset planning algorithm based on the constraint conditions to solve the unknown configuration parameters and obtain the target configuration parameters.

[0077] Optionally, the configuration task includes the following configuration properties: activation value status, large model processing stage, and parallel mode; among them, the activation value status includes whether activation value recalculation is enabled or not; enabling activation value recalculation includes full activation value recalculation and selective activation value recalculation; the large model processing stage includes the training stage and the inference stage; the parallel modes include: data parallelism, pipeline parallelism, tensor parallelism, zero-redundancy parallelism, 3D parallelism, sequence parallelism, and no parallelism.

[0078] Optionally, the demand determination module 320 is further configured to:

[0079] Determine the computing power requirements based on the activation value status, large model processing stage, and configuration parameters; or

[0080] Determine bandwidth requirements based on parallelization methods, large model processing stages, and configuration parameters; or,

[0081] Determine the video memory requirements based on the activation value status, large model processing stage, parallelization method and configuration parameters.

[0082] Optionally, the demand determination module 320 is further configured to:

[0083] Determine the forward propagation computing power requirements based on the full batch size, context length, number of Transformer layers, hidden layer dimensions, and vocabulary length;

[0084] Determine the computing power requirements based on the forward propagation computing power requirements, activation value status, and large model processing stage.

[0085] Optionally, the demand determination module 320 is further configured to:

[0086] When the large model processing stage is the training stage, if the parallel mode is tensor parallel, the bandwidth requirement is determined according to the micro-batch size, context length, hidden layer dimension, number of Transformer layers, full batch size, pipeline parallelism, and data parallelism; if the parallel mode is pipeline parallel, the bandwidth requirement is determined according to the micro-batch size, context length, hidden layer dimension, tensor parallelism, full batch size, and data parallelism; if the parallel mode is data parallel, the bandwidth requirement is determined according to the number of Transformer layers, hidden layer dimension, pipeline parallelism, and tensor parallelism; if the parallel mode is zero-redundancy parallel mode, the bandwidth requirement is determined according to the number of model parameters; among which, the number of model parameters is determined according to the number of Transformer layers, hidden layer dimension, vocabulary length, and context length;

[0087] When the large model processing stage is the inference stage, if the parallelization method is tensor parallelism, the bandwidth requirement is determined based on the context length, number of Transformer layers, hidden layer dimension, and pipeline parallelism; if the parallelization method is pipeline parallelism, the bandwidth requirement is determined based on the context length, hidden layer dimension, and tensor parallelism.

[0088] Optionally, the demand determination module 320 is further configured to:

[0089] When the large model processing stage is the training stage, determine the combination of activation value state and parallel mode, determine the required configuration parameters based on the combination, and determine the video memory requirement based on the configuration parameters;

[0090] When the large model processing stage is the inference stage, the video memory requirement is determined based on the number of model parameters, data size, micro-batch size, number of attention heads, hidden layer dimension, context length, data parallelism, and number of intermediate variables.

[0091] Optionally, the constraint condition and objective function construction module 330 is further used to:

[0092] Determine the computing power constraint based on the relationship between the computing power requirement and the target computing power;

[0093] Determine bandwidth constraints based on the relationship between bandwidth requirements and target bandwidth;

[0094] Determine storage constraints based on the size relationship between video memory requirements and target video memory;

[0095] Determine cost constraints based on the relationship between cost requirements and target costs;

[0096] The training or inference speed constraint is determined based on the relationship between the training or inference speed and the target speed.

[0097] Optionally, the configuration task includes a configuration task of determining model parameters based on known software and hardware information, a configuration task of determining model scale based on known software and hardware, a configuration task of determining software and hardware information based on cost, and a configuration task of determining software and hardware information based on time consumption.

[0098] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not fully described in this embodiment, please refer to the methods provided by all the above embodiments of the present invention.

[0099] Example 3

[0100] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, the connections and relationships between the components, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0101] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0102] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0103] The processor 11 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the resource configuration method for a large model cluster.

[0104] In some embodiments, the resource configuration method for a large model cluster may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the resource configuration method for the large model cluster described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the resource configuration method for the large model cluster by any other appropriate means (e.g., by means of firmware).

[0105] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0106] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0107] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0108] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0109] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0110] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0111] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the resource configuration method for a large model cluster as provided in any embodiment of the present application.

[0112] The computer program product may be implemented by writing computer program code for performing the operations of the present invention in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0113] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0114] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A resource configuration method for a large model cluster, characterized in that: include: Determine configuration parameters based on the configuration task selected by the user; wherein the configuration parameters include: one or more of: hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, number of model parameters, data parallelism, pipeline parallelism, tensor parallelism, and zero-redundancy parallelism mode; and some of the configuration parameters are known configuration parameters and others are unknown configuration parameters; Determine at least one of computing power requirements, bandwidth requirements, and video memory requirements based on the configuration task and the configuration parameters; Constructing a constraint condition and an objective function based on at least one of the computing power requirement, bandwidth requirement, and video memory requirement; wherein the constraint condition includes at least one of a computing power constraint condition, a bandwidth constraint condition, a storage constraint condition, a cost constraint condition, and a training or inference speed constraint condition; and the objective function includes a minimum time consumption objective function and a maximum cost ratio objective function; Based on the constraint conditions, the objective function is iteratively optimized using a preset planning algorithm to solve the unknown configuration parameters and obtain target configuration parameters.

2. The method according to claim 1, characterized in that The configuration task includes the following configuration properties: activation value status, large model processing stage, and parallel mode; wherein, the activation value status includes enabling activation value recalculation and disabling activation value recalculation; enabling activation value recalculation includes full activation value recalculation and selective activation value recalculation; the large model processing stage includes training stage and inference stage; the parallel mode includes: data parallelism, pipeline parallelism, tensor parallelism, zero-redundancy parallelism, 3D parallelism, sequence parallelism, and no parallelism.

3. The method according to claim 2, characterized in that Determining at least one of a computing power requirement, a bandwidth requirement, and a video memory requirement based on the configuration task and the configuration parameters includes: Determine the computing power requirement based on the activation value state, the large model processing stage and the configuration parameters; or, Determine the bandwidth requirement based on the parallel mode, the large model processing stage and the configuration parameters; or, The video memory requirement is determined according to the activation value state, the large model processing stage, the parallelization method and the configuration parameters.

4. The method according to claim 3, characterized in that Determining computing power requirements according to the activation value state, the large model processing stage, and the configuration parameters includes: Determine the forward propagation computing power requirements based on the full batch size, context length, number of Transformer layers, hidden layer dimensions, and vocabulary length; The computing power requirement is determined based on the forward propagation computing power requirement, the activation value state, and the large model processing stage.

5. The method according to claim 3, characterized in that Determining bandwidth requirements according to the parallelization mode, the large model processing stage, and the configuration parameters includes: When the large model processing stage is the training stage, if the parallel mode is tensor parallel, the bandwidth requirement is determined according to the micro-batch size, context length, hidden layer dimension, number of Transformer layers, full batch size, pipeline parallelism and data parallelism; if the parallel mode is pipeline parallel, the bandwidth requirement is determined according to the micro-batch size, context length, hidden layer dimension, tensor parallelism, full batch size and data parallelism; if the parallel mode is data parallel, the bandwidth requirement is determined according to the number of Transformer layers, hidden layer dimension, pipeline parallelism and tensor parallelism; if the parallel mode is zero-redundancy parallel mode, the bandwidth requirement is determined according to the model parameter quantity; wherein the model parameter quantity is determined according to the number of Transformer layers, hidden layer dimension, vocabulary length and context length; When the large model processing stage is the inference stage, if the parallelization method is tensor parallelism, the bandwidth requirement is determined based on the context length, number of Transformer layers, hidden layer dimension, and pipeline parallelism; if the parallelization method is pipeline parallelism, the bandwidth requirement is determined based on the context length, hidden layer dimension, and tensor parallelism.

6. The method according to claim 3, characterized in that Determining the video memory requirement according to the activation value state, the large model processing stage, the parallelization method, and the configuration parameters includes: When the large model processing stage is a training stage, determining a combination of the activation value state and the parallel mode, determining required configuration parameters based on the combination, and determining a video memory requirement based on the configuration parameters; When the large model processing stage is the inference stage, the video memory requirement is determined based on the number of model parameters, data size, micro-batch size, number of attention heads, hidden layer dimension, context length, data parallelism, and number of intermediate variables.

7. The method according to claim 1, characterized in that Constructing a constraint condition based on at least one of the computing power requirement, bandwidth requirement, and video memory requirement includes: Determine the computing power constraint condition based on the relationship between the computing power requirement and the target computing power; Determining a bandwidth constraint condition based on a size relationship between the bandwidth requirement and the target bandwidth; Determining a storage constraint condition based on a size relationship between the video memory requirement and the target video memory; Determine cost constraints based on the relationship between cost requirements and target costs; The training or inference speed constraint is determined based on the relationship between the training or inference speed and the target speed.

8. The method according to claim 1, characterized in that The configuration tasks include configuration tasks of determining model parameters based on known software and hardware information, configuration tasks of determining model scale based on known software and hardware, configuration tasks of determining software and hardware information based on cost, and configuration tasks of determining software and hardware information based on time consumption.

9. A resource configuration device for a large model cluster, characterized in that: include: A configuration parameter determination module is used to determine configuration parameters based on the configuration task selected by the user; wherein the configuration parameters include: one or more of: hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism, and zero-redundancy parallel mode; and some of the configuration parameters are known configuration parameters and others are unknown configuration parameters; a demand determination module, configured to determine at least one of a computing power requirement, a bandwidth requirement, and a video memory requirement based on the configuration task and the configuration parameters; a constraint condition and objective function construction module, configured to construct constraint conditions and objective functions based on at least one of the computing power requirement, bandwidth requirement, and video memory requirement; wherein the constraint conditions include at least one of a computing power constraint, a bandwidth constraint, a storage constraint, a cost constraint, and a training or inference speed equation; and the objective function includes a shortest time objective function and a highest cost ratio objective function; The target configuration parameter acquisition module is used to iteratively optimize the objective function based on the constraint conditions using a preset planning algorithm to solve the unknown configuration parameters and obtain the target configuration parameters.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to execute the resource configuration method for a large model cluster according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the resource configuration method for a large model cluster according to any one of claims 1 to 8 when executed.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the resource configuration method for a large model cluster according to any one of claims 1 to 8.

Images