Resource allocation method, device and equipment for large model cluster, storage medium and program product

Abstract

The invention discloses a resource allocation method and device for a large model cluster, equipment, a storage medium and a program product. Comprises: determining configuration parameters based on a configuration task selected by a user; determining at least one of a computing power demand, a bandwidth demand and a video memory demand based on the configuration task and the configuration parameters; constructing a constraint condition and a target function according to at least one of the computing power demand, the bandwidth demand and the video memory demand; and carrying out iterative optimization on the target function by utilizing a preset planning algorithm based on the constraint condition so as to solve the unknown configuration parameter and obtain a target configuration parameter. According to the resource allocation method for the large model cluster, provided by the embodiment of the invention, the optimal large model learning parameter or hardware resource allocation can be determined, and the efficiency and reliability of resource allocation of the large model cluster are improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular, to a resource configuration method, device, equipment, storage medium and program product for a large model cluster. Background Art

[0002] The field of artificial intelligence has made significant progress in recent years, and large models have attracted widespread attention due to their powerful learning capabilities and wide range of application scenarios. However, large models are more complex than deep learning. If you want to determine the hardware configuration of a large model training or reasoning cluster, you usually have to consider various factors, such as parallel configurations, data parallelism, model parallelism, pipeline parallelism, and memory optimization technology, and there are also multiple ways for communication equipment between nodes and GPUs. The different scales of models have different requirements for video memory, the balance between model acceleration and performance, etc. These factors also affect each other. The methods commonly used for large model hardware and software configuration are: 1. Retrieve selection guides through the Internet: For example, refer to professional hardware selection guides, which usually provide suggestions and recommended configurations on GPU selection, memory requirements, storage requirements, etc. 2. Manual experience configuration: This usually involves selecting hardware configurations based on historical data and professional knowledge. 3. Performance testing and evaluation: Evaluate the impact of different hardware configurations on large model training and reasoning tasks through actual performance testing. This may include benchmarking different GPU models, memory sizes, storage solutions, etc. to determine the best hardware combination. 4. Formula calculation: Estimate the required hardware resources through mathematical formulas.

[0003] Existing methods have the following defects: 1. Retrieving selection guides through the Internet: Guides may provide general suggestions, but may not be suitable for all specific scenarios; hardware is developing rapidly, and guides may not be updated in time to reflect the latest technology; guides may be based on specific suppliers or technology preferences, and are more targeted. 2. Manual experience configuration: highly dependent on personal or team experience, may not be friendly to novices; configuration selection may be subjective, and it is difficult to ensure objectivity and universal applicability; technology is developing rapidly, and personal experience may not keep up with the latest hardware development and best practices. 3. Performance testing and evaluation: Conducting exhaustive performance testing may be very time-consuming, increase project cycles, and may require multiple trials and errors to find the best configuration, which is time-consuming and resource-intensive; test results may not fully predict performance in actual applications. 4. Formula calculation: accurate input data is required, and inaccurate data will lead to biased results; it is easy to ignore physical limitations and compatibility issues in actual hardware configurations; over-theorizing, sometimes out of touch with actual application scenarios. Summary of the invention

[0004] The embodiments of the present invention provide a resource configuration method, apparatus, device, storage medium and program product for a large model cluster, which can determine the optimal large model learning parameters or hardware resource configuration, and improve the efficiency and reliability of the large model cluster resource configuration.

[0005] In a first aspect, an embodiment of the present invention provides a resource configuration method for a large model cluster, comprising:

[0006] Determine configuration parameters based on the configuration task selected by the user; wherein the configuration parameters include: one or more of hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism and zero-redundancy parallel mode; and some of the configuration parameters are known configuration parameters, and the other part are unknown configuration parameters;

[0007] Determine at least one of computing power requirement, bandwidth requirement and video memory requirement based on the configuration task and the configuration parameters;

[0008] Constructing a constraint condition and an objective function according to at least one of the computing power requirement, bandwidth requirement and video memory requirement; wherein the constraint condition includes at least one of a computing power constraint condition, a bandwidth constraint condition, a storage constraint condition, a cost constraint condition and a training or reasoning speed equation; the objective function includes a shortest time objective function and a highest cost ratio objective function;

[0009] Based on the constraint conditions, the objective function is iteratively optimized using a preset planning algorithm to solve the unknown configuration parameters and obtain target configuration parameters.

[0010] In a second aspect, an embodiment of the present invention further provides a resource configuration device for a large model cluster, including:

[0011] A configuration parameter determination module is used to determine configuration parameters based on the configuration task selected by the user; wherein the configuration parameters include: one or more of hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism and zero-redundancy parallel mode; and some of the configuration parameters are known configuration parameters, and the other part are unknown configuration parameters;

[0012] A demand determination module, configured to determine at least one of a computing power demand, a bandwidth demand, and a video memory demand based on the configuration task and the configuration parameters;

[0013] A constraint condition and objective function construction module, used to construct constraint conditions and objective functions according to at least one of the computing power requirements, bandwidth requirements and video memory requirements; wherein the constraint conditions include at least one of computing power constraints, bandwidth constraints, storage constraints, cost constraints and training or inference speed equations; the objective function includes a shortest time objective function and a highest cost ratio objective function;

[0014] The target configuration parameter acquisition module is used to iteratively optimize the target function based on the constraint conditions using a preset planning algorithm to solve the unknown configuration parameters and obtain the target configuration parameters.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the resource configuration method for the large model cluster described in the embodiment of the present invention.

[0019] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the resource configuration method for a large model cluster described in an embodiment of the present invention when executed.

[0020] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the resource configuration method for a large model cluster as described in an embodiment of the present invention.

[0021] The embodiment of the present invention discloses a resource configuration method, device, equipment, storage medium and program product for a large model cluster. The method comprises: determining configuration parameters based on a configuration task selected by a user; wherein the configuration parameters include: one or more of hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism and zero-redundancy parallel mode; and part of the configuration parameters are known configuration parameters, and the other part are unknown configuration parameters; determining at least one of computing power requirements, bandwidth requirements and video memory requirements based on the configuration task and the configuration parameters; constructing constraints and objective functions according to at least one of computing power requirements, bandwidth requirements and video memory requirements; wherein the constraints include at least one of computing power constraints, bandwidth constraints, storage constraints, cost constraints and training or inference speed equations; the objective function includes the shortest time-consuming objective function and the highest cost ratio objective function; iteratively optimizing the objective function based on the constraints using a preset planning algorithm to solve the unknown configuration parameters and obtain the target configuration parameters. The resource configuration method for a large model cluster provided in an embodiment of the present invention can determine the optimal large model learning parameters or hardware resource configuration, thereby improving the efficiency and reliability of the resource configuration of the large model cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of a resource configuration method of a large model cluster in the first embodiment of the present invention;

[0023] Figure 2 It is a structural diagram of a resource configuration system of a large model cluster in the first embodiment of the present invention;

[0024] Figure 3 It is a structural schematic diagram of a resource configuration device for a large model cluster in the second embodiment of the present invention;

[0025] Figure 4 It is a structural schematic diagram of an electronic device in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0027] Embodiment 1

[0028] Figure 1A flowchart of a resource configuration method for a large model cluster provided in Example 1 of the present invention. This embodiment is applicable to situations where the resource configuration of a large model cluster is determined. The method can be executed by a resource configuration device for a large model cluster, and the device can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC or a server, etc.

[0029] Figure 2 is a schematic diagram of a resource configuration system for a large model cluster provided by an embodiment of the present invention. Figure 2 The system includes: a large model cluster optimal configuration task management module, a scale calculation module, a configuration optimization module, a calculation strategy learning module and a large model cluster management module; wherein the configuration optimization module includes a parameter management submodule, a configuration optimization submodule and a calculation strategy submodule.

[0030] Among them, the large model cluster optimal configuration task management module is used to manage configuration tasks. Users select configuration tasks and input relevant parameters through this module. This module sends the tasks selected by users to the configuration optimization module. The scale calculation module consists of multiple scale calculation functions, which are used to calculate scale parameters. The input is large model parameters and training corpus, and the output is scale parameters; scale parameters include: model parameter scale, forward and backward computing power requirements, and corpus scale. The configuration optimization module consists of three submodules: parameter management submodule, configuration optimization submodule, and computing strategy submodule. The input is configuration tasks and their corresponding scale parameters. It outputs the optimal configuration scheme to the large model cluster management module, receives cluster configuration information (computing power, video memory) returned by the large model cluster management module, receives the new computing strategy returned by the computing strategy learning module, and finally outputs the optimized optimal configuration to the large model cluster optimal configuration task management module. The module is responsible for model parameter configuration and optimization calculation. The parameter management submodule consists of model network configuration, hyperparameter configuration, batch size, and parallel configuration. The configuration optimization module consists of optimization modules such as linear programming, nonlinear programming, hybrid programming, and dynamic programming. The computing strategy module consists of computing power requirement, bandwidth requirement, and video memory requirement structural equation management submodules, computing, storage, bandwidth and other constraint management submodules, and computing strategy management submodules.

[0031] The computational strategy learning module consists of a learning algorithm and a strategy update mechanism. Its input is real-time status data and its output is the updated computational strategy. The module is responsible for learning and updating the computational strategy based on historical data and real-time feedback.

[0032] The large model cluster management module is composed of homogeneous or heterogeneous hardware clusters. The input is the optimal configuration, and the output is cluster configuration information (computing power, video memory). The module is responsible for calling cluster resources to test the large model parameter configuration.

[0033] The modules achieve efficient operation through real-time data transmission and collaboration. For example, the large model cluster optimal configuration task management module and the scale calculation module collaborate to complete the calculation preparation for the optimal configuration, and under a specific configuration task, jointly complete the input of the configuration optimization module.

[0034] The configuration optimization module, large model cluster management module and computing strategy learning module collaborate to achieve and optimize the optimal configuration. The configuration optimization module calculates the optimal configuration based on the input and passes it to the large model cluster management module. The large model cluster management module tests the optimal configuration of the large model cluster under the corresponding hardware and software environment, feeds back the cluster configuration information (computing power, video memory) to the configuration optimization module, and outputs real-time status data to the computing strategy learning module. The computing strategy learning module calculates a new computing strategy function based on the real-time status data and learning mechanism and passes it to the configuration optimization module. The configuration optimization module recalculates the optimal configuration based on feedback and updates, iterates according to this process, and sets a convergence standard (such as performance improvement less than a certain threshold, or reaching the expected goal) to determine whether to stop iteration. If the performance no longer improves significantly after multiple iterations, it is considered that the optimal configuration has been reached and the iteration can be stopped.

[0035] The submodules of the configuration optimization module, namely the parameter management module, the configuration optimization module and the calculation strategy module, collaborate to achieve the optimal configuration. The parameter management module determines the parameters to be configured according to the required tasks, the calculation strategy management module determines the calculation strategy according to the required tasks or updates the calculation strategy according to the calculation strategy learning module, the configuration optimization module provides the required parameters according to the parameter management module, and the calculation strategy module provides the structural equation, constraint equation and objective function to determine the planning method and calculate the optimal configuration.

[0036] Specifically, Figure 1 As shown, the method comprises the following steps:

[0037] S110: Determine configuration parameters based on the configuration task selected by the user.

[0038] Among them, the configuration parameters include: one or more of hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism and zero redundancy parallel (ZeRO) mode; and some of the configuration parameters are known configuration parameters, and the other part is unknown configuration parameters. Hyperparameters include the number of iterations and the number of training cycles. Micro-batch size and full batch size can be collectively referred to as batch size; the number of Transformer layers and hidden layer dimensions can be collectively referred to as model structure parameters; data parallelism, pipeline parallelism, tensor parallelism and zero redundancy parallel mode can be collectively referred to as parallel configuration parameters.

[0039] In this embodiment, the configuration tasks are set according to different goals and scenarios, including but not limited to the following types: configuration tasks for determining model parameters based on known software and hardware information, configuration tasks for determining model scale based on known software and hardware, configuration tasks for determining software and hardware information based on cost, and configuration tasks for determining software and hardware information based on time consumption. Among them, the goal of the configuration task for determining model parameters based on known software and hardware information is to adjust the configuration parameters of the model so that the model can achieve the shortest time consumption when training or reasoning under the existing software and hardware configuration. The goal of the configuration task for determining model scale based on known software and hardware is to determine the maximum model scale that can be supported under the current software and hardware conditions. The goal of the configuration task for determining software and hardware information based on cost is to compare the cost and performance of different software and hardware configurations under the premise of maintaining budget constraints, and select the most cost-effective software and hardware configuration suitable for large model training or reasoning. The goal of the configuration task for determining software and hardware information based on time consumption is to select the software and hardware combination that can minimize the running time by evaluating the computing speed and response time of various software and hardware configurations under the premise of maintaining budget constraints.

[0040] Specifically, when the configuration task selected by the user is a configuration task that determines the model parameters based on known software and hardware information, the determined configuration parameters include: hyperparameters: number of iterations, number of training cycles; batch size: micro-batch size, full batch size; parallel configuration parameters: data parallelism, pipeline parallelism, tensor parallelism and zero-redundancy parallel mode. When the configuration task selected by the user is a configuration task that determines the model scale based on known software and hardware, the determined configuration parameters include: model parameter quantity; vocabulary length; context length; model structure parameters: number of Transformer layers, hidden layer dimension; parallel configuration parameters: data parallelism, pipeline parallelism, tensor parallelism and zero-redundancy parallel mode. When the configuration task selected by the user is a configuration task that determines the software and hardware information based on cost, the determined configuration parameters include: hyperparameters: number of iterations, number of training cycles; batch size: micro-batch size, full batch size; parallel configuration parameters: data parallelism, pipeline parallelism, tensor parallelism and zero-redundancy parallel mode. When the configuration task selected by the user is a configuration task that determines the software and hardware information based on time consumption, the determined configuration parameters include: parameters: number of iterations, number of training cycles; batch size: micro-batch size, full batch size; parallel configuration parameters: data parallelism, pipeline parallelism, tensor parallelism and zero redundancy parallel mode.

[0041] S120, determining at least one of a computing power requirement, a bandwidth requirement, and a video memory requirement based on the configuration task and the configuration parameters.

[0042] Among them, computing power requirements (Compute Requirements) are used to indicate the computing power resources required for large models during the training or reasoning stage; bandwidth requirements (Bandwidth Requirements) are used to indicate the bandwidth resources required for large models during the training or reasoning stage; memory requirements (Memory Requirements) are used to indicate the storage resources required for large models during the training or reasoning stage.

[0043] In this embodiment, the computing power requirement, bandwidth requirement and video memory requirement are affected by the configuration attributes of the configuration task. Different configuration attributes will result in different configuration parameters and calculation methods required to determine the computing power requirement, bandwidth requirement and video memory requirement.

[0044] The configuration task includes the following configuration attributes: activation value state, large model processing stage, and parallel mode. The activation value state includes whether activation value recalculation is enabled or disabled; enabling activation value recalculation includes full activation value recalculation and selective activation value recalculation; large model processing stage includes training stage and inference stage; parallel modes include: data parallelism (DP), pipeline parallelism (PP), tensor parallelism (TP), zero redundancy parallelism, 3D parallelism, sequence parallelism, and no parallelism.

[0045] Optionally, the process of determining at least one of computing power requirements, bandwidth requirements and video memory requirements based on configuration tasks and configuration parameters may be: determining computing power requirements according to activation value status, large model processing stage and configuration parameters; or, determining bandwidth requirements according to parallel mode, large model processing stage and configuration parameters; or, determining video memory requirements according to activation value status, large model processing stage, parallel mode and configuration parameters.

[0046] In this embodiment, in order to facilitate the determination of computing power requirements, bandwidth requirements and video memory requirements, each configuration parameter is represented by a letter, namely: micro-batch size is represented as b, full batch size is represented as B, vocabulary length is represented as v, context length is represented as s, number of Transformer layers is represented as l, hidden layer dimension is represented as h, model parameter quantity is represented as P, data parallelism is represented as d, pipeline parallelism is represented as p, and tensor parallelism is represented as t.

[0047] Optionally, the method of determining the computing power requirements based on the activation value status, large model processing stage and configuration parameters can be: determining the forward propagation computing power requirements based on the full batch size, context length, number of Transformer layers, hidden layer dimension and vocabulary length; determining the computing power requirements based on the forward propagation computing power requirements, activation value status and large model processing stage.

[0048] Among them, the calculation formula for determining the forward propagation computing power requirement based on the full batch size, context length, number of Transformer layers, hidden layer dimension, and vocabulary length can be expressed as:

[0049] The process of determining the computing power requirement according to the forward propagation computing power requirement, the activation value state and the large model processing stage can be: when the large model processing stage is the training stage, if the activation value state is that the activation value recalculation is not turned on, the computing power requirement is three times the forward propagation computing power requirement, and the formula can be expressed as: C 1 =3C 前 ; If the activation value state is to enable activation value recalculation, the computing power requirement is four times the forward propagation computing power requirement, and the formula can be expressed as: C 2 =4C 前 When the large model processing stage is the inference stage, the computing power requirement is the forward propagation computing power requirement, and the formula can be expressed as: C 3 =C 前 .

[0050] Optionally, the method of determining the bandwidth requirement according to the parallel mode, the large model processing stage and the configuration parameters can be: when the large model processing stage is the training stage, if the parallel mode is tensor parallelism, the bandwidth requirement is determined according to the micro-batch size, context length, hidden layer dimension, number of Transformer layers, full batch size, pipeline parallelism and data parallelism; if the parallel mode is pipeline parallelism, the bandwidth requirement is determined according to the micro-batch size, context length, hidden layer dimension, tensor parallelism, full batch size and data parallelism; if the parallel mode is data parallelism, the bandwidth requirement is determined according to the number of Transformer layers, hidden layer dimension, pipeline parallelism and tensor parallelism; if the parallel mode is zero redundancy parallel mode, the bandwidth requirement is determined according to the number of model parameters; when the large model processing stage is the inference stage, if the parallel mode is tensor parallelism, the bandwidth requirement is determined according to the context length, number of Transformer layers, hidden layer dimension and pipeline parallelism; if the parallel mode is pipeline parallelism, the bandwidth requirement is determined according to the context length, hidden layer dimension and tensor parallelism.

[0051] The model parameters are determined according to the number of Transformer layers, hidden layer dimensions, vocabulary length, and context length. The zero-redundancy parallel mode includes zero-redundancy parallel mode 1 (ZeRO-1), zero-redundancy parallel mode 2 (ZeRO-2), and zero-redundancy parallel mode 3 (ZeRO-3).

[0052] When the large model processing stage is the training stage, if the parallel mode is tensor parallelism, the calculation formula for bandwidth requirements can be expressed as: T 1= bsh*((4lB)) / (pdb); If the parallel mode is pipeline parallel, the calculation formula of bandwidth requirement can be expressed as: T 2 =(bsh / t)*(2B) / db; If the parallel mode is data parallel, the calculation formula of bandwidth requirement can be expressed as: T 3 =(12lh 2 ) / pt; If the parallel mode is zero-redundancy parallel mode 1, the calculation formula for bandwidth requirement can be expressed as: T 4 =2P; if the parallel mode is zero-redundancy parallel mode 2, the calculation formula for bandwidth requirement can be expressed as: T 5 =2P; if the parallel mode is zero-redundancy parallel mode 3, the calculation formula for bandwidth requirement can be expressed as: T 6 =3P. The calculation formula of the model parameter P can be expressed as: P = 12lh 2 +13lh+vh+sh. When the large model processing stage is the inference stage, if the parallel mode is tensor parallelism, the calculation formula for bandwidth requirements can be expressed as: T 7 =sh*(2l / p); If the parallel mode is pipeline parallel, the calculation formula of bandwidth requirement can be expressed as: T 8 =sh / t.

[0053] Optionally, the method of determining the video memory requirement based on the activation value state, the large model processing stage, the parallel mode and the configuration parameters can be: when the large model processing stage is the training stage, determining the combination of the activation value state and the parallel mode, determining the required configuration parameters based on the combination, and determining the video memory requirement based on the configuration parameters; when the large model processing stage is the inference stage, determining the video memory requirement based on the model parameter quantity, data size, micro-batch size, number of attention heads, hidden layer dimension, context length, data parallelism, and the number of intermediate variables.

[0054] Among them, the combinations of activation value states and parallel modes include the following: zero-redundancy basic parallel mode (baseline), zero-redundancy parallel mode 1, zero-redundancy parallel mode 2, zero-redundancy parallel mode 3, no parallelism, 3D parallelism, 3D parallelism + sequence parallelism, 3D parallelism + selective activation value recalculation, 3D parallelism + sequence parallelism + selective activation value recalculation, 3D parallelism + full activation value recalculation.

[0055] Specifically, when the combination mode is the zero-redundancy basic parallel mode, the determined configuration parameters include the model parameter quantity P, the pipeline parallelism p and the tensor parallelism t, and the calculation formula of the video memory requirement can be expressed as: M 1 =20P / (pt); When the combination mode is zero-redundancy parallel mode 1, the determined configuration parameters include: data parallelism d, model parameter quantity P, pipeline parallelism p and tensor parallelism t, and the calculation formula of video memory requirement can be expressed as: M2 =(4+16d)P / (pt); When the combination mode is zero-redundancy parallel mode 2, the determined configuration parameters include: data parallelism d, model parameter quantity P, pipeline parallelism p and tensor parallelism t, and the calculation formula of video memory requirement can be expressed as: M 3 =(2+18d)P / (pt); When the combination mode is zero-redundancy parallel mode 3, the determined configuration parameters include: data parallelism d, model parameter quantity P, pipeline parallelism p and tensor parallelism t, and the calculation formula of video memory requirement can be expressed as: M 4 =20P / (dpt); When the combination mode is no parallelism, the determined configuration parameters include: micro-batch size b, context length s, hidden layer dimension h, number of Transformer layers l, number of attention heads a, and the calculation formula of video memory requirement can be expressed as: M 5 =bshl(34+5sa / h); When the combination mode is 3D parallel, the determined configuration parameters include: micro-batch size b, context length s, hidden layer dimension h, number of Transformer layers l, tensor parallelism t, number of attention heads a, and the calculation formula of video memory requirement can be expressed as: M 6 =bshl(10+24 / t+5sa / (ht)); When the combination mode is 3D parallelism + sequence parallelism, the determined configuration parameters include: micro-batch size b, context length s, hidden layer dimension h, number of Transformer layers l, tensor parallelism t, number of attention heads a, and the calculation formula of video memory requirement can be expressed as: M 7 =bshl(34 / t+5sa / (ht)); When the combination mode is 3D parallelism + selective activation value recalculation, the determined configuration parameters include: micro-batch size b, context length s, hidden layer dimension h, Transformer layer number l, tensor parallelism t, and the calculation formula of video memory requirement can be expressed as: M 8 =bshl(10+24 / t); When the combination is 3D parallelism + sequence parallelism + selective activation value recalculation, the determined configuration parameters include: micro-batch size b, context length s, hidden layer dimension h, Transformer layer number l, tensor parallelism t, and the calculation formula for video memory requirements can be expressed as: M 9 =bshl(34 / t); When the combination mode is 3D parallel + full activation value recalculation, the determined configuration parameters include: micro-batch size b, context length s, hidden layer dimension h, and the calculation formula for video memory requirement is: M 10 =2bsh.

[0056] Specifically, when the large model processing stage is the inference stage, the calculation formula for the video memory requirement can be expressed as: M 11=P*n+b×(2×1×a×h×s×n)+b×(s×d×c×n), where n represents the data size (number of bytes) and c represents the number of intermediate variables.

[0057] Optionally, you can also average the number of contexts (tokens) on each sample in one iteration S average , the total number of samples in one iteration n total and the number of iterations n epoch Determine the corpus size S, the calculation formula can be expressed as: S = S average ×n total ×n epoch .

[0058] S130, constructing constraint conditions and objective functions according to at least one of computing power requirements, bandwidth requirements, and video memory requirements.

[0059] Among them, the constraints include at least one of computing power constraints, bandwidth constraints, storage constraints, cost constraints and training or inference speed constraints; the objective function includes the shortest time objective function and the highest cost ratio objective function.

[0060] Among them, the computing power constraint can be understood as the total computing power required by the large model during training or reasoning cannot exceed the available computing power of the system; the bandwidth constraint can be understood as the total bandwidth requirement required by the large model during training or reasoning cannot exceed the bandwidth capacity of the system; the storage constraint can be understood as the total storage requirement required by the large model during training or reasoning cannot exceed the storage capacity of the system; the cost constraint can be understood as the hardware cost selected by the large model during training or reasoning cannot exceed the budget; the training or reasoning speed constraint can be understood as the training or reasoning speed of the hardware configuration of the large model during training or reasoning must meet the time requirements of the task.

[0061] Specifically, the method of constructing constraints based on at least one of the computing power requirements, bandwidth requirements and video memory requirements can be: determining the computing power constraint based on the relationship between the computing power requirement and the target computing power; determining the bandwidth constraint based on the relationship between the bandwidth requirement and the target bandwidth; determining the storage constraint based on the relationship between the video memory requirement and the target video memory; determining the cost constraint based on the relationship between the cost requirement and the target cost; determining the training or inference speed constraint based on the relationship between the training or inference speed and the target speed.

[0062] Among them, the target computing power can be understood as the available computing power of the system; the target bandwidth can be understood as the bandwidth capacity of the system; the target video memory can be understood as the storage capacity of the system; the target cost can be understood as the budget; and the target speed can be understood as the estimated reasoning or training speed. In this embodiment, the size relationship between the computing power demand and the target computing power is that the computing power demand is less than or equal to the target computing power, that is, the computing power constraint can be expressed as: computing power demand ≤ target computing power. The size relationship between the bandwidth demand and the target bandwidth is that the bandwidth demand is less than or equal to the target bandwidth, that is, the bandwidth constraint can be expressed as: bandwidth demand ≤ target bandwidth. The size relationship between the video memory demand and the target video memory is that the video memory demand is less than or equal to the target video memory, that is, the storage constraint can be expressed as: video memory demand ≤ target video memory. The size relationship between the cost demand and the target cost is that the cost demand is less than or equal to the target cost, that is, the cost constraint can be expressed as: cost demand ≤ target cost. The size relationship between the training or reasoning speed and the target speed is that the training or reasoning speed is greater than or equal to the target speed, that is, the training or reasoning constraint can be expressed as: training or reasoning speed ≥ target speed.

[0063] In this embodiment, the shortest time-consuming objective function can be expressed as: training time = 8TP / nX, where T is the total number of contexts (tokens), P is the number of large model parameters, n is the total number of GPUs, X is the throughput, and X is the observed value. The cost ratio can be understood as the ratio between hardware performance and cost, so the highest cost ratio objective function can be expressed as: cost ratio = hardware performance / hardware cost.

[0064] S140, iteratively optimizing the objective function using a preset planning algorithm based on the constraint conditions to solve unknown configuration parameters and obtain target configuration parameters.

[0065] Among them, the planning algorithm can be linear programming, nonlinear programming, hybrid programming, and dynamic programming. In this embodiment, the expressions of computing power requirements, bandwidth requirements, and video memory requirements are selected based on the configuration tasks and configuration parameters, and then the constraints are determined based on the selected expressions of computing power requirements, bandwidth requirements, and video memory requirements. The process of iterative optimization of the objective function can be: setting initial values ​​for unknown configuration parameters, and then iteratively solving the objective function based on the constraints using the selected planning algorithm, and finally obtaining the optimal solution for the unknown configuration parameters, and the optimal solution and the known configuration parameters constitute the target configuration parameters.

[0066] Exemplarily, for any of the configuration tasks of determining model parameters based on known software and hardware information, determining model scale based on known software and hardware, determining software and hardware information based on cost, and determining software and hardware information based on time consumption, the configuration attributes and configuration parameters of the configuration task are selected from C 1 -C 3 Select the computing power requirement from T1 -T 8 Select the bandwidth requirement from M 1 -M 11 Select the video memory requirements, and build constraints based on the selected computing power requirements, bandwidth requirements, and video memory requirements, and select the shortest objective function. The planning algorithm is nonlinear programming (such as gradient descent method, interior point method). Finally, based on the constraints, use the nonlinear programming algorithm to optimize and iterate the shortest objective function to obtain the optimal solution.

[0067] Optionally, after obtaining the target configuration parameters, i.e., the optimal configuration parameters, if the hardware and software environment is input by the large model cluster optimal configuration task management module, the test is directly performed in the specified environment. Configure and load the required operating system, drivers, and other necessary software components. If the hardware and software environment is configured by the configuration optimization module, select multiple different hardware and software combinations for testing based on the optimal configuration. The hardware and software combinations include different computing resources (such as different models of GPUs, CPUs) and storage structures (such as SSDs, HDDs). Perform independent tests on each environment to evaluate the versatility and adaptability of the configuration. Start the large model in the cluster environment, perform load tests based on the configuration, and evaluate the performance of the model in different environments. Collect performance data in multiple dimensions including throughput, latency, energy consumption, and video memory usage. At the same time, monitor the health status of the cluster (such as temperature and power consumption) to prevent hardware overload or failure. Store all data generated during the test (including performance indicators, system logs, error reports, etc.) in a central database for subsequent analysis and policy adjustment. Generate a detailed test report for each test environment, including configuration parameters, test results, and descriptions of any exceptions or problems.

[0068] The real-time status data is transmitted to the calculation strategy learning module for dynamic strategy adjustment. After receiving the real-time status data, the calculation strategy learning module will generate a new strategy function F(x) based on this data. F(x) is a performance prediction function that comprehensively considers different configurations in the current environment and predicts their impact on system performance. Specifically, F(x) can be expressed as follows: F(x) = α*f(x) + β, where f(x) is a performance function based on the existing configuration, and α and β are coefficients used for adjustment and correction to ensure that the prediction function is more consistent with the actual situation. Through continuous learning and adjustment, this strategy function can be gradually optimized to improve the overall performance of the system. After receiving the updated strategy function F(x), the configuration optimization module will use it to update the existing optimization strategy. The new optimization strategy will adjust the system configuration based on the latest prediction function to achieve the best performance. This optimization process is dynamic and can be adjusted as the environment changes and the system status is updated in real time, thereby ensuring that the system is always running in the best state.

[0069] Using the calculation strategy in the above process, the resource configuration method of the large model cluster in this embodiment is used to reconfigure and generate new target configuration parameters. Then redeploy the large model in the cluster and test it to evaluate the effect of the new configuration. Repeat the above process and perform multiple iterative calculations. The results of each iteration are used to further optimize the configuration and strategy. Through multiple iterations, the optimal solution is gradually approached to ensure that the final configuration can perform best in all tested hardware and software environments. Set a convergence standard (such as performance improvement less than a certain threshold, or reaching the expected goal) to determine whether to stop iteration. If the performance is no longer significantly improved after multiple iterations, it is considered that the optimal configuration has been reached and the iteration can be stopped. The final configuration results (including model parameters, resource allocation, strategy, etc.) after multiple iterations are returned to the large model cluster optimal configuration task management module. Generate a final configuration report containing detailed records of all tests and iterations for administrator review and decision-making. The administrator reviews the final configuration report to confirm that the configuration is reasonable and the performance meets expectations. If confirmed to be correct, the final configuration is deployed to the production environment; if there are any questions or further optimization is required, you can return to the above steps for adjustment.

[0070] The technical solution of this embodiment determines the configuration parameters based on the configuration task selected by the user; wherein the configuration parameters include: one or more of hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism and zero redundancy parallel mode; and part of the configuration parameters are known configuration parameters, and the other part is unknown configuration parameters; at least one of computing power requirements, bandwidth requirements and video memory requirements is determined based on the configuration task and configuration parameters; constraints and objective functions are constructed according to at least one of computing power requirements, bandwidth requirements and video memory requirements; wherein the constraints include computing power constraints, bandwidth constraints, storage constraints, cost constraints and at least one of training or reasoning speed equations; the objective function includes the shortest time-consuming objective function and the highest cost ratio objective function; based on the constraints, the objective function is iteratively optimized using a preset planning algorithm to solve the unknown configuration parameters and obtain the target configuration parameters. The resource configuration method of the large model cluster provided in the embodiment of the present invention can determine the optimal large model learning parameters or hardware resource configuration, and improve the efficiency and reliability of the resource configuration of the large model cluster.

[0071] Embodiment 2

[0072] Figure 3 is a schematic diagram of a resource configuration device for a large model cluster provided in the second embodiment of the invention. Figure 3 As shown, the device comprises:

[0073] The configuration parameter determination module 310 is used to determine the configuration parameters based on the configuration task selected by the user; wherein the configuration parameters include: one or more of hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism and zero-redundancy parallel mode; and some of the configuration parameters are known configuration parameters, and the other part are unknown configuration parameters;

[0074] A demand determination module 320, configured to determine at least one of a computing power demand, a bandwidth demand, and a video memory demand based on the configuration task and the configuration parameters;

[0075] The constraint condition and objective function construction module 330 is used to construct the constraint condition and the objective function according to at least one of the computing power requirement, the bandwidth requirement and the video memory requirement; wherein the constraint condition includes at least one of the computing power constraint condition, the bandwidth constraint condition, the storage constraint condition, the cost constraint condition and the training or reasoning speed equation; the objective function includes the shortest time consumption objective function and the highest cost ratio objective function;

[0076] The target configuration parameter acquisition module 340 is used to iteratively optimize the target function based on the constraint conditions using a preset planning algorithm to solve the unknown configuration parameters and obtain the target configuration parameters.

[0077] Optionally, the configuration task includes the following configuration properties: activation value status, large model processing stage, and parallel mode; wherein, the activation value status includes enabling activation value recalculation and disabling activation value recalculation; enabling activation value recalculation includes full activation value recalculation and selective activation value recalculation; the large model processing stage includes the training stage and the inference stage; the parallel modes include: data parallelism, pipeline parallelism, tensor parallelism, zero redundancy parallelism, 3D parallelism, sequence parallelism, and no parallelism.

[0078] Optionally, the demand determination module 320 is further configured to:

[0079] Determine the computing power requirements based on the activation value status, large model processing stage and configuration parameters; or,

[0080] Determine bandwidth requirements based on parallelism, large model processing stages, and configuration parameters; or,

[0081] Determine the video memory requirements based on the activation value status, large model processing stage, parallelization method and configuration parameters.

[0082] Optionally, the demand determination module 320 is further configured to:

[0083] Determine the forward propagation computing power requirements based on the full batch size, context length, number of Transformer layers, hidden layer dimensions, and vocabulary length;

[0084] Determine the computing power requirements based on the forward propagation computing power requirements, activation value status, and large model processing stage.

[0085] Optionally, the demand determination module 320 is further configured to:

[0086] When the large model processing stage is the training stage, if the parallel mode is tensor parallelism, the bandwidth requirement is determined according to the micro-batch size, context length, hidden layer dimension, number of Transformer layers, full batch size, pipeline parallelism and data parallelism; if the parallel mode is pipeline parallelism, the bandwidth requirement is determined according to the micro-batch size, context length, hidden layer dimension, tensor parallelism, full batch size and data parallelism; if the parallel mode is data parallelism, the bandwidth requirement is determined according to the number of Transformer layers, hidden layer dimension, pipeline parallelism and tensor parallelism; if the parallel mode is zero-redundancy parallel mode, the bandwidth requirement is determined according to the model parameter quantity; among which, the model parameter quantity is determined according to the number of Transformer layers, hidden layer dimension, vocabulary length and context length;

[0087] When the large model processing stage is the inference stage, if the parallel mode is tensor parallelism, the bandwidth requirement is determined based on the context length, the number of Transformer layers, the hidden layer dimension, and the pipeline parallelism; if the parallel mode is pipeline parallelism, the bandwidth requirement is determined based on the context length, the hidden layer dimension, and the tensor parallelism.

[0088] Optionally, the demand determination module 320 is further configured to:

[0089] When the large model processing stage is the training stage, determine the combination of the activation value state and the parallel mode, determine the required configuration parameters based on the combination, and determine the video memory requirement based on the configuration parameters;

[0090] When the large model processing stage is the inference stage, the video memory requirement is determined based on the model parameter quantity, data size, micro-batch size, number of attention heads, hidden layer dimension, context length, data parallelism, and number of intermediate variables.

[0091] Optionally, the constraint condition and objective function construction module 330 is further used to:

[0092] Determine the computing power constraint conditions based on the relationship between the computing power requirement and the target computing power;

[0093] Determine bandwidth constraints based on the relationship between bandwidth requirements and target bandwidth;

[0094] Determine storage constraints based on the size relationship between video memory requirements and target video memory;

[0095] Determine cost constraints based on the relationship between cost requirements and target costs;

[0096] The training or inference speed constraint is determined based on the relationship between the training or inference speed and the target speed.

[0097] Optionally, the configuration tasks include a configuration task of determining model parameters based on known software and hardware information, a configuration task of determining model scale based on known software and hardware, a configuration task of determining software and hardware information based on cost, and a configuration task of determining software and hardware information based on time consumption.

[0098] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not described in detail in this embodiment, please refer to the methods provided by all the above embodiments of the present invention.

[0099] Embodiment 3

[0100] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, the connections and relationships between the components, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0101] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0102] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0103] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a resource configuration method for a large model cluster.

[0104] In some embodiments, the resource configuration method for a large model cluster may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the resource configuration method for the large model cluster described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the resource configuration method for the large model cluster in any other appropriate manner (e.g., by means of firmware).

[0105] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0106] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0107] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0108] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0109] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0110] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0111] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the resource configuration method for a large model cluster as provided in any embodiment of the present application.

[0112] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0113] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0114] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A resource configuration method for a large model cluster, characterized in that: include: Determine configuration parameters based on the configuration task selected by the user; wherein the configuration parameters include: one or more of hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism and zero-redundancy parallel mode; and some of the configuration parameters are known configuration parameters, and the other part are unknown configuration parameters; Determine at least one of computing power requirement, bandwidth requirement and video memory requirement based on the configuration task and the configuration parameters; Constructing a constraint condition and an objective function according to at least one of the computing power requirement, bandwidth requirement and video memory requirement; wherein the constraint condition includes at least one of a computing power constraint condition, a bandwidth constraint condition, a storage constraint condition, a cost constraint condition and a training or reasoning speed constraint condition; and the objective function includes a shortest time objective function and a highest cost ratio objective function; Based on the constraint conditions, the objective function is iteratively optimized using a preset planning algorithm to solve the unknown configuration parameters and obtain target configuration parameters.

2. The method according to claim 1, characterized in that The configuration task includes the following configuration properties: activation value status, large model processing stage, and parallel mode; wherein, the activation value status includes enabling activation value recalculation and disabling activation value recalculation; enabling activation value recalculation includes full activation value recalculation and selective activation value recalculation; the large model processing stage includes training stage and inference stage; the parallel mode includes: data parallelism, pipeline parallelism, tensor parallelism, zero redundancy parallelism, 3D parallelism, sequence parallelism, and no parallelism.

3. The method according to claim 2, characterized in that Determining at least one of a computing power requirement, a bandwidth requirement, and a video memory requirement based on the configuration task and the configuration parameters includes: Determine the computing power requirement according to the activation value state, the large model processing stage and the configuration parameters; or, Determine the bandwidth requirement according to the parallel mode, the large model processing stage and the configuration parameters; or, The video memory requirement is determined according to the activation value state, the large model processing stage, the parallel mode and the configuration parameters.

4. The method according to claim 3, characterized in that Determining computing power requirements according to the activation value state, the large model processing stage, and the configuration parameters includes: Determine the forward propagation computing power requirements based on the full batch size, context length, number of Transformer layers, hidden layer dimensions, and vocabulary length; The computing power requirement is determined according to the forward propagation computing power requirement, the activation value state and the large model processing stage.

5. The method according to claim 3, characterized in that: Determining bandwidth requirements according to the parallel mode, the large model processing stage and the configuration parameters includes: When the large model processing stage is the training stage, if the parallel mode is tensor parallelism, the bandwidth requirement is determined according to the micro-batch size, context length, hidden layer dimension, number of Transformer layers, full batch size, pipeline parallelism and data parallelism; if the parallel mode is pipeline parallelism, the bandwidth requirement is determined according to the micro-batch size, context length, hidden layer dimension, tensor parallelism, full batch size and data parallelism; if the parallel mode is data parallelism, the bandwidth requirement is determined according to the number of Transformer layers, hidden layer dimension, pipeline parallelism and tensor parallelism; if the parallel mode is zero-redundancy parallel mode, the bandwidth requirement is determined according to the model parameter quantity; wherein the model parameter quantity is determined according to the number of Transformer layers, hidden layer dimension, vocabulary length and context length; When the large model processing stage is the inference stage, if the parallel mode is tensor parallelism, the bandwidth requirement is determined based on the context length, the number of Transformer layers, the hidden layer dimension, and the pipeline parallelism; if the parallel mode is pipeline parallelism, the bandwidth requirement is determined based on the context length, the hidden layer dimension, and the tensor parallelism.

6. The method according to claim 3, characterized in that Determining the video memory requirement according to the activation value state, the large model processing stage, the parallel mode and the configuration parameters includes: When the large model processing stage is a training stage, determining a combination mode of the activation value state and the parallel mode, determining required configuration parameters based on the combination mode, and determining video memory requirements based on the configuration parameters; When the large model processing stage is the inference stage, the video memory requirement is determined based on the model parameter quantity, data size, micro-batch size, number of attention heads, hidden layer dimension, context length, data parallelism, and number of intermediate variables.

7. The method according to claim 1, characterized in that Constructing a constraint condition according to at least one of the computing power requirement, bandwidth requirement, and video memory requirement includes: Determine the computing power constraint condition according to the relationship between the computing power requirement and the target computing power; Determine a bandwidth constraint condition according to the size relationship between the bandwidth requirement and the target bandwidth; Determining a storage constraint condition according to a size relationship between the video memory requirement and the target video memory; Determine cost constraints based on the relationship between cost requirements and target costs; The training or inference speed constraint is determined based on the relationship between the training or inference speed and the target speed.

8. The method according to claim 1, characterized in that: The configuration tasks include a configuration task of determining model parameters based on known software and hardware information, a configuration task of determining model scale based on known software and hardware, a configuration task of determining software and hardware information based on cost, and a configuration task of determining software and hardware information based on time consumption.

9. A resource configuration device for a large model cluster, characterized in that: include: A configuration parameter determination module is used to determine configuration parameters based on the configuration task selected by the user; wherein the configuration parameters include: one or more of hyperparameters, micro-batch size, full batch size, vocabulary length, context length, number of Transformer layers, hidden layer dimension, model parameter quantity, data parallelism, pipeline parallelism, tensor parallelism and zero-redundancy parallel mode; and some of the configuration parameters are known configuration parameters, and the other part are unknown configuration parameters; A demand determination module, configured to determine at least one of a computing power demand, a bandwidth demand, and a video memory demand based on the configuration task and the configuration parameters; A constraint condition and objective function construction module, used to construct a constraint condition and an objective function according to at least one of the computing power requirement, bandwidth requirement and video memory requirement; wherein the constraint condition includes at least one of a computing power constraint condition, a bandwidth constraint condition, a storage constraint condition, a cost constraint condition and a training or reasoning speed equation; the objective function includes a shortest time-consuming objective function and a highest cost ratio objective function; The target configuration parameter acquisition module is used to iteratively optimize the target function based on the constraint conditions using a preset planning algorithm to solve the unknown configuration parameters and obtain the target configuration parameters.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the resource configuration method for a large model cluster according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the resource configuration method for a large model cluster according to any one of claims 1 to 8 when executed.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the resource configuration method for a large model cluster according to any one of claims 1 to 8.

Images