Target training strategy determination method for single-model chip computing power cluster and related device

Abstract

The invention provides a target training strategy determination method for a single-model chip computing power cluster and a related device, and relates to the technical fields of graphic processing chips, computing power clusters, generative large language models, model training, training strategy optimization and the like. The method comprises the following steps: determining a target model chip used by a computing power cluster formed by single model chips; determining performance evaluation of the target model chip under a preset performance type; determining weight distribution of each preset performance type under each alternative training strategy; calculating comprehensive training evaluation corresponding to each alternative training strategy according to performance evaluation obtained after weighting is carried out on each corresponding performance type according to each weight distribution; and determining a target training strategy based on the evaluation parameter of each comprehensive training evaluation. According to the scheme, the target training strategy suitable for being used when the computing power cluster composed of the target model chips trains the generative large language model can be rapidly determined at low cost, and the model training efficiency is indirectly improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, specifically to the technical fields of graphics processing chips, computing power clusters, generative large language models, model training, training strategy optimization, and more particularly to a method, device, electronic device, computer-readable storage medium, and computer program product for determining a target training strategy for a single-model chip computing power cluster. Background Art

[0002] The popularity of ChatGPT has triggered a new round of AI (Artificial Intelligence) technology development. In order to improve the competitiveness of their products, companies have begun to pursue more effective generative large language models. In order to improve the model effect, the scale of the model and data needs to be expanded, which brings great challenges to the infrastructure.

[0003] In order to overcome the scalability issues brought about by large generative language models, current AI frameworks have introduced optimization strategies such as tensor parallelism, recomputation, memory offload (memory reasoning technology, which can also be understood as hybrid deployment or hybrid reasoning technology, such as unloading objects to external resources or distributing them across multiple devices to reduce the memory load of the main device), and sequence parallelism to support larger and larger models, longer and longer inputs, and special model architectures.

[0004] At the same time, due to the differences in model size and data scale, the scale of the computing power cluster built by multiple graphics processing units (GPUs) at the bottom layer is also different. Under different model sizes, data scales, and computing power clusters of different sizes, there will be obvious differences in the performance of different training strategy combinations.

[0005] How to quickly and cost-effectively select a suitable training strategy from a variety of alternative training strategies to train the required generative large language model through a computing power cluster according to the training task is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0006] The embodiments of the present disclosure propose a method, device, electronic device, computer-readable storage medium and computer program product for determining a target training strategy for a single-model chip computing power cluster.

[0007] In the first aspect, an embodiment of the present disclosure proposes a method for determining a target training strategy for a computing power cluster of a single-model chip, including: determining a target model chip used by a computing power cluster composed of a single-model chip; determining a performance evaluation of the target model chip under a preset performance category; determining a weight distribution of each preset performance category under each alternative training strategy; calculating a comprehensive training evaluation corresponding to each alternative training strategy based on the performance evaluation after weighting each corresponding performance category according to each weight distribution; and determining a target training strategy based on evaluation parameters of each comprehensive training evaluation.

[0008] In the second aspect, the embodiment of the present disclosure proposes a target training strategy determination device for a single-model chip computing power cluster, including: a target model chip determination unit, configured to determine the target model chip used by the computing power cluster composed of a single model chip; a performance evaluation determination unit, configured to determine the performance evaluation of the target model chip under a preset performance type; a weight distribution determination unit, configured to determine the weight distribution of each preset performance type under each alternative training strategy; a comprehensive training evaluation determination unit, configured to calculate the comprehensive training evaluation corresponding to each alternative training strategy according to the performance evaluation after weighting each corresponding performance type according to each weight distribution; the target training strategy determination unit, configured to determine the target training strategy based on the evaluation parameters of each comprehensive training evaluation.

[0009] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, it can implement the target training strategy determination method for a single-model chip computing power cluster as described in the first aspect.

[0010] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement the target training strategy determination method for a single-model chip computing power cluster as described in the first aspect when executed.

[0011] In a fifth aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, can implement the target training strategy determination method for a single-model chip computing power cluster as described in the first aspect.

[0012] The target training strategy determination scheme for a single-model chip computing power cluster provided by the present disclosure, after receiving a training task instructing to train a generative large language model, first determines the computing power cluster composed of a single-model chip used for training and the target model chip specifically used by the computing power cluster, then determines the performance evaluation of the target model chip under a preset performance type, and then combines the weight distribution of each preset performance type under different alternative training strategies, and uses the weight distribution to weight the performance evaluation of the corresponding performance type, and calculates the comprehensive training evaluation corresponding to each alternative training strategy based on the weighted performance evaluations, and finally selects a suitable target training strategy based on the evaluation parameters of each comprehensive training evaluation.

[0013] That is, by applying the technical solution provided by the present invention, it is not necessary to determine the appropriate target training strategy by trial training according to each alternative training strategy one by one, thereby reducing the resource overhead and time overhead required for trial training. The target training strategy suitable for training a generative large language model in a computing power cluster composed of target model chips can be determined quickly at a low cost, so that the training task can be completed with less time and resource overhead, thereby indirectly improving the model training efficiency.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0016] Figure 1 is an exemplary system architecture in which the present disclosure may be applied;

[0017] Figure 2 A flowchart of a method for determining a target training strategy for a single-model chip computing cluster provided in an embodiment of the present disclosure;

[0018] Figure 3 A flowchart of a method for determining a performance evaluation of a target model chip under a preset performance category provided by an embodiment of the present disclosure;

[0019] Figure 4 A flowchart of a method for training a performance prediction model provided in an embodiment of the present disclosure;

[0020] Figure 5 A flowchart of a method for determining a comprehensive training evaluation provided by an embodiment of the present disclosure;

[0021] Figure 6 A flowchart of a method for scoring and determining a comprehensive training evaluation based on an embodiment of the present disclosure;

[0022] Figure 7-1 A schematic diagram of the structural flow of an optimal training strategy determination system in combination with a specific application scenario provided by an embodiment of the present disclosure;

[0023] Figure 7-2 A schematic diagram of the structural flow of a system for comparing different training strategies to determine an optimal training strategy in combination with a specific application scenario provided by an embodiment of the present disclosure;

[0024] Figure 8 A structural block diagram of a device for determining a target training strategy for a single-model chip computing cluster provided in an embodiment of the present disclosure;

[0025] Fig. 9 A schematic diagram of the structure of an electronic device suitable for executing a method for determining a target training strategy for a single-model chip computing power cluster provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments may be combined with each other without conflict.

[0027] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0028] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the method, apparatus, electronic device, and computer-readable storage medium for determining a target training strategy for a single-model chip computing power cluster of the present disclosure can be applied.

[0029] like Figure 1 As shown, the system architecture 100 may include a server 110, a network 120, and a computing cluster 130. The computing cluster 130 includes multiple graphics processors, such as 131, 132, 133, 134, and 135 (not just 5, Figure 1Only a few are used as examples); the network 120 is used as a medium for providing a communication link between the server 110 and the computing power cluster 130. The network 120 may include various connection types, such as wired, wireless communication links or fiber optic cables, etc.

[0030] The user can use the terminal device to interact with the server 110 through the network 120 to receive or send messages, etc. The server 110, the computing cluster 130 and the terminal device can be installed with various applications for realizing information communication between the two, such as training task delivery applications, training strategy optimization applications, instant messaging applications, etc.

[0031] The terminal device and the server 110 can be hardware or software. When the terminal device is hardware, it can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.; when the terminal device is software, it can be installed in the electronic devices listed above, which can be implemented as multiple software or software modules, or as a single software or software module, without specific limitation here. When the server 110 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server; when the server 110 is software, it can be implemented as multiple software or software modules, or as a single software or software module, without specific limitation here. The graphics processor constituting the computing power cluster 130 is usually manifested as hardware. Of course, in special scenarios (such as simulation scenarios), it can also be manifested as software or software running products, which are not specifically limited here.

[0032] The server 110 can provide various services through various built-in applications. Taking the training strategy optimization application that can provide training strategy optimization service as an example, the server 110 can achieve the following effects when running the training strategy optimization application: first, determine the target model chip used by the computing power cluster 130 composed of a single model chip; then, determine the performance evaluation of the target model chip under the preset performance type; then, determine the weight distribution of each preset performance type under each alternative training strategy; next, calculate the comprehensive training evaluation corresponding to each alternative training strategy according to the performance evaluation after weighting each corresponding performance type according to each weight distribution; finally, determine the target training strategy based on the evaluation parameters of each comprehensive training evaluation.

[0033] Furthermore, after determining the target training strategy, the server 110 may also send the information to the computing power cluster 130 through the network 120, so that the computing power cluster 130 trains the large model by using the target training strategy.

[0034] Since selecting the best one from a variety of alternative training strategies requires more computing resources and stronger computing power, the target training strategy determination method for a single-model chip computing power cluster provided in the subsequent embodiments of the present disclosure is generally executed by a server 110 with stronger computing power and more computing resources. Correspondingly, the target training strategy determination device for a single-model chip computing power cluster is generally also set in the server 110.

[0035] It should be understood that Figure 1 The number of servers, networks, computing clusters, and graphics cards in the example is only illustrative. Any number of servers, networks, computing clusters, and graphics cards may be provided as required.

[0036] Please refer to Figure 2 , Figure 2 A flowchart of a method for determining a target training strategy for a single-model chip computing cluster provided by an embodiment of the present disclosure, wherein process 200 includes the following steps:

[0037] Step 201: Determine the target chip model used by the computing power cluster composed of a single chip model;

[0038] This step aims to determine the execution subject of the method by the target training strategy for a single chip computing cluster (for example Figure 1 The server 110 shown determines the target model chip used by the computing power cluster composed of a single model chip.

[0039] Among them, it should be noted that the computing power cluster is determined based on the large model training task received in advance (for example, the training task clearly records or carries relevant information indicating the need to use the computing power cluster, such as the computing power cluster number, name, etc.), that is, the computing power cluster is needed to specifically perform the training task of the generative large language model. In the present disclosure, the computing power cluster is composed of multiple graphics processing chips of the same model, that is, composed of a single model chip, so this step actually determines the model of the graphics processing chip used by the computing power cluster used to perform the large model training task later - the target model, and then determines the graphics processing chip with the target model as the target model chip described in this step.

[0040] Step 202: Determine the performance evaluation of the target model chip under a preset performance category;

[0041] On the basis of step 201, this step aims to determine the performance evaluation of the target model chip under the preset performance category by the above-mentioned execution subject, and the performance evaluation can be expressed as a specific performance level, performance score, etc. Among them, the preset performance category refers to a pre-set or selected performance category of the evaluation target model chip, such as computing performance, storage performance, communication performance, etc., that is, the preset performance category may include: at least one of computing performance, storage performance and communication performance. The following will respectively describe these three performances in detail:

[0042] Computing performance is one of the core advantages of GPU, which is usually measured by the following aspects:

[0043] 1) Number of cores and parallel computing power: GPUs contain thousands of small computing cores (for example, NVIDIA's A100 GPU has 6912 CUDA cores). These cores are a parallel computing platform and programming model that can perform a large number of simple computing tasks in parallel. Unlike the few powerful cores of the CPU (Central Processing Unit), the GPU is designed to maximize parallel processing performance. This architecture enables the GPU to far exceed the computing power of the CPU when processing large-scale matrix and vector operations. When training deep learning models (such as neural networks), millions of matrix multiplication and addition operations are just right for the GPU's parallel processing mode.

[0044] 2) Floating Point Operations Per Second (FLOPs): Floating point operations are the basic type of calculations in deep learning training, and the computing performance of GPUs is usually measured in FLOPs. The computing performance of GPUs is relatively high. For example, the FP32 (32-bit Single Precision Floating Point) computing power of the A100 GPU is 19.5TFLOPs (19.5 trillion floating point operations per second), and under mixed precision (FP16, 16-bit Single Precision Floating Point, 16-bit half-precision floating point), the performance of A100 can reach 156TFLOPs. Such powerful computing power makes GPUs more efficient than CPUs when training large-scale neural networks.

[0045] 3) Tensor Core: A hardware unit dedicated to accelerating tensor calculations (matrix multiplication) in deep learning. Tensor Core can support more efficient mixed precision calculations, such as 16-bit floating point numbers (FP16), and significantly improve the training speed of deep learning tasks. Especially in large model training, the application of Tensor Core can reduce training time.

[0046] 4) CUDA architecture and library support: The CUDA platform provides developers with programming interfaces and optimized libraries, enabling GPUs to efficiently execute computing tasks in deep learning frameworks (such as TensorFlow and PyTorch). Using dedicated programming frameworks such as CUDA, developers can fully tap the computing potential of GPUs.

[0047] Storage performance mainly includes memory bandwidth and memory capacity, both of which are crucial for training large-scale models:

[0048] 1) GPU memory bandwidth determines the speed at which data can be transferred between the computing core and the memory. With the increase in large-scale deep learning models and data sets, memory bandwidth has become particularly important. GPUs typically use high-speed video memory (such as GDDR6, HBM2, etc.) to achieve high bandwidth. Taking A100 as an example, its HBM2 video memory provides a bandwidth of 1555GB / s, which means that the GPU can read and write large amounts of data very quickly, which is very important for training deep neural networks, especially when processing high-resolution images or large-scale data.

[0049] 2) The size of the video memory determines the size of the data set that the GPU can process. In deep learning tasks, especially when training large-scale neural networks, the video memory needs to be able to accommodate the entire model and training data set. The video memory capacity is usually the determining factor in whether the GPU can process large models on a single card. For example, the A100 is equipped with 40GB or 80GB of video memory, which enables it to process large pre-trained models such as BERT or GPT, while smaller GPU video memory may cause frequent data exchange and reduce performance.

[0050] In addition, memory access patterns also affect storage performance to a certain extent. The memory in the GPU is usually divided into different levels, such as global memory, shared memory, constant memory, etc. The access pattern of each memory is different, which affects the computing efficiency. For example, the shared memory of the GPU usually has lower latency, while the global memory is suitable for storing large amounts of data, but the access speed is slower. Therefore, in large-scale model training, how to optimize the use and access pattern of memory is the key to improving GPU performance.

[0051] The communication performance of the GPU, especially the synchronization mechanism in data transmission and parallel computing, directly affects the efficiency of large-scale model training:

[0052] 1) Inter-GPU communication (NVLink / PCIe, two different communication methods): Multi-card training is particularly common in distributed deep learning. In this scenario, multiple GPUs work together, so the data transmission performance between GPUs is particularly important. In distributed training, the synchronization speed of data between different GPUs (such as gradient synchronization) directly affects the training time and model convergence speed.

[0053] 2) PCIe 4.0 / 5.0: PCIe 4.0 and PCIe 5.0 are important communication interfaces between GPU and CPU, and GPU and storage devices. The bandwidth of PCIe 4.0 is 16GT / s, and PCIe 5.0 is increased to 32GT / s. For single-card systems, the bandwidth of these interfaces is usually sufficient for most tasks, but when training multiple cards in parallel, the bandwidth may become a bottleneck.

[0054] 3) Network bandwidth (for distributed training): In distributed training, GPUs usually communicate with each other through high-speed networks (such as InfiniBand or high-speed Ethernet). Network bandwidth and latency will directly affect the efficiency of training, especially in synchronous training mode where gradients or parameters need to be exchanged frequently. Low-latency and high-bandwidth networks can reduce the communication waiting time between GPUs and improve the overall training speed.

[0055] In addition to the three performance factors mentioned above that affect the GPU's ability to train large models, the following performance factors also affect the GPU's ability to train large models to a certain extent:

[0056] 1) Co-optimization of hardware and software: The performance of the GPU is not only determined by the hardware itself, but software optimization is also crucial. Deep learning frameworks (such as TensorFlow, PyTorch) accelerate specific computing tasks through highly optimized libraries (such as cuDNN, cuBLAS, the full English name corresponds to: CUDA Deep Neural Network library, CUDA Basic Linear AlgebraSubprograms, the former is optimized for deep learning frameworks, and the latter is optimized for linear algebra operations). These libraries are deeply optimized for GPU hardware and can fully utilize the computing and storage capabilities of the GPU. In addition, optimizations at the operating system, driver, compiler and other levels will also have an important impact on the final performance of the GPU.

[0057] 2) Power and thermal management: When training large-scale models, the high-load operation of the GPU generates a lot of heat, so the GPU's heat dissipation capacity and power stability are also factors that affect performance. Insufficient heat dissipation may cause the GPU to overheat, thereby reducing performance (for example, frequency reduction) or causing system instability. Efficient thermal design and power management are critical to keeping the GPU running efficiently for a long time.

[0058] 3) Precision and mixed precision computing: For deep learning tasks, especially large model training, mixed precision computing (such as FP16, BF16, where BF16 is also a 16-bit floating point number, but has fewer bits than FP16) can significantly improve computing speed and efficiency while reducing memory consumption. GPUs usually support mixed precision computing to improve performance without significantly losing model accuracy.

[0059] 4) Hardware acceleration features: In addition to Tensor Cores, GPUs may also include other hardware acceleration units, such as accelerators for specific tasks (such as video encoding and decoding, encryption and decryption, etc.). These hardware acceleration features can reduce the burden on the GPU in specific scenarios, thereby improving training efficiency.

[0060] In general, the performance of GPU plays a vital role in training large-scale deep learning models, mainly reflected in computing performance, storage performance and communication performance. The powerful parallel computing capability, huge memory bandwidth, low-latency communication mechanism, and coordinated optimization of hardware and software of GPU determine its advantages in training large models. In addition, factors such as precision optimization, heat dissipation and power management will also affect the performance of GPU. In practical applications, choosing a suitable GPU and optimizing related configurations can significantly improve training efficiency and speed up the training of large models.

[0061] Step 203: determining the weight distribution of each preset performance category under each candidate training strategy;

[0062] This step is intended to determine the weight distribution of each preset performance type under each alternative training strategy by the above-mentioned execution subject. Specifically, the above-mentioned execution subject may first determine the different effects of different training methods provided by different alternative training strategies on different types of performance, and then determine the weight distribution of each preset performance type under each alternative training strategy based on the different effects of different training methods on different types of performance.

[0063] Simply put, the reason why there are multiple training strategies is that each training strategy calls or processes the computing power cluster differently for the same large model training task, that is, each training strategy provides a different training method, and the differences between different training methods will be reflected in the degree of impact on the preset performance types corresponding to the GPU. Still taking the three performances of computing performance, storage performance and communication performance as examples, the corresponding computing performance refers to the part of operations that need to be calculated in the training method, the storage performance corresponds to the part of operations that need to store certain data, and the communication performance refers to the part of operations that transmit certain data within the GPU or between GPUs. Different training methods usually decompose the same training task into the above three parts, and different proportions will be obtained, that is, some training methods may result in a larger number of operations related to calculations, and some training methods may result in a larger number of operations related to communications.

[0064] Therefore, this step is to determine the weight distribution of each preset performance type under each alternative training strategy based on the different effects of different training methods provided by different alternative training strategies on different preset performance types. Generally speaking, if a certain performance type is affected more (positively, that is, it is affected by increasing the number of operations corresponding to this type of performance), it can also be understood that the weight of this performance type is relatively larger than that of other performance types. On the contrary, if the greater the degree of impact refers to a negative impact (that is, it refers to the impact of reducing the number of operations corresponding to this type of performance), it can also be adaptively understood that the weight of this performance type is relatively smaller than that of other performance types.

[0065] Step 204: Calculate the comprehensive training evaluation corresponding to each candidate training strategy according to the performance evaluation weighted for each corresponding performance type according to each weight distribution;

[0066] The purpose of this step is to use the weight distribution of each alternative training strategy determined by step 203 to weight the original performance evaluation of each corresponding performance type to obtain a weighted performance evaluation, and then calculate the comprehensive training evaluation corresponding to each alternative training strategy based on the weighted performance evaluation of each preset performance type.

[0067] That is, the above three steps first clarify the several performance types of GPU that mainly affect the ability to train large models, and obtain the original performance evaluation, and then determine the weight distribution based on the influence of different training methods provided by different alternative training strategies on these performance types, and use the weighted calculation method to clarify the actual contribution of these performance types to the ability to train large models under each alternative training strategy, and finally obtain a relatively more accurate comprehensive training evaluation corresponding to each alternative training strategy by using the correct weighted calculation method. Similarly, the comprehensive training evaluation can also be an evaluation level or score evaluation.

[0068] Step 205: Determine a target training strategy based on the evaluation parameters of each comprehensive training evaluation.

[0069] On the basis of step 204, this step aims to determine the target training strategy that better meets the actual needs based on the evaluation parameters of each comprehensive training evaluation by the above-mentioned execution subject. The actual needs can be specifically manifested as the demand indicators that minimize the overall training time and consume the least resources in the overall training process. The alternative training strategy with the comprehensive evaluation parameters that better meet the demand indicators can be determined as the target training strategy.

[0070] Considering that under normal circumstances, we only need to determine the alternative training strategy with the best evaluation parameters as the target training strategy, then we can simply determine the alternative training strategy with the highest evaluation parameters as the target training strategy; among which, the highest evaluation parameters include: the highest evaluation level, the highest evaluation score, and the highest evaluation degree.

[0071] The target training strategy determination method for a single-model chip computing power cluster provided by the embodiment of the present disclosure, after receiving a training task instructing to train a generative large language model, first determines the computing power cluster composed of a single-model chip used for training and the target model chip specifically used by the computing power cluster, then determines the performance evaluation of the target model chip under a preset performance type, and then combines the weight distribution of each preset performance type under different alternative training strategies, and uses the weight distribution to weight the performance evaluation of the corresponding performance type, and calculates the comprehensive training evaluation corresponding to each alternative training strategy based on the weighted performance evaluations, and finally selects a suitable target training strategy based on the evaluation parameters of each comprehensive training evaluation.

[0072] That is, by applying the technical solution provided by the present invention, it is not necessary to determine the appropriate target training strategy by trial training according to each alternative training strategy one by one, thereby reducing the resource overhead and time overhead required for trial training. The target training strategy suitable for training a generative large language model in a computing power cluster composed of target model chips can be determined quickly at a low cost, so that the training task can be completed with less time and resource overhead, thereby indirectly improving the model training efficiency.

[0073] Please refer to Figure 3 , Figure 3 A flowchart of a method for determining a performance evaluation of a target model chip under a preset performance category provided by an embodiment of the present disclosure, wherein process 300 includes the following steps:

[0074] Step 301: Determine a target performance prediction model corresponding to a target model chip;

[0075] This step aims to determine the target performance prediction model corresponding to the target model chip by the above-mentioned execution subject. The target performance prediction model is used to characterize the corresponding relationship between the performance evaluations of different models of chips for various data sizes under preset performance categories, and the target performance prediction model can be pre-trained using training samples with the above-mentioned corresponding relationship.

[0076] Furthermore, considering that different types of chips under the same chip architecture often have the same chip architecture, the predicted performance between different types of chips has continuity or inertia. That is, the performance prediction model trained using the training samples of the A1 type chip and the A2 type chip under the A chip architecture can often be used when actually determining the performance evaluation of the new A3 type chip, and a relatively high-accuracy predicted performance evaluation can be obtained. However, different types of chips under different chip architectures cannot be accurately predicted due to different chip architectures. Therefore, in the case of considering the chip architecture, the matching performance prediction model can also be determined through the following steps:

[0077] First, determine the target chip architecture used by the target model chip, and then determine the performance prediction model corresponding to the target chip architecture as the target performance prediction model; wherein a matching performance prediction model is pre-trained for each chip architecture, that is, assuming that there are three different chip architectures A, B, and C, a performance prediction model is trained for each of the three different chip architectures A, B, and C, for example, performance prediction model Z1 corresponds to chip architecture A, performance prediction model Z2 corresponds to chip architecture B, and performance prediction model Z3 corresponds to chip architecture C.

[0078] Step 302: Input the chip model of the target model chip into the target performance prediction model to obtain an output performance evaluation under a preset performance category.

[0079] On the basis of step 301, this step aims at having the above-mentioned execution subject input the chip model of the target model chip into the target performance prediction model, and obtain the performance evaluation under the preset performance category output by the target performance prediction model.

[0080] Specifically, the training samples used in the training of the target performance prediction model may include real performance evaluations that exceed the types of performance that are actually required to be output, that is, assuming that the target performance prediction model currently only needs to output predicted performance evaluations under computing performance, storage performance, and communication performance, then the training samples used in the training of the target performance prediction model may only include actual performance evaluations under computing performance, storage performance, and communication performance, and may also further include actual performance evaluations of other performance types. This allows the target performance prediction model to directly meet its requirements if it is required to output performance evaluations under more performance types in the future.

[0081] This embodiment provides a solution of first determining a target performance prediction model that matches a target model chip, and then using the target performance prediction model to output the required performance evaluation of the target model chip under a preset performance category through steps 301 and 302. That is, by pre-training a target performance pre-stored model that can characterize the corresponding relationship between the performance evaluations of different model chips for various data sizes under preset performance categories, it is possible to quickly obtain a relatively accurate predicted performance evaluation without the need for actual testing when the actual performance evaluation of the preset performance category of the target model chip is not clearly known, thereby reducing the time consumption of determination in the actual use stage and improving the determination efficiency.

[0082] Furthermore, in order to deepen the understanding of how to pre-train performance prediction models for different chip architectures, Figure 4 A performance prediction model training method is provided, wherein the process 400 includes the following steps:

[0083] Step 401: Obtain known chip models under the same chip architecture;

[0084] For example, the A1 chip and the A2 chip under the A chip architecture in the above example, where it is assumed that the A2 chip has more CUDA units and larger video memory than the A1 chip.

[0085] Step 402: obtaining actual performance evaluations of image processing chips of known chip models for various data sizes under preset performance categories;

[0086] For example, obtain the actual performance evaluation of A1 model chips and A2 model chips for various data sizes under preset performance categories, such as the actual computing performance evaluation, actual storage performance evaluation and actual communication performance evaluation of 2MB, 4MB, 8MB or even larger data in terms of computing performance, storage performance and communication performance respectively.

[0087] Step 403: Taking the known chip model as sample input and the corresponding actual performance evaluation as expected sample output, constructing a training sample;

[0088] That is, the training samples include multiple sample pairs, each of which is composed of a sample input of a known chip model and an expected sample output of the actual performance evaluation corresponding to the known chip model. Specifically, the sample input is not just a model name, but the core parameters corresponding to the model (such as video memory capacity, video memory type, number of cores, and computing power accuracy, etc.), so that the prediction model can learn the corresponding relationship between the differences in core parameters of different models and the degree of difference between performance parameters by using such training samples.

[0089] Step 404: Use the training samples to train the prediction model to be trained to obtain a trained performance prediction model.

[0090] This step aims to use the training samples to train the prediction model to be trained to obtain a trained performance prediction model.

[0091] It should be noted that Figure 4 The execution subject of the solution for training the performance prediction model provided may be the same execution subject as the above embodiments, for example, the same execution subject performs the same training as the above embodiments at a certain stage before executing the above embodiments. Figure 4 The performance prediction model training operation shown in the figure may also be another execution subject different from that used in the above embodiments, for example, different from Figure 1 Other servers besides the server 110 shown, for example, a server dedicated to training a performance prediction model.

[0092] In order to deepen the understanding of how to use the weighted calculation method to determine the comprehensive training evaluation corresponding to each candidate training strategy, this embodiment also uses Figure 5 A flow chart of a method for determining a comprehensive training evaluation is shown, wherein the flow 500 includes the following steps:

[0093] Step 501: Determine the performance evaluation of the target model chip in terms of computing, storage and communication, and obtain computing performance evaluation, storage performance evaluation and communication performance evaluation accordingly;

[0094] This step is based on the assumption that the preset performance types specifically include computing, storage and communication, and the computing performance evaluation, storage performance evaluation and communication performance evaluation of the target model chip are then determined by the above-mentioned execution subject.

[0095] The computing performance evaluation, storage performance evaluation and communication performance evaluation may be evaluation levels that are differentiated by semantic description, or may be evaluation scores that are differentiated by numerical values, which are not specifically limited here.

[0096] Step 502: Determine the weight distribution of computing performance, storage performance and communication performance under each candidate training strategy according to the different effects of different training methods on computing performance, storage performance and communication performance.

[0097] This step aims to further determine the weight distribution of computing performance, storage performance and communication performance under each alternative training strategy according to the different effects of different training methods on computing performance, storage performance and communication performance. For example, the weight distribution can be expressed as: computing performance: storage performance: communication performance = 2:2:1, and it can be seen that the alternative training strategy corresponding to this weight distribution will contain relatively more calculation-related operations and storage-related operations, and relatively fewer communication-related operations.

[0098] Step 503: For each candidate training strategy, the following comprehensive training evaluation determination steps are performed:

[0099] Step 5031: Determine the current weight distribution corresponding to the current candidate training strategy;

[0100] Step 5032: using the current weight distribution to determine the current weights corresponding to the computing performance, storage performance, and communication performance respectively;

[0101] Continuing with the above example, assume that the current weights of the computing performance, storage performance, and communication performance are 2:2:1 respectively.

[0102] Step 5033: using the current weights corresponding to the computing performance, storage performance and communication performance, respectively, weighting the corresponding computing performance evaluation, storage performance evaluation and communication performance evaluation, to obtain weighted computing performance evaluation, storage performance evaluation and communication performance evaluation;

[0103] Continuing with the above example, if the sum of the total weights needs to satisfy unit 1, the weight corresponding to the computing performance will be adaptively changed to: 0.4, that is, 0.4×computing performance evaluation; the weight corresponding to the storage performance will be adaptively changed to: 0.4, that is, 0.4×storage performance evaluation; the weight corresponding to the communication performance will be adaptively changed to: 0.2, that is, 0.2×communication performance evaluation.

[0104] That is, 0.4+0.4+0.2=1. If there is no need to adaptively change the original weight size according to unit 1, the weighted performance evaluation needs to be standardized to unify the comparison standard.

[0105] Step 5034: Based on the weighted computing performance evaluation, storage performance evaluation, and communication performance evaluation corresponding to the target model chip, a comprehensive training evaluation corresponding to the current candidate training strategy is calculated.

[0106] Based on step 5033, this step aims to enable the above-mentioned execution subject to calculate a comprehensive training evaluation corresponding to the current candidate training strategy based on the weighted computing performance evaluation, storage performance evaluation and communication performance evaluation corresponding to the target model chip.

[0107] An implementation including but not limited to can be found in Figure 6 A flow chart of a method for determining a comprehensive training evaluation based on performance scoring is shown, wherein the process 600 includes the following steps:

[0108] Step 601: quantizing the weighted evaluation parameters of computing performance evaluation, storage performance evaluation and communication performance evaluation corresponding to the target model chip to obtain a computing performance score, a storage performance score and a communication performance score;

[0109] That is, this embodiment quantifies the evaluation parameters of the weighted computing performance evaluation, storage performance evaluation, and communication performance evaluation to obtain computing performance scores, storage performance scores, and communication performance scores that are easier to distinguish between high and low performance and the degree of high and low performance. There are many ways to quantify, as long as they can meet the needs, and no specific restrictions are made here.

[0110] Step 602: Calculate the sum of the computing performance score, the storage performance score and the communication performance score to obtain a performance score sum;

[0111] This step aims to obtain the sum of the computing performance score, the storage performance score and the communication performance score by the above-mentioned execution subject through addition operation to obtain the performance score sum.

[0112] Step 603: Determine a comprehensive training evaluation corresponding to the current candidate training strategy based on the performance score.

[0113] On the basis of step 602, this step aims to enable the above-mentioned execution subject to determine the comprehensive training evaluation corresponding to the current alternative training strategy based on the performance score and, that is, the comprehensive evaluation training is at least determined based on the performance score and, for example, the performance score and can be directly used as the comprehensive training evaluation corresponding to the current alternative training strategy, or the comprehensive training evaluation can be determined based on the performance score and in combination with some other coefficients or bases.

[0114] On the basis of any of the above embodiments, in order to reduce the amount of calculation as much as possible and avoid invalid calculations, before entering the calculation comprehensive training evaluation provided by step 204, the alternative training strategies that do not meet the requirements of the actual received training tasks (for example, they obviously do not meet the requirements at the training rule level) and / or the alternative training strategies with obvious video memory conflicts (for example, strategies in which the required storage amount obviously exceeds the available video memory ratio) can be removed from all the alternative training strategies, thereby obtaining the remaining training strategies, so that the subsequent calculation only needs to be based on the performance evaluation after weighting each corresponding performance type according to each weight distribution, and the comprehensive training evaluation corresponding to each remaining training strategy can be calculated.

[0115] Based on any of the above embodiments, considering that the above steps of determining the comprehensive training evaluation corresponding to each candidate training strategy actually perform data calculation operations, and compared with the GPU, the CPU obviously has more powerful data calculation capabilities, so all or part of the calculation tasks of the comprehensive training evaluation can be assigned to the CPU to perform:

[0116] For example, the execution subject may control the CPU to create multiple parallel computing threads, and control each computing thread to execute the following processing steps:

[0117] First, a weight distribution corresponding to any candidate training strategy to be calculated is obtained; then, based on the performance evaluation after weighting each corresponding performance type according to the weight distribution, a comprehensive training evaluation corresponding to the currently selected candidate training strategy is calculated and output.

[0118] That is, a message queue can be set up in advance, and the alternative training strategies that need to be comprehensively trained and evaluated can be stored in the message queue, and the CPU can enable each of its computing threads to take out a specific alternative training strategy from the message queue respectively, so as to complete the calculation operation of comprehensively training evaluating the obtained alternative training strategies in each computing thread.

[0119] If only some of the alternative training strategies to be subjected to comprehensive training evaluation are stored in the message queue, it is equivalent to the CPU playing an auxiliary computing role in this process. If all the alternative training strategies to be subjected to comprehensive training evaluation are stored in the message queue, it is equivalent to the CPU replacing the GPU to play an accelerated computing role in this process.

[0120] On the basis of any of the above embodiments, after selecting the target training strategy suitable for the computing cluster composed of the target model chips, the above execution subject (for example Figure 1 The server 110 shown in FIG. 1 controls a computing cluster (eg, Figure 1 The computing cluster 130 shown in the figure performs the training task of the generative large language model according to the target training strategy. It should be noted that since the computing cluster used in each embodiment of the present disclosure is constructed by multiple graphics processing chips of the same model, each graphics processing chip has the same processing power, then when executing the training task, the processing tasks (including computing tasks, storage tasks, and communication tasks) assigned to each target model chip will be consistent and uniform, and each processing task is decomposed from the overall training task facing the computing cluster.

[0121] To deepen understanding, the present disclosure also combines specific application scenarios and provides a complete set of implementation solutions, in order to solve the existing technical defects that the existing related technologies may not be able to accurately determine the appropriate training strategy even at a high cost:

[0122] To address the defects of the prior art, this embodiment first abstracts the time spent on training a specified large model under a certain strategy in one iter (which can be simply understood as a training stage or one iterative training), and the formula is as follows:

[0123]

[0124] Among them, time_cost: the time spent on an iter, which can be converted into training throughput or computing power MFU for training tasks. The smaller the index, the more in line with the actual needs and the lower the cost, and the more likely it is to be selected as the target training strategy;

[0125] gemm_ops: the number of computational operations performed on each matrix;

[0126] flops: the actual floating-point computing capability per cycle of the chip at each matrix size;

[0127] mem_ops: the number of memory access operations performed by each non-matrix operation;

[0128] mem_bws: memory access bandwidth of the chip at various memory access operation sizes;

[0129] comm_ops: the amount of communication required under different communication modes;

[0130] comm_bws: Communication bandwidth of the chip under different communication models and sizes.

[0131] In order to obtain some indicators in the above formula, the following method is designed, see Figure 7-1 ,exist Figure 7-1 In the implementation shown, the following mechanisms are mainly used to achieve the above time estimation:

[0132] 1. Basic chip performance:

[0133] 1) Offline testing of the computing, storage, and communication performance of various chips under various data sizes;

[0134] 2) An XGBoost model (an integrated learning model based on the gradient boosting framework, which iteratively generates multiple decision trees to improve the prediction accuracy of the model. The basic idea is to combine multiple tree models with low classification accuracy into a model with higher accuracy) is used for fitting. In this way, the computing MFU CostModel, memory bandwidth CostModel, and communication bandwidth CostModel under different chips are obtained, which are used to subsequently calculate the performance of different data volumes under different strategies.

[0135] That is, in the early offline stage, it is necessary to use the real performance parameters of some known chip models to train and build the XGBoost model, so that for the new chip models input later, the XGBoost model that has learned the relevant knowledge can output the predicted performance parameters for the relevant performance types according to the chip hardware parameters that constitute the new chip models.

[0136] 2. Strategy abstraction:

[0137] 1) According to the logic of various parallel strategies, the computational graph of the entire model is divided to obtain the required computational load, memory access, communication volume, storage volume, and pipeline bubble under different strategies;

[0138] 2) To consider the scenario of insufficient video memory, the size of the video memory system buffer, temporary variables, and other spaces are obtained based on the offline profile mechanism (which can be understood as a trial run mechanism, that is, by providing similar offline running conditions to clarify the video memory parameters), so as to determine the proportion of video memory available for model training.

[0139] 3. Strategy space:

[0140] 1) Traverse all strategy combinations and filter them through some simple strategies (such as strategies that obviously do not comply with the rules), thereby reducing the strategy space that needs to be calculated;

[0141] 2) Traverse all candidate strategies, calculate the required storage amount, compare it with the available video memory ratio, and exclude strategies that will cause video memory OOM (insufficient video memory);

[0142] 3) Traverse the remaining strategies, call the strategy abstraction to obtain the required computing amount, memory access amount, and communication amount, and then calculate the time_cost of the above formula with the basic performance set of the chip, and select the strategy with the smallest time_cost, that is, the strategy with the best performance. This process can be seen in Figure 7-2 (Taking training strategies A and B as examples, the entire task execution time is calculated based on the operator execution time + waiting time of the last group of chips, and these parameters are specifically summarized by the bubbles before and after the previous chip, the execution time of each type of operator, the synchronization point communication waiting time, the recompute execution time and the recalculation time, etc.)

[0143] In the implementation scheme provided in the above content of this embodiment, compared with the existing related technologies, in addition to traversing test and profile related operations, it also has the following advantages and characteristics:

[0144] 1) In addition to the basic offline performance test in the early stage, no actual resources are required during the model development process to search for the optimal parallel strategy;

[0145] 2) The entire process is purely computational, and the strategies are not dependent on each other, so the CPU can be used for parallel computing, and the optimal strategy can be found in minutes;

[0146] 3) The coverage strategy is wider than the previous solution, and the new strategy only requires logical abstraction, so the accuracy is very high, reaching 95%.

[0147] Further references Figure 8 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for determining a target training strategy for a single-model chip computing cluster. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0148] like Figure 8As shown, the target training strategy determination device 800 for a single-model chip computing power cluster of this embodiment may include: a target model chip determination unit 801, a performance evaluation determination unit 802, a weight distribution determination unit 803, a comprehensive training evaluation determination unit 804, and a target training strategy determination unit 805. Among them, the target model chip determination unit 801 is configured to determine the target model chip used by the computing power cluster composed of a single model chip; the performance evaluation determination unit 802 is configured to determine the performance evaluation of the target model chip under a preset performance category; the weight distribution determination unit 803 is configured to determine the weight distribution of each preset performance category under each alternative training strategy; the comprehensive training evaluation determination unit 804 is configured to calculate the comprehensive training evaluation corresponding to each alternative training strategy according to the performance evaluation after weighting each corresponding performance category according to each weight distribution; the target training strategy determination unit 805 is configured to determine the target training strategy based on the evaluation parameters of each comprehensive training evaluation.

[0149] In this embodiment, in the target training strategy determination device 800 for a single-model chip computing power cluster: the specific processing of the target model chip determination unit 801, the performance evaluation determination unit 802, the weight distribution determination unit 803, the comprehensive training evaluation determination unit 804 and the target training strategy determination unit 805 and the technical effects thereof can be referred to respectively. Figure 2 The relevant descriptions of steps 201 - 205 in the corresponding embodiment are not repeated here.

[0150] In some optional implementations of this embodiment, the target model chip determination unit 801 may be further configured to:

[0151] Determine a target model of an image processing chip used by a computing power cluster consisting of a preset number of image processing chips of a single model;

[0152] An image processing chip having a target model is determined as a target model chip.

[0153] In some optional implementations of this embodiment, the performance evaluation determination unit 802 may include:

[0154] The target performance prediction model determination subunit is configured to determine a target performance prediction model corresponding to a target model chip; wherein the target performance prediction model is used to characterize the corresponding relationship between performance evaluations of different models of chips for various data sizes under preset performance categories;

[0155] The model using subunit is configured to input the chip model of the target model chip into the target performance prediction model to obtain an output performance evaluation under a preset performance category.

[0156] In some optional implementations of this embodiment, the target performance prediction model determination subunit may be further configured to:

[0157] Determine the target chip architecture used by the target model chip;

[0158] A performance prediction model corresponding to the target chip architecture is determined as a target performance prediction model; wherein a matching performance prediction model is pre-trained for each chip architecture.

[0159] In some optional implementations of this embodiment, the target training strategy determination device 800 for a single-model chip computing power cluster may further include: a performance prediction model training unit, and the performance prediction model training unit may be further configured to:

[0160] Get known chip models under the same chip architecture;

[0161] Obtaining actual performance evaluation of image processing chips of known chip models for various data sizes under preset performance categories;

[0162] The known chip model is used as sample input and the corresponding actual performance evaluation is used as the expected sample output to construct the training sample;

[0163] The training samples are used to train the prediction model to be trained to obtain a trained performance prediction model.

[0164] In some optional implementations of this embodiment, the weight distribution determining unit 803 may include:

[0165] The different impact determination subunit is configured to determine the different impacts of different training methods provided by different alternative training strategies on different types of performance;

[0166] The weight distribution determination subunit is configured to determine the weight distribution of each preset performance type under each alternative training strategy according to the different impacts of different training methods on different types of performance; wherein the weight of any type of performance is proportional to the degree of impact.

[0167] In some optional implementations of this embodiment, the preset performance type includes: at least one of computing performance, storage performance, and communication performance;

[0168] Correspondingly, the performance evaluation determination unit 802 may be further configured to:

[0169] Determine the performance evaluation of the target model chip in terms of computing, storage and communication, and obtain computing performance evaluation, storage performance evaluation and communication performance evaluation accordingly;

[0170] Correspondingly, the weight distribution determination subunit may be further configured as follows:

[0171] According to the different impacts of different training methods on computing performance, storage performance and communication performance, the weight distribution of computing performance, storage performance and communication performance under each alternative training strategy is determined.

[0172] In some optional implementations of this embodiment, the comprehensive training evaluation determination unit 804 may be further configured to:

[0173] For each candidate training strategy, the following comprehensive training evaluation determination steps are performed:

[0174] Determining a current weight distribution corresponding to a current candidate training strategy;

[0175] Determine the current weights corresponding to the computing performance, storage performance, and communication performance respectively using the current weight distribution;

[0176] Using the current weights corresponding to the computing performance, storage performance and communication performance, respectively, weighted processing is performed on the corresponding computing performance evaluation, storage performance evaluation and communication performance evaluation to obtain weighted computing performance evaluation, storage performance evaluation and communication performance evaluation;

[0177] Based on the weighted computing performance evaluation, storage performance evaluation and communication performance evaluation corresponding to the target model chip, a comprehensive training evaluation corresponding to the current candidate training strategy is calculated.

[0178] In some optional implementations of this embodiment, based on the weighted computing performance evaluation, storage performance evaluation, and communication performance evaluation corresponding to the target model chip, a comprehensive training evaluation corresponding to the current candidate training strategy is calculated, which may include:

[0179] Quantifying the evaluation parameters of the weighted computing performance evaluation, storage performance evaluation, and communication performance evaluation corresponding to the target model chip to obtain a computing performance score, a storage performance score, and a communication performance score;

[0180] Calculate the sum of the computing performance score, the storage performance score, and the communication performance score to obtain a performance score sum;

[0181] A comprehensive training evaluation is performed based on performance scoring and determination of the corresponding candidate training strategies currently in place.

[0182] In some optional implementations of this embodiment, the comprehensive training evaluation corresponding to the current candidate training strategy is determined based on the performance score, including:

[0183] The performance scores and are used as comprehensive training evaluations corresponding to the current candidate training strategies.

[0184] In some optional implementations of this embodiment, the target training strategy determination device 800 for a single-model chip computing cluster may also include:

[0185] The abnormal training strategy removal unit is configured to remove the candidate training strategies that do not meet the requirements of the training tasks actually received and / or remove the candidate training strategies that have obvious video memory conflicts from all the candidate training strategies before calculating the comprehensive training evaluations corresponding to each candidate training strategy according to the performance evaluations after weighting the corresponding performance types according to the weight distributions, so as to obtain the remaining training strategies;

[0186] Correspondingly, the comprehensive training evaluation determination unit 804 may be further configured to:

[0187] Based on the performance evaluation after weighting the corresponding performance categories according to each weight distribution, the comprehensive training evaluation corresponding to each remaining training strategy is calculated.

[0188] In some optional implementations of this embodiment, the comprehensive training evaluation determination unit 804 may be further configured to:

[0189] Control the central processing unit to create multiple parallel computing threads, and control each computing thread to execute the following processing steps:

[0190] Obtaining a weight distribution corresponding to any candidate training strategy to be calculated;

[0191] Based on the performance evaluation after weighting each corresponding performance type according to the weight distribution, the comprehensive training evaluation corresponding to the currently selected candidate training strategy is calculated and output.

[0192] In some optional implementations of this embodiment, the target training strategy determination unit 805 may be further configured to:

[0193] The alternative training strategy with the highest evaluation parameter is determined as the target training strategy; wherein the highest evaluation parameter includes: the highest evaluation level, the highest evaluation score, and the highest evaluation degree.

[0194] In some optional implementations of this embodiment, the target training strategy determination device 800 for a single-model chip computing cluster may also include:

[0195] The large model training unit is configured to control the computing power cluster to execute the training task of the generative large language model according to the target training strategy; among them, each target model chip under the computing power cluster will be evenly assigned processing tasks, and the processing tasks are decomposed from the training tasks.

[0196] This embodiment exists as an apparatus embodiment corresponding to the above method embodiment. The target training strategy determination device for a single-model chip computing power cluster provided in this embodiment, after receiving a training task indicating training of a generative large language model, first determines the computing power cluster composed of a single-model chip used for training and the target model chip specifically used by the computing power cluster, then determines the performance evaluation of the target model chip under a preset performance category, and then combines the weight distribution of each preset performance category under different alternative training strategies, and uses the weight distribution to perform weighted processing on the performance evaluation of the corresponding performance category, and calculates the comprehensive training evaluation corresponding to each alternative training strategy according to the weighted performance evaluation, and finally selects a suitable target training strategy according to the evaluation parameters of each comprehensive training evaluation. That is, by applying the technical solution provided by the present disclosure, it is not necessary to determine the suitable target training strategy by trial training according to each alternative training strategy one by one, reducing the resource overhead and time overhead consumed by trial training, and can quickly determine the target training strategy suitable for the computing power cluster composed of the target model chip to train the generative large language model at a low cost, so that only less time and resource overhead are required to complete the training task, indirectly improving the model training efficiency.

[0197] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the target training strategy determination method for a single-model chip computing power cluster described in any of the above embodiments when executing.

[0198] According to an embodiment of the present disclosure, the present disclosure also provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the target training strategy determination method for a single-model chip computing power cluster described in any of the above embodiments when executed.

[0199] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which, when executed by a processor, can implement the target training strategy determination method for a single-model chip computing power cluster described in any of the above embodiments.

[0200] Fig. 9A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0201] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0202] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0203] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as a target training strategy determination method for a single model chip computing power cluster. For example, in some embodiments, the target training strategy determination method for a single model chip computing power cluster may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the target training strategy determination method for a single model chip computing power cluster described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured in any other appropriate manner (for example, by means of firmware) to execute a target training strategy determination method for a single-model chip computing power cluster.

[0204] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0205] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0206] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0207] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0208] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0209] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0210] According to the technical solution of the embodiment of the present disclosure, after receiving the training task instructing to train the generative large language model, the computing power cluster composed of a single model chip and the target model chip specifically used by the computing power cluster are first determined, and then the performance evaluation of the target model chip under the preset performance category is determined, and then the weight distribution of each preset performance category under different alternative training strategies is combined, and the performance evaluation of the corresponding performance category is weighted by using the weight distribution, and the comprehensive training evaluation corresponding to each alternative training strategy is calculated according to the weighted performance evaluation, and finally the appropriate target training strategy is selected according to the evaluation parameters of each comprehensive training evaluation. That is, by applying the technical solution provided by this embodiment, it is not necessary to determine the appropriate target training strategy by trial training according to each alternative training strategy one by one, reducing the resource overhead and time overhead consumed by trial training, and can quickly determine the target training strategy suitable for the computing power cluster composed of the target model chip to train the generative large language model at a relatively low cost, so that only less time and resource overhead are required to complete the training task, which indirectly improves the model training efficiency.

[0211] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0212] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for determining a target training strategy for a single-model chip computing cluster, comprising: Determine the target chip model used by the computing power cluster composed of a single chip model; Determine the performance evaluation of the target model chip under a preset performance category; Determining the weight distribution of each of the preset performance categories under each alternative training strategy; Calculating the comprehensive training evaluation corresponding to each of the candidate training strategies according to the performance evaluation after weighting each corresponding performance type according to each weight distribution; Based on the evaluation parameters of each of the comprehensive training evaluations, a target training strategy is determined.

2. The method according to claim 1, wherein: The step of determining a target chip model used by a computing cluster composed of chips of a single model includes: Determine a target model of an image processing chip used by a computing power cluster consisting of a preset number of image processing chips of a single model; An image processing chip having the target model is determined as the target model chip.

3. The method according to claim 1, wherein: Determining the performance evaluation of the target model chip under a preset performance category includes: Determine a target performance prediction model corresponding to the target model chip; wherein the target performance prediction model is used to characterize the corresponding relationship between the performance evaluations of different model chips for various data sizes under the preset performance category; The chip model of the target model chip is input into the target performance prediction model to obtain an output performance evaluation under the preset performance category.

4. The method according to claim 3, wherein: The step of determining a target performance prediction model corresponding to the target chip model includes: Determine the target chip architecture adopted by the target model chip; A performance prediction model corresponding to the target chip architecture is determined as the target performance prediction model; wherein a matching performance prediction model is pre-trained for each chip architecture.

5. The method according to claim 4, wherein: The performance prediction model is trained by the following steps: Get known chip models under the same chip architecture; Obtaining actual performance evaluation of the image processing chip of the known chip model for various data sizes under the preset performance category; The known chip model is used as a sample input and the corresponding actual performance evaluation is used as an expected sample output to construct a training sample; The training samples are used to train the prediction model to be trained to obtain a trained performance prediction model.

6. The method according to claim 1, wherein: The determining of the weight distribution of each of the preset performance categories under each candidate training strategy includes: Determine the different effects of different training approaches provided by different alternative training strategies on different types of performance; According to the different impacts of the different training methods on different types of performance, the weight distribution of each of the preset performance types under each of the alternative training strategies is determined; wherein the weight of any type of performance is proportional to the degree of impact.

7. The method according to claim 6, wherein: The preset performance type includes: at least one of computing performance, storage performance and communication performance; Determining the performance evaluation of the target model chip under a preset performance category includes: Determine the performance evaluation of the target model chip in terms of computing, storage and communication, respectively, and obtain computing performance evaluation, storage performance evaluation and communication performance evaluation accordingly; Determining the weight distribution of each of the preset performance types under each of the alternative training strategies according to the different effects of the different training methods on the different types of performance includes: According to the different effects of the different training methods on the computing performance, the storage performance and the communication performance, respectively, the weight distribution of the computing performance, the storage performance and the communication performance under each of the alternative training strategies is determined.

8. The method according to claim 7, wherein: The step of calculating the comprehensive training evaluation corresponding to each of the candidate training strategies based on the performance evaluation weighted by each corresponding performance type according to each weight distribution includes: For each of the candidate training strategies, the following comprehensive training evaluation determination steps are performed: Determining a current weight distribution corresponding to a current candidate training strategy; Determine the current weights corresponding to the computing performance, the storage performance, and the communication performance respectively by using the current weight distribution; Using the current weights corresponding to the computing performance, the storage performance, and the communication performance, respectively, weighted processing is performed on the corresponding computing performance evaluation, storage performance evaluation, and communication performance evaluation to obtain weighted computing performance evaluation, storage performance evaluation, and communication performance evaluation; Based on the weighted computing performance evaluation, storage performance evaluation and communication performance evaluation corresponding to the target model chip, a comprehensive training evaluation corresponding to the current candidate training strategy is calculated.

9. The method according to claim 8, wherein: The comprehensive training evaluation corresponding to the current candidate training strategy is calculated based on the weighted computing performance evaluation, storage performance evaluation and communication performance evaluation corresponding to the target model chip, including: Quantifying the weighted evaluation parameters of the computing performance evaluation, storage performance evaluation, and communication performance evaluation corresponding to the target model chip to obtain a computing performance score, a storage performance score, and a communication performance score; Calculating the sum of the computing performance score, the storage performance score and the communication performance score to obtain a performance score sum; Based on the performance score and determining a comprehensive training evaluation corresponding to the current candidate training strategy.

10. The method according to claim 9, wherein: The comprehensive training evaluation based on the performance score and determining the corresponding candidate training strategy includes: The performance scores and the sum are used as a comprehensive training evaluation corresponding to the current candidate training strategy.

11. The method according to claim 1, wherein: Before calculating the comprehensive training evaluation corresponding to each of the candidate training strategies according to the performance evaluation after weighting the corresponding performance types according to the weight distribution, the method further includes: Removing the candidate training strategies that do not meet the requirements of the actually received training tasks and / or removing the candidate training strategies that have obvious video memory conflict problems from all the candidate training strategies to obtain the remaining training strategies; The step of calculating the comprehensive training evaluation corresponding to each of the candidate training strategies based on the performance evaluation weighted by each corresponding performance type according to each weight distribution includes: Based on the performance evaluation after weighting each corresponding performance type according to each weight distribution, a comprehensive training evaluation corresponding to each of the remaining training strategies is calculated.

12. The method according to claim 1, wherein: The step of calculating the comprehensive training evaluation corresponding to each of the candidate training strategies based on the performance evaluation weighted by each corresponding performance type according to each weight distribution includes: Control the central processing unit to create multiple parallel computing threads, and control each computing thread to execute the following processing steps: Obtaining a weight distribution corresponding to any candidate training strategy to be calculated; Based on the performance evaluation after weighting the corresponding performance types according to the weight distribution, a comprehensive training evaluation corresponding to the currently selected candidate training strategy is calculated and output.

13. The method according to claim 1, wherein: The step of determining a target training strategy based on the evaluation parameters of each comprehensive training evaluation comprises: The candidate training strategy with the highest evaluation parameter is determined as the target training strategy; wherein the highest evaluation parameter includes: the highest evaluation level, the highest evaluation score, and the highest evaluation degree.

14. The method according to any one of claims 1 to 13, further comprising: The computing power cluster is controlled to execute the training task of the generative large language model according to the target training strategy; wherein each target model chip under the computing power cluster will be evenly assigned processing tasks, and the processing tasks are decomposed from the training tasks.

15. A device for determining a target training strategy for a single-model chip computing cluster, comprising: A target model chip determination unit is configured to determine a target model chip used by a computing power cluster composed of a single model chip; A performance evaluation determination unit, configured to determine a performance evaluation of the target model chip under a preset performance category; A weight distribution determining unit, configured to determine the weight distribution of each of the preset performance categories under each candidate training strategy; A comprehensive training evaluation determination unit is configured to calculate a comprehensive training evaluation corresponding to each of the candidate training strategies according to the performance evaluation after weighting the corresponding performance types according to the weight distributions; The target training strategy determination unit is configured to determine the target training strategy based on the evaluation parameters of each of the comprehensive training evaluations.

16. The device according to claim 15, wherein: The target model chip determination unit is further configured to: Determine a target model of an image processing chip used by a computing power cluster consisting of a preset number of image processing chips of a single model; An image processing chip having the target model is determined as the target model chip.

17. The device according to claim 15, wherein: The performance evaluation determination unit comprises: The target performance prediction model determination subunit is configured to determine a target performance prediction model corresponding to the target model chip; wherein the target performance prediction model is used to characterize the corresponding relationship between the performance evaluations of different model chips for various data sizes under the preset performance category; The model using subunit is configured to input the chip model of the target model chip into the target performance prediction model to obtain an output performance evaluation under the preset performance category.

18. The device according to claim 17, wherein: The target performance prediction model determination subunit is further configured to: Determine the target chip architecture adopted by the target model chip; A performance prediction model corresponding to the target chip architecture is determined as the target performance prediction model; wherein a matching performance prediction model is pre-trained for each chip architecture.

19. The apparatus according to claim 18, further comprising: A performance prediction model training unit, wherein the performance prediction model training unit is further configured to: Get known chip models under the same chip architecture; Obtaining actual performance evaluation of the image processing chip of the known chip model for various data sizes under the preset performance category; The known chip model is used as a sample input and the corresponding actual performance evaluation is used as an expected sample output to construct a training sample; The training samples are used to train the prediction model to be trained to obtain a trained performance prediction model.

20. The device according to claim 15, wherein: The weight distribution determining unit comprises: The different impact determination subunit is configured to determine the different impacts of different training methods provided by different alternative training strategies on different types of performance; The weight distribution determination subunit is configured to determine the weight distribution of each of the preset performance types under each of the alternative training strategies according to the different impacts of the different training methods on different types of performance; wherein the weight of any type of performance is proportional to the degree of impact.

21. The device according to claim 20, wherein: The preset performance types include: at least one of computing performance, storage performance and communication performance; Correspondingly, the performance evaluation determination unit is further configured to: Determine the performance evaluation of the target model chip in terms of computing, storage and communication, respectively, and obtain computing performance evaluation, storage performance evaluation and communication performance evaluation accordingly; Correspondingly, the weight distribution determination subunit is further configured to: According to the different effects of the different training methods on the computing performance, the storage performance and the communication performance, respectively, the weight distribution of the computing performance, the storage performance and the communication performance under each of the alternative training strategies is determined.

22. The device according to claim 21, wherein The comprehensive training evaluation determination unit is further configured to: For each of the candidate training strategies, the following comprehensive training evaluation determination steps are performed: Determining a current weight distribution corresponding to a current candidate training strategy; Determine the current weights corresponding to the computing performance, the storage performance, and the communication performance respectively by using the current weight distribution; Using the current weights corresponding to the computing performance, the storage performance, and the communication performance, respectively, weighted processing is performed on the corresponding computing performance evaluation, storage performance evaluation, and communication performance evaluation to obtain weighted computing performance evaluation, storage performance evaluation, and communication performance evaluation; Based on the weighted computing performance evaluation, storage performance evaluation and communication performance evaluation corresponding to the target model chip, a comprehensive training evaluation corresponding to the current candidate training strategy is calculated.

23. The device according to claim 22, wherein: The comprehensive training evaluation corresponding to the current candidate training strategy is calculated based on the weighted computing performance evaluation, storage performance evaluation and communication performance evaluation corresponding to the target model chip, including: Quantifying the weighted evaluation parameters of the computing performance evaluation, storage performance evaluation, and communication performance evaluation corresponding to the target model chip to obtain a computing performance score, a storage performance score, and a communication performance score; Calculating the sum of the computing performance score, the storage performance score and the communication performance score to obtain a performance score sum; Based on the performance score and determining a comprehensive training evaluation corresponding to the current candidate training strategy.

24. The device according to claim 23, wherein: The comprehensive training evaluation based on the performance score and determining the corresponding candidate training strategy includes: The performance scores and the sum are used as a comprehensive training evaluation corresponding to the current candidate training strategy.

25. The apparatus of claim 15, further comprising: The abnormal training strategy removal unit is configured to remove the alternative training strategies that do not meet the requirements of the training task actually received and / or remove the alternative training strategies due to obvious video memory conflict problems from all the alternative training strategies before calculating the comprehensive training evaluation corresponding to each of the alternative training strategies according to the performance evaluation after weighting the corresponding performance types according to the weight distributions, so as to obtain the remaining training strategies; Correspondingly, the comprehensive training evaluation determination unit is further configured to: Based on the performance evaluation after weighting each corresponding performance type according to each weight distribution, a comprehensive training evaluation corresponding to each of the remaining training strategies is calculated.

26. The device according to claim 15, wherein: The comprehensive training evaluation determination unit is further configured to: Control the central processing unit to create multiple parallel computing threads, and control each computing thread to execute the following processing steps: Obtaining a weight distribution corresponding to any candidate training strategy to be calculated; Based on the performance evaluation after weighting the corresponding performance types according to the weight distribution, a comprehensive training evaluation corresponding to the currently selected candidate training strategy is calculated and output.

27. The device according to claim 15, wherein: The target training strategy determination unit is further configured to: The candidate training strategy with the highest evaluation parameter is determined as the target training strategy; wherein the highest evaluation parameter includes: the highest evaluation level, the highest evaluation score, and the highest evaluation degree.

28. The apparatus according to any one of claims 15 to 27, further comprising: The large model training unit is configured to control the computing power cluster to perform the training task of the generative large language model according to the target training strategy; wherein each target model chip under the computing power cluster will be evenly assigned processing tasks, and the processing tasks are decomposed from the training tasks.

29. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the target training strategy determination method for a single-model chip computing power cluster described in any one of claims 1-14.

30. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the target training strategy determination method for a single-model chip computing power cluster described in any one of claims 1-14.

31. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method for determining a target training strategy for a single-model chip computing power cluster according to any one of claims 1 to 14.

Images