Heterogeneous cluster performance prediction method, system, device, equipment, medium and product

Abstract

The invention relates to a heterogeneous cluster performance prediction method, system and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining respective operation data of at least one type of accelerators in a heterogeneous cluster, wherein the operation data is generated when a target model is iteratively trained through the heterogeneous cluster; based on the respective operation data of the at least one type of accelerators, performing statistics on the data processing amount and the data processing duration of the heterogeneous cluster during at least one round of training, and determining the actual measurement force utilization rate of the heterogeneous cluster according to the data processing amount and the data processing duration; an isomorphic cluster matched with the heterogeneous cluster is determined, and the theoretical computing power utilization rate of the isomorphic cluster is predicted according to the total computing power value of the isomorphic cluster and the network structure of the target model; and comparing the actual calculation power utilization rate with the theoretical calculation power utilization rate to obtain a performance prediction result of the heterogeneous cluster. By adopting the method, the heterogeneous cluster performance can be effectively evaluated.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, system, apparatus, computer device, computer-readable storage medium, and computer program product for predicting heterogeneous cluster performance. Background Art

[0002] With the development of the field of artificial intelligence, the demand for training large models built based on artificial intelligence technology has increased dramatically. Due to the limited resources of a single type of accelerator and the complex parameters and data processing requirements of such models, in general, heterogeneous clusters integrating different types of accelerators can be selected to implement model training to meet the training needs of such models.

[0003] However, due to the differences in computing power, memory capacity, communication bandwidth, etc. between different types of accelerators, the combined use of different types of accelerators often leads to inefficiency and uneven resource utilization in heterogeneous clusters, affecting model training efficiency. In order to improve model training efficiency, cluster performance can usually be evaluated and heterogeneous clusters can be optimized based on performance evaluation results. However, most commonly used cluster performance evaluation methods are based on homogeneous clusters (composed of a single type of accelerator) and cannot effectively evaluate the performance of heterogeneous clusters. Summary of the invention

[0004] Based on this, the present application at least provides a heterogeneous cluster performance prediction method, system, apparatus, computer equipment, computer-readable storage medium and computer program product that can effectively evaluate the performance of heterogeneous clusters.

[0005] In a first aspect, the present application provides a method for predicting the performance of a heterogeneous cluster, comprising: obtaining operating data of each of at least two types of accelerators in a heterogeneous cluster, the operating data being generated when a target model is iteratively trained through the heterogeneous cluster; based on the operating data of each of the at least two types of accelerators, counting the data processing volume and data processing time of the heterogeneous cluster during at least one round of training, and determining the actual computing power utilization of the heterogeneous cluster according to the data processing volume and the data processing time; determining a homogeneous cluster matching the heterogeneous cluster, and predicting the theoretical computing power utilization of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model; comparing the actual computing power utilization with the theoretical computing power utilization to obtain a performance prediction result of the heterogeneous cluster.

[0006] In one of the embodiments, determining a homogeneous cluster that matches a heterogeneous cluster includes: obtaining a total computing power value of the heterogeneous cluster; and using the total computing power value of the heterogeneous cluster as the total computing power value of the homogeneous cluster that matches the heterogeneous cluster.

[0007] In this embodiment, a homogeneous cluster with the same computing power as the heterogeneous cluster is introduced to obtain the performance prediction result of the heterogeneous cluster by comparing the performance differences between the heterogeneous cluster and the homogeneous cluster under the same computing power. This is convenient for users to refer to the performance prediction results, adopt optimization solutions suitable for the heterogeneous cluster, and improve the model training efficiency of the heterogeneous cluster.

[0008] In one of the embodiments, the theoretical computing power utilization of the homogeneous cluster is predicted based on the total computing power value of the homogeneous cluster and the network structure of the target model, including: predicting a first computing power value consumed in each round of iterative training of the target model by the homogeneous cluster based on the network structure of the target model; determining a second computing power value used by the homogeneous cluster to process each data based on the first computing power value and the data processing amount; and obtaining the theoretical computing power utilization of the homogeneous cluster based on the total computing power value and the second computing power value of the homogeneous cluster.

[0009] In this embodiment, for a homogeneous cluster with the same computing power as a heterogeneous cluster, the computing power consumption of the homogeneous cluster in processing each data can be determined by the computing power consumption and data processing of the theoretical homogeneous cluster when training the target model, and then combined with the total computing power value of the homogeneous cluster, the theoretical computing power utilization of the homogeneous cluster can be accurately calculated, so that the performance difference between the heterogeneous cluster and the homogeneous cluster with the same computing power can be accurately compared by comparing the measured computing power utilization and the theoretical computing power utilization, thereby accurately obtaining the performance prediction results of the heterogeneous cluster.

[0010] In one of the embodiments, the measured computing power utilization and the theoretical computing power utilization are compared to obtain a performance prediction result of a heterogeneous cluster, including: comparing the measured computing power utilization and the theoretical computing power utilization to obtain a utilization comparison result, and performing accelerator performance prediction based on the operating data of at least two types of accelerators to determine the accelerator performance prediction result; based on the accelerator performance prediction result and the utilization comparison result, obtaining the performance prediction result of the heterogeneous cluster.

[0011] In this embodiment, based on the utilization comparison results, accelerator performance prediction is also performed on the heterogeneous cluster. By combining the accelerator performance prediction results and the utilization comparison results, the accuracy of the heterogeneous cluster performance prediction results can be improved, which is beneficial for users to more accurately adopt optimization solutions suitable for heterogeneous clusters based on the performance prediction results, thereby effectively improving the model training efficiency of heterogeneous clusters.

[0012] In one of the embodiments, at least two types of accelerators include a first type of accelerator and a second type of accelerator; accelerator performance prediction is performed based on respective operating data of the at least two types of accelerators to determine accelerator performance prediction results, including: extracting a first data processing volume per unit time of the first type of accelerator, an accelerator computing power value of the second type of accelerator, and a second data processing volume per unit time of the heterogeneous cluster from respective operating data of the at least two types of accelerators; determining a data processing volume difference between the second data processing volume and the first data processing volume; and determining the accelerator performance prediction result based on the data processing volume difference and the accelerator computing power value of the second type of accelerator.

[0013] In this embodiment, the data processing volume difference caused by the second type of accelerator can be determined based on the operating data of the first type of accelerator in the heterogeneous cluster and the operating data of the heterogeneous cluster as a whole, so as to effectively evaluate the impact of the second type of accelerator on the performance of the heterogeneous cluster based on the data processing volume difference and the accelerator computing power value of the second type of accelerator, that is, determine an accelerator performance prediction result for the heterogeneous cluster, so that the accelerator performance prediction result and the utilization comparison result can be combined to improve the accuracy of the heterogeneous cluster performance prediction result.

[0014] In one embodiment, at least two types of accelerators include a first type of accelerator and a second type of accelerator; accelerator performance prediction is performed based on respective operating data of the at least two types of accelerators to determine an accelerator performance prediction result, including: extracting a first computing efficiency and a second computing efficiency of the first type of accelerator from the operating data of the first type of accelerator; the first computing efficiency is the computing efficiency when the first type of accelerator is used alone; the second computing efficiency is the computing efficiency when the first type of accelerator and the second type of accelerator are used in combination; and the first computing efficiency and the second computing efficiency are compared to determine the accelerator performance prediction result.

[0015] In this embodiment, the computing efficiency of the first type of accelerator in the heterogeneous cluster in a non-mixed training environment and a non-mixed training environment can be used to effectively evaluate the impact of multiple types of accelerators in the heterogeneous cluster on the performance of the heterogeneous cluster compared to a single type of accelerator, that is, determine the performance prediction result of another accelerator for the heterogeneous cluster, so that the accelerator performance prediction result and the utilization comparison result can be combined subsequently to improve the accuracy of the heterogeneous cluster performance prediction result.

[0016] In the second aspect, the present application also provides a heterogeneous cluster performance prediction system, including: a heterogeneous cluster for training a target model; an accelerator performance monitoring tool for monitoring the operating data of at least two types of accelerators in the heterogeneous cluster during the training of the target model; a heterogeneous cluster performance prediction device for obtaining the operating data of at least two types of accelerators in the heterogeneous cluster from the accelerator performance monitoring tool; based on the operating data of at least two types of accelerators, counting the data processing volume and data processing time of the heterogeneous cluster in at least one round of training, and determining the measured computing power utilization of the heterogeneous cluster based on the data processing volume and data processing time; determining a homogeneous cluster matching the heterogeneous cluster, and predicting the theoretical computing power utilization of the homogeneous cluster based on the total computing power value of the homogeneous cluster and the network structure of the target model; comparing the measured computing power utilization with the theoretical computing power utilization to obtain the performance prediction result of the heterogeneous cluster.

[0017] In the third aspect, the present application also provides a heterogeneous cluster performance prediction device, including: an operation data acquisition module, used to obtain the operation data of each of at least two types of accelerators in the heterogeneous cluster, and the operation data is generated when the target model is iteratively trained through the heterogeneous cluster; a measured computing power utilization determination module, used to count the data processing volume and data processing time of the heterogeneous cluster in at least one round of training based on the operation data of each of at least two types of accelerators, and determine the measured computing power utilization of the heterogeneous cluster according to the data processing volume and data processing time; a theoretical computing power utilization determination module, used to determine a homogeneous cluster matching the heterogeneous cluster, and predict the theoretical computing power utilization of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model; a performance prediction result acquisition module, used to compare the measured computing power utilization with the theoretical computing power utilization to obtain the performance prediction result of the heterogeneous cluster.

[0018] In a fourth aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned heterogeneous cluster performance prediction method when executing the computer program.

[0019] In a fifth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned heterogeneous cluster performance prediction method is implemented.

[0020] In a sixth aspect, the present application also provides a computer program product, including a computer program, which implements the above-mentioned heterogeneous cluster performance prediction method when executed by a processor.

[0021] The above-mentioned heterogeneous cluster performance prediction method, system, device, computer equipment, computer-readable storage medium and computer program product first obtain the operating data of at least two types of accelerators in the heterogeneous cluster, wherein the operating data is generated when the target model is iteratively trained through the heterogeneous cluster, and then based on the operating data of at least two types of accelerators, the data processing volume and data processing time of the heterogeneous cluster in at least one round of training are counted, so as to determine the measured computing power utilization of the heterogeneous cluster according to the data processing volume and data processing time, that is, the performance evaluation index of the heterogeneous cluster. Further, a homogeneous cluster matching the heterogeneous cluster is determined, so as to predict the theoretical computing power utilization of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model, that is, to determine the performance evaluation index of the homogeneous cluster. Furthermore, by comparing the measured computing power utilization with the theoretical computing power utilization, and comparing the performance differences between heterogeneous clusters and homogeneous clusters with the same computing power, we can obtain the performance prediction results of heterogeneous clusters, which is convenient for users to refer to the performance prediction results, evaluate the performance of heterogeneous clusters, and then adopt optimization solutions suitable for heterogeneous clusters to improve the model training efficiency of heterogeneous clusters.

[0022] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] In addition, the drawings herein are incorporated into the specification of this application and constitute a part of the specification of this application. These drawings illustrate embodiments consistent with the present application and are used together with the specification of this application to illustrate the technical solution of the present application.

[0025] FIG1 is an application environment diagram of a heterogeneous cluster performance prediction method in one embodiment;

[0026] FIG2 is a schematic diagram of a process flow of a heterogeneous cluster performance prediction method in one embodiment;

[0027] FIG3 is a schematic diagram of a process for predicting the theoretical computing power utilization rate of a homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model in one embodiment;

[0028] FIG4 is a schematic diagram of a process for obtaining a performance prediction result of a heterogeneous cluster based on an accelerator performance prediction result and a utilization comparison result in one embodiment;

[0029] 5 is a schematic diagram of a process of performing accelerator performance prediction based on respective operating data of at least two types of accelerators and determining an accelerator performance prediction result in one embodiment;

[0030] FIG6 is a flow chart of performing accelerator performance prediction based on respective operating data of at least two types of accelerators and determining an accelerator performance prediction result in another embodiment:

[0031] FIG7 is a schematic diagram of a process of performing performance prediction on a first type of accelerator and a second type of accelerator in a heterogeneous cluster in one embodiment;

[0032] FIG8 is a structural block diagram of a heterogeneous cluster performance prediction device in one embodiment;

[0033] FIG. 9 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] The heterogeneous cluster performance prediction method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and Internet of Things devices. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. The data storage system can store the data that the server 104 needs to process. A heterogeneous cluster, a target model, and an accelerator performance monitoring tool can be deployed on the server 104. The accelerator performance monitoring tool can be used to monitor the operation of each type of accelerator in the heterogeneous cluster in real time, including the measured accelerator's computing power value (number of floating-point operations per second), computing load, number of tasks processed, running time, etc. The accelerator performance monitoring tools used in this embodiment include but are not limited to: NVML (NVIDIA Management Library), ROCm (Radeon Open Compute), NVIDIA-SMI (NVIDIA System Management Interface). Among them, NVML is a dynamic library that can provide an interface for managing and monitoring accelerators; ROCm is an open source accelerator computing platform that can provide a set of tools for developing, running, and monitoring accelerators; NVIDIA-SMI is a command line utility that can be used to manage and monitor accelerators.

[0036] Exemplarily, the execution subject of the heterogeneous cluster performance prediction method can be a server 104 (for example, a server group composed of multiple physical servers), and the user (such as a tester) can trigger the model training instruction through the terminal 102, and the terminal 102 can communicate with the server 104 through the network, and send the model training instruction to the server 104, and the server 104 can receive and respond to the model training instruction sent by the terminal 102, and start iterative training of the target model. In the process of iterative training of the target model, the server 104 can monitor each type of accelerator in the heterogeneous cluster deployed on the server 104 through the accelerator performance monitoring tool deployed on the server 104, so as to obtain the operating data of at least two types of accelerators in the heterogeneous cluster, wherein the operating data is generated when the target model is iteratively trained through the heterogeneous cluster. Further, the server 104 can count the data processing volume and data processing time of the heterogeneous cluster during at least one round of training based on the operating data of at least two types of accelerators, and determine the measured computing power utilization rate of the heterogeneous cluster based on the data processing volume and data processing time. Furthermore, the server 104 can determine the homogeneous cluster that matches the heterogeneous cluster, and thus predict the theoretical computing power utilization of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model, and then obtain the performance prediction result of the heterogeneous cluster by comparing the measured computing power utilization with the theoretical computing power utilization, and feed back the performance prediction result to the terminal 102 for user reference.

[0037] Alternatively, the execution subject of the heterogeneous cluster performance prediction method is server 104 as an example for explanation. The user may also trigger the model training instruction to the server 104 through the instruction trigger channel directly connected to the server 104 instead of the terminal 102. Taking the server 104 as a physical server as an example, the user may trigger the model training instruction by pressing a button or touch key set on the server 104. The server 104 may also feedback the performance prediction result of the heterogeneous cluster to the user through the result feedback channel directly connected to the server 104 instead of the terminal 102. Taking the server 104 as a physical server and a display panel set on the server 104 as an example, the server 104 may display the performance prediction result of the heterogeneous cluster on the display panel for the user to view.

[0038] In an optional implementation, the execution subject of the heterogeneous cluster performance prediction method may also be the terminal 102, which may be a device group composed of multiple devices. Taking the case where a heterogeneous cluster, a target model, and an accelerator performance monitoring tool are deployed on the terminal 102 as an example, the user may trigger the model training instruction on the terminal 102, and the terminal 102 may respond to the model training instruction to start iterative training of the target model, and in the process of iterative training of the target model, the accelerator performance monitoring tool deployed on the terminal 102 may be used to monitor the accelerators in the heterogeneous cluster, thereby obtaining the respective operating data of at least two types of accelerators in the heterogeneous cluster. Furthermore, the terminal 102 may count the data processing volume and data processing time of the heterogeneous cluster during at least one round of training based on the respective operating data of at least two types of accelerators, and determine the measured computing power utilization rate of the heterogeneous cluster based on the data processing volume and data processing time. Furthermore, the terminal 102 can determine the homogeneous cluster that matches the heterogeneous cluster, and thus predict the theoretical computing power utilization of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model, and then obtain the performance prediction result of the heterogeneous cluster by comparing the measured computing power utilization with the theoretical computing power utilization, and feedback the performance prediction result.

[0039] The above description of various embodiments in the application environment involved in the heterogeneous cluster performance prediction method tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other and will not be repeated here.

[0040] In an exemplary embodiment, as shown in FIG2 , a heterogeneous cluster performance prediction method is provided, and the method is applied to the server 104 in FIG1 as an example for description, including the following steps 202 to 208. Among them:

[0041] Step 202 , obtaining operating data of at least two types of accelerators in the heterogeneous cluster, where the operating data is generated when the target model is iteratively trained through the heterogeneous cluster.

[0042] Among them, accelerators include but are not limited to at least one of the following: GPU (Graphics Processing Unit), CPU (Central Processing Unit), NPU (Neural Network Processing Unit), TPU (Tensor Processing Unit), which is a microprocessor designed for performing complex mathematical and geometric calculations, and can be used to accelerate various computing-intensive tasks, such as graphics rendering, scientific simulation, deep learning training, etc. Heterogeneous clusters can specifically refer to: clusters composed of multiple types of accelerators. Users can flexibly combine different accelerator resources according to needs to build heterogeneous clusters to meet complex high-performance computing and data processing needs. The operating data of the accelerator includes but is not limited to: the measured computing power value of the accelerator (the number of floating-point operations per second), the amount of data processed, the time point when the operation starts, the time point when the operation ends, etc. The target model includes but is not limited to: complex models built based on artificial intelligence technology, or complex models in other fields.

[0043] Optionally, a heterogeneous cluster, a target model, and an accelerator performance monitoring tool are deployed on the server. Based on this, the server can schedule multiple types of accelerators in the heterogeneous cluster to perform training tasks for the target model according to the received model training instructions. During the process of iterative training of the target model by the heterogeneous cluster, each type of accelerator in the heterogeneous cluster will generate a large amount of operating data. The server can schedule the accelerator performance monitoring tool to monitor each type of accelerator in the heterogeneous cluster separately, thereby obtaining the operating data of at least two types of accelerators in the heterogeneous cluster.

[0044] Step 204 , based on the respective operating data of at least two types of accelerators, statistics are collected on the data processing volume and data processing time of the heterogeneous cluster during at least one round of training, and the measured computing power utilization rate of the heterogeneous cluster is determined according to the data processing volume and the data processing time.

[0045] Among them, in the data processing amount during at least one round of training, the data processing amount of the heterogeneous cluster during each round of training can specifically refer to: the amount of input data processed by the heterogeneous cluster during each round of training, which can be represented by the number of tokens. In model training, a token usually refers to: a minimum data unit in the input data of the model, that is, a basic data unit. This minimum data unit or basic data unit can be a word, a character, or a word or phrase after word segmentation, etc., depending on the training method of the model. In the model training process, at the input end of the model, a specific conversion method suitable for the model is usually used to convert a large number of tokens (that is, input data) input to the model into a representation form that the model can understand and process. In the data processing time during at least one round of training, the data processing time of the heterogeneous cluster during each round of training can specifically refer to: in each round of training, the data processing time consumed by the accelerator in the heterogeneous cluster from the beginning of processing the input data of the round to the end of processing the input data of the round.

[0046] Optionally, during the process of iterative training of the target model in the heterogeneous cluster, the server can monitor the operation data generated by each type of accelerator in the heterogeneous cluster during at least one round of training through the accelerator performance monitoring tool, and then extract the data processing volume, data processing start time and data processing end time of each type of accelerator in each round of training from the operation data of each type of accelerator. Furthermore, for each round, the server can take the sum of the data processing volumes of all accelerators in the heterogeneous cluster in that round as the data processing volume of the heterogeneous cluster in that round, thereby obtaining the data processing volume of the heterogeneous cluster in each round of training, and can obtain the average data processing volume of the heterogeneous cluster to complete an iterative training process by averaging the data processing volume of the heterogeneous cluster in at least one round. Furthermore, for each round, the server can use the earliest data processing start time of the accelerator in the round as the data processing start time of the heterogeneous cluster in the round, and the latest data processing end time of the accelerator in the round as the data processing end time of the heterogeneous cluster in the round, so as to obtain the data processing time of the heterogeneous cluster in each round of training, and can obtain the average data processing time of the heterogeneous cluster to complete an iterative training process by averaging the data processing time of the heterogeneous cluster in at least one round. Based on this, the server can calculate the measured computing power utilization of the heterogeneous cluster based on the average data processing amount and average data processing time of the heterogeneous cluster to complete an iterative training process.

[0047] Alternatively, when the relevant data calculation program has been set in the accelerator performance monitoring tool, the server can also directly extract the average data processing volume and average data processing time of the heterogeneous cluster completing one iterative training process from the operating data fed back by the accelerator performance monitoring tool. Alternatively, the server determines the average data processing volume and average data processing time of the heterogeneous cluster completing one iterative training process by analyzing the log files generated during the iterative training process. Among them, the log file is a file that can record events that occur when the system or application is running.

[0048] For example, the server may use the ratio of the average data processing volume to the average data processing time as the measured computing power utilization of the heterogeneous cluster, which may be specifically shown in formula (1):

[0049] (1)

[0050] Among them, the average data processing volume can specifically refer to: during the process of iterative training of the target model by the heterogeneous cluster, the number of tokens processed by the heterogeneous cluster in one iterative training process. The average data processing volume can be used to evaluate the computing power of the heterogeneous cluster. The average data processing time can specifically refer to: during the process of iterative training of the target model by the heterogeneous cluster, the time required for the heterogeneous cluster to complete one iterative training process. The average data processing time can represent: during the actual training process, the computing efficiency of the heterogeneous cluster in executing the model training task. The measured computing power utilization rate can specifically refer to: during the actual training process, the number of tokens that the heterogeneous cluster can process per second, which can reflect the overall computing efficiency of the heterogeneous cluster. Even when the cluster is reduced to a single accelerator, the calculation method of the measured computing power utilization rate is still applicable, and the overall computing efficiency of the cluster can continue to be evaluated.

[0051] Step 206, determining a homogeneous cluster that matches the heterogeneous cluster, and predicting the theoretical computing power utilization of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model.

[0052] Specifically, a homogeneous cluster can be: a cluster composed of a single type of accelerator. A homogeneous cluster matching a heterogeneous cluster can refer to: a homogeneous cluster whose theoretical total computing power is equal to that of a heterogeneous cluster, and in the case where a homogeneous cluster is composed of a certain type of accelerator, the heterogeneous cluster also includes this type of accelerator, so as to facilitate the subsequent comparison of the impact of multiple types of accelerators in heterogeneous clusters and single accelerators in homogeneous clusters on cluster performance, so as to evaluate whether multiple types of accelerators effectively improve the performance of heterogeneous clusters. Taking the target model as a model based on a neural network as an example, the network structure of the target model includes but is not limited to: convolutional layers, fully connected layers, pooling layers, etc. The convolutional layer is used to extract features from the input data. The fully connected layer is usually located at the end of the network structure and is used to map the features extracted by the previous layer to the final output space. The pooling layer can be used to perform feature dimensionality reduction and feature compression on the features of the input pooling layer. Theoretical computing power utilization can specifically refer to the number of tokens that can be processed per second when a homogeneous cluster theoretically iterates to train the target model, which can represent the theoretical computing power upper limit of the homogeneous cluster.

[0053] Optionally, in order to evaluate whether multiple types of accelerators have effectively improved the performance of heterogeneous clusters, the server can first determine a homogeneous cluster whose total computing power value is equal to that of the heterogeneous cluster and is composed of a certain accelerator included in the heterogeneous cluster, so that the performance of the heterogeneous cluster can be evaluated by comparing the difference between the computing power utilization of the heterogeneous cluster and the homogeneous cluster under the same computing power. Furthermore, the server can analyze the network structure of the target model and predict the theoretical predicted computing power consumption of the homogeneous cluster in the scenario of iterative training of the target model by the homogeneous cluster, thereby predicting the theoretical computing power utilization of the homogeneous cluster when iteratively training the target model based on the total computing power value of the homogeneous cluster and the predicted computing power consumption of the homogeneous cluster.

[0054] Alternatively, in addition to predicting the theoretical computing power utilization of the homogeneous cluster by analyzing the network structure of the target model, when the total computing power value of the homogeneous cluster is equal to that of the heterogeneous cluster, that is, when the total computing power value of the homogeneous cluster is known, the server can also predict the theoretical computing power utilization of the homogeneous cluster when iteratively training the target model through tools such as torchprofile deployed on the server. Among them, torchprofile is a lightweight performance profiler specially designed for the field of deep learning. It can accurately count the MACs (Multiply–Accumulate Operations) of the model, TFLOPS, and the number of tokens that can be processed per second based on the number of trillions of floating-point operations performed per second. Among them, the number of tokens that can be processed per second is the theoretical computing power utilization of the homogeneous cluster.

[0055] Step 208 , compare the measured computing power utilization and the theoretical computing power utilization to obtain the performance prediction result of the heterogeneous cluster.

[0056] The performance prediction results of the heterogeneous cluster may include multi-dimensional analysis results, including but not limited to: the utilization comparison results of the measured computing power utilization and the theoretical computing power utilization.

[0057] Optionally, the server can preliminarily obtain the performance prediction results of the heterogeneous cluster by comparing the measured computing power utilization with the theoretical computing power utilization.

[0058] For example, the example in which at least two types of accelerators in a heterogeneous cluster include a Class A accelerator and a non-Class A accelerator, while the homogeneous cluster is composed of a single Class A accelerator is used for explanation: (1) If the measured computing power utilization of the heterogeneous cluster is higher than the theoretical computing power utilization of the homogeneous cluster, the server can feedback the following performance prediction results: integrating the Class A accelerator with the non-Class A accelerator can effectively improve the cluster performance through the non-Class A accelerator, so that the user can refer to the performance prediction results and optimize the heterogeneous cluster, for example, increase the proportion of non-Class A accelerators in the heterogeneous cluster. (2) If the measured computing power utilization of the heterogeneous cluster is lower than the theoretical computing power utilization of the homogeneous cluster, the server can feedback the following performance prediction results: integrating the Class A accelerator with the non-Class A accelerator will reduce the cluster performance, so that the user can refer to the performance prediction results and optimize the heterogeneous cluster, for example, reduce the proportion of non-Class A accelerators in the heterogeneous cluster. Specifically, the Class A accelerator can be an NVIDIA accelerator commonly used in the computer field.

[0059] The above description of the various embodiments involved in step 202 to step 208 tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other and will not be repeated for the sake of brevity.

[0060] The above heterogeneous cluster performance prediction method first obtains the operating data of at least two types of accelerators in the heterogeneous cluster, wherein the operating data is generated when the target model is iteratively trained through the heterogeneous cluster, and then based on the operating data of at least two types of accelerators, the data processing volume and data processing time of the heterogeneous cluster during at least one round of training are counted, so as to determine the measured computing power utilization of the heterogeneous cluster according to the data processing volume and data processing time, that is, the performance evaluation index of the heterogeneous cluster. Further, a homogeneous cluster matching the heterogeneous cluster is determined, so as to predict the theoretical computing power utilization of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model, that is, to determine the performance evaluation index of the homogeneous cluster. Furthermore, by comparing the measured computing power utilization with the theoretical computing power utilization, and comparing the performance difference between the heterogeneous cluster and the homogeneous cluster with the same computing power, the performance prediction result of the heterogeneous cluster is obtained, which is convenient for users to refer to the performance prediction result, evaluate the performance of the heterogeneous cluster, and then adopt an optimization scheme suitable for the heterogeneous cluster to improve the model training efficiency of the heterogeneous cluster.

[0061] In an exemplary embodiment, the homogeneous cluster matching the heterogeneous cluster may be determined in the following manner: a total computing power value of the heterogeneous cluster is obtained, and the total computing power value of the heterogeneous cluster is used as the total computing power value of the homogeneous cluster matching the heterogeneous cluster.

[0062] Among them, the total computing power of the homogeneous cluster can be represented by: the number of trillions of floating-point operations that the homogeneous cluster can theoretically perform per second.

[0063] Optionally, the server can obtain the computing power value of each type of accelerator in the heterogeneous cluster (which can be the measured computing power value of the accelerator extracted from the running data or the theoretical computing power value of the accelerator), and add up the total computing power value of the heterogeneous cluster (which can be the measured total computing power value or the theoretical total computing power value), so as to use the total computing power value of the heterogeneous cluster as the total computing power value of the homogeneous cluster matching the heterogeneous cluster, so as to determine a homogeneous cluster with the same computing power as the heterogeneous cluster in theory. Among them, the computing power value of the accelerator (whether it is the measured computing power value or the theoretical computing power value) can characterize the computing power of the accelerator, and the total computing power value of the heterogeneous cluster can characterize the computing power of the heterogeneous cluster as a whole. The computing power value can be specifically: the number of floating point operations that can be executed per second (FLOPS, Floating Point Operations Per Second). Considering that the number of floating point operations that can be executed per second by the accelerator or heterogeneous cluster is extremely large, the computing power value in this embodiment can be characterized by TFLOPS (trillions of floating point operations per second) in the FLOPS system.

[0064] For example, the server uses the total computing power value of the heterogeneous cluster as the total computing power value of the homogeneous cluster. The server can query and obtain the theoretical computing power value of each accelerator in the heterogeneous cluster according to the model of each accelerator, and accumulate the theoretical computing power values ​​of all accelerators in the heterogeneous cluster to obtain the theoretical total computing power value of the heterogeneous cluster, and use the theoretical total computing power value of the heterogeneous cluster as the total computing power value of the homogeneous cluster. The theoretical computing power value of the accelerator can specifically refer to: based on the accelerator model, the computing power value marked by the accelerator manufacturer for the accelerator can be stored in the server for query.

[0065] Exemplarily, taking the example of a server using the total computing power value measured by a heterogeneous cluster as the total computing power value of a homogeneous cluster, the server can obtain the measured computing power value of each accelerator from the operating data of each accelerator, thereby accumulating the measured computing power values ​​of all accelerators in the heterogeneous cluster to obtain the total computing power value measured by the heterogeneous cluster, and use the total computing power value measured by the heterogeneous cluster as the total computing power value of the homogeneous cluster.

[0066] In this embodiment, the total computing power value of the heterogeneous cluster is used as the total computing power value of the homogeneous cluster, so that the computing power utilization rates of the heterogeneous cluster and the homogeneous cluster can be compared later when the total computing power is equal, thereby evaluating the performance difference between the heterogeneous cluster and the homogeneous cluster.

[0067] In one embodiment, as shown in FIG3 , a flow chart of predicting the theoretical computing power utilization of a homogeneous cluster based on the total computing power value of the homogeneous cluster and the network structure of the target model is provided, wherein:

[0068] Step 302 , based on the network structure of the target model, predict the first computing power value consumed in each round when the homogeneous cluster iteratively trains the target model.

[0069] Among them, the first computing power value can refer to: the number of trillion floating-point operations consumed in each round when the homogeneous cluster iteratively trains the target model, that is, the theoretical amount of computing used in a single round when the homogeneous cluster theoretically iterates the target model.

[0070] Optionally, for each layer in the network structure of the target model, the server can first determine the type and parameters of each layer, and then predict the theoretical amount of computation required for the homogeneous cluster to perform the operations involved in each layer in each round of the homogeneous cluster training target model according to the type and parameters of each layer, and then obtain the theoretical amount of computation required for each round (single round) when the homogeneous cluster iteratively trains the target model by accumulating the theoretical amount of computation required for all layers in the target model, that is, obtain the first computing power value consumed by the homogeneous cluster in each round when training the target model, so as to subsequently calculate the theoretical computing power utilization rate of the homogeneous cluster according to the total computing power value provided by the homogeneous cluster and the theoretical amount of computation required for the homogeneous cluster to train the target model in a single round. Among them, the type of each layer in the target model includes but is not limited to: convolution layer, fully connected layer, pooling layer. Taking the fully connected layer as an example, the parameters of the fully connected layer include but are not limited to: input size and output size. Taking the convolution layer as an example, the parameters of the convolution layer include but are not limited to: convolution kernel width, convolution kernel height, number of input channels, number of output channels, output feature map width, and output feature map height. Taking the pooling layer as an example, since the operation of the pooling layer usually does not involve floating-point operations, the theoretical amount of operations required by the pooling layer can be assumed to be zero. If the operation of the pooling layer involves floating-point operations, the parameters of the pooling layer include but are not limited to: pooling kernel size, step size, output feature map width, and output feature map length.

[0071] In a possible implementation, how to determine the type and parameters of each layer in the target model: if the server pre-stores the annotation information for the network structure of the target model, and the type and parameters of each layer in the target model are marked in the annotation information, the server can obtain the type and parameters of each layer in the target model from the stored annotation information. If the server does not store the annotation information for the network structure of the target model, the server needs to analyze the network structure of the target model to determine the type and parameters of each layer in the target model. For example, the server can traverse all layers of the target model through the model structure traversal script deployed on the server (the script can be customized) to determine the type and parameters of each layer in the target model, or identify the type and parameters of each layer in the target model through the model visualization tool deployed on the server. Among them, the model visualization tool includes but is not limited to TensorBoard, Netron, etc. TensorBoard can provide rich visualization functions and can clearly display the type and parameters of each layer in the model structure. Netron is a cross-platform tool that supports model files of multiple deep learning frameworks and can display the type and parameters of each layer in the model in a graphical way.

[0072] Exemplarily, how to predict the theoretical amount of operations required for a homogeneous cluster to perform operations involved in each layer based on the type and parameters of each layer: If a fully connected layer exists in the target model, the server can determine the theoretical amount of operations involved in executing a fully connected layer = 2×(input size×output size) based on the input size and output size of the fully connected layer. If a convolutional layer exists in the target model, the server can determine the theoretical amount of operations involved in executing a convolutional layer = 2×(convolutional kernel width×convolutional kernel height×input channels×output channels×output feature map width×output feature map height) based on the convolutional kernel width, convolution kernel height, input channel number, output channel number, output feature map width, and output feature map height of the convolutional layer. If there is a pooling layer in the target model, but the operation of the pooling layer does not involve floating-point operations, the theoretical amount of operations involved in executing the pooling layer once = zero. If the operation of the pooling layer involves floating-point operations, the server can determine the theoretical amount of operations involved in executing the pooling layer once = (pooling kernel size - 1) × (output feature map width ÷ step length) × (output feature map length ÷ step length) based on the pooling kernel size, step length, output feature map width, and output feature map length of the pooling layer. Based on this, the server can accumulate the theoretical amount of operations involved in executing all layers in the target model once to determine the theoretical amount of operations consumed in each round when the homogeneous cluster iteratively trains the target model, that is, the first computing power value consumed by the homogeneous cluster to theoretically complete an iterative training process.

[0073] Step 304: Determine a second computing power value used by the homogeneous cluster to process each data according to the first computing power value and the data processing amount.

[0074] The second computing power value represents: the theoretical amount of computing power used by the homogeneous cluster to process each token, that is, the number of floating-point operations required to process each token.

[0075] Optionally, during the process of training the target model of the heterogeneous cluster, the server can determine the data processing amount of the heterogeneous cluster in each round of training through the operation data monitored by the accelerator performance monitoring tool, that is, the number of input data (the number of tokens) processed by the heterogeneous cluster in each round of training, thereby obtaining the average data processing amount of the heterogeneous cluster to complete an iterative training process, that is, the average number of input data of the heterogeneous cluster in an iterative training process. Further, the server can use the average number of input data of the heterogeneous cluster in an iterative training process as the theoretical number of input data of the homogeneous cluster in an iterative training process (theoretically the number of input tokens). Further, after determining the first computing power value consumed by the homogeneous cluster to theoretically complete an iterative training process and the theoretical number of input data of the homogeneous cluster in an iterative training process, the server can calculate the second computing power value required for the homogeneous cluster to process each data during the execution of an iterative training by the homogeneous cluster, that is, the number of floating-point operations required for the homogeneous cluster to process each token.

[0076] For example, the server can specifically calculate the second computing power value required by the homogeneous cluster to process each data during one iteration of training performed by the homogeneous cluster by using formula (2):

[0077] (2)

[0078] The first computing power value specifically refers to the number of floating-point operations consumed by a homogeneous cluster in theoretically executing one iterative training process. The theoretical number of input data specifically refers to the number of tokens that a homogeneous cluster needs to process in theoretically one iterative training process. The second computing power value specifically refers to the number of floating-point operations consumed by a homogeneous cluster in processing each token in theoretically executing one iterative training process.

[0079] Step 306: Obtain a theoretical computing power utilization rate of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the second computing power value.

[0080] Optionally, the server can calculate the theoretical computing power utilization of the homogeneous cluster based on the total computing power that the homogeneous cluster can provide (teraflops) and the number of floating-point operations required for the homogeneous cluster to process each data during one iterative training.

[0081] For example, the server may use the ratio of the total computing power value of the homogeneous cluster to the second computing power value as the theoretical computing power utilization rate of the homogeneous cluster, which may be specifically shown as formula (3):

[0082] (3)

[0083] The theoretical computing power utilization rate specifically refers to: theoretically, when a homogeneous cluster is used to iteratively train the target model, the number of tokens that can be processed per second by the homogeneous cluster can represent the overall computing efficiency of the homogeneous cluster.

[0084] In this embodiment, for a homogeneous cluster with the same computing power as a heterogeneous cluster, the computing power consumption (first computing power value) of the homogeneous cluster in processing each data can be determined by the computing power consumption (first computing power value) and data processing (theoretical number of input data) of the theoretical homogeneous cluster when training the target model, and then combined with the total computing power value of the homogeneous cluster, the theoretical computing power utilization rate of the homogeneous cluster can be accurately calculated, so that the performance difference between the heterogeneous cluster and the homogeneous cluster with the same computing power can be accurately compared by comparing the measured computing power utilization rate with the theoretical computing power utilization rate, thereby accurately obtaining the performance prediction result of the heterogeneous cluster.

[0085] In an optional implementation, as shown in FIG. 4 , a flow chart of obtaining a performance prediction result of a heterogeneous cluster based on an accelerator performance prediction result and a utilization comparison result is provided, that is, step 208 in FIG. 2 may also include steps 402 to 404, wherein:

[0086] Step 402, compare the measured computing power utilization and the theoretical computing power utilization to obtain a utilization comparison result, and perform accelerator performance prediction based on the operating data of at least two types of accelerators to determine the accelerator performance prediction result.

[0087] Among them, the accelerator performance prediction result can be at least one.

[0088] Optionally, the server can obtain a utilization comparison result by comparing the measured computing power utilization and the theoretical computing power utilization, and determine the accelerator performance evaluation parameters of at least two types of accelerators respectively, and / or the accelerator performance evaluation parameters for the accelerators in the entire heterogeneous cluster based on the respective operating data of at least two types of accelerators. Further, according to the preset accelerator performance prediction rules, based on the accelerator performance evaluation parameters of the two types of accelerators respectively, and / or, for the accelerator performance evaluation parameters of the accelerators in the entire heterogeneous cluster, the accelerator performance prediction is performed on the accelerators in the heterogeneous cluster, and the accelerator performance prediction result is calculated.

[0089] Step 404: Based on the accelerator performance prediction result and the utilization comparison result, a performance prediction result of the heterogeneous cluster is obtained.

[0090] Optionally, the server may combine the accelerator performance prediction result and the utilization comparison result to obtain the performance prediction result of the heterogeneous cluster, and feed back the obtained performance prediction result.

[0091] For example, a heterogeneous cluster includes at least two types of accelerators, including a Class A accelerator and a non-Class A accelerator, while a homogeneous cluster consists of a single Class A accelerator. (1) If the measured computing power utilization of the heterogeneous cluster is higher than the theoretical computing power utilization of the homogeneous cluster, and the accelerator performance prediction result indicates that the non-Class A accelerator improves the computing efficiency of the heterogeneous cluster, or that when the Class A accelerator and the non-Class A accelerator run together, the computing efficiency of the Class A accelerator is higher, the server can feedback the following performance prediction result: integrating the Class A accelerator with the non-Class A accelerator can effectively improve cluster performance. Based on this, the user can refer to the performance prediction result to optimize the heterogeneous cluster, for example, increase the proportion of non-Class A accelerators in the heterogeneous cluster. (2) If the measured computing power utilization of the heterogeneous cluster is lower than the theoretical computing power utilization of the homogeneous cluster, and the accelerator performance prediction results indicate that the non-Class A accelerator reduces the computing efficiency of the heterogeneous cluster, or that when the Class A accelerator and the non-Class A accelerator are run together, the computing efficiency of the Class A accelerator is lower, the server can feedback the following performance prediction results: integrating the Class A accelerator with the non-Class A accelerator will reduce cluster performance. Based on this, users can refer to the performance prediction results to optimize the heterogeneous cluster, for example, reducing the proportion of non-Class A accelerators in the heterogeneous cluster. Specifically, the Class A accelerator can be an NVIDIA accelerator commonly used in the computer field.

[0092] In this embodiment, based on the utilization comparison results, accelerator performance prediction is also performed on the heterogeneous cluster. The accelerator performance prediction results and the utilization comparison results can be combined to improve the accuracy of the performance prediction results of the heterogeneous cluster, which is beneficial for users to more accurately adopt optimization solutions suitable for heterogeneous clusters based on the performance prediction results, thereby effectively improving the model training efficiency of heterogeneous clusters.

[0093] It should be noted that there is no unique way to implement accelerator performance prediction. In an optional implementation, the at least two types of accelerators include a first type of accelerator and a second type of accelerator. As shown in FIG5 , a flow chart of performing accelerator performance prediction based on respective operating data of at least two types of accelerators and determining accelerator performance prediction results is provided, which mainly includes the following steps:

[0094] Step 502 : extracting, from the respective operating data of at least two types of accelerators, a first data processing amount per unit time of the first type of accelerator, an accelerator computing power value of the second type of accelerator, and a second data processing amount per unit time of the heterogeneous cluster.

[0095] Among them, the first data processing volume of the first type of accelerator, the accelerator computing power value of the second type of accelerator, and the second data processing volume of the heterogeneous cluster can all be measured accelerator performance evaluation parameters extracted from the operating data to evaluate the performance of the accelerator in the heterogeneous cluster. The first data processing volume of the first type of accelerator per unit time can refer to: the measured number of tokens processed by the first type of accelerator per second. The accelerator computing power value of the second type of accelerator can be: the measured number of trillions of floating-point operations that can be performed by the second type of accelerator per second. The second data processing volume of the heterogeneous cluster per unit time can be: the measured number of tokens processed by the heterogeneous cluster per second.

[0096] Optionally, in the process of the accelerator performance monitoring tool monitoring the heterogeneous cluster, the accelerator performance monitoring tool can monitor the operating data of at least two types of accelerators (for example, the measured accelerator computing power value, the measured first data processing volume per unit time), and monitor the operating data of the entire heterogeneous cluster (for example, the measured second data processing volume per unit time of the entire heterogeneous cluster), and feed back the monitored operating data to the server. Considering that each type of accelerator is part of the heterogeneous cluster, the accelerator performance monitoring tool can feed back the real-time operating data of the entire heterogeneous cluster in the real-time operating data of each type of accelerator fed back to the server. Based on this, the server can extract the first data processing volume of the first type of accelerator per unit time from the operating data of the first type of accelerator, and extract the accelerator computing power value of the second type of accelerator from the operating data of the second type of accelerator, and the second data processing volume per unit time of the heterogeneous cluster can be extracted from the operating data of the first type of accelerator, or from the operating data of the second type of accelerator.

[0097] Alternatively, the accelerator performance monitoring tool can also feed back the operating data of each type of accelerator and the operating data of the entire heterogeneous cluster to the server separately, so that the server can extract the first data processing volume of the first type of accelerator per unit time, the accelerator computing power value of the second type of accelerator, and the second data processing volume of the heterogeneous cluster per unit time from the received operating data.

[0098] Step 504: determine a data processing amount difference between the second data processing amount and the first data processing amount.

[0099] Optionally, the server may further calculate a data processing volume difference between the second data processing volume and the first data processing volume.

[0100] It should be noted that if the data processing volume difference is low (such as less than the preset difference) or approaches zero, it indicates that: in the working process of the heterogeneous cluster (the first type of accelerator and the second type of accelerator operate together), when executing the same task, it is mainly the first type of accelerator that performs the computing task, and the computing resources of the second type of accelerator are not fully utilized. The integration of the first type of accelerator and the second type of accelerator does not effectively improve the overall computing efficiency of the cluster. If the data processing volume difference is large (such as reaching the preset difference), it indicates that: in the working process of the heterogeneous cluster, when executing the same task, the second type of accelerator can undertake certain computing tasks. The larger the data processing volume difference, the more computing tasks the second type of accelerator undertakes. The integration of the first type of accelerator and the second type of accelerator can effectively improve the overall computing efficiency of the cluster.

[0101] Step 506: Determine the accelerator performance prediction result according to the data processing amount difference and the accelerator computing power value of the second type of accelerator.

[0102] Optionally, there is no unique way to determine the accelerator performance prediction result based on the data processing volume difference and the accelerator computing power value of the second type of accelerator.

[0103] (1) Regardless of whether there are multiple second-subclass accelerators in the second-class accelerator, when the second-class accelerator is regarded as a whole, the server can use the ratio of the data processing volume difference and the accelerator computing power value of the second-class accelerator as the accelerator performance prediction result to evaluate whether the performance of the second-class accelerator is effectively exerted by integrating the second-class accelerator on the basis of the first-class accelerator, that is, to evaluate whether the performance of the heterogeneous cluster is effectively improved compared with the cluster composed of a single first-class accelerator. For example, the larger the accelerator performance prediction result of the second-class accelerator, the greater the performance effect of the second-class accelerator in the heterogeneous cluster, and the greater the effect on improving the overall performance of the cluster. Based on this, the server can specifically calculate the accelerator performance prediction result through formula (4):

[0104] (4)

[0105] Among them, the accelerator performance prediction result in formula (4) represents: after the second accelerator is regarded as a whole, the amount of data processed by the second type of accelerator based on the unit floating point computing power. If the total computing power value of the heterogeneous cluster is TFLOPS_fuse (trillion floating point operations per second), the accelerator computing power value of the first type of accelerator is TFLOPS_nv (trillion floating point operations per second), and the accelerator computing power value of the second type of accelerator is TFLOPS_other (trillion floating point operations per second), then TFLOPS_fuse=TFLOPS_nv+TFLOPS_other, the first data processing volume can be specifically: the computing power of the first type of accelerator based on TFLOPS_nv, the number of tokens processed per second, and the second data processing volume can be specifically: the computing power of the heterogeneous cluster based on TFLOPS_fuse, the number of tokens processed per second. Data processing volume difference = second data processing volume - first data processing volume. In addition, TFLOPS_fuse, TFLOPS_nv, TFLOPS_other, the first data processing volume, and the second data processing volume are all operating data that can be directly monitored by the accelerator performance monitoring tool and fed back to the server.

[0106] Alternatively, the server may also calculate the data processing volume difference based on the sum of the number of tokens processed per second by all second sub-category accelerators monitored by the accelerator performance monitoring tool. However, considering that the second-category accelerator generally includes a variety of second sub-category accelerators (of various types), this embodiment adopts the method of calculating the difference between the second data processing volume and the first data processing volume to determine the data processing volume difference.

[0107] (2) When there are multiple second-subclass accelerators in the second-class accelerator, the multiple second-subclass accelerators may not be regarded as a whole, but the accelerator performance prediction may be performed for each second-subclass accelerator separately. For each second-subclass accelerator, the server may extract the accelerator computing power value of the second-subclass accelerator from the operating data monitored for the second-subclass accelerator, and then use the ratio of the data processing volume difference to the accelerator computing power value of the second-subclass accelerator as the accelerator performance prediction result of the second-subclass accelerator to evaluate whether the performance of the second-subclass accelerator is effectively exerted, that is, to evaluate whether the integration of the second-subclass accelerator on the basis of the first-class accelerator can effectively improve the cluster performance. For example, the larger the value of the accelerator performance prediction result of the second-subclass accelerator, the greater the performance effect of the second-subclass accelerator in the heterogeneous cluster, and the greater the effect on improving the overall performance of the cluster. Based on this, the server can specifically calculate the accelerator performance prediction result of each second-subclass accelerator through formula (5):

[0108] (5)

[0109] The accelerator performance prediction result in formula (5) represents the maximum amount of data processed by the second-class sub-accelerator based on the unit floating-point computing power. The accelerator computing power value of the second-class accelerator can be monitored by the accelerator performance monitoring tool and fed back to the server. Users can refer to the accelerator performance prediction results of each second-class accelerator to find the second-class accelerator with lower performance in the heterogeneous cluster.

[0110] In this embodiment, the data processing volume difference caused by the second type of accelerator can be determined based on the operating data of the first type of accelerator in the heterogeneous cluster and the operating data of the heterogeneous cluster as a whole. Therefore, based on the data processing volume difference and the accelerator computing power value of the second type of accelerator, the impact of the second type of accelerator in the heterogeneous cluster on the performance of the heterogeneous cluster can be effectively evaluated, providing a reference for optimizing the second type of accelerator in the heterogeneous cluster. An accelerator performance prediction result for the heterogeneous cluster can also be obtained, so that the accelerator performance prediction result and the utilization comparison result can be combined in the subsequent process to improve the accuracy of the heterogeneous cluster performance prediction result.

[0111] In another optional embodiment, as shown in FIG6 , another flow chart of performing accelerator performance prediction based on respective operating data of at least two types of accelerators and determining accelerator performance prediction results is provided, which mainly includes the following steps:

[0112] Step 602: extract a first computing efficiency and a second computing efficiency of the first type of accelerator from the operation data of the first type of accelerator; the first computing efficiency is the computing efficiency when the first type of accelerator is used alone; the second computing efficiency is the computing efficiency when the first type of accelerator and the second type of accelerator are used in combination.

[0113] Among them, the first computing efficiency can specifically refer to: the computing efficiency of the model feedforward unit when the first type of accelerator is used alone during the iterative training of the target model in the heterogeneous cluster. The second computing efficiency can specifically refer to: the computing efficiency of the model feedforward unit when the first type of accelerator and the second type of accelerator are used in combination during the iterative training of the target model in the heterogeneous cluster. The model feedforward unit computing efficiency is MFU (Model FLOPs Utilization), which represents: the ratio of the matrix computing power consumed by the target model to complete a forward and reverse calculation to the machine computing power. The accelerator performance monitoring tool can generally monitor the accelerators in the heterogeneous cluster, automatically analyze the MFU, and feed it back to the server as the accelerator's operating data.

[0114] Optionally, the server can schedule the heterogeneous cluster to iteratively train the target model in two environments respectively, and obtain the operating data monitored by the accelerator performance monitoring tool for the first type of accelerator during the training process. The two environments refer to: the first type of accelerator in the heterogeneous cluster is used alone, and the first type of accelerator and the second type of accelerator in the heterogeneous cluster are used together. Based on this, the server can extract the first computing efficiency of the first type of accelerator from the operating data generated when the first type of accelerator is used alone, and extract the second computing efficiency of the first type of accelerator from the operating data generated when the first type of accelerator and the second type of accelerator are used together. Among them, the accelerator performance monitoring tools that can be used to monitor MFU include but are not limited to: NSight Systems, NVML. NSight Systems is a performance analysis tool that can be used to identify optimization opportunities for GPUs, and NVML is a dynamic library that provides interfaces for managing and monitoring accelerators. Both can be used to monitor and calculate MFU during model training.

[0115] Optionally, for accelerator performance monitoring tools that cannot automatically analyze and calculate MFU, the server can calculate the machine computing power consumed by the target model to complete a forward reverse calculation = the sum of the accelerator computing power values ​​of various accelerators used × one iteration training time by extracting one iteration training time from the running data fed back by the accelerator performance monitoring tool and the accelerator computing power values ​​of various accelerators used. Furthermore, the server can calculate the matrix computing power consumed by the target model to complete a forward reverse calculation based on the network structure of the target model, and then use the ratio of matrix computing power to machine computing power as the calculated MFU.

[0116] Step 604: compare the first computing efficiency and the second computing efficiency to determine an accelerator performance prediction result.

[0117] The first computational efficiency characterization is the cluster performance when a single accelerator is running (non-mixed training). The second computational efficiency characterization is the cluster performance when multiple types of accelerators are running (mixed training).

[0118] Optionally, the server may determine the accelerator performance prediction result by comparing the first computing efficiency and the second computing efficiency.

[0119] For example, the server can use "the second computing efficiency ÷ the first computing efficiency" as the accelerator performance prediction result. The larger the ratio, the more effective the multiple types of accelerators are in improving cluster performance relative to a single accelerator. The server can also use "the second computing efficiency - the first computing efficiency" as the accelerator performance prediction result. The larger the difference, the more effective the multiple types of accelerators are in improving cluster performance relative to a single accelerator.

[0120] In this embodiment, the computing power utilization of the first type of accelerator in the heterogeneous cluster in a non-mixed training environment and a non-mixed training environment can be used to effectively evaluate the multi-type accelerator configuration in the heterogeneous cluster, and the performance improvement or degradation compared to the single type of accelerator configuration can be obtained, that is, another accelerator performance prediction result for the heterogeneous cluster is obtained, so that the accelerator performance prediction result and the utilization comparison result can be combined in the subsequent process to improve the accuracy of the heterogeneous cluster performance prediction result.

[0121] Based on the above embodiments, taking a heterogeneous cluster integrating multiple types of accelerators such as NVIDIA GPU, Intel CPU, AMD CPU, etc. as an example, if the server (group) is the executor of the above heterogeneous cluster performance prediction method, the performance prediction result of the heterogeneous cluster when training the target model can be obtained in the following way:

[0122] (1) The server can schedule all accelerators in the heterogeneous cluster to run to train the target model. During the training process, the server can schedule the accelerator performance monitoring tool to monitor the various accelerators in the heterogeneous cluster.

[0123] (2) On the one hand, the server can calculate the measured computing power utilization of the heterogeneous cluster based on the operating data of each type of accelerator in the heterogeneous cluster, and further predict the theoretical computing power utilization of the homogeneous cluster with the same computing power as the heterogeneous cluster, so as to obtain the utilization comparison result for the heterogeneous cluster by comparing the measured computing power utilization with the theoretical computing power utilization, so as to preliminarily judge whether the combined use of multiple types of accelerators can effectively improve the performance of the heterogeneous cluster as a whole. If the measured computing power utilization is less than the theoretical computing power utilization, and the gap between the two reaches the preset condition, the server can feedback the first prompt information indicating that the overall performance of the heterogeneous cluster does not meet expectations (the quantitative calculation results can be fed back); otherwise, feedback the second prompt information indicating that the overall performance of the heterogeneous cluster meets expectations (the quantitative calculation results can be fed back).

[0124] (3) On the other hand, if we want to evaluate whether the performance of Intel CPU, AMD CPU and other accelerators is effectively utilized by combining them with NVIDIA GPU, we can take NVIDIA GPU as the first type of accelerator and Intel CPU, AMD CPU and other accelerators as the second type of accelerators. From the monitored operation data, we can extract the first data processing volume of NVIDIA GPU per unit time, the actual computing power values ​​of Intel CPU, AMD CPU and other accelerators, and the second data processing volume of the entire heterogeneous cluster per unit time. Furthermore, if we regard Intel CPU, AMD CPU and other accelerators as a whole, the server can calculate an accelerator performance prediction result based on the data processing volume difference between the second data processing volume and the first data processing volume and the sum of the computing power values ​​of Intel CPU, AMD CPU and other accelerators as a whole by formula (4). This accelerator performance prediction result can represent whether the overall computing efficiency of the resulting cluster can be effectively improved by combining Intel CPU, AMD CPU and other accelerators with NVIDIA GPU when Intel CPU, AMD CPU and other accelerators are regarded as a whole.

[0125] For example, if the accelerator performance prediction result after considering Intel CPU, AMD CPU, etc. as a whole is close to 0 or less than a preset value, the server can feedback the third prompt information (quantitative calculation results can be fed back) indicating that the performance of accelerators such as Intel CPU, AMD CPU, etc. does not meet expectations, to instruct the user to optimize accelerators such as Intel CPU, AMD CPU, etc.; otherwise, feedback the fourth prompt information (quantitative calculation results can be fed back) indicating that the performance of accelerators such as Intel CPU, AMD CPU, etc. meets expectations.

[0126] Furthermore, if it is desired to further determine which type of accelerators such as Intel CPU and AMD CPU has the lowest performance, the server can use formula (5) to specifically calculate the accelerator performance prediction index corresponding to each type of accelerators such as Intel CPU and AMD CPU, and determine the accelerator corresponding to the minimum accelerator performance prediction index as the accelerator with the lowest performance among the accelerators such as Intel CPU and AMD CPU, and feedback the fifth prompt information (which can be a quantitative calculation result) representing the optimization of the accelerator with the lowest performance.

[0127] (4) In order to ensure the accuracy of the performance prediction results, the server can use other types of accelerators in the above-mentioned heterogeneous cluster, such as Intel CPU, AMD CPU, etc., as the first type of accelerator for prediction in turn, so as to ensure the accuracy of the performance prediction results of the heterogeneous cluster through multiple repeated predictions.

[0128] Based on this, users can integrate the prompt information fed back by the server and develop performance optimization plans for accelerators in heterogeneous clusters to improve the model training efficiency of heterogeneous clusters.

[0129] In some embodiments, as shown in FIG. 7 , a schematic diagram of a process for predicting the performance of a first type of accelerator and a second type of accelerator in a heterogeneous cluster is provided, which mainly includes the following steps:

[0130] Step 702, obtaining respective operating data of at least two types of accelerators in the heterogeneous cluster;

[0131] Step 704, based on the operating data of at least two types of accelerators, the data processing volume and data processing time of the heterogeneous cluster during at least one round of training are counted;

[0132] Step 706, determining the measured computing power utilization of the heterogeneous cluster according to the data processing amount and the data processing time;

[0133] Step 708, obtaining the total computing power value of the heterogeneous cluster, and using the total computing power value of the heterogeneous cluster as the total computing power value of the homogeneous cluster matching the heterogeneous cluster;

[0134] Step 710, based on the network structure of the target model, predicting the first computing power value consumed in each round when the homogeneous cluster iteratively trains the target model;

[0135] Step 712, determining a second computing power value used by the homogeneous cluster to process each data according to the first computing power value and the data processing amount;

[0136] Step 714, obtaining a theoretical computing power utilization rate of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the second computing power value;

[0137] Step 716, comparing the measured computing power utilization rate with the theoretical computing power utilization rate to obtain a utilization comparison result;

[0138] Step 718, extracting a first data processing amount per unit time of the first type of accelerator, an accelerator computing power value of the second type of accelerator, and a second data processing amount per unit time of the heterogeneous cluster from the respective operating data of at least two types of accelerators;

[0139] Step 720, determining a data processing amount difference between the second data processing amount and the first data processing amount;

[0140] Step 722, determining an accelerator performance prediction result according to the data processing amount difference and the accelerator computing power value of the second type of accelerator;

[0141] Step 724, extracting a first computing efficiency and a second computing efficiency of the first type of accelerator from the operation data of the first type of accelerator;

[0142] Step 726, comparing the first computing efficiency and the second computing efficiency to determine an accelerator performance prediction result;

[0143] Step 728: Based on the accelerator performance prediction result and the utilization comparison result, a performance prediction result of the heterogeneous cluster is obtained.

[0144] In this embodiment, the utilization comparison results and the performance prediction results of the two accelerators can be combined to obtain the performance prediction results of the heterogeneous cluster in a targeted manner, so as to quantify the configuration differences between multiple types of accelerators and a single accelerator, and the specific impact on the cluster performance, so that users can refer to the performance prediction results and accurately identify accelerators with poor performance in the heterogeneous cluster, so as to formulate accelerator optimization plans suitable for heterogeneous clusters, improve the computing efficiency and accelerator resource utilization of heterogeneous clusters, and further improve the efficiency of training models using heterogeneous clusters.

[0145] The performance prediction methods involved in the above embodiments can be applied to the following scenarios: If there is a certain model training demander, there is a large model that needs to be trained, such as the next generation language processing model, complex image recognition model, etc., due to the complexity of the model and the huge amount of training data, the model training demander usually chooses a heterogeneous cluster integrated with GPUs of different performances to perform model training tasks. Furthermore, before using the heterogeneous cluster training model, the model training demander can first use the above-mentioned heterogeneous cluster performance prediction method to perform performance prediction on the heterogeneous cluster planned to be used, so as to accurately identify the accelerator with poor performance in the heterogeneous cluster according to the performance prediction results, so as to formulate an accelerator optimization plan suitable for the heterogeneous cluster, improve the computing performance of the heterogeneous cluster, and then improve the efficiency of the subsequent use of the heterogeneous cluster training model, speed up the development progress, and shorten the model iteration test cycle. Among them, the fields involved by the model training demander include but are not limited to: high-performance computing, commercial data centers, artificial intelligence and machine learning, enterprise-level infrastructure, cloud services, intelligent manufacturing and automated manufacturing, etc.

[0146] In an embodiment of the present application, a heterogeneous cluster performance prediction system is also provided, including:

[0147] Heterogeneous clusters, used to train the target model;

[0148] An accelerator performance monitoring tool, used to monitor the operating data of at least two types of accelerators in a heterogeneous cluster during the training of a target model;

[0149] A heterogeneous cluster performance prediction device is used to obtain the operating data of at least two types of accelerators in a heterogeneous cluster from an accelerator performance monitoring tool; based on the operating data of at least two types of accelerators, the data processing volume and data processing time of the heterogeneous cluster in at least one round of training are counted, and the actual computing power utilization of the heterogeneous cluster is determined according to the data processing volume and the data processing time; a homogeneous cluster matching the heterogeneous cluster is determined, and the theoretical computing power utilization of the homogeneous cluster is predicted according to the total computing power value of the homogeneous cluster and the network structure of the target model; and the actual computing power utilization is compared with the theoretical computing power utilization to obtain a performance prediction result of the heterogeneous cluster.

[0150] It should be noted that the heterogeneous cluster performance prediction device can be a server (group) or a terminal device (group). A heterogeneous cluster can be deployed on the heterogeneous cluster performance prediction device, that is, multiple types of accelerators for building a heterogeneous cluster can be installed. In addition, an accelerator performance monitoring tool can also be deployed on the heterogeneous cluster performance prediction device, so that when the heterogeneous cluster performance prediction device starts the heterogeneous cluster and uses the computing power of the heterogeneous cluster to train the target model, the heterogeneous cluster performance prediction device can start the accelerator performance monitoring tool to monitor the operating data of each accelerator in the heterogeneous cluster, and then execute the above-mentioned heterogeneous cluster performance prediction method based on the operating data obtained from the accelerator performance monitoring tool to perform performance prediction on various types of accelerators in the heterogeneous cluster and obtain performance prediction results for the heterogeneous cluster.

[0151] The above-mentioned heterogeneous cluster performance prediction system can accurately compare the performance differences between heterogeneous clusters and homogeneous clusters with the same computing power, and output the performance prediction results of heterogeneous clusters, so that users can refer to the performance prediction results to evaluate the performance of heterogeneous clusters, adopt optimization solutions suitable for heterogeneous clusters, and improve the model training efficiency of heterogeneous clusters.

[0152] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps. Based on the same inventive concept, the embodiment of the present application also provides a heterogeneous cluster performance prediction device for implementing the heterogeneous cluster performance prediction method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific definition in one or more heterogeneous cluster performance prediction device embodiments provided below can refer to the above definition of the heterogeneous cluster performance prediction method, which will not be repeated here.

[0153] In an exemplary embodiment, as shown in FIG8 , a heterogeneous cluster performance prediction device is provided, including: an operation data acquisition module 802, a measured computing power utilization determination module 804, a theoretical computing power utilization determination module 806, and a performance prediction result acquisition module 808, wherein:

[0154] An operation data acquisition module, used to acquire operation data of at least two types of accelerators in the heterogeneous cluster, where the operation data is generated when the target model is iteratively trained through the heterogeneous cluster;

[0155] A measured computing power utilization determination module is used to count the data processing volume and data processing time of the heterogeneous cluster during at least one round of training based on the respective operating data of at least two types of accelerators, and determine the measured computing power utilization of the heterogeneous cluster according to the data processing volume and data processing time;

[0156] Theoretical computing power utilization determination module is used to determine the homogeneous cluster that matches the heterogeneous cluster, and predict the theoretical computing power utilization of the homogeneous cluster based on the total computing power value of the homogeneous cluster and the network structure of the target model;

[0157] The performance prediction result acquisition module is used to compare the measured computing power utilization with the theoretical computing power utilization to obtain the performance prediction results of the heterogeneous cluster.

[0158] The above-mentioned heterogeneous cluster performance prediction device first obtains the operating data of at least two types of accelerators in the heterogeneous cluster, wherein the operating data is generated when the target model is iteratively trained through the heterogeneous cluster, so as to count the data processing volume and data processing time of the heterogeneous cluster during at least one round of training based on the operating data of at least two types of accelerators, so as to determine the measured computing power utilization of the heterogeneous cluster according to the data processing volume and data processing time, that is, the performance evaluation index of the heterogeneous cluster. Further, a homogeneous cluster matching the heterogeneous cluster is determined, so as to predict the theoretical computing power utilization of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model, that is, to determine the performance evaluation index of the homogeneous cluster. Furthermore, by comparing the measured computing power utilization with the theoretical computing power utilization, and comparing the performance difference between the heterogeneous cluster and the homogeneous cluster with the same computing power, the performance prediction result of the heterogeneous cluster is obtained, so that the user can refer to the performance prediction result, evaluate the performance of the heterogeneous cluster, and then adopt an optimization scheme suitable for the heterogeneous cluster to improve the model training efficiency of the heterogeneous cluster.

[0159] In one of the embodiments, the theoretical computing power utilization determination module is further used to: obtain the total computing power value of the heterogeneous cluster; and use the total computing power value of the heterogeneous cluster as the total computing power value of the homogeneous cluster that matches the heterogeneous cluster.

[0160] In one of the embodiments, the theoretical computing power utilization determination module is also used to: predict the first computing power value consumed in each round of iterative training of the target model by the homogeneous cluster based on the network structure of the target model; determine the second computing power value used by the homogeneous cluster to process each data based on the first computing power value and the data processing amount; and obtain the theoretical computing power utilization of the homogeneous cluster based on the total computing power value and the second computing power value of the homogeneous cluster.

[0161] In one of the embodiments, the performance prediction result obtaining module also includes: a utilization comparison result obtaining unit, which is used to compare the measured computing power utilization and the theoretical computing power utilization to obtain the utilization comparison result; an accelerator performance prediction result obtaining unit, which is used to perform accelerator performance prediction based on the operating data of at least two types of accelerators and determine the accelerator performance prediction result; a prediction result aggregation unit, which is used to obtain the performance prediction result of the heterogeneous cluster based on the accelerator performance prediction result and the utilization comparison result.

[0162] In one of the embodiments, at least two types of accelerators include a first type of accelerator and a second type of accelerator, and the accelerator performance prediction result obtaining unit is specifically used to: extract a first data processing volume of the first type of accelerator per unit time, an accelerator computing power value of the second type of accelerator, and a second data processing volume of the heterogeneous cluster per unit time from the respective operating data of the at least two types of accelerators; determine a data processing volume difference between the second data processing volume and the first data processing volume; and determine an accelerator performance prediction result based on the data processing volume difference and the accelerator computing power value of the second type of accelerator.

[0163] In one of the embodiments, at least two types of accelerators include a first type of accelerator and a second type of accelerator; the accelerator performance prediction result obtaining unit can also be used to: extract a first computing efficiency and a second computing efficiency of the first type of accelerator from the operating data of the first type of accelerator; the first computing efficiency is the computing efficiency when the first type of accelerator is used alone; the second computing efficiency is the computing efficiency when the first type of accelerator and the second type of accelerator are used in combination; compare the first computing efficiency and the second computing efficiency to determine the accelerator performance prediction result.

[0164] Each module in the above heterogeneous cluster performance prediction device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0165] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG9. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store heterogeneous cluster performance prediction data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a heterogeneous cluster performance prediction method is implemented. Those skilled in the art will understand that the structure shown in FIG. 9 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0166] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above-mentioned heterogeneous cluster performance prediction method is implemented when the processor executes the computer program. A computer-readable storage medium may also be provided, on which a computer program is stored, and the above-mentioned heterogeneous cluster performance prediction method is implemented when the computer program is executed by the processor. A computer program product is also provided, including a computer program, and the above-mentioned heterogeneous cluster performance prediction method is implemented when the computer program is executed by the processor.

[0167] It should be noted that the user information (including but not limited to user device information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0168] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0169] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application. The above embodiments only express several implementation methods of the present application, and the description is relatively specific and detailed, but it cannot be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several deformations and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the attached claims.

Claims

1. A heterogeneous cluster performance prediction method, characterized in that: The method comprises: Acquire operating data of at least two types of accelerators in a heterogeneous cluster, where the operating data is generated when a target model is iteratively trained through the heterogeneous cluster; Based on the respective operating data of the at least two types of accelerators, counting the data processing volume and data processing duration of the heterogeneous cluster during at least one round of training, and determining the measured computing power utilization rate of the heterogeneous cluster according to the data processing volume and the data processing duration; Determine a homogeneous cluster that matches the heterogeneous cluster, and predict the theoretical computing power utilization rate of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model; The measured computing power utilization rate is compared with the theoretical computing power utilization rate to obtain a performance prediction result of the heterogeneous cluster.

2. The method according to claim 1, characterized in that The determining of a homogeneous cluster matching the heterogeneous cluster includes: Obtaining the total computing power value of the heterogeneous cluster; The total computing power value of the heterogeneous cluster is used as the total computing power value of the homogeneous cluster matching the heterogeneous cluster.

3. The method according to claim 1, characterized in that The predicting the theoretical computing power utilization rate of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model includes: Based on the network structure of the target model, predicting a first computing power value consumed in each round when the homogeneous cluster iteratively trains the target model; Determining, according to the first computing power value and the data processing amount, a second computing power value used by the homogeneous cluster to process each data; The theoretical computing power utilization rate of the homogeneous cluster is obtained according to the total computing power value of the homogeneous cluster and the second computing power value.

4. The method according to claim 1, characterized in that The comparing the measured computing power utilization and the theoretical computing power utilization to obtain the performance prediction result of the heterogeneous cluster includes: Comparing the measured computing power utilization rate with the theoretical computing power utilization rate to obtain a utilization comparison result, and performing accelerator performance prediction according to the respective operating data of the at least two types of accelerators to determine an accelerator performance prediction result; Based on the accelerator performance prediction result and the utilization comparison result, a performance prediction result of the heterogeneous cluster is obtained.

5. The method according to claim 4, characterized in that The at least two types of accelerators include a first type of accelerator and a second type of accelerator; and performing accelerator performance prediction according to respective operating data of the at least two types of accelerators to determine the accelerator performance prediction result includes: Extracting, from the respective operating data of the at least two types of accelerators, a first data processing amount of the first type of accelerator per unit time, an accelerator computing power value of the second type of accelerator, and a second data processing amount of the heterogeneous cluster per unit time; determining a data processing volume difference between the second data processing volume and the first data processing volume; An accelerator performance prediction result is determined according to the data processing amount difference and the accelerator computing power value of the second type of accelerator.

6. The method according to claim 4, characterized in that The at least two types of accelerators include a first type of accelerator and a second type of accelerator; and performing accelerator performance prediction according to respective operating data of the at least two types of accelerators to determine the accelerator performance prediction result includes: Extracting a first computing efficiency and a second computing efficiency of the first type of accelerator from the operation data of the first type of accelerator; the first computing efficiency is the computing efficiency when the first type of accelerator is used alone; the second computing efficiency is the computing efficiency when the first type of accelerator and the second type of accelerator are used in combination; The first computing efficiency and the second computing efficiency are compared to determine an accelerator performance prediction result.

7. A heterogeneous cluster performance prediction system, characterized in that: The system comprises: Heterogeneous clusters, used to train the target model; An accelerator performance monitoring tool, used to monitor the operating data of at least two types of accelerators in the heterogeneous cluster during the training of the target model; A heterogeneous cluster performance prediction device is used to obtain the operating data of at least two types of accelerators in the heterogeneous cluster from the accelerator performance monitoring tool; based on the operating data of the at least two types of accelerators, count the data processing volume and data processing time of the heterogeneous cluster during at least one round of training, and determine the measured computing power utilization of the heterogeneous cluster according to the data processing volume and the data processing time; determine a homogeneous cluster matching the heterogeneous cluster, and predict the theoretical computing power utilization of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model; compare the measured computing power utilization with the theoretical computing power utilization to obtain a performance prediction result of the heterogeneous cluster.

8. A heterogeneous cluster performance prediction device, characterized in that: The device comprises: An operation data acquisition module, used to acquire operation data of at least two types of accelerators in a heterogeneous cluster, wherein the operation data is generated when a target model is iteratively trained through the heterogeneous cluster; A measured computing power utilization determination module, used to count the data processing volume and data processing time of the heterogeneous cluster during at least one round of training based on the respective operating data of the at least two types of accelerators, and determine the measured computing power utilization of the heterogeneous cluster according to the data processing volume and the data processing time; A theoretical computing power utilization determination module is used to determine a homogeneous cluster that matches the heterogeneous cluster, and predict the theoretical computing power utilization of the homogeneous cluster according to the total computing power value of the homogeneous cluster and the network structure of the target model; The performance prediction result obtaining module is used to compare the measured computing power utilization rate with the theoretical computing power utilization rate to obtain the performance prediction result of the heterogeneous cluster.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images