Modeling method and apparatus for nodes in multivariate heterogeneous computing system, device, and medium

Abstract

The present application relates to the technical field of computers, and discloses a modeling method and apparatus for nodes in a multivariate heterogeneous computing system, a device, and a medium. The method comprises: using test data to carry out performance test on heterogeneous computing nodes in a multivariate heterogeneous computing system to obtain test result data, and extracting training data from the test result data, the training data comprising a training task feature, an operator feature, and the computing consumed time of computing operators; and constructing an initial model corresponding to each heterogeneous computing node, and using the training data to train the initial model to obtain a corresponding heterogeneous computing node model, so that according to feature parameters corresponding to a training task to be predicted, the heterogeneous computing node model simulates a training time line of an operator level when the heterogeneous computing node operates said training task. The present application can reduce costs and accurately predict a working time line of an operator level.

Description

Modeling method, device, equipment and medium for nodes in multi-heterogeneous computing system

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on June 16, 2023, with application number 202310714547.7, and entitled “Modeling method, device, equipment and medium for nodes in a multi-heterogeneous computing system”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for modeling nodes in a multi-heterogeneous computing system. Background Art

[0004] Currently, the advantages of large neural network models are gradually becoming apparent, and academics and industry are investing heavily in research on neural network models with larger parameter counts. This, in turn, rapidly increases the difficulty of training large neural network models. To address the challenges of training large neural networks, both academics and industry are generally using distributed training methods. This involves splitting training data or large network models and deploying the split data and sub-training tasks across multiple computing chips, making it possible to train large models. In a multi-layered heterogeneous computing system, heterogeneous computing resources with varying computing performance are connected to the same distributed computing system and used to collaboratively train large neural network models. However, since large neural network training is time-consuming and energy-intensive, how to properly configure training tasks to enable efficient collaborative computation across multiple heterogeneous computing resources has become a key issue that requires research before executing large neural network training tasks. To address this issue, related technologies use simulation modeling to evaluate and test the configuration of training tasks in multi-layered heterogeneous computing systems before actual deployment. Modeling a multi-layered heterogeneous computing system primarily involves two steps: modeling the heterogeneous computing nodes within it and modeling the interconnections between these nodes. Among them, modeling heterogeneous computing nodes is more difficult. This is because the hardware design, optimization level, and software and hardware ecosystem of computing chips from different manufacturers are different, resulting in different computing capabilities and computing processes of computing chips from different manufacturers. It is difficult to represent them with consistent modeling, and it is also difficult to simulate their actual workflow.

[0005] Summary of the Invention

[0006] In view of this, the purpose of this application is to provide a method, apparatus, device, and medium for modeling nodes in a multi-heterogeneous computing system, which can accurately and cost-effectively predict the working timeline at the operator level. The specific solution is as follows:

[0007] In a first aspect, the present application discloses a method for modeling nodes in a multi-heterogeneous computing system, comprising:

[0008] Using test data to perform performance testing on heterogeneous computing nodes within a multi-heterogeneous computing system to obtain test result data, and extracting training data from the test result data; the training data includes training task characteristics, operator characteristics, and the computational time of the operator;

[0009] Construct an initial model corresponding to the heterogeneous computing node, and use the training data to train the initial model to obtain the corresponding heterogeneous computing node model, so that the heterogeneous computing node model can simulate the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted.

[0010] In some embodiments, constructing an initial model corresponding to a heterogeneous computing node includes:

[0011] The initial model is constructed in the order of embedding layer, multi-layer artificial neural network and output layer.

[0012] In some embodiments, training the initial model using training data to obtain a corresponding heterogeneous computing node model includes:

[0013] The initial model is trained with the training task features and operator features in the training data as training inputs and the computational time of the computational operators in the training data as labels to obtain a heterogeneous computing node model that takes the training task features and operator features as inputs and the computational time of the computational operators as outputs.

[0014] In some embodiments, before the heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted, the model further includes:

[0015] The feature acquisition module obtains the feature parameters corresponding to the training task to be predicted; the feature parameters include training task characteristics, operator calculation order and operator characteristics; the feature acquisition module and the heterogeneous computing node have the same artificial intelligence computing framework.

[0016] In some embodiments, obtaining feature parameters corresponding to the training task to be predicted by the feature acquisition module includes:

[0017] According to the task information of the training task to be predicted and the artificial intelligence computing framework of the feature acquisition module, the operator calculation order and operator characteristics are determined.

[0018] In some embodiments, obtaining feature parameters corresponding to the training task to be predicted by the feature acquisition module includes:

[0019] Extracting training task features based on task information of the training task to be predicted;

[0020] Alternatively, the training task feature information input by the user is obtained to obtain the training task features.

[0021] In some embodiments, the training task characteristics include any one or more of the number of single training samples, input data size, input data storage type, and model data accuracy.

[0022] In some embodiments, the operator features include any one or more of the operator name, operator internal features, operator input size features, and operator output size features.

[0023] In some embodiments, before the heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted, the model further includes:

[0024] Obtain the training task to be predicted; the training task to be predicted is a training task for a complete neural network or a training task for a partial neural network.

[0025] In some embodiments, using test data to perform performance testing on heterogeneous computing nodes in a multi-heterogeneous computing system includes:

[0026] Use the data generation module to generate multiple artificial intelligence training tasks to obtain a test task set;

[0027] Use test task sets to perform performance testing on heterogeneous computing nodes.

[0028] In some embodiments, using test data to perform performance testing on heterogeneous computing nodes in a multi-heterogeneous computing system to obtain test result data, and extracting training data from the test result data, includes:

[0029] Using the test data, the performance of each type of heterogeneous computing node is tested to obtain test result data corresponding to different types of heterogeneous computing nodes, and training data corresponding to different types of heterogeneous computing nodes are extracted from the test result data;

[0030] Accordingly, an initial model corresponding to the heterogeneous computing node is constructed, and the initial model is trained using the training data to obtain the corresponding heterogeneous computing node model, including:

[0031] Initial models corresponding to different types of heterogeneous computing nodes are constructed, and the corresponding initial models are trained using training data to obtain heterogeneous computing node models corresponding to different types of heterogeneous computing nodes.

[0032] In some embodiments, the test result data includes training task characteristics, as well as computation time and operator characteristics of each computation operator.

[0033] In some embodiments, after the initial model is trained using the training data to obtain a corresponding heterogeneous computing node model, the method further includes:

[0034] Obtain the node parameters of the target heterogeneous computing node to be simulated, and determine the corresponding target heterogeneous computing node model from the model library according to the node parameters; the model library contains heterogeneous computing node models corresponding to different types of heterogeneous computing nodes.

[0035] In some embodiments, the node parameters include data processing unit type, artificial intelligence accelerator type, artificial intelligence computing framework, and artificial intelligence computing framework version.

[0036] In some embodiments, the heterogeneous computing node model simulates the operator-level training timeline of the heterogeneous computing node when running the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted, including:

[0037] Based on characteristic parameters, a heterogeneous computing node model is used to simulate the time consumption of each operator when the heterogeneous computing node runs the training task to be predicted. The characteristic parameters include training task characteristics, operator calculation order, and operator characteristics.

[0038] Based on the time consumption of each operator, predict the operator-level training timeline generated when running the training task to be predicted.

[0039] In some embodiments, a heterogeneous computing node model is used according to characteristic parameters to simulate the time consumption of each operator when the heterogeneous computing node runs the training task to be predicted, including:

[0040] Based on the characteristics of the training task and the operator, the time required for each operator to run the training task to be predicted on the heterogeneous computing node is predicted.

[0041] In some embodiments, based on the time consumption of each operator, predicting the operator-level training timeline generated when running the training task to be predicted includes:

[0042] Based on the time consumption of each operator and the order of operator calculation, the operator-level training timeline generated when running the training task to be predicted is predicted.

[0043] In a second aspect, the present application discloses a device for modeling nodes in a multi-heterogeneous computing system, comprising:

[0044] The performance testing module is used to use test data to perform performance testing on heterogeneous computing nodes in a multi-heterogeneous computing system to obtain test result data and extract training data from the test result data; the training data includes training task characteristics, operator characteristics, and the computation time of the operator;

[0045] The model construction module is used to construct an initial model corresponding to the heterogeneous computing node and use the training data to train the initial model to obtain the corresponding heterogeneous computing node model, so that the heterogeneous computing node model can simulate the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted.

[0046] In a third aspect, the present application discloses an electronic device, comprising:

[0047] Memory, used to store computer programs;

[0048] The processor is used to execute a computer program to implement the aforementioned method for modeling nodes in a multi-heterogeneous computing system.

[0049] In a fourth aspect, the present application discloses a computer non-volatile readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned method for modeling nodes within a multi-heterogeneous computing system.

[0050] In this application, test data is used to perform performance testing on heterogeneous computing nodes in a multi-heterogeneous computing system to obtain test result data, and training data is extracted from the test result data; the training data includes training task characteristics, operator characteristics and computational time of the operator; an initial model corresponding to the heterogeneous computing node is constructed, and the initial model is trained using the training data to obtain a corresponding heterogeneous computing node model, so that the heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted.

[0051] It can be seen that by performing performance tests on heterogeneous computing nodes in a multi-heterogeneous computing system, training data is extracted based on the test result data, and the initial model is trained using the training data to obtain the corresponding heterogeneous computing node model. Using this heterogeneous computing node model, the operator-level training task timeline when the node runs different training tasks can be simulated without actually running the heterogeneous computing node. That is, the specific time when each operator starts and ends during the training task is simulated, and the operator-level work timeline prediction is achieved accurately and at low cost, thereby assisting the overall modeling of the multi-heterogeneous computing system. At the same time, it can also assist the multi-heterogeneous computing system in task segmentation, cluster deployment, and other tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0053] FIG1 is a flow chart of a method for modeling nodes in a multi-heterogeneous computing system provided by the present application;

[0054] FIG2 is a schematic diagram of a specific multi-heterogeneous computing system provided by this application;

[0055] FIG3 is a schematic diagram of a specific heterogeneous computing node model structure provided by this application;

[0056] FIG4 is a schematic diagram of a specific modeling and application architecture of nodes in a multi-heterogeneous computing system provided by the present application;

[0057] FIG5 is a flowchart of a specific method for modeling nodes in a multi-heterogeneous computing system provided by the present application;

[0058] FIG6 is a schematic diagram of a specific operator calculation sequence provided in this application;

[0059] FIG7 is a flowchart of a specific method for modeling nodes in a multi-heterogeneous computing system provided by the present application;

[0060] FIG8 is a schematic diagram of the structure of a node modeling device in a multi-heterogeneous computing system provided by the present application;

[0061] FIG9 shows an electronic device structure provided by this application. DETAILED DESCRIPTION

[0062] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0063] In related technologies, modeling heterogeneous computing nodes is difficult. This is because different vendors' computing chips have different hardware designs, optimization levels, and software and hardware ecosystems. This leads to different computing capabilities and processes, making it difficult to represent them with consistent modeling and simulating their actual workflows. To overcome these technical issues, this application proposes a modeling and application method for nodes within a multi-layered heterogeneous computing system, which can simulate the operator-level training task timeline when heterogeneous computing nodes run different training tasks.

[0064] The present application discloses a method for modeling nodes in a multi-heterogeneous computing system. Referring to FIG1 , the method may include the following steps:

[0065] Step S11: Use test data to perform performance testing on heterogeneous computing nodes in the multi-heterogeneous computing system to obtain test result data, and extract training data from the test result data; the training data includes training task characteristics, operator characteristics and calculation time of the calculation operator.

[0066] In some embodiments, test data is first used to perform performance testing on heterogeneous computing nodes within a multi-heterogeneous computing system to obtain test result data output after the performance test, and then training data is extracted from the above test result data, where the training data may include operator characteristics, node training task characteristics and operator calculation time.

[0067] It is understandable that, for example, FIG2 shows a specific schematic diagram of a multi-heterogeneous computing system, including multiple heterogeneous computing nodes, such as heterogeneous computing node 1, heterogeneous computing node 2, heterogeneous computing node 3, heterogeneous computing node 4, and communication between different heterogeneous computing nodes, wherein the communication can be within the server or between servers. FIG2 also includes a heterogeneous computing node provided in this embodiment. Since the performance of heterogeneous computing nodes in processing training tasks is affected by many factors such as data I / O (Input / Output), hardware performance, and software optimization, a heterogeneous computing node includes a data processing unit, an artificial intelligence accelerator (AI accelerator), and a deployed artificial intelligence framework (AI framework). In actual deployment, a server can contain one heterogeneous computing node or multiple heterogeneous computing nodes, and a heterogeneous computing system can be composed of one or more servers. Among them, the data processing unit can be an XPU (Processing Unit, a general term for various processors, CPU, DPU, etc.), and the AI ​​accelerator can be a CPU (Central Processing Unit, central processing unit), FPGA (Field Programmable Gate Array, field programmable logic gate array), or GPU (Graphics Processing Unit, graphics processing unit).

[0068] In some embodiments, using test data to perform performance testing on heterogeneous computing nodes within a multi-heterogeneous computing system may include: using a data generation module to generate multiple artificial intelligence training tasks to obtain a test task set; and using the test task set to perform performance testing on the heterogeneous computing nodes. That is, the data generation module generates a test task set consisting of a batch of AI training tasks, and inputs these training tasks one by one into the heterogeneous computing nodes; the specific method for generating test tasks may use a pre-defined test task set, the goal of which is to test the computing performance of as many heterogeneous computing nodes as possible for different neural network layers and computing operators. The heterogeneous computing nodes execute the test tasks one by one, and the results to be collected are used as test result data.

[0069] In some embodiments, using test data to perform performance testing on heterogeneous computing nodes within a multi-layered heterogeneous computing system to obtain test result data, and extracting training data from the test result data, may include: using the test data to perform performance testing on each type of heterogeneous computing node, obtaining test result data corresponding to different types of heterogeneous computing nodes, and extracting training data corresponding to different types of heterogeneous computing nodes from the test result data; accordingly, constructing initial models corresponding to the heterogeneous computing nodes, and training the initial models using the training data to obtain corresponding heterogeneous computing node models, may include: constructing initial models corresponding to different types of heterogeneous computing nodes, and training the corresponding initial models using the training data to obtain heterogeneous computing node models corresponding to different types of heterogeneous computing nodes. It is understood that for multiple heterogeneous computing nodes within a multi-layered heterogeneous computing system, since the performance of the nodes is affected by the type of data processing unit, AI accelerator type, AI framework, and AI framework version within the nodes, in some embodiments, the heterogeneous computing nodes are classified based on the above factors; that is, the data processing unit type, AI accelerator type, AI framework, and AI framework version identify the same type of heterogeneous computing node; if any of the three factors are different, the heterogeneous computing nodes are not of the same type. Then, by using the test data to perform performance tests on each type of heterogeneous computing node, test result data corresponding to different types of heterogeneous computing nodes are obtained, so that different models can be trained using the test result data to construct heterogeneous computing node models corresponding to different types of heterogeneous computing nodes.

[0070] In some embodiments, the test result data includes training task characteristics, as well as the computational time and operator characteristics of each computing operator. That is, when testing heterogeneous computing nodes, the main features of the training task characteristics, as well as the computational time and operator characteristics of each computing operator are collected. This is because these features are the main features for building a heterogeneous computing node model, and model building through these features can enable the constructed model to have the same features as the actual heterogeneous computing node. The collection of test result data can be collected and detected using the profiling tool profiler that comes with the AI ​​framework, or it can be collected by writing a program by yourself, that is, the computational time of different computing operators of heterogeneous computing nodes is collected through the profiling tool, and the corresponding operator characteristics and training task characteristics are collected, and the collected features are stored as training data; when the test is completed, the test result data collection module uses all the training data to train the model of the heterogeneous computing node.

[0071] Step S12: Construct an initial model corresponding to the heterogeneous computing node, and use the training data to train the initial model to obtain the corresponding heterogeneous computing node model, so that the heterogeneous computing node model can simulate the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted.

[0072] In some embodiments, after obtaining the training data, the training data is used to construct a heterogeneous computing node model corresponding to the heterogeneous computing node. That is, by using the training data for training, a modeling model of the heterogeneous computing node can be obtained. The model can predict the time consumption of the heterogeneous computing node to run various operators. Then, the model of the heterogeneous computing node will be stored in the heterogeneous computing node model library for subsequent application.

[0073] In some embodiments, constructing an initial model corresponding to a heterogeneous computing node may include: constructing an initial model in the order of an embedding layer, a multi-layer artificial neural network, and an output layer; and obtaining a heterogeneous computing node model by training the initial model. The training of the above-mentioned heterogeneous computing node model depends on the test result data, and the model structure used for training can be as shown in FIG3. It should be noted that the specific modules and the number of modules used in the model are not limited here. Specifically, the above-mentioned training data is used as the training input feature to obtain a high-dimensional vector of the feature through the embedding layer (embedding layer), and then pass through a multi-layer artificial neural network. The above-mentioned multi-layer artificial neural network can be a multi-layer MLP multi-layer perceptron (MLP, MultilayerPerceptron) or a multi-layer transformer encoder (transformer encoder), and finally pass through the output layer (such as the MLP output layer) to obtain the operator time.

[0074] In some embodiments, training the initial model using training data to obtain a corresponding heterogeneous computing node model may include: using the training task features and operator features in the training data as training inputs, using the computational time of the computational operators in the training data as labels, and training the initial model to obtain a heterogeneous computing node model that uses the training task features and operator features as inputs and uses the computational time of the computational operators as outputs. That is, when training the initial model, the training task features and operator features in the training data are used as training inputs, using the computational time of the computational operators in the training data as labels, and through multiple iterative training, obtaining a heterogeneous computing node model that uses the training task features and operator features as inputs and uses the computational time of the computational operators as outputs.

[0075] In some embodiments, characteristic parameters corresponding to the training task to be predicted are obtained, where the characteristic parameters include training task characteristics, operator calculation order, and operator characteristics. It is understood that the training task to be predicted is the task that is actually intended to be trained on the actual heterogeneous computing node.

[0076] In some embodiments, before simulating the operator-level training timeline of the heterogeneous computing node running the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted, the heterogeneous computing node model may further include: obtaining the training task to be predicted; the training task to be predicted may be a training task for a complete neural network or a training task for a partial neural network. That is, in some embodiments, the training task to be predicted may be a training task for a complete neural network or a training task for a partial neural network, such as a virtual training task consisting of several convolutional layers in a neural network.

[0077] In some embodiments, after obtaining the characteristic parameters corresponding to the training task to be predicted, the characteristic parameters are input into the heterogeneous computing node model, and the heterogeneous computing node is the heterogeneous computing node that you want to simulate, so as to simulate the operator-level training timeline of the heterogeneous computing node when running the training task to be predicted, that is, the timeline with the operator as the indicator, forming the operator calculation order according to time, and the time consuming of each operator calculation. It is understandable that the deep learning algorithm is composed of individual computing units, and these computing units are called operators (Operator, referred to as OP). Thus, by testing to obtain the modeling of heterogeneous computing nodes, the modeling can be used when performing strategy optimization of large-scale training tasks without the need for actual deployment, which greatly reduces costs.

[0078] In some embodiments, after training the initial model using training data to obtain a corresponding heterogeneous computing node model, the following steps may also be performed: obtaining node parameters of the target heterogeneous computing node to be simulated, and determining the corresponding target heterogeneous computing node model from a model library based on the node parameters; the model library contains heterogeneous computing node models corresponding to different types of heterogeneous computing nodes. Specifically, a determination is first made as to whether the desired target heterogeneous computing node model exists among all constructed heterogeneous computing node models. The target heterogeneous computing node is the heterogeneous computing node to be simulated. If so, the target heterogeneous computing node model can be directly obtained; if not, the target heterogeneous computing node model must be constructed according to the aforementioned model construction sequence.

[0079] In some embodiments, node parameters include data processing unit type, AI accelerator type, AI computing framework, and AI computing framework version. The AI ​​computing framework can be, for example, TensorFlow, PyTorch, Caffe, or other AI computing frameworks. In other words, heterogeneous computing node models are selected based on the classification criteria for heterogeneous computing nodes.

[0080] As can be seen from the above, in some embodiments, test data is used to perform performance testing on heterogeneous computing nodes in a multi-heterogeneous computing system to obtain test result data, and training data is extracted from the test result data; the training data includes training task characteristics, operator characteristics and computational time of the operator; an initial model corresponding to the heterogeneous computing node is constructed, and the initial model is trained using the training data to obtain a corresponding heterogeneous computing node model, so that the heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted. It can be seen that by performing performance tests on heterogeneous computing nodes in a multi-heterogeneous computing system, training data is extracted based on the test result data, and the initial model is trained using the training data to obtain the corresponding heterogeneous computing node model. Using this heterogeneous computing node model, the operator-level training task timeline when the node runs different training tasks can be simulated without actually running the heterogeneous computing node. That is, the specific time when each operator starts and ends during the training task is simulated, and the operator-level work timeline prediction is achieved accurately and at low cost, thereby assisting the overall modeling of the multi-heterogeneous computing system. At the same time, it can also assist the multi-heterogeneous computing system in task segmentation, cluster deployment, and other tasks.

[0081] The present application discloses a specific node modeling and application architecture within a multi-component heterogeneous computing system. As shown in Figure 4, this architecture includes five modules: the first three modules pertain to the model building process for heterogeneous computing nodes, and the last two modules pertain to the model application process for heterogeneous computing nodes. The names and functions of each module are as follows: 1. A test data generation module, which generates training data and inputs it into heterogeneous computing nodes to test their performance. 2. A test result data collection module, which collects test result data for heterogeneous computing nodes. The collected data includes operator characteristics, node training task characteristics, and operator computation time. 3. A training modeling module, which uses the test result data to model heterogeneous computing nodes, obtains models for the heterogeneous computing nodes, and stores the trained heterogeneous computing node models in a heterogeneous computing node model library. 4. An operator order and feature acquisition module, which obtains node training task characteristics and the operator computation order for the heterogeneous computing nodes based on the actual training task to be predicted, as well as the operator computation time for each operator. 5. The time-consuming prediction module is used to extract the model corresponding to the required heterogeneous computing node, and simulate the operator-level neural network training timeline of the heterogeneous computing node running various training tasks based on the operator calculation order, operator characteristics, and node training task characteristics. That is, after the target heterogeneous computing node model is transferred to the time-consuming prediction module, the time-consuming prediction module is used to perform time consumption prediction to obtain the final operator-level timeline simulation result, and the result is output to various applications with demand.

[0082] The embodiment of the present application discloses a specific method for modeling nodes in a multi-heterogeneous computing system. Referring to FIG5 , the method may include the following steps:

[0083] Step S21: Use test data to perform performance testing on heterogeneous computing nodes in the multi-heterogeneous computing system to obtain test result data, and extract training data from the test result data; the training data includes training task characteristics, operator characteristics and calculation time of the calculation operator.

[0084] Step S22: construct an initial model corresponding to the heterogeneous computing node, and train the initial model using the training data to obtain the corresponding heterogeneous computing node model.

[0085] Step S23: Acquire feature parameters corresponding to the training task to be predicted through the feature acquisition module; the feature acquisition module includes the same artificial intelligence computing framework as the heterogeneous computing node; the feature parameters include training task characteristics, operator calculation order and operator characteristics.

[0086] In some embodiments, feature parameter acquisition can be specifically performed through a pre-built feature acquisition module to obtain corresponding feature parameters based on the training task to be predicted. The feature acquisition module includes the same artificial intelligence computing framework as the heterogeneous computing node. In other words, in addition to the processor, the feature acquisition module also needs to include the same artificial intelligence computing framework as the heterogeneous computing node. This ensures that the extracted feature parameters are the characteristics of the training task to be predicted under the heterogeneous computing node. The feature parameters include training task characteristics, operator calculation order, and operator characteristics.

[0087] That is, the feature acquisition module obtains the operator calculation order, operator features, and node training task features based on the actual training task, and inputs them into the time consumption prediction module. Specifically, the feature acquisition module is also a computing node that can run independently. It can be a server without an AI accelerator, or a server configured with any AI accelerator. Its function is only to obtain the operator calculation order, operator features, and node training task features, but it does not obtain the operator calculation time. The acquisition of operator order and features depends on deploying the same AI computing framework as the heterogeneous computing node, so as to ensure that the operator calculation order and features are consistent with the original heterogeneous computing node.

[0088] Taking the operator calculation sequence for a prediction training task under the artificial intelligence computing framework PyTorch as an example, the various operators in the backend operator library (aten) of the artificial intelligence computing framework PyTorch are specifically used. The arrow direction represents the time sequence direction, and the operator calculation sequence is obtained as shown in Figure 6. These are the operator calculation sequence for the forward transmission operator (forward 2D convolution layer operator, forward batch normalization layer operator, forward linear rectifier function (ReLU) activation operator, forward summation operator, forward 2D convolution layer operator), and the operator calculation sequence for the reverse transmission operator (reverse 2D convolution operator, reverse summation operator, reverse linear rectifier function (ReLU) activation operator, reverse batch normalization layer operator, and reverse 2D convolution layer operator). Different artificial intelligence computing frameworks have different operator libraries, and the operator calculation sequence may also be different after the computing framework is optimized.

[0089] In some embodiments, obtaining the feature parameters corresponding to the training task to be predicted through the feature acquisition module may include: determining the operator calculation order and operator features based on the task information of the training task to be predicted and the artificial intelligence computing framework of the feature acquisition module. In some embodiments, the operator features include any one or more of the operator name, operator internal features, operator input size features, and operator output size features. For example, a fully connected operator only has input and output size features, but no operator internal features, and its input features and output features are both the sizes of one-dimensional vectors. For example, a two-dimensional convolution operator, in addition to the input and output size features, also includes operator internal features. Its input features are the length, width, and number of channels of the input data, and the output features are the length, width, and number of channels of the input data of the output data. Its operator internal features include the convolution kernel size, the number of convolution kernels, the convolution step size, and the padding method. Other operators that include operator internal features include pooling layer operators, batch normalization layer operators, etc., and they each have their own special internal features. That is, the task information of the training task to be predicted and the artificial intelligence computing framework of the feature acquisition module are combined to determine the operator calculation order and operator features. The operator features may include but are not limited to the operator name, operator internal features, operator input size features, and operator output size features.

[0090] In some embodiments, obtaining feature parameters corresponding to the training task to be predicted through the feature acquisition module may include: extracting training task features based on task information of the training task to be predicted; or obtaining training task feature information input by the user to obtain training task features. That is, node training task features can be obtained directly from the training task or based on manual input. In some embodiments, the training task features include any one or more of the number of single training samples (batch_size), input data size, input data storage type, and model data accuracy.

[0091] Step S24: The heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted.

[0092] Among them, the specific processes of the above-mentioned steps S21, S22, and S24 can refer to the corresponding contents disclosed in the above-mentioned embodiments, and will not be repeated here.

[0093] As can be seen from the above, in some embodiments, test data is used to perform performance testing on heterogeneous computing nodes in a multi-heterogeneous computing system to obtain test result data, and training data is extracted from the test result data; the training data includes training task characteristics, operator characteristics and computational time of the operator; an initial model corresponding to the heterogeneous computing node is constructed, and the initial model is trained using the training data to obtain a corresponding heterogeneous computing node model; feature parameters corresponding to the training task to be predicted are obtained through a feature acquisition module; the feature acquisition module includes the same artificial intelligence computing framework as the heterogeneous computing node; the feature parameters include training task characteristics, operator calculation order and operator characteristics; the heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the feature parameters corresponding to the training task to be predicted. It can be seen that by using a feature acquisition module with the same artificial intelligence computing framework as the heterogeneous computing node, the feature parameters corresponding to the training task to be predicted are obtained, and the feature parameters are sent as input to the heterogeneous computing node model to obtain the operator-level training timeline output by the heterogeneous computing node model, thereby realizing the operator-level work timeline prediction at a low cost and accurately, thereby assisting the overall modeling of the multi-heterogeneous computing system, and also assisting the multi-heterogeneous computing system in task segmentation, cluster deployment and other tasks.

[0094] The present application embodiment discloses a specific method for modeling nodes in a multi-heterogeneous computing system. Referring to FIG7 , the method may include the following steps:

[0095] Step S31: Use test data to perform performance testing on heterogeneous computing nodes in the multi-heterogeneous computing system to obtain test result data, and extract training data from the test result data; the training data includes training task characteristics, operator characteristics and calculation time of the calculation operator.

[0096] Step S32: constructing an initial model corresponding to the heterogeneous computing node, and training the initial model using training data to obtain a corresponding heterogeneous computing node model.

[0097] Step S33: The heterogeneous computing node model simulates the time consumption of each operator when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted; the characteristic parameters include training task characteristics, operator calculation order and operator characteristics.

[0098] In some embodiments, simulating the time consumption of each operator when the heterogeneous computing node runs the training task to be predicted using a heterogeneous computing node model based on characteristic parameters may include: predicting the time consumption of each operator when the heterogeneous computing node runs the training task to be predicted based on the training task characteristics and the operator characteristics. Specifically, the time consumption of each operator is predicted based on the training task characteristics and the operator characteristics.

[0099] Step S34: Based on the time consumption of each operator, predict the operator-level training timeline generated when running the training task to be predicted.

[0100] In some embodiments, after obtaining the time consumption of each operator, an operator-level training timeline is generated when the training task to be predicted is run based on the time consumption prediction of each operator. Specifically, based on the operator calculation order, operator characteristics, and node training task characteristics, and using the obtained heterogeneous computing node model, the time consumption prediction module predicts the time consumption of each operator, ultimately obtaining the operator-level training task timeline for any training task of the heterogeneous computing node as the final output.

[0101] In some embodiments, based on the time consumption of each operator, predicting the operator-level training timeline generated when running the training task to be predicted can include: predicting the operator-level training timeline generated when running the training task to be predicted based on the time consumption of each operator and the operator calculation order. That is, the role of the model is to predict the calculation time consumption of each operator in the target training task by the heterogeneous computing node, and finally construct the timeline of the entire training task. For example, the operator calculation order of a training task is OP1->OP2->OP3. Through the extracted operator features and node training task features, the model can predict the calculation time consumption of each operator, and finally simulate the operator-level neural network training timeline as OP1(t1)->OP2(t2)->OP3(t3).

[0102] For the specific processes of the above-mentioned steps S31 and S32, reference may be made to the corresponding contents disclosed in the above-mentioned embodiments, which will not be described in detail here.

[0103] As can be seen from the above, in some embodiments, test data is used to perform performance testing on heterogeneous computing nodes within a multi-component heterogeneous computing system to obtain test result data, and training data is extracted from the test result data; the training data includes training task characteristics, operator characteristics, and the computational time of the operators; an initial model corresponding to the heterogeneous computing node is constructed, and the initial model is trained using the training data to obtain the corresponding heterogeneous computing node model; a heterogeneous computing node model corresponding to the heterogeneous computing node is constructed based on the test data; the heterogeneous computing node model simulates the time consumption of each operator when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted; the characteristic parameters include training task characteristics, operator computation order, and operator characteristics; based on the time consumption of each operator, the operator-level training timeline generated when running the training task to be predicted is predicted. As can be seen, by calculating the time consumption of each operator and predicting the operator-level training timeline generated when running the training task to be predicted, low-cost and accurate prediction of operator-level working timelines is achieved.

[0104] Based on the above embodiment, the embodiment of the present application also discloses a timeline simulation process of a complete neural network. If the neural network is composed of two-dimensional convolution, two-dimensional batchnorm (batch normalization), pooling (pooling layer), MLP, transformerEncoder (transformer encoding) and other neural network layers, the simulated node training task characteristics are batch_size = 32, input data size = 32x32, input data storage type is jpge, and model data accuracy is float32. The three parts of the heterogeneous computing node include: the data processing unit is the Applem2 chip cpu part; the AI ​​accelerator is the Applem2 chip gpu part; the AI ​​framework is pytorch1.13.1. The specific process of building the heterogeneous computing node model is as follows:

[0105] 1. The test data generation module generates training tasks including various operators based on NAS dataset.

[0106] 2. The heterogeneous computing nodes execute the test tasks from the test data generation module and send the results required by the test result data collection module to the test result data collection module.

[0107] 3. The test result data collection module uses a performance analyzer (PytorchProfiler) to collect the computing time, operator characteristics, and node training task characteristics of heterogeneous computing nodes for different computing operators, and stores them as training data.

[0108] 4. The model training module uses the training data to train and obtain a model for the heterogeneous computing node. This model can predict the time required to run various operators on the heterogeneous computing node. The heterogeneous computing node model is then stored in the heterogeneous computing node model library for future use.

[0109] Based on the above modeling, we can simulate the operator-level training task timeline of various training tasks on the heterogeneous computing nodes without actually running the nodes. The specific process is as follows:

[0110] 5. The operator order and feature acquisition module obtains the operator calculation order, operator features, and node training task features according to the actual training task, and inputs them into the time-consuming prediction module. For this module, this embodiment uses an ordinary CPU server with the AI ​​framework: pytorch1.13.1 installed. In this server, the same training task as the model to be predicted is deployed, and then a small number of steps are quickly trained with a lower batch size. Finally, PytorchProfiler is used to obtain the operator calculation order and operator features. For the acquisition of node training task features, this embodiment uses manual input.

[0111] 6. According to the heterogeneous computing node you want to simulate, that is, the data processing unit: Applem2 chip CPU part, AI accelerator Applem2 chip GPU part, AI framework: pytorch1.13.1; the heterogeneous computing node model library will transfer the heterogeneous computing node model of the heterogeneous computing node to the time-consuming prediction module.

[0112] 7. Based on the operator calculation order, operator characteristics and node training task characteristics, the time consumption prediction module uses the obtained heterogeneous computing node model to predict the time consumption of each operator one by one, and finally obtains the operator-level training task timeline of the heterogeneous computing node for the above-mentioned neural network training task as the final output of this embodiment.

[0113] The above embodiment describes how to simulate the training of a complete neural network by a heterogeneous computing node through the modeling method of a node in a multi-heterogeneous computing system disclosed in this embodiment, and generate an operator-level training timeline. It can be seen that this embodiment is aimed at single-node testing and modeling in a multi-heterogeneous computing system, and uses AI technology to accurately model the performance of heterogeneous computing nodes in a multi-heterogeneous computing system, thereby making it possible to model the entire multi-heterogeneous computing system and solving the defects of previous related work. The working performance of heterogeneous computing nodes for any training task can be accurately modeled. This modeling can not only accurately predict the computational time of heterogeneous computing nodes running operators in different training tasks, but also predict the overall training time of heterogeneous computing nodes for various unseen neural networks.

[0114] In addition to simulating a complete neural network, it can also be used to simulate heterogeneous computing nodes to train several layers in a complete neural network (for example, several convolutional layers of a neural network) and generate operator-level training timelines. This embodiment can be used for subsequent simulations of model parallelism or pipeline parallelism in distributed training tasks. The difference between this embodiment and the previous embodiment lies in step 5 in the heterogeneous computing node model application process, that is, when simulating a partial neural network, step 5 of the previous embodiment is modified as follows:

[0115] 5. The operator order and feature acquisition module acquires the operator calculation order, operator features, and node training task features based on the actual training task and inputs them into the time-consuming prediction module. This example uses a standard CPU server with PyTorch 1.13.1 installed. A dummy training task consisting of several layers to be predicted is deployed on this server. Training is then performed quickly for a small number of steps using a low batch size. Finally, the operator calculation order and operator features are acquired using PyTorchProfiler.

[0116] Accordingly, the embodiment of the present application further discloses a device for modeling nodes in a multi-heterogeneous computing system. Referring to FIG8 , the device includes:

[0117] The performance testing module 11 is used to perform performance testing on heterogeneous computing nodes in the multi-heterogeneous computing system using test data to obtain test result data, and to extract training data from the test result data; the training data includes training task characteristics, operator characteristics, and computation time of the computation operators;

[0118] The model construction module 12 is used to construct an initial model corresponding to the heterogeneous computing node, and use the training data to train the initial model to obtain the corresponding heterogeneous computing node model, so that the heterogeneous computing node model can simulate the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted.

[0119] As can be seen from the above, in some embodiments, test data is used to perform performance testing on heterogeneous computing nodes in a multi-heterogeneous computing system to obtain test result data, and training data is extracted from the test result data; the training data includes training task characteristics, operator characteristics and computational time of the operator; an initial model corresponding to the heterogeneous computing node is constructed, and the initial model is trained using the training data to obtain a corresponding heterogeneous computing node model, so that the heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted. It can be seen that by performing performance tests on heterogeneous computing nodes in a multi-heterogeneous computing system, training data is extracted based on the test result data, and the initial model is trained using the training data to obtain the corresponding heterogeneous computing node model. Using this heterogeneous computing node model, the operator-level training task timeline when the node runs different training tasks can be simulated without actually running the heterogeneous computing node. That is, the specific time when each operator starts and ends during the training task is simulated, and the operator-level work timeline prediction is achieved accurately and at low cost, thereby assisting the overall modeling of the multi-heterogeneous computing system. At the same time, it can also assist the multi-heterogeneous computing system in task segmentation, cluster deployment, and other tasks.

[0120] In some specific embodiments, the model building module 12 may specifically include:

[0121] The initial model construction unit is used to construct the initial model in the order of embedding layer, multi-layer artificial neural network and output layer.

[0122] In some specific embodiments, the model building module 12 may specifically include:

[0123] The model training unit is used to train the initial model using the training task features and operator features in the training data as training inputs and the computational time of the computational operators in the training data as labels, so as to obtain a heterogeneous computing node model using the training task features and operator features as inputs and the computational time of the computational operators as outputs.

[0124] In some specific embodiments, the modeling device for nodes in a multi-heterogeneous computing system may specifically include:

[0125] The feature parameter acquisition unit is used to obtain the feature parameters corresponding to the training task to be predicted through the feature acquisition module; the feature parameters include training task characteristics, operator calculation order and operator characteristics; the feature acquisition module and the heterogeneous computing node have the same artificial intelligence computing framework.

[0126] In some specific embodiments, the characteristic parameter acquisition unit may specifically include:

[0127] The operator calculation sequence determination unit is used to determine the operator calculation sequence and operator features based on the task information of the training task to be predicted and the artificial intelligence calculation framework of the feature acquisition module.

[0128] In some specific embodiments, the characteristic parameter acquisition unit may specifically include:

[0129] A first training task feature acquisition unit is configured to extract training task features based on task information of the training task to be predicted;

[0130] The second training task feature acquisition unit acquires the training task feature information input by the user to obtain the training task feature.

[0131] In some specific embodiments, the training task characteristics may specifically include any one or more of the number of single training samples, input data size, input data storage type, and model data accuracy.

[0132] In some specific embodiments, the operator features may specifically include any one or more of the operator name, operator internal features, operator input size features, and operator output size features.

[0133] In some specific embodiments, the modeling device for nodes in a multi-heterogeneous computing system may specifically include:

[0134] The training task acquisition unit is used to acquire the training task to be predicted; the training task to be predicted is a training task for a complete neural network, or a training task for a part of the neural network.

[0135] In some specific embodiments, the performance testing module 11 may specifically include:

[0136] A training task generation unit, configured to generate a plurality of artificial intelligence training tasks using a data generation module to obtain a test task set;

[0137] The performance testing unit is used to perform performance testing on heterogeneous computing nodes using a test task set.

[0138] In some specific embodiments, the performance testing module 11 may specifically include:

[0139] A performance testing unit is used to perform performance testing on each type of heterogeneous computing node using test data, obtain test result data corresponding to different types of heterogeneous computing nodes, and extract training data corresponding to different types of heterogeneous computing nodes from the test result data;

[0140] Accordingly, the model building module 12 includes:

[0141] The model building unit is used to build initial models corresponding to different types of heterogeneous computing nodes, and use training data to train the corresponding initial models to obtain heterogeneous computing node models corresponding to different types of heterogeneous computing nodes.

[0142] In some specific embodiments, the test result data may specifically include training task characteristics, as well as the computation time and operator characteristics of each computing operator.

[0143] In some specific embodiments, the modeling device for nodes in a multi-heterogeneous computing system may specifically include:

[0144] The node parameter acquisition unit is used to obtain the node parameters of the target heterogeneous computing node to be simulated, and determine the corresponding target heterogeneous computing node model from the model library based on the node parameters; the model library contains heterogeneous computing node models corresponding to different types of heterogeneous computing nodes.

[0145] In some specific embodiments, the node parameters may specifically include the data processing unit type, the artificial intelligence accelerator type, the artificial intelligence computing framework, and the artificial intelligence computing framework version.

[0146] In some specific embodiments, the model building module 12 may specifically include:

[0147] An operator time-consuming calculation unit is used to simulate the time consumption of each operator when the target heterogeneous computing node runs the training task to be predicted based on characteristic parameters using the target heterogeneous computing node model; the characteristic parameters include training task characteristics, operator calculation order, and operator characteristics;

[0148] The training timeline prediction unit is used to predict the operator-level training timeline generated when running the training task to be predicted based on the time consumption of each operator.

[0149] In some specific embodiments, the operator time consumption calculation unit can be specifically used to predict the time consumption of each operator when the heterogeneous computing node runs the training task to be predicted based on the training task characteristics and the operator characteristics.

[0150] In some specific embodiments, the training timeline prediction unit may be specifically configured to predict the operator-level training timeline generated when running the training task to be predicted based on the time consumption of each operator and the operator calculation order.

[0151] Furthermore, an embodiment of the present application also discloses an electronic device, as shown in FIG9 . The content in the figure cannot be considered as any limitation on the scope of use of the present application.

[0152] Figure 9 is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the method for modeling nodes in a multi-heterogeneous computing system disclosed in any of the aforementioned embodiments.

[0153] In some embodiments, the computer program implements the following steps when it is executed: using test data to perform performance testing on heterogeneous computing nodes within a multi-heterogeneous computing system to obtain test result data, and extracting training data from the test result data; the training data includes training task characteristics, operator characteristics, and computational time of the operator; constructing an initial model corresponding to the heterogeneous computing node, and using the training data to train the initial model to obtain a corresponding heterogeneous computing node model, so that the heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted. It can be seen that by performing performance tests on heterogeneous computing nodes in a multi-heterogeneous computing system, training data is extracted based on the test result data, and the initial model is trained using the training data to obtain the corresponding heterogeneous computing node model. Using this heterogeneous computing node model, the operator-level training task timeline when the node runs different training tasks can be simulated without actually running the heterogeneous computing node. That is, the specific time when each operator starts and ends during the training task is simulated, and the operator-level work timeline prediction is achieved accurately and at low cost, thereby assisting the overall modeling of the multi-heterogeneous computing system. At the same time, it can also assist the multi-heterogeneous computing system in task segmentation, cluster deployment, and other tasks.

[0154] In some embodiments, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of this application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0155] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223 including test data, etc. The storage method can be temporary storage or permanent storage.

[0156] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to calculate and process the massive data 223 in the memory 22. The operating system 221 can be Windows Server, NetWare, Unix, Linux, etc. In addition to including computer programs capable of implementing the method for modeling nodes in a multi-heterogeneous computing system executed by the electronic device 20 as disclosed in any of the aforementioned embodiments, the computer programs 222 can further include computer programs capable of performing other specific tasks.

[0157] Furthermore, an embodiment of the present application also discloses a computer storage medium, which stores computer-executable instructions. When the computer-executable instructions are loaded and executed by a processor, the steps of the modeling method for nodes in a multi-heterogeneous computing system disclosed in any of the aforementioned embodiments are implemented.

[0158] In some embodiments, when the computer executable instructions are executed, the following steps are implemented: using test data to perform performance testing on heterogeneous computing nodes in a multi-heterogeneous computing system to obtain test result data, and extracting training data from the test result data; the training data includes training task characteristics, operator characteristics and computational time of the operator; constructing an initial model corresponding to the heterogeneous computing node, and using the training data to train the initial model to obtain a corresponding heterogeneous computing node model, so that the heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted based on the characteristic parameters corresponding to the training task to be predicted. It can be seen that by performing performance tests on heterogeneous computing nodes in a multi-heterogeneous computing system, training data is extracted based on the test result data, and the initial model is trained using the training data to obtain the corresponding heterogeneous computing node model. Using this heterogeneous computing node model, the operator-level training task timeline when the node runs different training tasks can be simulated without actually running the heterogeneous computing node. That is, the specific time when each operator starts and ends during the training task is simulated, and the operator-level work timeline prediction is achieved accurately and at low cost, thereby assisting the overall modeling of the multi-heterogeneous computing system. At the same time, it can also assist the multi-heterogeneous computing system in task segmentation, cluster deployment, and other tasks.

[0159] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0160] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0161] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0162] The above is a detailed introduction to the modeling method, device, equipment and medium for nodes in a multi-heterogeneous computing system provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. At the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for modeling nodes in a multi-heterogeneous computing system, characterized in that: include: Using the test data to perform performance testing on heterogeneous computing nodes in the multi-heterogeneous computing system to obtain test result data, and extracting training data from the test result data; The training data includes training task features, operator features, and computation time of the computation operators; An initial model corresponding to the heterogeneous computing node is constructed, and the initial model is trained using the training data to obtain a corresponding heterogeneous computing node model, so that the heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted according to the characteristic parameters corresponding to the training task to be predicted.

2. The method for modeling nodes in a multi-heterogeneous computing system according to claim 1, characterized in that: The constructing the initial model corresponding to the heterogeneous computing node includes: The initial model is constructed in the order of embedding layer, multi-layer artificial neural network and output layer.

3. The method for modeling nodes in a multi-heterogeneous computing system according to claim 2, characterized in that: The using the training data to train the initial model to obtain a corresponding heterogeneous computing node model includes: The initial model is trained with the training task features and operator features in the training data as training inputs and the computational time of the computational operators in the training data as labels to obtain a heterogeneous computing node model with the training task features and operator features as inputs and the computational time of the computational operators as outputs.

4. The method for modeling nodes in a multi-heterogeneous computing system according to claim 1, characterized in that: The heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted according to the characteristic parameters corresponding to the training task to be predicted, and further includes: The feature acquisition module is used to acquire feature parameters corresponding to the training task to be predicted; the feature parameters include training task features, operator calculation order and operator features; the feature acquisition module and the heterogeneous computing node have the same artificial intelligence computing framework.

5. The method for modeling nodes in a multi-heterogeneous computing system according to claim 4, characterized in that: The acquiring of the feature parameters corresponding to the training task to be predicted by the feature acquisition module includes: The operator calculation sequence and operator features are determined according to the task information of the training task to be predicted and the artificial intelligence calculation framework of the feature acquisition module.

6. The method for modeling nodes in a multi-heterogeneous computing system according to claim 4, characterized in that: The acquiring of the feature parameters corresponding to the training task to be predicted by the feature acquisition module includes: Extracting the training task features according to the task information of the training task to be predicted; Or, obtain the training task feature information input by the user to obtain the training task feature.

7. The method for modeling nodes in a multi-heterogeneous computing system according to claim 4, characterized in that: The training task characteristics include any one or more of the number of single training samples, input data size, input data storage type and model data accuracy.

8. The method for modeling nodes in a multi-heterogeneous computing system according to claim 4, characterized in that: The operator features include any one or more of the operator name, operator internal features, operator input size features, and operator output size features.

9. The method for modeling nodes in a multi-heterogeneous computing system according to claim 1, characterized in that: The heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted according to the characteristic parameters corresponding to the training task to be predicted, and further includes: Obtain a training task to be predicted; the training task to be predicted is a training task for a complete neural network, or a training task for a partial neural network.

10. The method for modeling nodes in a multi-heterogeneous computing system according to claim 1, characterized in that: The method of using the test data to perform performance testing on heterogeneous computing nodes in the multi-heterogeneous computing system includes: Use the data generation module to generate multiple artificial intelligence training tasks to obtain a test task set; The test task set is used to perform performance testing on heterogeneous computing nodes.

11. The method for modeling nodes in a multi-heterogeneous computing system according to claim 1, characterized in that: The method of using the test data to perform a performance test on the heterogeneous computing nodes in the multi-heterogeneous computing system to obtain test result data, and extracting training data from the test result data, includes: Using the test data, respectively, a performance test is performed on each type of heterogeneous computing node, to obtain test result data corresponding to different types of heterogeneous computing nodes, and training data corresponding to different types of heterogeneous computing nodes are extracted from the test result data; Accordingly, the constructing of the initial model corresponding to the heterogeneous computing node and training the initial model using the training data to obtain the corresponding heterogeneous computing node model includes: Initial models corresponding to different types of heterogeneous computing nodes are constructed, and the corresponding initial models are trained using the training data to obtain heterogeneous computing node models corresponding to different types of heterogeneous computing nodes.

12. The method for modeling nodes in a multi-heterogeneous computing system according to claim 1, characterized in that: The test result data includes training task characteristics, as well as the computation time and operator characteristics of each computing operator.

13. The method for modeling nodes in a multi-heterogeneous computing system according to claim 1, characterized in that: After the initial model is trained using the training data to obtain a corresponding heterogeneous computing node model, the method further includes: Node parameters of a target heterogeneous computing node to be simulated are obtained, and a corresponding target heterogeneous computing node model is determined from a model library according to the node parameters; the model library contains heterogeneous computing node models corresponding to different types of heterogeneous computing nodes.

14. The method for modeling nodes in a multi-heterogeneous computing system according to claim 13, characterized in that: The node parameters include data processing unit type, artificial intelligence accelerator type, artificial intelligence computing framework and artificial intelligence computing framework version.

15. The method for modeling nodes in a multi-heterogeneous computing system according to any one of claims 1 to 14, characterized in that: The heterogeneous computing node model simulates the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted according to the characteristic parameters corresponding to the training task to be predicted, including: Using a heterogeneous computing node model according to the characteristic parameters, simulating the time consumption of each operator when the heterogeneous computing node runs the training task to be predicted; the characteristic parameters include training task characteristics, operator calculation order and operator characteristics; Based on the time consumption of each operator, the operator-level training timeline generated when running the training task to be predicted is predicted.

16. The method for modeling nodes in a multi-heterogeneous computing system according to claim 15, characterized in that: The using a heterogeneous computing node model according to the characteristic parameters to simulate the time consumption of each operator when the heterogeneous computing node runs the training task to be predicted includes: The time consumption of each operator when the heterogeneous computing node runs the training task to be predicted is predicted according to the training task characteristics and the operator characteristics.

17. The method for modeling nodes in a multi-heterogeneous computing system according to claim 15, characterized in that: The predicting, based on the time consumption of each operator, the operator-level training timeline generated when running the training task to be predicted includes: According to the time consumption of each operator and the calculation order of the operators, the operator-level training timeline generated when the training task to be predicted is run is predicted.

18. A modeling device for nodes in a multi-heterogeneous computing system, characterized in that: include: A performance testing module is used to perform performance testing on heterogeneous computing nodes in a multi-heterogeneous computing system using test data to obtain test result data, and to extract training data from the test result data; The training data includes training task features, operator features, and computation time of the computation operators; A model building module is used to build an initial model corresponding to the heterogeneous computing node, and use the training data to train the initial model to obtain a corresponding heterogeneous computing node model, so that the heterogeneous computing node model can simulate the operator-level training timeline when the heterogeneous computing node runs the training task to be predicted according to the characteristic parameters corresponding to the training task to be predicted.

19. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the method for modeling nodes in a multi-heterogeneous computing system as described in any one of claims 1 to 17.

20. A computer non-volatile readable storage medium, characterized in that: Used to store computer programs; wherein when the computer programs are executed by a processor, a method for modeling nodes within a multi-heterogeneous computing system as described in any one of claims 1 to 17 is implemented.

Images