A cross-domain cooperative training task computing efficiency simulation method and related device

Abstract

The application discloses a cross-domain cooperative training task calculation efficiency simulation method and related equipment, and relates to the technical field of simulation methods. The calculation efficiency simulation method comprises the following steps: inputting a cross-domain cooperative training model and related parameters; initializing simulation models of five types of calculation stages in cross-domain cooperative training according to the cross-domain cooperative training model and the related parameters; calculating time points of different stage calculation tasks in each cooperative training period according to the cross-domain cooperative training rounds in the related parameters; and calculating the calculation efficiency index of the cross-domain cooperative training task according to the time points in all cooperative training periods. After the simulation models of the five types of calculation stages in cross-domain cooperative training are established, the time points in all cooperative training periods are calculated, and then the calculation efficiency index of the cross-domain cooperative training task is calculated, so that the simulation efficiency of the cross-domain cooperative training task is calculated quickly, and the accuracy of the calculation efficiency simulation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of simulation, in particular to a cross-domain collaborative training task computing efficiency simulation method and related equipment. BACKGROUND

[0002] In recent years, as an important part of new infrastructure, the intelligent computing center has become an intelligent base for the development of digital economy, which will accelerate the deep integration of intelligent technology and traditional industry, create a competitive digital industry cluster, and is of great significance to the digital and intelligent transformation of industry. The intelligent computing center is a full-stack artificial intelligence capability that provides from bottom chip computing power release to top application enablement, which can be used for artificial intelligence deep learning model development, training and inference in various industries. In China, with the rapid development of artificial intelligence technology and application, the intelligent computing center has been included in the key layout and planning of major cities across the country.

[0003] The intelligent computing network can connect the distributed intelligent computing center nodes in various places to form a network that perceives, allocates, and schedules resources such as computing power, storage, data, and models between multiple intelligent computing centers, which is an important direction for the further development of intelligent computing centers and will help to build an artificial intelligence research and development ecology in a regional or even national range. Collaborative computing enables data, computing power, models, networks, and services through intelligent computing network infrastructure to complete safe, compliant, and free resource circulation and sharing in the market, and thus realize new computing paradigms and business scenarios such as cross-domain collaborative training of large models, multi-party model aggregation, and federated learning. Among them, one of the most important computing tasks in the intelligent computing network is the cross-domain collaborative training task involving multiple artificial intelligence centers across regions, which is also a typical implementation scenario of the federated learning mode. Therefore, how to accurately evaluate the computing efficiency of the computing task under different scales of hardware, network, and other resource conditions before starting the computing task is beneficial to users to analyze and predict the required resources and actual efficiency of the computing task in advance, reducing unnecessary resource waste.

[0004] Currently, for local training cost modeling, researchers have proposed various methods to implement deep learning model cost-model evaluation. Among them, FlexFlow (FlexFlow is a deep learning framework that uses an operator graph G to describe all operations and states in DNN), RaNNC (RaNNC is a neural network pipeline parallel scheme search method), and other works have conducted simple cost-model modeling work. The basic idea is to calculate the execution time of each operator in the deep learning model computation graph according to the local training, thereby completing the computing efficiency modeling of the entire local training process.

[0005] However, these methods are all proposed for distributed computing tasks in local clusters and cannot be directly applied to collaborative training tasks across artificial intelligence centers. In addition, Flower (Flower is a framework for building federated learning systems, which has the characteristics of customization, good scalability, adaptation to multiple machine learning frameworks, easy understanding and maintenance) implements a federated learning framework that can adapt to multiple AI frameworks and provides federated learning simulation functions under the condition of small models and a large number of participants. This method realizes federated learning simulation under the condition of a large number of participants based on the distributed computing framework Ray, but the purpose of simulation is to simulate the accuracy of the federated learning model, not the efficiency of the computing task. When the computing efficiency of cross-domain collaborative training tasks needs to be simulated and evaluated, this method is no longer applicable.

[0006] Therefore, the prior art still needs to be improved and improved. SUMMARY

[0007] The main purpose of the present application is to provide a cross-domain collaborative training task computing efficiency simulation method and related equipment, which aims to solve the problem that the existing technology cannot simulate and evaluate the computing efficiency when performing cross-domain collaborative training tasks.

[0008] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0009] A cross-domain collaborative training task computing efficiency simulation method, the cross-domain collaborative training task computing efficiency simulation method comprising the following steps:

[0010] Inputting a cross-domain collaborative training model and related parameters;

[0011] Initializing simulation models of five types of computing stages in cross-domain collaborative training according to the cross-domain collaborative training model and the related parameters;

[0012] According to the number of cross-domain collaborative training rounds in the related parameters, the time points of different stage computing tasks in each collaborative training cycle are calculated;

[0013] According to the time points in all collaborative training cycles, the computing efficiency indicators of the cross-domain collaborative training task are calculated.

[0014] The cross-domain collaborative training task computing efficiency simulation method, the step of inputting the cross-domain collaborative training model and the related parameters before comprising:

[0015] Dividing the cross-domain collaborative training task into multiple computing cycles, and dividing each computing cycle into five types of computing tasks;

[0016] The five types of computing tasks are simulated and modeled respectively to obtain simulation models of five types of computing stages in the cross-domain collaborative training, wherein the five types of computing stages include local training, data serialization or deserialization, data transmission, model parameter fusion, and model evaluation and parameter updating.

[0017] In the cross-domain collaborative training task computing efficiency simulation method, the step of respectively simulating and modeling the five types of computing tasks to obtain simulation models of five types of computing stages in the cross-domain collaborative training specifically includes:

[0018] In the local training stage, the input parameters are a deep learning model M, a number of computing devices D, a communication bandwidth C between computing cards, a distributed training strategy P, a data set B, a data volume DataNum, and training hyperparameters E, and the output is a model local training time consumption t1, and the simulation model of the local training stage is defined as: t1=F1(M,D,C,P,B,DataNum,E); wherein the deep learning model M includes the cross-domain collaborative training model.

[0019] In the data serialization or deserialization stage, the input parameters are a model parameter volume ParaNum and a computing device Device, and the output is a model parameter serialization and deserialization computing time consumption t2, and the simulation model of the data serialization or deserialization stage is defined as: t2=F2(ParaNum,Device), wherein F2 is a linear regression model.

[0020] In the data transmission stage, the input parameters are a communication delay δ, a communication bandwidth C, and a data volume DataNum, and the output is a data transmission time consumption t3, and the simulation model of the data transmission stage is defined as: t3=F3(C,DataNum,δ), wherein F3 is a linear regression model.

[0021] In the model parameter fusion stage, the input parameters are a model parameter volume ParaNum and a parameter fusion strategy S, and the output is a model parameter fusion time consumption t4, and the simulation model of the model parameter fusion stage is defined as: t4=F4(ParaNum,S), wherein F4 is a linear regression model.

[0022] In the model evaluation and parameter updating stage, the input parameters are a model parameter volume ParaNum and an evaluation data volume DataNum, and the output is a model evaluation and parameter updating time consumption t5, and the simulation model of the model evaluation and parameter updating stage is defined as: t5=F5(ParaNum,DataNum), wherein F5 is a linear regression model.

[0023] The cross-domain co-training task computing efficiency simulation method, the step of initializing the simulation model of the five types of computing stages in the cross-domain co-training according to the cross-domain co-training model and the related parameters specifically includes:

[0024] The cross-domain co-training model and the related parameters are input into the simulation model of the five types of computing stages in the cross-domain co-training, and an initialization operation is performed.

[0025] The cross-domain co-training task computing efficiency simulation method, the step of calculating the time point of the computing task in each co-training cycle according to the cross-domain co-training round in the related parameters specifically includes:

[0026] The cross-domain co-training round in the related parameters input for simulation is read;

[0027] The simulation model of the five types of computing stages in the cross-domain co-training is used to calculate the computing time point of the cross-domain co-training task of the client in each co-training cycle;

[0028] The simulation model of the five types of computing stages in the cross-domain co-training is used to calculate the computing time point of the cross-domain co-training task of the server in each co-training cycle;

[0029] According to the computing time point of the cross-domain co-training task of the client in each co-training cycle and the computing time point of the cross-domain co-training task of the server in each co-training cycle, the time point of the computing task in each co-training cycle is obtained.

[0030] The cross-domain co-training task computing efficiency simulation method, the step of calculating the computing time point of the cross-domain co-training task of the client in each co-training cycle using the simulation model of the five types of computing stages in the cross-domain co-training specifically includes:

[0031] In the local training stage, the computing time and computing efficiency of each operator are calculated using a method based on a computation graph representation, and the local training time and local training computing efficiency of the client are obtained;

[0032] In the data serialization or deserialization stage, the time consumption of the client for converting and compressing Numpy format data into a binary file, and the time consumption of the client for deserializing and parsing the binary file into Numpy format data are calculated;

[0033] In the data transmission stage, the time consumption of the client for transmitting the binary file to the server is calculated, and the time consumption of the client for receiving the binary file from the server is calculated;

[0034] In the model evaluation and parameter update stage, the time consumption of the client for evaluating and updating the performance of the model after the model parameter fusion is calculated;

[0035] The computing time points of the client for performing the cross-domain collaborative training task in all stages in a collaborative training cycle are counted.

[0036] In the cross-domain collaborative training task computing efficiency simulation method, the step of calculating the computing time points of the server for performing the cross-domain collaborative training task in each collaborative training cycle by using the simulation model of the five types of computing stages in the cross-domain collaborative training specifically includes:

[0037] In the data serialization or deserialization stage, the time consumption of the server for converting and compressing the Numpy format data into a binary file and deserializing and parsing the binary file into the Numpy format data is calculated;

[0038] In the data transmission stage, the time consumption of the server for transmitting the binary file to the client is calculated, and the time consumption of the server for receiving the binary file from the client is calculated;

[0039] In the model parameter fusion stage, the time consumption of the server for performing model parameter fusion after receiving the model parameters is calculated;

[0040] The computing time points of the server for performing the cross-domain collaborative training task in all stages in a collaborative training cycle are counted.

[0041] In the cross-domain collaborative training task computing efficiency simulation method, the step of calculating the computing efficiency index of the cross-domain collaborative training task according to the time points in all collaborative training cycles specifically includes:

[0042] The total computing time points of the client for performing the cross-domain collaborative training task in all stages in all collaborative training cycles and the total computing time points of the server for performing the cross-domain collaborative training task in all stages in all collaborative training cycles are calculated;

[0043] According to the total computing time points of the client for performing the cross-domain collaborative training task in all stages in all collaborative training cycles and the total computing time points of the server for performing the cross-domain collaborative training task in all stages in all collaborative training cycles, the computing efficiency index of the client and the server for performing the cross-domain collaborative training task in all collaborative training cycles is calculated.

[0044] The related parameters include a starting time of each collaborative training round, a training round number, a collaborative training round number, a deep learning model, a number of computing devices, a communication bandwidth between computing cards, a distributed training strategy, a data set, a data volume, a local training hyperparameter, a model parameter volume, a communication delay, a communication bandwidth, a parameter fusion strategy, and an evaluation data volume.

[0045] The computing efficiency indicators include a total task consumption time, a computing power utilization efficiency, a communication consumption time proportion, a data serialization and deserialization consumption time proportion.

[0046] A cross-domain collaborative training task computing efficiency simulation system, comprising:

[0047] A parameter input module configured to input a cross-domain collaborative training model and related parameters.

[0048] An initialization module configured to initialize simulation models of five types of computing phases in cross-domain collaborative training according to the cross-domain collaborative training model and the related parameters.

[0049] A computing time point output module configured to calculate time points of different phase computing tasks in each collaborative training cycle according to a cross-domain collaborative training round number in the related parameters.

[0050] A computing efficiency indicator output module configured to statistically obtain computing efficiency indicators of the cross-domain collaborative training task according to the time points in all collaborative training cycles.

[0051] A controller, comprising a memory, a processor, a display, and a cross-domain collaborative training task computing efficiency simulation program stored on the memory and executable on the processor, wherein the cross-domain collaborative training task computing efficiency simulation program, when executed by the processor, implements the steps of the cross-domain collaborative training task computing efficiency simulation method described above.

[0052] A computer readable storage medium storing a cross-domain collaborative training task computing efficiency simulation program, wherein the cross-domain collaborative training task computing efficiency simulation program, when executed by a processor, implements the steps of the cross-domain collaborative training task computing efficiency simulation method described above.

[0053] Compared with the prior art, the cross-domain collaborative training task computing efficiency simulation method and the related device provided by the application, the computing efficiency simulation method comprises: inputting a cross-domain collaborative training model and related parameters; initializing simulation models of five types of computing stages in cross-domain collaborative training according to the cross-domain collaborative training model and the related parameters; calculating time points of different stage computing tasks in each collaborative training cycle according to the number of cross-domain collaborative training rounds in the related parameters; and calculating the computing efficiency index of the cross-domain collaborative training task according to the time points in all collaborative training cycles. By establishing the simulation models of the five types of computing stages in the cross-domain collaborative training, calculating the time points in all collaborative training cycles, and then calculating the computing efficiency index of the cross-domain collaborative training task, the simulation efficiency of the cross-domain collaborative training task is calculated quickly, and the accuracy of the computing efficiency simulation is improved. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 The flowchart of the preferred embodiment of the cross-domain collaborative training task computing efficiency simulation method provided by the application;

[0055] Figure 2 The flowchart before step S100 in the preferred embodiment of the cross-domain collaborative training task computing efficiency simulation method provided by the application;

[0056] Figure 3 The flowchart of step S20 in the preferred embodiment of the cross-domain collaborative training task computing efficiency simulation method provided by the application;

[0057] Figure 4 The data transmission and processing process diagram of the cross-domain collaborative training task in a training cycle provided by the application;

[0058] Figure 5 The flowchart of step S300 in the preferred embodiment of the cross-domain collaborative training task computing efficiency simulation method provided by the application;

[0059] Figure 6 The main time point diagram in a computing cycle of the cross-domain collaborative training task provided by the application;

[0060] Figure 7 The flowchart of step S320 in the preferred embodiment of the cross-domain collaborative training task computing efficiency simulation method provided by the application;

[0061] Figure 8 The flowchart of step S330 in the preferred embodiment of the cross-domain collaborative training task computing efficiency simulation method provided by the application;

[0062] Figure 9The flow chart of step S400 in the preferred embodiment of the cross-domain cooperative training task computing efficiency simulation method provided by the present application;

[0063] Figure 10 The architecture relationship diagram of the cross-domain cooperative training task computing efficiency simulation system provided by the present application;

[0064] Figure 11 The running environment schematic diagram of the preferred embodiment of the controller provided by the present application. DETAILED DESCRIPTION

[0065] In order to make the objectives, technical solutions and effects of the present application clearer and more explicit, the present application is further described in detail below with reference to the drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0066] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an" and "the" used herein also include the plural forms. It should be further understood that the phrase "comprising" used in the specification of the present application means that the features, integers, steps, operations, elements and / or components exist, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be intermediate elements. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0067] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as that generally understood by those skilled in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have meanings consistent with those in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as such.

[0068] In order to facilitate the understanding of the embodiments of the present application, first, the related elements involved in the embodiments of the present application are introduced.

[0069] The cross-domain collaborative training task participated by multiple artificial intelligence centers becomes an important computing task in the artificial intelligence computing network. How to relatively accurately simulate and evaluate the computing efficiency of the task before the actual execution of the computing task, provide the user with the computing efficiency evaluation result reference under different devices and network conditions, realize the maximum utilization of resources for the collaborative training task, and reduce the resource waste caused by multiple debugging is very necessary and meaningful, especially for the cross-cluster training task of the super large scale parameter model. One-time error debugging will cause great resource loss and affect the computing efficiency of the task. Therefore, how to effectively and relatively accurately simulate and evaluate the computing efficiency of the task will provide an important simulation reference for the cross-domain collaborative training, and further improve the development and execution efficiency of the cross-domain collaborative training.

[0070] The core subject of the application is to analyze the computing efficiency of the deep learning model for the cross-region collaborative training of multiple artificial intelligence computing centers. A multi-stage modeling cross-domain collaborative training task computing efficiency simulation analysis method and system are proposed. The purpose of the application is to propose a multi-stage modeling efficiency simulation method for the computing efficiency simulation of this kind of cross-domain collaborative training task. First, the multi-stage modeling of the cross-region collaborative training task between multiple artificial intelligence centers is carried out, including five main stages: local training, data serialization or deserialization, data transmission, model parameter fusion, and model evaluation and parameter update. Then, the efficiency modeling of the five stages is carried out, realizing the simulation calculation of the cross-domain collaborative training task efficiency of different deep learning models on different scale hardware resources, supporting the user to better analyze and predict the resources and time consumption required for the computing task.

[0071] Therefore, the application provides a cross-domain collaborative training task computing efficiency simulation method and related equipment. In the application, the simulation model of the five types of computing stages in the cross-domain collaborative training is initialized by inputting the cross-domain collaborative training model and related parameters. Then, the time points of the computing task in different stages in each collaborative training cycle are calculated. After calculating the time points in all collaborative training cycles, the computing efficiency index of the cross-domain collaborative training task is calculated, so that the simulation efficiency of the entire cross-domain collaborative training task is calculated simply and quickly, and the accuracy of the computing simulation efficiency is effectively improved.

[0072] The cross-domain collaborative training task computing efficiency simulation method design scheme will be described below through specific exemplary embodiments. It should be noted that the following embodiments are only used to explain and describe the technical scheme of the application, and are not limited to specific embodiments:

[0073] Please refer to Figure 1The application provides a cross-domain collaborative training task computing efficiency simulation method, which comprises the following steps:

[0074] S100, input a cross-domain collaborative training model and related parameters.

[0075] The related parameters comprise a starting time of each collaborative training round, a training round number, a collaborative training round number, a deep learning model, a computing device number, a computing card intercommunication bandwidth, a distributed training strategy, a data set, a data volume, a local training hyperparameter, a model parameter volume, a communication delay, a communication bandwidth, a parameter fusion strategy, and an evaluation data volume.

[0076] Specifically, when simulating the cross-domain collaborative training task computing efficiency, a user needs to input the cross-domain collaborative training model and the related parameters, for example, the starting time of each collaborative training round, the training round number, the collaborative training round number, the deep learning model, the computing device number, the computing card intercommunication bandwidth, the distributed training strategy, the data set, the data volume, the local training hyperparameter, the model parameter volume, the communication delay, the communication bandwidth, the parameter fusion strategy, and the evaluation data volume, so as to initialize the simulation model of the five types of computing stages in the cross-domain collaborative training.

[0077] Further, referring to Figure 2 , the step of S100, inputting the cross-domain collaborative training model and the related parameters, comprises the following steps:

[0078] S10, divide the cross-domain collaborative training task into a plurality of computing periods, and divide each computing period into five types of computing tasks;

[0079] S20, respectively simulate the five types of computing tasks to obtain the simulation model of the five types of computing stages in the cross-domain collaborative training; wherein the five types of computing stages comprise local training, data serialization or deserialization, data transmission, model parameter fusion, and model evaluation and parameter updating.

[0080] Specifically, before simulating the cross-domain collaborative training task computing efficiency, the simulation model of the five types of computing stages in the cross-domain collaborative training needs to be established, specifically: first, when a server end and any client end perform cross-domain collaborative training task transmission, the cross-domain collaborative training task is divided into a plurality of computing periods, and each computing period is divided into five types of computing tasks; then, the five types of computing tasks are respectively simulated to obtain the simulation model of the five types of computing stages (local training, data serialization or deserialization, data transmission, model parameter fusion, and model evaluation and parameter updating) in the cross-domain collaborative training.

[0081] The cross-domain collaborative training task in each calculation period is divided into five types of calculation tasks according to stage characteristics, then, modeling is respectively carried out, and simulation models of five types of calculation stages are obtained, so that the cross-domain collaborative training task is effectively divided into stages and modeled respectively, the calculation simulation efficiency is realized in stages and periods, and the calculation efficiency and calculation accuracy are improved.

[0082] Further, referring to Figure 3 , the step of S20, simulation modeling of five types of calculation tasks, obtaining simulation models of five types of calculation stages in cross-domain collaborative training, specifically comprises:

[0083] S21, in the local training stage, the input parameters are: a deep learning model M, a number of computing devices D, a communication bandwidth C between computing cards, a distributed training strategy P, a data set B, a data volume DataNum and training hyperparameters E, and the output is: a model local training time consumption t1, the simulation model of the local training stage is defined as: t1=F1(M,D,C,P,B,DataNum,E); wherein, the deep learning model M comprises the cross-domain collaborative training model;

[0084] S22, in the data serialization or deserialization stage, the input parameters are: a model parameter volume ParaNum and a computing device Device, and the output is: a model parameter serialization and deserialization calculation time consumption t2, the simulation model of the data serialization or deserialization stage is defined as: t2=F2(ParaNum,Device), wherein, F2 is a linear regression model;

[0085] S23, in the data transmission stage, the input parameters are: a communication delay δ, a communication bandwidth C and a data volume DataNum, and the output is: a data transmission time consumption t3, the simulation model of the data transmission stage is defined as: t3=F3(C,DataNum,δ), wherein, F3 is a linear regression model;

[0086] S24, in the model parameter fusion stage, the input parameters are: a model parameter volume ParaNum and a parameter fusion strategy S, and the output is: a model parameter fusion time consumption t4, the simulation model of the model parameter fusion stage is defined as: t4=F4(ParaNum,S), wherein, F4 is a linear regression model;

[0087] S25, in the model evaluation and parameter updating stage, the input parameters are: a model parameter volume ParaNum and an evaluation data volume DataNum, and the output is: a model evaluation and parameter updating time consumption t5, the simulation model of the model evaluation and parameter updating stage is defined as: t5=F5(ParaNum,DataNum), wherein, F5 is a linear regression model.

[0088] Specifically, the specific process of simulating and modeling five types of computing tasks in stages is as follows:

[0089] First, the process of establishing a simulation model by simulation software in the local training stage is described as follows:

[0090] The parameters input by the user are: a deep learning model M (the cross-domain collaborative training model in the present application, such as an ONNX model or a Pytorch model), the number of computing devices D, the communication bandwidth C between computing cards, the distributed training strategy P (such as data parallelism, model parallelism or pipeline parallelism), the batch size B of the data set, the data volume DataNum and the training hyperparameters E. Among them, the batch size represents the number of parameters passed to the program for training at a time; for example, if the batch size is set to 100 when there are 1000 data in the training set, the program will first use the first 100 parameters in the data set, i.e., the first 100 data, to train the model.

[0091] Output: model local training time consumption t1; the purpose is to calculate the efficiency of training under these configuration inputs;

[0092] The simulation model of the local training stage is defined as: t1=F1(M,D,C,P,B,DataNum,E); wherein the function F1 realizes the calculation of the time consumption of all operators in the deep learning model under different devices and communication conditions.

[0093] The process of establishing a simulation model by simulation software in the data serialization or data deserialization stage is described as follows:

[0094] The parameters input by the user are: the number of model parameters ParaNum and the computing device Device.

[0095] Output: model parameter serialization and deserialization calculation time consumption t2.

[0096] The simulation model of the data serialization or data deserialization stage is defined as: t2=F2(ParaNum,Device); wherein the function F2 is a linear regression model (a statistical analysis method for determining the quantitative relationship between two or more variables by using regression analysis in mathematical statistics).

[0097] The process of establishing a simulation model by simulation software in the data transmission stage is described as follows:

[0098] The parameters input by the user are: the communication delay δ, the communication bandwidth C and the data volume DataNum.

[0099] Output: data transmission time consumption t3;

[0100] The simulation model of the data transmission stage is defined as: t3=F3(C, DataNum, δ); wherein, the function F3 is a linear regression model.

[0101] Fourthly, the process of the parameter fusion stage for establishing the simulation model by the simulation software is described as:

[0102] The user input parameters are: model parameter quantity ParaNum and parameter fusion strategy S.

[0103] Output: model parameter fusion time consumption t4;

[0104] The simulation model of the parameter fusion stage is defined as: t4=F4(ParaNum, S); wherein, the function F4 is a linear regression model.

[0105] Fifthly, the process of the model evaluation and parameter updating stage for establishing the simulation model by the simulation software is described as:

[0106] The user input parameters are: model parameter quantity ParaNum and evaluation data quantity DataNum.

[0107] Output: model evaluation and parameter updating time consumption t5;

[0108] The simulation model of the model evaluation and parameter updating stage is defined as: t5=F5(ParaNum, DataNum); wherein, F5 is a linear regression model.

[0109] Finally, the simulation models of the five types of computing stages in the cross-domain collaborative training are obtained as: the simulation model of the local training stage: t1=F1(M, D, C, P, B, DataNum, E), the simulation model of the data serialization or data deserialization stage: t2=F2(ParaNum, Device), the simulation model of the data transmission stage: t3=F3(C, DataNum, δ), the simulation model of the parameter fusion stage: t4=F4(ParaNum, S), and the simulation model of the model evaluation and parameter updating stage: t5=F5(ParaNum, DataNum).

[0110] Further, please continue to refer to Figure 1 S200, according to the cross-domain collaborative training model and the related parameters, initialize the simulation models of the five types of computing stages in the cross-domain collaborative training.

[0111] Specifically, after the user inputs the cross-domain co-training model and the related parameters, the cross-domain co-training model and the related parameters are used to initialize the simulation model of the five types of calculation stages in the cross-domain co-training, that is, the process of assigning initial values to the parameters in the simulation model of the five types of calculation stages in the cross-domain co-training, effectively realizing the process of initializing the parameters of the simulation model.

[0112] Further, the step S200 of initializing the simulation model of the five types of calculation stages in the cross-domain co-training according to the cross-domain co-training model and the related parameters specifically comprises:

[0113] S210, input the cross-domain co-training model and the related parameters into the simulation model of the five types of calculation stages in the cross-domain co-training, and perform initialization operation.

[0114] Specifically, after the user inputs the cross-domain co-training model and the related parameters, and obtains the simulation model of the five types of calculation stages in the cross-domain co-training, the cross-domain co-training model and the related parameters are input into the simulation model of the five types of calculation stages in the cross-domain co-training, and the related parameters in the simulation model are initialized, that is, the variables in the simulation model are assigned to default values, the controls are set to default states, and the preparations are made ready, thereby effectively realizing the initialization operation of the simulation model of the five types of calculation stages.

[0115] Further, please continue to refer to Figure 1 S300, according to the number of cross-domain co-training rounds in the related parameters, calculate the time points of different stage calculation tasks in each co-training period.

[0116] Wherein, the data transmission and processing process of the cross-domain co-training task in a training period is as shown in Figure 4 Here, taking an example of a server side (Server) simultaneously communicating with two client sides (Client-1 and Client-2), the data transmission and processing process is described as follows:

[0117] Firstly, after all the Client terminals receive the binary model parameters (binary files) sent by the Server terminal, the Numpy format model parameters are obtained by data deserialization parsing on each Client terminal, and the local model training operation is started on the Numpy format model parameters (Numpy format data); then, after the Client terminal completes the local training, the Numpy format training model parameters are serialized, and the obtained binary training model parameters are sent to the Server terminal, and after the Server terminal receives the binary training model parameters of all the Client terminals, the model parameter fusion operation is performed.

[0118] Secondly, after the model parameter fusion operation is completed, the binary fusion model parameters are serialized and sent to the Client terminal; further, after the Client terminal receives and parses the Numpy format fusion model parameters, the model is locally updated and evaluated; finally, all the Client terminals send the model evaluation result data to the Server terminal, and after the Server terminal receives the model evaluation result, the next cooperative training cycle is started.

[0119] Specifically, after the initialization process of the simulation model is completed, according to the cross-domain cooperative training rounds in the input related parameters, the time points of different stage calculation tasks in each cooperative training cycle are calculated, that is, the time points of five stage calculation tasks in each cross-domain cooperative training round are calculated, so that the time points of settlement tasks in different stages of each cycle are accurately and quickly calculated.

[0120] Further, referring to Figure 5 , the step of S300 of calculating the time points of different stage calculation tasks in each cooperative training cycle according to the cross-domain cooperative training rounds in the related parameters specifically includes:

[0121] S310, reading the cross-domain cooperative training rounds for simulation in the input related parameters;

[0122] S320, calculating the calculation time points of the client terminals for the cross-domain cooperative training tasks in each cooperative training cycle by using the simulation models of the five types of calculation stages in the cross-domain cooperative training;

[0123] S330, calculating the calculation time points of the server terminals for the cross-domain cooperative training tasks in each cooperative training cycle by using the simulation models of the five types of calculation stages in the cross-domain cooperative training;

[0124] S340, according to the calculation time point of the cross-domain cooperative training task of the client in each cooperative training period and the calculation time point of the cross-domain cooperative training task of the server in each cooperative training period, the time point of the calculation task in each cooperative training period is obtained.

[0125] Specifically, after completing the initialization process of the simulation model, the number of cross-domain cooperative training rounds in the related parameters input previously is read to confirm the number of cross-domain cooperative training rounds that need to be simulated; and in each round, i.e., in each cooperative training period, there are five types of calculation stages, so it is necessary to calculate the calculation time point of the cross-domain cooperative training task of the client on the simulation model of the five types of calculation stages in each cooperative training period, and the calculation time point of the cross-domain cooperative training task of the server on the simulation model of the five types of calculation stages in each cooperative training period; finally, according to the calculation time point of the cross-domain cooperative training task of the client and the calculation time point of the cross-domain cooperative training task of the server in each cooperative training period, the time point of the calculation task in each cooperative training period is obtained.

[0126] In a calculation period of a cross-domain cooperative training task, the main time points are as shown in Figure 6 Figure 6 The main time points in a cross-domain cooperative training task period are defined as follows: the main time points {T1, T2, T3, T4, T5, T6, T7, T8, T9, T10} of the client (Client) in the current period are calculated according to the five-stage model once; the main time points {t1, t2, t3, t4, t5, t6} of the server (Server) in the current period are calculated according to the five-stage model once; wherein Fit (function) is a kind of fitting function, evaluate (abbreviation eval) means model evaluation, and round means period or round.

[0127] Therefore, in each cooperative training period, the calculation time point of the cross-domain cooperative training task of the server (Server) is defined as follows: (1) Δt1=t2-t1: representing the Server measuring the time from initiating the execution of the training task to the Client (client) to receiving all the model parameters returned by the Client; (2) Δt2=t3-t2: representing the calculation time required by the Server model parameter fusion; (3) Δt3=t4-t3: representing the Server measuring the time from initiating the execution of the evaluation task to the Client to receiving all the evaluation results returned by the Client.

[0128] ​And in each collaborative training cycle, the client performs cross-domain collaborative training task computing time point definition is: (1) ΔT1=T2-T1: indicates that the training function is executed before the Client receives the deserialization time of the model parameter; (2) ΔT2=T3-T2: indicates that the Client local training required computing time; (3) ΔT3=T4-T3: indicates that the Client local training is completed, the model parameter serialization required time; (4) ΔT4=T5-T4: indicates that the Client model parameter sending required time; (5) ΔT5=T6-T5: indicates that the evaluation function is executed before the Client receives the deserialization time of the fused model parameter; (6) ΔT6=T7-T6: indicates that the Client model evaluation required time; (7) ΔT7=T8-T7: indicates that the evaluation function evaluation result data serialization time; (8) ΔT8=T9-T8: indicates that the Client evaluation result sending time; (9) ΔT9=T1-S_T1(S_T1 is equal to the Server's t1): indicates that the training function is executed before receiving the model parameter from the Server required communication time; (10) ΔT 10 =S_T2-T4(S_T2 is equal to the Server's t2): indicates that the training function is executed after the Client to the Server sending model parameter required communication time; (11) ΔT 11 =T6-S_T4(S_T4 is equal to the Server's t4): indicates that the evaluation function is executed before receiving the model parameter from the Server required communication time; (12) ΔT 12 =S_T5-T9(S_T5 is equal to the Server's t5: indicates that the evaluation function is executed after the Client to the Server sending evaluation results required communication time.

[0129] Further, please refer to Figure 7 , the S320, using the simulation model of five types of computing phase in cross-domain collaborative training to calculate the computing time point of the client in each collaborative training cycle for cross-domain collaborative training task specific steps include:

[0130] S321, in the local training phase, using a method based on the computation graph representation for each operator to calculate the time and computational efficiency statistics, the local training time and local training efficiency of the Client;

[0131] S322, in the data serialization or deserialization phase, the client converts the Numpy format data to a binary file, and the time required for the binary file to be deserialized and parsed into Numpy format data is calculated;

[0132] S323, in the data transmission phase, the time consumption of the client transmitting the binary file to the server end is calculated, and the time consumption of the client receiving the binary file from the server end is calculated;

[0133] S324, in the model evaluation and parameter updating phase, the time consumption of the client evaluating and updating the parameters of the model after the model parameter fusion is calculated;

[0134] S325, the computing time points of the client performing the cross-domain collaborative training task in all stages in a collaborative training cycle are counted.

[0135] Specifically, in each collaborative training cycle, the simulation model of the five types of computing stages in cross-domain collaborative training is used to calculate the computing time points of the client performing the cross-domain collaborative training task, and only part of the simulation model of the five types of computing stages in cross-domain collaborative training is used for the client, specifically:

[0136] The local training process is the computing process of the client training one or more Epoch (operators) locally, which usually adopts a method based on a computation graph representation (for example: a deep learning cost-model method) to calculate the local training time consumption and local training computing efficiency of each operator in the obtained efficiency simulation model, and to obtain the comprehensive training time consumption and comprehensive computing efficiency by comprehensively calculating the computing time consumption of each operator, the communication time consumption between local multi-processes, etc. The time consumption of the client performing the local training process in each collaborative training cycle is calculated. In the present application, the cross-domain collaborative training task is modeled by stages, and the deep learning model cost-model method is used to model the local training part of the collaborative training task, which effectively improves the accuracy of the computing efficiency simulation.

[0137] The main influencing factors of the local training operation process include: model parameter quantity, computing device quantity and type, device network topology and communication bandwidth, etc. Among them, the larger the model parameter, the longer the local training time consumption; the more the local training device quantity, the smaller the local training time consumption; the larger the network topology and communication bandwidth between local training devices, the smaller the local training time consumption.

[0138] The data serialization process is to convert and compress the Numpy (NumPy: Numerical Python, which is an open source numerical calculation extension of Python, a basic package for scientific calculation) format data into a binary file, which is convenient for cross-domain transmission of the calculation process through gRPC (gRPC is a high-performance, general-purpose open source RPC framework); and the data deserialization process is the reverse process of data serialization, which refers to the process of parsing the received binary file into Numpy format data. The main factors affecting the data serialization and deserialization process calculation time are data volume and execution device type. The time consumed by the client in each collaborative training cycle for data serialization or deserialization process calculation.

[0139] The data serialization or deserialization stage exists when the multi-party participants (clients) and the server communicate, and the data operation needs to be adopted by the data sender or receiver. A linear regression model is used for modeling, and the main factors affecting the data serialization and deserialization process calculation time are data volume (including the amount of model parameters that need to be communicated) and execution device type (such as computing device type).

[0140] The communication between the server and the client is a data transmission process, which refers to the calculation process required for data transmission and reception between Server and Client. The main factors affecting the calculation time of this process are the amount of data communicated, communication delay, communication bandwidth, etc.

[0141] Then, for the client, the time consumed by the client in each collaborative cycle for data transmission stage calculation is the time consumed by the client for transmitting binary files to the server, and the time consumed by the client for receiving binary files from the server.

[0142] The model evaluation and parameter update process refers to the calculation process of updating the local model after the client receives the fused model parameters, and testing and evaluating the performance of the local model. The calculation time of this process is mainly related to the amount of model parameters, the amount of test data, the type of computing device, etc.; the time consumed by the client in each collaborative training cycle for model evaluation and parameter update process calculation.

[0143] Finally, the time points of the client's cross-domain collaborative training task calculation in all stages in a collaborative training cycle are calculated, including the time consumed by the local training, data serialization or deserialization, data transmission, and model evaluation and parameter update.

[0144] Further, please refer to Figure 8The step of calculating the time point of the server side for performing the cross-domain collaborative training task in each collaborative training cycle by using the simulation model of the five types of calculation stages in the cross-domain collaborative training specifically includes:

[0145] S331, in the data serialization or deserialization stage, calculating the time consumption of the server side for converting and compressing the Numpy format data into a binary file and for deserializing and parsing the binary file into the Numpy format data;

[0146] S332, in the data transmission stage, calculating the time consumption of the server side for transmitting the binary file to the client side and calculating the time consumption of the server side for receiving the binary file from the client side;

[0147] S333, in the model parameter fusion stage, calculating the time consumption of the server side for performing the model parameter fusion after receiving the model parameters;

[0148] S334, calculating the time point of the server side for performing the cross-domain collaborative training task in all stages in one collaborative training cycle.

[0149] Specifically, similarly, in each collaborative training cycle, by using the simulation model of the five types of calculation stages in the cross-domain collaborative training, when calculating the time point of the server side for performing the cross-domain collaborative training task, the server side also only uses part of the simulation model of the five types of calculation stages in the cross-domain collaborative training, specifically:

[0150] Since the data serialization or deserialization stage exists in the data operation required by the data sending side or receiving side when the multi-participant (client) and the server side communicate, on the server side, there is also a data serialization process, and the time consumption of the server side for performing the data serialization or deserialization process in each collaborative training cycle needs to be calculated.

[0151] Moreover, since the communication between the server side and the client side is a data transmission process, for the server side, calculating the time consumption of the server side in the data transmission stage in each collaborative cycle is to calculate the time consumption of the server side for transmitting the binary file to the client side and the time consumption of the server side for receiving the binary file from the client side. The time consumption of the server side for performing the data serialization or deserialization process in each collaborative training cycle needs to be calculated.

[0152] The model parameter fusion process refers to the calculation process of model parameter fusion performed by the server in the collaborative training when receiving the uploaded model parameters of the participants (clients) through a certain fusion strategy. The influencing factors of the model parameter fusion operation include the model parameter quantity and the parameter fusion strategy. Typical parameter fusion strategies include the federated average method (federated average method (FedAvg) runs stochastic gradient descent (SGD) in parallel on a small part of all devices and averages the sequence every certain period of time) and the federated Adam method.

[0153] Finally, the calculation time points of the server in the cross-domain collaborative training task in all stages in a collaborative training cycle are counted, including data serialization or deserialization, data transmission, and the time consumption of model parameter fusion.

[0154] Further, please refer to Figure 1 , S400, the calculation efficiency indicators of the cross-domain collaborative training task are counted according to the time points in all collaborative training cycles. The calculation efficiency indicators include the total time consumption, the computing power utilization efficiency, the communication time consumption proportion, and the data serialization and deserialization time consumption proportion.

[0155] Specifically, after the calculation time points of the server in the cross-domain collaborative training task in all stages in a collaborative training cycle and the calculation time points of the server in the cross-domain collaborative training task in all stages in a collaborative training cycle are calculated, the calculation time points of the server in the cross-domain collaborative training task in all stages in all collaborative training cycles and the calculation time points of the server in the cross-domain collaborative training task in all stages in all collaborative training cycles are calculated. Finally, the calculation efficiency indicators of the cross-domain collaborative training task are counted according to the time points in all collaborative training cycles.

[0156] Further, please refer to Figure 9 , the S400, the calculation efficiency indicators of the cross-domain collaborative training task are counted according to the time points in all collaborative training cycles. The calculation efficiency indicators include the total time consumption, the computing power utilization efficiency, the communication time consumption proportion, and the data serialization and deserialization time consumption proportion.

[0157] S410, the total calculation time points of the clients in the cross-domain collaborative training task in all stages in all collaborative training cycles and the total calculation time points of the servers in the cross-domain collaborative training task in all stages in all collaborative training cycles are calculated.

[0158] S420, according to the total computing time points of the client side in all stages of all collaborative training cycles and the total computing time points of the server side in all stages of all collaborative training cycles, the computing efficiency index of the client side and the server side in all collaborative training cycles is calculated.

[0159] Specifically, after the computing time points of the server side in all stages of one collaborative training cycle and the computing time points of the server side in all stages of one collaborative training cycle are calculated respectively, the total computing time points of the client side in all stages of all collaborative training cycles and the total computing time points of the server side in all stages of all collaborative training cycles are calculated, and finally, according to the total computing time points of the client side and the server side in all stages of all collaborative training cycles, the computing efficiency index of the client side and the server side in all collaborative training cycles is calculated. The efficiency index of the cross-domain collaborative training task is defined as follows:

[0160] Total task time (comprehensive training time): T≈∑ N (t6-t1); wherein N represents N cycles;

[0161] Client side computing power utilization efficiency (defined as the ratio of the actual model training or evaluation time occupied by the client to the total task time):

[0162] Client side communication time consumption ratio:

[0163] Client side data serialization and deserialization time consumption ratio:

[0164] Therefore, the entire cross-domain collaborative training task computing efficiency simulation process is as follows:

[0165] The user input parameters are: the number of collaborative training rounds N, the deep learning model M (such as ONNX model, Pytorch model), the number of computing devices D, the communication bandwidth between computing cards C, the distributed training strategy P (such as data parallel, model parallel or pipeline parallel), the data set batch size B, the data volume DataNum, the local training hyperparameter E, the model parameter volume ParaNum, the communication delay δ, the communication bandwidth C, the parameter fusion strategy S, and the evaluation data volume DataNum.

[0166] Output: total task duration, Client computing power utilization efficiency, Client communication time consumption ratio, Client data serialization and deserialization time consumption ratio;

[0167] Set the starting time T0;

[0168] Cycle cooperative training rounds N:

[0169] For Client:

[0170] Cycle all participating Clients:

[0171] According to the five-stage model, calculate the main time points {T1, T2, T3, T4, T5, T6, T7, T8, T9, T10} of the Server in the current cycle once;

[0172] For Server: According to the five-stage model, calculate the main time points {t1, t2, t3, t4, t5, t6} of the Server in the current cycle once;

[0173] Record all simulation time points of the Server and all Clients in the current Nth round;

[0174] Set the real-time time point of the next round of cooperative training;

[0175] According to the time points calculated by all simulations, calculate the efficiency indicators: total task duration, Client computing power utilization efficiency, Client communication time consumption ratio, Client data serialization and deserialization time consumption ratio.

[0176] As can be seen, only by inputting various input parameters, the total task duration, Client computing power utilization efficiency, Client communication time consumption ratio, Client data serialization and deserialization time consumption ratio, etc. can be directly outputted.

[0177] Further, please refer to Figure 10 The application provides a cross-domain cooperative training task computing efficiency simulation system, which comprises:

[0178] A parameter input module 1 is used for inputting a cross-domain cooperative training model and related parameters;

[0179] An initialization module 2 is used for initializing simulation models of five types of computing stages in cross-domain cooperative training according to the cross-domain cooperative training model and the related parameters;

[0180] A computing time point output module 3 is used for calculating the time points of different stage computing tasks in each cooperative training cycle according to the cross-domain cooperative training rounds in the related parameters.

[0181] a computing efficiency index output module 4, configured to calculate the computing efficiency index of the cross-domain collaborative training task according to the time points in all the collaborative training cycles.

[0182] Specifically, the cross-domain collaborative training task computing efficiency simulation system is used to implement the cross-domain collaborative training task computing efficiency simulation method, and the cross-domain collaborative training task computing efficiency simulation method is introduced as follows:

[0183] When simulating the computing efficiency of the cross-domain collaborative training task, the user needs to first input the cross-domain collaborative training model and related parameters, such as the starting time of each collaborative training round, the training round number, the collaborative training round number, the deep learning model, the number of computing devices, the communication bandwidth between computing cards, the distributed training strategy, the data set, the data volume, the local training hyperparameter, the model parameter volume, the communication delay, the communication bandwidth, the parameter fusion strategy, the evaluation data volume, etc. Then, the simulation model of the five types of computing stages in the cross-domain collaborative training obtained is initialized by using the cross-domain collaborative training model and the related parameters, that is, the process of assigning initial values to the parameters in the simulation model of the five types of computing stages in the cross-domain collaborative training.

[0184] Secondly, according to the cross-domain collaborative training round number in the input related parameters, the time points of the computing task in different stages in each collaborative training cycle are calculated, that is, the time points of the five stage computing tasks in each cross-domain collaborative training round are calculated. Finally, the computing time points of the cross-domain collaborative training task on the server side in all stages in all collaborative training cycles in the cross-domain collaborative training round number are calculated, and the computing time points of the cross-domain collaborative training task on the server side in all stages in all collaborative training cycles are calculated, and the computing efficiency index of the cross-domain collaborative training task is calculated according to the time points in all the collaborative training cycles.

[0185] In summary, the present application has the following characteristics: 1. Unlike the existing analysis of the efficiency of deep learning computing tasks, which mostly focuses on single-cluster distributed training efficiency modeling of deep learning models, the present application first proposes a complete efficiency simulation scheme for the computing efficiency of cross-domain collaborative training tasks involving resources of multiple cross-regional artificial intelligence centers; 2. The cross-domain collaborative training task is modeled in stages, and the deep learning model cost-model method is used to model the local training part of the collaborative training task, effectively improving the accuracy of the computing efficiency simulation; 3. A large amount of actual experimental data is used to construct the computing efficiency simulation prediction linear regression model of the data serialization or deserialization, data transmission, model parameter fusion and other computing processes.

[0186] As Figure 11As shown, based on the above cross-domain co-training task computing efficiency simulation method, the application also correspondingly provides a controller, which comprises a processor 10, a memory 20 and a display 30. Figure 11 Only part of the components of the controller is shown, but it should be understood that all the shown components are not required to be implemented, and more or less components can be alternatively implemented.

[0187] The memory 20 can be an internal storage unit of the controller in some embodiments, such as a hard disk or a memory of the controller. The memory 20 can also be an external storage device of the controller in other embodiments, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the controller. Further, the memory 20 can include both the internal storage unit and the external storage device of the controller. The memory 20 is used to store application software and various data installed on the controller, such as program codes of the installed controller, etc. The memory 20 can also be used to temporarily store data that has been output or will be output. In an embodiment, the memory 20 stores a cross-domain co-training task computing efficiency simulation program 40, which can be executed by the processor 10, so as to implement the cross-domain co-training task computing efficiency simulation method in the application.

[0188] The processor 10 can be a central processing unit (CPU), a microprocessor or other data processing chip in some embodiments, which is used to run program codes or process data stored in the memory 20, such as to execute the cross-domain co-training task computing efficiency simulation method, etc.

[0189] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. in some embodiments. The display 30 is used to display information of the controller and to display a visualized user interface. The components 10-30 of the controller communicate with each other through a system bus.

[0190] In an embodiment, the processor 10 implements the steps of the cross-domain co-training task computing efficiency simulation method as described above when executing the cross-domain co-training task computing efficiency simulation program 40 in the memory 20. Since the steps of the cross-domain co-training task computing efficiency simulation method are described in detail above, no further description is given here.

[0191] Further, the present application also provides a computer readable storage medium, which stores a cross-domain co-training task computing efficiency simulation program, and the cross-domain co-training task computing efficiency simulation program realizes the steps of the cross-domain co-training task computing efficiency simulation method when executed by a processor.

[0192] To sum up, the present application provides a cross-domain co-training task computing efficiency simulation method and related equipment, the computing efficiency simulation method comprises: inputting a cross-domain co-training model and related parameters; initializing simulation models of five types of computing stages in cross-domain co-training according to the cross-domain co-training model and the related parameters; calculating time points of different stage computing tasks in each co-training cycle according to the number of cross-domain co-training rounds in the related parameters; and calculating the computing efficiency index of the cross-domain co-training task according to the time points in all co-training cycles. By establishing the simulation models of the five types of computing stages in cross-domain co-training, calculating the time points in all co-training cycles, and then calculating the computing efficiency index of the cross-domain co-training task, the simulation efficiency of the cross-domain co-training task is calculated quickly, and the accuracy of the computing efficiency simulation is improved.

[0193] It can be understood that, for those skilled in the art, equivalent replacements or changes can be made according to the technical solutions and the inventive concept of the present application, and all these changes or replacements shall belong to the protection scope of the appended claims of the present application.

Claims

1. A method for simulating computing efficiency of cross-domain collaborative training tasks, characterized in that, The cross-domain co-training task computation efficiency simulation method comprises the following steps: input the cross-domain co-training model and related parameters; initialize the simulation model of the five types of computation stages in cross-domain co-training according to the cross-domain co-training model and the related parameters; According to the number of cross-domain co-training rounds in the related parameters, calculate the time points of different stage computation tasks in each co-training cycle; According to the time points in all co-training cycles, the computation efficiency index of cross-domain co-training task is calculated; The steps of respectively simulating the five types of computation tasks to obtain the simulation model of the five types of computation stages in cross-domain co-training include: In the local training stage, the input parameters are: deep learning model M, number of computing devices D, communication bandwidth between computing cards C, distributed training strategy P, data set B, data volume DataNum and training hyperparameters E, and the output is: model local training time t1, the simulation model of the local training stage is defined as: t1=F1 (M,D,C,P,B,DataNum,E); wherein, the deep learning model M includes the cross-domain co-training model, and F1 is a function of computation time statistics; In the data serialization or deserialization stage, the input parameters are: model parameter quantity ParaNum and computing device Device, and the output is: model parameter serialization and deserialization computation time t2, the simulation model of the data serialization or deserialization stage is defined as: t2=F2 (ParaNum,Device); wherein, F2 is a linear regression model; In the data transmission stage, the input parameters are: communication delay δ, communication bandwidth C and data volume DataNum, and the output is: data transmission time t3, the simulation model of the data transmission stage is defined as: t3=F3 (C,DataNum,δ); wherein, F3 is a linear regression model; In the model parameter fusion stage, the input parameters are: model parameter quantity ParaNum and parameter fusion strategy S, and the output is: model parameter fusion time t4, the simulation model of the model parameter fusion stage is defined as: t4=F4 (ParaNum,S); wherein, F4 is a linear regression model; In the model evaluation and parameter updating stage, the input parameters are: model parameter quantity ParaNum and evaluation data volume DataNum, and the output is: model evaluation and parameter updating time t5, the simulation model of the model evaluation and parameter updating stage is defined as: t5=F5 (ParaNum,DataNum); wherein, F5 is a linear regression model. 2.The cross-domain coordinated training task computation efficiency simulation method according to claim 1, characterized in that, The step of inputting the cross-domain co-training model and related parameters includes: Divide the cross-domain co-training task into multiple computation cycles, and divide each computation cycle into five types of computation tasks; The simulation modeling is respectively performed on five types of computing tasks to obtain simulation models of five types of computing stages in the cross-domain collaborative training, wherein the five types of computing stages include local training, data serialization or deserialization, data transmission, model parameter fusion, and model evaluation and parameter updating. 3.The cross-domain coordinated training task computation efficiency simulation method according to claim 1, characterized in that, The step of initializing the simulation models of the five types of computing stages in the cross-domain collaborative training according to the cross-domain collaborative training model and the related parameters specifically includes: The cross-domain collaborative training model and the related parameters are input into the simulation models of the five types of computing stages in the cross-domain collaborative training for initialization operation.

4. The cross-domain co-training task computation efficiency simulation method of claim 2, wherein, The step of calculating the time points of the computing tasks in different stages in each collaborative training cycle according to the number of cross-domain collaborative training rounds in the related parameters specifically includes: The number of cross-domain collaborative training rounds for simulation in the input related parameters is read; The computing time points of the cross-domain collaborative training tasks of the client in each collaborative training cycle are calculated by using the simulation models of the five types of computing stages in the cross-domain collaborative training; The computing time points of the cross-domain collaborative training tasks of the server in each collaborative training cycle are calculated by using the simulation models of the five types of computing stages in the cross-domain collaborative training; The time points of the computing tasks in different stages in each collaborative training cycle are obtained according to the computing time points of the cross-domain collaborative training tasks of the client in each collaborative training cycle and the computing time points of the cross-domain collaborative training tasks of the server in each collaborative training cycle.

5. The cross-domain co-training task computation efficiency simulation method of claim 4, wherein, The step of calculating the computing time points of the cross-domain collaborative training tasks of the client in each collaborative training cycle by using the simulation models of the five types of computing stages in the cross-domain collaborative training specifically includes: In the local training stage, the computing time consumption and computing efficiency of each operator are calculated by using a method based on a computation graph representation to obtain the local training time consumption and local training computing efficiency of the client; In the data serialization or deserialization stage, the time consumption of converting and compressing Numpy format data into a binary file and the time consumption of deserializing and parsing the binary file into Numpy format data are calculated; In the data transmission stage, the time consumption of transmitting the binary file from the client to the server and the time consumption of receiving the binary file from the server are calculated; In the model evaluation and parameter updating stage, the time consumption of evaluating and updating the parameters of the model after the model parameter fusion is calculated; The computing time points of the cross-domain collaborative training tasks of the client in all stages in one collaborative training cycle are calculated.

6. The cross-domain co-training task computation efficiency simulation method of claim 5, wherein, The step of calculating the computing time points of the cross-domain collaborative training tasks of the server in each collaborative training cycle by using the simulation models of the five types of computing stages in the cross-domain collaborative training specifically includes: In the data serialization or deserialization stage, the time consumption of converting and compressing Numpy format data into a binary file and the time consumption of deserializing and parsing the binary file into Numpy format data are calculated; calculating a time consumption of the server side for transmitting the binary file to the client side, and calculating a time consumption of the server side for receiving the binary file from the client side; calculating a time consumption of the server side for receiving the model parameters and performing model parameter fusion; counting the time points of the server side for performing the cross-domain collaborative training task in all stages in a collaborative training cycle.

7. The cross-domain co-training task computation efficiency simulation method of claim 6, wherein, The step of calculating the computing efficiency index of the cross-domain collaborative training task according to the time points in all collaborative training cycles comprises: counting the total time points of the client side for performing the cross-domain collaborative training task in all stages in all collaborative training cycles, and counting the total time points of the server side for performing the cross-domain collaborative training task in all stages in all collaborative training cycles; calculating the computing efficiency index of the client side and the server side for performing the cross-domain collaborative training task in all collaborative training cycles according to the total time points of the client side for performing the cross-domain collaborative training task in all stages in all collaborative training cycles, and according to the total time points of the server side for performing the cross-domain collaborative training task in all stages in all collaborative training cycles. 8.The cross-domain coordinated training task computation efficiency simulation method according to claim 3, characterized in that, The related parameters comprise a starting time, a training round number, a collaborative training round number, a deep learning model, a number of computing devices, a communication bandwidth between computing cards, a distributed training strategy, a data set, a data volume, a local training hyperparameter, a model parameter volume, a communication delay, a communication bandwidth, a parameter fusion strategy, and an evaluation data volume. 9.The cross-domain coordinated training task computation efficiency simulation method of claim 7, wherein, The computing efficiency index comprises a total time consumption, a computing power utilization efficiency, a communication time consumption proportion, a data serialization and deserialization time consumption proportion.

10. A cross-domain co-training task computing efficiency simulation system, characterized in that, The cross-domain collaborative training task computing efficiency simulation system is applied to the cross-domain collaborative training task computing efficiency simulation method in any one of claims 1-9, and the cross-domain collaborative training task computing efficiency simulation system comprises: a parameter input module configured to input a cross-domain collaborative training model and related parameters; an initialization module configured to initialize simulation models of five types of computing stages in cross-domain collaborative training according to the cross-domain collaborative training model and the related parameters; a computing time point output module configured to calculate time points of computing tasks in different stages in each collaborative training cycle according to a cross-domain collaborative training round number in the related parameters; a computing efficiency index output module configured to calculate a computing efficiency index of the cross-domain collaborative training task according to the time points in all collaborative training cycles.

11. A controller characterized by comprising: The controller comprises a memory, a processor, a display, and a cross-domain collaborative training task computing efficiency simulation program stored on the memory and executable on the processor, and the cross-domain collaborative training task computing efficiency simulation program, when executed by the processor, implements the steps of the cross-domain collaborative training task computing efficiency simulation method in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a cross-domain co-training task computing efficiency simulation program, and the cross-domain co-training task computing efficiency simulation program, when executed by the processor, implements the steps of the cross-domain co-training task computing efficiency simulation method in any one of claims 1-9.

Images