Model training simulation method and electronic equipment

Abstract

The invention discloses a model training simulation method and electronic equipment, and the method comprises the steps: obtaining a to-be-simulated and trained target model; determining processor parameters or network configuration parameters used for training the target model; determining the network configuration parameters under the condition that the training efficiency of the target model is determined to meet a first efficiency threshold value based on the processor parameters and the hierarchical parameters of the target model; or, on the basis of the network configuration parameters and the hierarchical parameters of the target model, determining the processor parameters under the condition that the training efficiency of the target model is determined to meet a second efficiency threshold value.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of electronic technology, and are related to but not limited to a model training simulation method and electronic equipment. Background Art

[0002] With the rapid development of large models, the demand for artificial intelligence (AI) computing power has increased dramatically. However, the cost of training large models is often high. Taking the generative pre-training of deep learning models GPT3 as an example, if the training is completed within a week, it will require about 600 8-card A100 servers for training. The total GPU resource consumption for each training is about 35,000 cards / day.

[0003] The high cost of training leads to a strong demand for algorithm, software or hardware redesign, usually by enhancing algorithms and software, or using hardware acceleration options. However, designing an efficient distributed training system is challenging because there are many design choices, such as parallelization strategy, GPU performance, GPU memory bandwidth, network topology, network bandwidth and scheduling strategy. In addition, these design choices are interdependent and require the coordinated design of the software and hardware of the training platform. However, in practice, it is difficult to infer the impact of changes and search for the optimal combination from a large parameter space. Therefore, a simulation-based mechanism is needed to quickly model and analyze the performance of distributed training platforms under different design spaces. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a model training simulation method and an electronic device.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a model training simulation method, comprising:

[0007] Obtain the target model to be simulated and trained;

[0008] Determine processor parameters or network configuration parameters used to train the target model;

[0009] Determining network configuration parameters when the training efficiency of the target model satisfies a first efficiency threshold based on the processor parameters and the hierarchical parameters of the target model;

[0010] or,

[0011] When it is determined based on the network configuration parameters and the hierarchical parameters of the target model that the training efficiency of the target model meets a second efficiency threshold, the processor parameters are determined.

[0012] In a second aspect, an embodiment of the present application provides a model training simulation device, comprising:

[0013] A first acquisition module is used to acquire a target model to be simulated and trained;

[0014] A first determination module is used to determine processor parameters or network configuration parameters used for training the target model;

[0015] A second determination module is used to determine the network configuration parameters when the training efficiency of the target model meets the first efficiency threshold based on the processor parameters and the hierarchical parameters of the target model;

[0016] or,

[0017] The third determination module is used to determine the processor parameters when the training efficiency of the target model meets the second efficiency threshold based on the network configuration parameters and the hierarchical parameters of the target model.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, the memory storing a computer program that can be run on the processor, the processor executing to obtain a target model to be simulated and trained; determining processor parameters or network configuration parameters used to train the target model; determining the network configuration parameters based on the processor parameters and the hierarchical parameters of the target model when the training efficiency of the target model meets a first efficiency threshold; or determining the processor parameters based on the network configuration parameters and the hierarchical parameters of the target model when the training efficiency of the target model meets a second efficiency threshold.

[0019] In a fourth aspect, an embodiment of the present application provides a storage medium storing executable instructions for implementing the above method when executed by a processor.

[0020] In a fifth aspect, a computer program product includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the steps in the above method are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1A A schematic diagram of an implementation flow of a model training simulation method provided in an embodiment of the present application;

[0022] Figure 1B A schematic diagram of a process of inputting a model into a simulator provided in an embodiment of the present application;

[0023] Figure 1C A schematic diagram of the structure of a custom model provided in an embodiment of the present application;

[0024] Figure 1D A schematic diagram of an implementation flow of determining GPU parameters provided in an embodiment of the present application;

[0025] Figure 1E A schematic diagram of an implementation flow of network parameter definition provided in an embodiment of the present application;

[0026] Figure 1F A schematic diagram of a one-dimensional network topology configuration library provided in an embodiment of the present application;

[0027] Figure 1G A schematic diagram of an implementation process of a custom network topology provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of an implementation flow of determining network configuration parameters provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of an implementation flow of determining processor parameters provided in an embodiment of the present application;

[0030] Figure 4A A schematic diagram of a simulator input and output provided in an embodiment of the present application;

[0031] Figure 4B A schematic diagram of an implementation flow of a training strategy search provided in an embodiment of the present application;

[0032] Figure 4C A schematic diagram of an implementation flow of obtaining a training strategy provided in an embodiment of the present application;

[0033] Figure 5 A schematic diagram of the structure of a model training simulation device provided in an embodiment of the present application;

[0034] Figure 6 A hardware entity schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the embodiments of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0036] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0037] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0039] The present application embodiment provides a model training simulation method, such as Figure 1A As shown, the method includes:

[0040] Step S110, obtaining a target model to be simulated and trained;

[0041] Here, the simulator for simulating model training can obtain the target model through user customization, and can also provide a model set for the user to select the target model to be simulated and trained.

[0042] In the implementation process, in order to evaluate the model performance in a fine-grained manner, a series of detailed parameters such as the shape, weight, activation, and pooling of the Deep Neural Networks (DNN) / Large Language Model (LLM) model layer can be provided to the user as the input of the custom model. For example, an offline model library can be designed, which includes the layer concepts covered by the basic scope of the industry model. Users can retrieve the required model layers in the offline model library and combine them to define a custom model structure for use as the input of the simulator.

[0043] Figure 1B A schematic diagram of a process of inputting a model into a simulator is provided in an embodiment of the present application, such as Figure 1B As shown, this can be achieved by following the steps below:

[0044] Step A, retrieving the offline model library;

[0045] Step B: Customize the model structure based on the offline model library;

[0046] Step C: input the completed customized model into the simulator.

[0047] Figure 1C A schematic diagram of a custom model provided in an embodiment of the present application is shown in FIG. Figure 1CAs shown, the structural diagram includes: residual connection & layer normalization (Add&Norm) 11, feedforward (FeedForward) 12, multi-head attention (Multi-Head Attention) 13, where,

[0048] Residual connection & layer normalization (Add&Norm)11 are two important components in the Transformer model. Residual connection is a connection technology used in deep neural networks. Through residual connection, the network can learn the residual representation between input and output. Layer Normalization is a data preprocessing technology that maps the output data of the neural network layer to a stable numerical range or distribution through linear transformation. In the Transformer model, Add&Norm can appear as an integral unit in each layer of the encoder and decoder.

[0049] FeedForward12 refers to a network structure or layer, namely a feedforward neural network or a feedforward layer. It is used to learn complex mapping relationships between input and output. In deep learning models such as Transformer, the FeedForward layer is an important component used to extract deep features of input data.

[0050] Multi-Head Attention13 is an extended form of attention mechanism that obtains the attention distribution of different subspaces of the input sequence by running multiple independent attention mechanisms in parallel. It is a key component in the Transformer model.

[0051] Figure 1C The shape parameters shown in the figure usually refer to the dimension or shape of data (such as input, weight, output, etc.) in neural networks. It describes the size of data in various dimensions and is the basis for understanding and processing data. Bias is a learnable parameter in neural networks, usually used with weights to adjust the output of neurons. The bias term is usually added after the product of weights and input data as one of the inputs of the activation function.

[0052] In the implementation process, an offline model layer library can be used, which corresponds one visual vector image to the model layer, including detailed parameters such as model layer type, shape, bias, activation function, pooling layer, etc. By dragging the icon representing the model to the interface, the complete model structure and the input and output dimensions of the model can be visually defined, such as Figure 1C shown.

[0053] In this way, an offline model is provided, which realizes most of the basic model layers, and the user can retrieve and customize the structure of the model layer in the model library according to the actual situation, and combine them as the input of the simulator for analysis.

[0054] Step S120, determining processor parameters or network configuration parameters used for training the target model;

[0055] Here, the processor parameters include at least one of the following: peak performance at different precisions, corresponding memory, bandwidth for reading memory data, etc. The network configuration parameters include at least one of the following: bandwidth, latency, efficiency, efficiency coefficient corresponding to communication semantics, and calculation bias, etc.

[0056] The processor may be at least one of the following: a graphics processing unit (GPU), a neural network processing unit (NPU), a tensor processing unit (TPU) and a central processing unit (CPU).

[0057] During the implementation process, in order to predict the model's operating performance, it is necessary to calibrate the parameters of the cluster infrastructure and network to be predicted. The type of calibrated parameters will affect the accuracy of the prediction. However, for different purposes, the types of calibrated parameters are also very different, and their accuracy has a significant impact on the estimation of model accuracy. For example, the network transmission bandwidth has different efficiency factors for data packets of different sizes.

[0058] Therefore, a calibration tool for processor and network parameters is provided. For existing measurable hardware, the tool can obtain calibration values ​​of specified parameters by performing use case tests on the hardware. For hardware that does not currently exist and cannot be measured, the tool can infer predicted calibration values ​​based on empirical values ​​in the parameter library.

[0059] For example, a GPU parameter library can be used to store the peak performance, video memory, video memory bandwidth, etc. of existing GPUs at different precisions. In actual applications, the infrastructure often cannot fully exert its computing power due to factors such as loss, competition, and power consumption. Therefore, it is necessary to calibrate the efficiency of peak performance and video memory bandwidth at different orders of magnitude, and the calibrated parameters will be stored in the parameter library.

[0060] For GPUs that are not calibrated but accessible, the corresponding parameters can be calibrated by running a standard model at different precisions, computing power, and data sizes.

[0061] NPU calibration parameters include: the processing performance of NPU corresponding to several calculation precisions (Floating Point Operations Per Second, FLOPS) and efficiency factors corresponding to different levels, memory capacity and efficiency factors corresponding to memory access levels: measurement of memory capacity, memory bandwidth, peer-to-peer (P2P) data transmission bandwidth, latency and other parameters.

[0062] Figure 1D A schematic diagram of an implementation flow of determining GPU parameters provided in an embodiment of the present application is shown as follows: Figure 1D As shown, this can be achieved by following the steps below:

[0063] Step A: Select GPU model;

[0064] Step B: Has it been calibrated?

[0065] Here, it can be determined based on the GPU parameter library whether the selected GPU has completed parameter calibration.

[0066] During the implementation process, when it is determined that the selected GPU is a GPU in the GPU parameter library that has completed parameter calibration, step J is executed; when it is determined that the selected GPU is not in the GPU parameter library, step C is executed.

[0067] Step C: Is the GPU accessible?

[0068] Here, it may be determined whether the GPU can be accessed.

[0069] During the implementation process, if it is determined that the GPU is accessible, step E is executed; if it is determined that the GPU is not accessible, step D is executed.

[0070] Step D: Reasoning based on empirical values;

[0071] During the implementation process, if the GPU cannot be measured, the predicted calibration value can be obtained by reasoning based on the empirical value of the parameter library.

[0072] Step E: Select a standard model in the offline model library to measure the parameters of the GPU;

[0073] For GPUs that are not calibrated but accessible, the corresponding parameters can be calibrated by running a standard model at different precisions, computing power, and data sizes.

[0074] Step F: Check whether the calculation accuracy has been traversed;

[0075] If it is determined that the calculation accuracy traversal is completed, execute step G; if the calculation accuracy traversal is not completed, continue to perform the calculation accuracy traversal.

[0076] Step G, obtaining the calculation accuracy calibration value under different computing power;

[0077] Step H: Check whether the video memory bandwidth has been traversed;

[0078] If it is determined that the video memory bandwidth traversal is completed, step I is executed; if the video memory bandwidth traversal is not completed, the video memory bandwidth traversal is continued.

[0079] Step I: obtain the memory bandwidth calibration value under different data sizes.

[0080] Step J: Output GPU parameters.

[0081] Here, the model training performance is also closely related to the network topology, and the performance under different network topologies and corresponding collective communication algorithms varies greatly. The embodiment of the present application provides a network topology definer that can be customized by developers. Developers can determine the running communication flow of the model on the network by designing a multi-dimensional network topology structure and communication algorithms on different topologies.

[0082] In this way, executing the above process can achieve that for existing measurable hardware, the calibration values ​​of specified parameters can be obtained by performing use case tests on the hardware; for hardware that does not currently exist, the predicted calibration values ​​can be inferred based on the empirical values ​​of the parameter library.

[0083] Figure 1E A schematic diagram of an implementation flow of a network parameter definition provided in an embodiment of the present application, such as Figure 1E As shown, this can be achieved by following the steps below:

[0084] Step A, inputting the network topology dimension, the specific dimension network topology and the specific dimension communication algorithm into the network topology definer;

[0085] During implementation, you can use Figure 1FThe one-dimensional network topology configuration library shown is used to build a multi-dimensional network topology, which includes the most basic network topology structures: ring network topology (Ring) 14, fully connected network topology (Fully-Connected) 15 and switch network topology (Switch) 16. By customizing the network topology dimension, sequence, and bandwidth and delay of each dimension, a two-dimensional, three-dimensional, or even N-dimensional network topology can be built from a one-dimensional network topology. Among them, the ring network topology is a closed link structure in which each node is connected to two adjacent nodes through a link. The fully connected network topology refers to a structure in which each node in the network is directly connected to all other nodes. The switch network topology refers to a network structure in which multiple nodes are connected through a switch.

[0086] Step B, generating a multi-dimensional network topology;

[0087] Step C: predict the number of algorithm hops.

[0088] The algorithm hop count in network topology is a key performance indicator, reflecting the number of routers or links that a message needs to pass through from the source node to the destination node in the network. Hop count refers to the number of routers or links that a message needs to pass through from the source to the destination. In network topology, routing from one node to the next node is called a hop. Therefore, the hop count can be simply understood as the number of "hops" that a message needs to pass through in the network.

[0089] In the implementation process, the algorithm hop count can be predicted based on the structural information of the generated multi-dimensional network topology. By designing the multi-dimensional network topology structure and the communication algorithms on different topologies, the running communication flow of the model on the network can be determined.

[0090] Figure 1G A schematic diagram of a process flow for implementing a custom network topology provided in an embodiment of the present application is shown in FIG. Figure 1G As shown, this can be achieved by following the steps below:

[0091] Step A: define a network dimension N, where N is an integer greater than or equal to 1;

[0092] Step B, processing the i-th dimension of the network, where i is an integer less than or equal to N;

[0093] Step C, determine whether i is less than N;

[0094] When it is determined that the i-th dimension of the network being processed is less than the N-th dimension, step D is executed; when it is determined that the i-th dimension of the network being processed is not less than the N-th dimension, that is, equal to the N-th dimension, step G is executed.

[0095] Step D, defining the number of nodes in the i-th dimension;

[0096] Step E, defining the i-th dimension topology type;

[0097] Here, topology types include Figure 1F The most basic network topologies shown are: a ring network topology (Ring) 14, a fully connected network topology (Fully-Connected) 15 and a switch network topology (Switch) 16.

[0098] Step F: Define the bandwidth and delay between nodes in the i-th dimension.

[0099] In this way, executing the above process can realize the simulation of any network topology.

[0100] During implementation, step S130 is executed when the processor parameters are determined; and step S140 is executed when the network configuration parameters are determined.

[0101] Step S130: determining the network configuration parameters when it is determined based on the processor parameters and the hierarchical parameters of the target model that the training efficiency of the target model meets a first efficiency threshold;

[0102] or,

[0103] Here, the first efficiency threshold can be set according to the actual training requirements. The hierarchical parameters of the target model can be the layer size, activation value size and weight size of each layer of the model under a specific parallel optimization strategy determined by disassembling the structure of the model. Among them, the parallel strategy includes at least one of the following: tensor parallel strategy, data parallel strategy, pipeline parallel strategy, batch size strategy, micro-batch size strategy, optimizer sharding strategy, redo strategy, etc. Tensor parallelism is a specific model parallel strategy that distributes tensor operations of a single neural network layer to multiple GPUs to share the computing burden and memory requirements among multiple devices. Data parallelism divides the data set into multiple subsets, each of which is assigned to a different computing node (GPU), and each node has a complete model copy. Pipeline parallelism divides the model into multiple stages in layer or module order, and each stage is assigned to a different computing node to form a pipeline. Batch Size refers to the number of samples used in each iteration during neural network training. Specifically, during the training process, the model will randomly select a portion of samples from the training data set for calculation and parameter update according to the specified Batch Size. Micro-Batch Size is a finer-grained batch processing parameter used to further divide data within a single GPU or computing node. In deep learning parallel computing, the optimizer is responsible for updating the parameters of the model to minimize the loss function. The optimizer's sharding strategy involves multiple aspects such as parameter grouping, gradient aggregation, and model update. The Redo strategy may involve multiple aspects such as repetition and iteration of the training process, recovery from training failures, model evaluation and verification, etc. in large model training.

[0104] During the implementation process, based on the acquired processor parameters and the hierarchical parameters of the target model, the network configuration parameters are determined when it is determined that the training efficiency of the target model can meet the first efficiency threshold. The processor parameters include at least one of the following: peak performance at different precisions, corresponding memory, bandwidth for reading memory data, etc. The network configuration parameters include at least one of the following: bandwidth, delay, efficiency, efficiency coefficient corresponding to communication semantics, and calculation bias, etc.

[0105] Step S140: Determine the processor parameters when it is determined based on the network configuration parameters and the hierarchical parameters of the target model that the training efficiency of the target model meets a second efficiency threshold.

[0106] Here, the second efficiency threshold can be set according to the actual training requirements. The hierarchical parameters of the target model can be the layer size, activation value size and weight size of each layer model under a specific parallel optimization strategy determined by disassembling the structure of the model. The second efficiency threshold can be the same as the first efficiency threshold or different from the first efficiency threshold.

[0107] During the implementation process, based on the acquired network configuration parameters and the hierarchical parameters of the target model, the processor parameters are determined when it is determined that the training efficiency of the target model can meet the second efficiency threshold.

[0108] In an embodiment of the present application, first obtain the target model to be simulated and trained; then determine the processor parameters or network configuration parameters used to train the target model; finally, determine the network configuration parameters based on the processor parameters and the hierarchical parameters of the target model when the training efficiency of the target model meets the first efficiency threshold; or, determine the processor parameters based on the network configuration parameters and the hierarchical parameters of the target model when the training efficiency of the target model meets the second efficiency threshold. In this way, a target model parameter can be input into the simulation trainer, and through simulation analysis, a simulation configuration environment that meets the efficiency threshold is obtained, including network configuration parameters and processor parameters, when the network configuration parameters or processor parameters are determined. Model performance prediction can be extrapolated to any network cluster size and hardware configuration in a very flexible way, without being limited to specific infrastructure or hardware.

[0109] In some embodiments, the above simulation model training method further includes the following steps:

[0110] Step S150, output at least one network configuration parameter and / or processor parameter used to train the target model, so as to determine a target network configuration parameter and a target processor parameter for training the target model, or to deploy a training environment for training the target model based on the target network configuration parameter and the target processor parameter.

[0111] During the implementation process, when at least one network configuration parameter and / or processor parameter used for training the target model is determined, at least one network configuration parameter and / or processor parameter corresponding to the training target model can be output, so that the simulator can determine a target network configuration parameter and / or target processor parameter for the training target model, or can complete the environmental deployment of the training target model based on the target network configuration parameter and / or target processor parameter.

[0112] In some embodiments, the network configuration parameters and / or processor parameters corresponding to each training target model can be displayed to the user in a visual form, so that the user can determine the target network configuration parameters and / or target processor parameters of a training target model based on the actual test environment, or the user can complete the environmental deployment of the training target model based on the target network configuration parameters and / or target processor parameters.

[0113] In the embodiment of the present application, the network configuration parameters and / or processor parameters used by at least one training target model are output. It can be achieved that the user can determine the target network configuration parameters and / or target processor parameters of a training target model based on the network configuration parameters and / or processor parameters used by at least one training target model, and deploy the training environment of the training target model based on the target network configuration parameters and / or target processor parameters.

[0114] In some embodiments, the above step S130 "determines the network configuration parameters when the training efficiency of the target model meets the first efficiency threshold based on the processor parameters and the hierarchical parameters of the target model" is as follows Figure 2 As shown, this can be achieved by following the steps below:

[0115] Step S210: determining the hierarchical computing time of the target model and the processor memory access time based on the processor parameters and the hierarchical parameters of the target model;

[0116] Here, we can determine the processor parameters, that is, fix the GPU computing power, and intend to search for the optimal network topology configuration within the range of feasible solutions.

[0117] The layered computing time is affected by many factors, including the size and complexity of the model, the scale of the training data, the performance of the hardware device, and the parallelization strategy.

[0118] During the implementation process, the corresponding computing FLOPs under different parallel strategies can be calculated based on the processor parameters, the shape of each layer model and the input and output size, thereby inferring the layered computing time.

[0119] Processor memory access time refers to the time difference between the processor (GPU) issuing a memory (video memory) access request and actually obtaining the data.

[0120] During implementation, the processor access memory time can be calculated based on the layer size, activation value size and weight size of each layer model under a specific parallel optimization strategy, and the calibration parameters can be processed to determine the required video memory capacity and efficiency.

[0121] Step S220: Determine the network communication time when it is determined that the training efficiency of the target model meets the first efficiency threshold based on the layered computing time and the processor access memory time;

[0122] Here, the training time of the overall target model consists of three parts: FLOPS computing time, memory access time, and communication time.

[0123] During the implementation process, the calculation FLOPS time needs to match the corresponding video memory bandwidth capacity, otherwise it will cause additional waiting time. In order to match the GPU computing power as much as possible, the communication efficiency should match the calculation time. If it is too high, the communication will occupy too many GPU resources and affect the computing efficiency. If it is too low, the GPU will wait for too much extra time for communication processing, resulting in too long overall time.

[0124] Therefore, the simulator can calculate the communication time that enables the model efficiency to reach the first efficiency threshold based on the tiered computing time and the processor access memory.

[0125] Step S230: Determine the network configuration parameters based on the network communication time.

[0126] During implementation, an optional network topology and a corresponding communication algorithm may be preset to determine implementable network configuration parameters in setting the optional network topology and the corresponding communication algorithm based on the network communication time.

[0127] In the embodiment of the present application, the hierarchical computing time and the processor access memory time of the target model are first determined based on the processor parameters and the hierarchical parameters of the target model; then the network communication time is determined based on the hierarchical computing time and the processor access memory time to determine that the training efficiency of the target model meets the first efficiency threshold; finally, the network configuration parameters are determined based on the network communication time. In this way, the corresponding network configuration parameters can be determined based on the determined processor parameters when the training efficiency of the target model meets the first efficiency threshold, providing the user with hardware and network configuration that can meet the training efficiency.

[0128] In some embodiments, the above step S210 "determining the hierarchical computing time and the processor memory access time of the target model based on the processor parameters and the hierarchical parameters of the target model" can be implemented by the following steps:

[0129] Step 211: Based on a preset parallel strategy set and the processor parameters, determine the hierarchical computing time corresponding to the hierarchical parameters of the target model under different types of parallel strategies;

[0130] During the implementation process, the parallel optimization strategy can be disassembled according to the hierarchical model structure, and the corresponding floating-point operations per second FLOPs under different parallel strategies can be calculated based on the shape of each layer of the model and the input and output size, so as to infer the hierarchical computing time. Among them, disassembly refers to starting from the 3D parallel dimension, aiming at the amount of calculation of each layer of the hierarchical model, to summarize and analyze the final 3D parallel strategy combination used. The optimization parallel strategy for optimizing spatial parameters includes at least one of the following: 3D parallel strategy, Batch Size, Micro-Batch Size, activation recalculation related parameters, pipeline parallelism, optimizer Sharding, Redo and other parallel training strategies. 3D parallel strategies include tensor parallelism, data parallelism and pipeline parallelism.

[0131] Step 212: Determine the processor memory access time based on the hierarchical parameters of the target model under the combination of different types of parallel strategies and the processor parameters.

[0132] During the implementation process, the required video memory capacity and efficiency can be calculated based on the layer size, activation value size, weight size, and processor parameters of each layer model under a specific parallel optimization strategy, that is, the processor access memory time can be determined.

[0133] In the embodiment of the present application, based on the preset parallel strategy set and processor parameters, the hierarchical computing time corresponding to the hierarchical parameters of the target model under the combination of different types of parallel strategies is determined; based on the hierarchical parameters of the target model under the combination of different types of parallel strategies and the processor parameters, the processor access memory time is determined. In this way, the hierarchical computing time corresponding to the hierarchical parameters of the target model and the processor access memory time can be determined.

[0134] In some embodiments, the above step 212 "determining the processor access memory time based on the hierarchical parameters of the target model under the combination of different types of parallel strategies and the processor parameters" can be implemented by the following steps:

[0135] Step 2121, obtaining the number of layer parameters, activation values ​​and weight parameters corresponding to the target model under the combination of different types of parallel strategies;

[0136] Here, the number of layer parameters refers to the total number of trainable parameters in a layer of a neural network, including weights and biases. The activation value is the output value of a neuron in a neural network after weighted summation and activation function processing, which determines whether the neuron should be activated and the degree of activation. The weight parameter is the parameter connecting different neurons in a neural network, which determines the contribution of the input signal to the output signal.

[0137] During the implementation process, since the layer parameters of the target model under different types of parallel strategies are different, the number of layer parameters, activation values ​​and weight parameters corresponding to the target model under different types of parallel strategies can be obtained.

[0138] Step 2122: determine the processor memory access time based on the number of layer parameters, activation values ​​and weight parameters corresponding to the target model under the combination of different types of parallel strategies and the processor parameters.

[0139] During implementation, the processor memory access time under different types of parallel strategies can be determined based on the number of layer parameters, activation values ​​and weight parameters corresponding to the target model under different types of parallel strategies, as well as processor parameters.

[0140] In the embodiment of the present application, firstly, the number of layer parameters, activation values ​​and weight parameters corresponding to the target model under the combination of different types of parallel strategies are obtained; then, the processor access memory time is determined based on the number of layer parameters, activation values ​​and weight parameters corresponding to the target model under the combination of different types of parallel strategies and the processor parameters. In this way, the processor access memory time under the combination of different types of parallel strategies can be determined.

[0141] In some embodiments, the above step S230 "determining the network parameter configuration based on the network communication time" can be implemented by the following steps:

[0142] Step 231: determining at least one preset network topology based on the network communication time, and a communication algorithm corresponding to each preset network topology;

[0143] During the implementation process, multiple optional network topologies and corresponding communication algorithms can be preset to select at least one preset network topology among the multiple optional network topologies and corresponding communication algorithms based on the network communication time, and the communication algorithm corresponding to each of the preset network topologies.

[0144] Step 232: determining the degree of overlap between computing and communication, and the amount of cross-node communication based on the communication algorithm corresponding to each of the preset network topologies;

[0145] Here, the degree of overlap between computing and communication (Overlap) refers to the partial or complete overlap in time between computing and communication operations in a distributed or parallel computing environment. This means that while computing tasks are being performed, data communication is also being transmitted, thereby improving the overall computing efficiency. When training large-scale neural networks in a distributed manner, communication volume and communication overlap are two key factors. By reducing the amount of communication between GPUs and achieving overlap between communication and computing, the training latency can be significantly reduced and the training efficiency can be improved. Cross-node communication volume refers to the amount of data transmitted between different computing nodes in a distributed or parallel computing environment.

[0146] During implementation, the overlap and cross-node communication volume can be determined based on the communication algorithm corresponding to each preset network topology. That is, the overlap degree of calculation and communication in the communication process and the size of cross-node communication volume can be determined based on the optional network topology and the collective communication algorithm used by the specific network topology, whether to segment the communication semantics, etc.

[0147] Step 233: Determine the network parameter configuration based on the overlap degree and the cross-node communication volume.

[0148] During implementation, the optimal network parameter configuration can be calculated based on the degree of overlap and the amount of cross-node communication.

[0149] In the embodiment of the present application, at least one preset network topology and the communication algorithm corresponding to each preset network topology are first determined based on the network communication time; then the overlap degree of calculation and communication and the cross-node communication volume are determined based on the communication algorithm corresponding to each preset network topology; finally, the network parameter configuration is determined based on the overlap degree and the cross-node communication volume. In this way, it is possible to subsequently determine the overlap degree of calculation and communication and the cross-node communication volume in the communication process based on the optional network topology and the collective communication algorithm used for the specific network topology, and finally calculate the network parameter configuration.

[0150] In some embodiments, the above step S140 "determines the processor parameters when the training efficiency of the target model meets the second efficiency threshold based on the network configuration parameters and the hierarchical parameters of the target model" is as follows Figure 3 As shown, this can be achieved by following the steps below:

[0151] Step S310: determining the network communication time and the degree of overlap between calculation and communication based on the network configuration parameters;

[0152] Here, the network configuration parameters are fixed, with the intention of searching for the optimal processor parameter configuration within the range of feasible solutions.

[0153] During implementation, under a specific network topology, i.e., network parameter (bandwidth, delay, etc.) configuration, the principle of matching communication efficiency with computing time should also be followed, and the network communication time and the degree of overlap between computing and communication should be determined based on the network configuration parameters.

[0154] Step S320: determining the computing power corresponding to the network communication based on the hierarchical parameters of the target model and the network configuration parameters;

[0155] During the implementation process, based on different combinations of parallel strategies, the hierarchical parameters of the target model can be allocated to the corresponding network nodes determined based on the network configuration parameters, and then the amount of data communicated between each node can be determined based on the hierarchical parameters of each node to determine the computing power corresponding to the network communication.

[0156] Step S330: Based on the network communication time, the overlap degree, and the computing power corresponding to the network communication, determine the processor parameters when the training efficiency of the target model meets the second efficiency threshold.

[0157] During the implementation process, the processing parameters used by the corresponding network communication can be obtained based on the network communication time and the computing power corresponding to the network communication, and then the processor parameters corresponding to the calculation under the condition that the training efficiency of the target model meets the second efficiency threshold can be determined based on the overlap degree. Finally, the processor parameters for training the target model can be determined based on the processor parameters used for network communication and the processing parameters corresponding to the calculation.

[0158] In some embodiments, the communication time for determining the network configuration parameters, as well as the communication efficiency and the degree of overlap between communication and computing are first calculated, thereby calculating the ratio of communication to computing power to determine the processor parameters (peak processing power, etc.) that should be matched under the same amount of computing.

[0159] In the embodiment of the present application, the network communication time, the overlap degree of calculation and communication are first determined based on the network configuration parameters; then the computing power corresponding to the network communication is determined based on the hierarchical parameters of the target model and the network configuration parameters; finally, based on the network communication time, the overlap degree and the computing power corresponding to the network communication, the processor parameters are determined when the training efficiency of the target model meets the second efficiency threshold. In this way, the corresponding processor parameters can be determined based on the determined network configuration parameters when the training efficiency of the target model meets the second efficiency threshold, providing the user with hardware and network configuration that can meet the training efficiency.

[0160] In some embodiments, the above step S310 "determining the network communication time, the degree of overlap between calculation and communication based on the network configuration parameters" can be implemented by the following steps:

[0161] Step 311, determining network intra-machine interconnection calibration parameters and inter-machine network parameters based on the network configuration parameters;

[0162] Here, the intra-machine interconnection calibration parameters refer to the computer's internal network interface and its related configurations. These parameters ensure that the computer's network interface can work properly and communicate with other devices. Inter-machine network parameters refer to the network parameters that connect different computers or devices. These parameters ensure that data can be correctly transmitted between devices in the network.

[0163] During the implementation process, the intra-machine interconnection calibration parameters and the inter-machine network parameters can be determined based on the preset network configuration parameters.

[0164] Step 312: Based on the intra-machine interconnection calibration parameters and inter-machine network parameters, determine the corresponding network communication time and the degree of overlap between calculation and communication under the combination of different types of parallel strategies.

[0165] During the implementation process, the number of inter-machine hops and inter-machine communication input and output can be determined based on the intra-machine interconnection calibration parameters and inter-machine network parameters; then the corresponding network communication time, the degree of overlap of calculation and communication under the combination of different types of parallel strategies can be determined based on the number of inter-machine hops and inter-machine communication input and output. Here, the number of inter-machine hops is a term in network communication technology, which refers to the number of intermediate nodes that information passes through in the process of transmitting from one node to another. The "hop" here can be understood as a forwarding process of a data packet in the network. The more hops, the longer the path for the data packet to be transmitted in the network, and it may need to pass through more devices such as routers or switches. Inter-machine communication input and output refers to the process of sending and receiving data during network communication. This includes steps such as encoding, transmission, and decoding of data, as well as various mechanisms to ensure data integrity and reliability.

[0166] In the embodiment of the present application, the network's intra-machine interconnection calibration parameters and inter-machine network parameters are first determined based on the network configuration parameters; then, based on the intra-machine interconnection calibration parameters and inter-machine network parameters, the corresponding network communication time, the overlap degree of calculation and communication under the combination of different types of parallel strategies are determined. In this way, the network communication time, the overlap degree of calculation and communication under the combination of different types of parallel strategies can be obtained based on the network configuration parameters.

[0167] In some embodiments, the above step S320 "determining the computing power corresponding to the network communication based on the hierarchical parameters of the target model and the network configuration parameters" can be implemented by the following steps:

[0168] Step 321, determining the execution network nodes corresponding to the hierarchical parameters of the target model under the combination of different types of parallel strategies based on the network configuration parameters;

[0169] Here, due to the combination of different types of parallel strategies, the execution nodes corresponding to the hierarchical parameters of the target model can be determined, and the network configuration parameters can also determine the topology of the network, that is, the configuration of the nodes.

[0170] During the implementation process, the corresponding execution network nodes of the hierarchical parameters of the target model under the combination of different types of parallel strategies can be determined based on the network configuration parameters.

[0171] Step 322: Determine the computing power corresponding to the network communication based on the corresponding execution network nodes under the combination of different types of parallel strategies based on the hierarchical parameters of the target model.

[0172] Here, based on the determined execution network nodes and the amount of operation data corresponding to each execution network node based on the hierarchical parameters, the amount of operation data transmitted between nodes can be determined to further determine the computing power corresponding to the network communication.

[0173] In the embodiment of the present application, firstly, the execution network nodes corresponding to the hierarchical parameters of the target model under the combination of different types of parallel strategies are determined based on the network configuration parameters; then, the computing power corresponding to the network communication is determined based on the execution network nodes corresponding to the hierarchical parameters of the target model under the combination of different types of parallel strategies. In this way, the computing power corresponding to the network communication under the combination of different types of parallel strategies can be determined based on the network configuration parameters and different types of parallel strategies.

[0174] In some embodiments, the above step S330 "determining the processor parameters when the training efficiency of the target model meets the second efficiency threshold based on the network communication time, the overlap degree and the computing power corresponding to the network communication" can be implemented by the following steps:

[0175] Step 331: Determine the proportion of processor computing power occupied by communication based on the network communication time and the overlap degree;

[0176] Here, the proportion of processor computing power occupied by communication can represent the proportion of computing power occupied by communication in the total processor computing power. The larger the amount of communication data, the more data the processor needs to process, which takes up more computing power. For example, when performing large-scale data transmission, the processor needs to process a large number of data packets, resulting in a significant increase in the processor computing power occupied. In order to match the processor computing power as much as possible, the communication efficiency should match the computing time. If it is too high, communication will occupy too many processor resources and affect computing efficiency. If it is too low, the processor will wait too much extra time for communication processing, resulting in too long overall time.

[0177] In the implementation process, the proportion of communication occupying the processor computing power can be determined based on the network signal time and the degree of overlap, and multiple factors can be considered comprehensively. Through performance monitoring, benchmarking and mathematical modeling, the occupation of processor computing power by communication can be more accurately evaluated.

[0178] Step 332: Based on the computing power corresponding to the network communication and the ratio of the processing computing power occupied by the communication, determine the processor parameters when the training efficiency of the target model meets the second efficiency threshold.

[0179] Here, when determining the computing power corresponding to network communication and the ratio of the processing computing power occupied by communication, the matching processor parameters can be determined based on the constraint that the training efficiency of the target model meets the second efficiency threshold.

[0180] In the embodiment of the present application, the proportion of the processor computing power occupied by communication is first determined based on the network communication time and the degree of overlap; then, based on the computing power corresponding to the network communication and the proportion of the processing computing power occupied by communication, the processor parameters are determined when the training efficiency of the target model meets the second efficiency threshold. In this way, the corresponding processor parameters can be determined when the training efficiency of the target model meets the second efficiency threshold.

[0181] Figure 4A A schematic diagram of a simulator input and output provided in an embodiment of the present application, such as Figure 4A As shown in the figure, the schematic diagram includes: the input of the simulator: model structure definition, GPU configuration, network configuration and parallel strategy; the output of the simulator: optimal configuration, performance evaluation and execution details, among which,

[0182] The model structure definition can utilize a designed offline model to implement most of the basic model layers. Users can retrieve and customize the structure of the model layer in the model library according to actual conditions, and combine them as input to the simulator for analysis.

[0183] For GPU configuration, a calibration tool for GPU and network parameters can be designed. For existing measurable hardware, the tool can obtain the calibration values ​​of specified parameters by performing use case tests on the hardware. For hardware that does not currently exist, the tool can infer the predicted calibration values ​​based on the empirical values ​​of the parameter library.

[0184] Network configuration can utilize a designed network topology definer. Developers can determine the communication flow of the model on the network by designing a multi-dimensional network topology structure and communication algorithms on different topologies.

[0185] The parallel strategy can be a collection of all parallel strategies that can be implemented.

[0186] During the implementation process, the model structure definition, GPU configuration, network configuration and parallel strategy can be input into the simulator, so that the simulator can perform parameter search in the huge parallel training strategy space, network infrastructure space, and GPU configuration space according to the customized model structure. The AI ​​simulator can determine the optimal system design in the joint space of software and hardware under given constraints.

[0187] Figure 4B A schematic diagram of an implementation flow of a training strategy search provided in an embodiment of the present application, such as Figure 4B As shown, this can be achieved by following the steps below:

[0188] The first step is to disassemble the parallel optimization strategy for the hierarchical model structure, and calculate the corresponding floating-point operations per second (FLOPs) under different parallel strategies based on the shape of each layer of the model and the input and output size, so as to estimate the computing time;

[0189] Step 2: Calculate the required video memory capacity and efficiency based on the layer size, activation value size, weight size, and NPU calibration parameters of each layer model under a specific parallel optimization strategy;

[0190] The third step is to calibrate the intra-machine interconnection parameters and inter-machine network parameters, as well as the intra-machine and inter-machine communication volume under a specific parallel optimization strategy and a specific network topology algorithm;

[0191] The fourth step is to search and compare the optimal parameters in the feasible optimization space parameters to obtain the best performance of the model under a specific parallel optimization strategy and network topology algorithm configuration.

[0192] Figure 4C A schematic diagram of an implementation flow of obtaining a training strategy provided in an embodiment of the present application is shown in FIG. Figure 4C As shown, this can be achieved by following the steps below:

[0193] Step S410, obtaining the GPU parameters, custom model and 3D parallel strategy that have been calibrated in the custom network topology allocation;

[0194] During implementation, a GPU parameter library may be set, and calibrated GPU parameters may be determined based on the GPU parameter library.

[0195] Step S420, determining the best training strategy based on allocating calibrated GPU parameters, custom models and 3D parallel strategies in the custom network topology.

[0196] During the implementation process, the cards are divided into nodes of networks of different dimensions according to the parallel strategy, and then it is determined whether the parallel strategy includes tensor parallelism, data parallelism and pipeline parallelism.

[0197] When tensor parallelism is determined, calculate the FLPOs and communication traffic in tensor parallelism

[0198] Based on the foregoing embodiments, the embodiments of the present application provide a model training simulation device, which includes the modules included, each module includes sub-modules, each sub-module includes a unit, which can be implemented by a processor in an electronic device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0199] Figure 5 A schematic diagram of the structure of the model training simulation device provided in the embodiment of the present application is shown in FIG. Figure 5 As shown, the device 500 includes:

[0200] A first acquisition module 510 is used to acquire a target model to be simulated and trained;

[0201] A first determination module 520, used to determine processor parameters or network configuration parameters used to train the target model;

[0202] A second determination module 530 is used to determine the network configuration parameters when determining that the training efficiency of the target model meets a first efficiency threshold based on the processor parameters and the hierarchical parameters of the target model;

[0203] or,

[0204] The third determination module 540 is used to determine the processor parameters when it is determined that the training efficiency of the target model meets the second efficiency threshold based on the network configuration parameters and the hierarchical parameters of the target model.

[0205] In some embodiments, the second determination module 530 includes a first determination submodule, a second determination submodule and a third determination submodule, wherein the first determination submodule is used to determine the hierarchical computing time and the processor access memory time of the target model based on the processor parameters and the hierarchical parameters of the target model; the second determination submodule is used to determine the network communication time when the training efficiency of the target model meets the first efficiency threshold based on the hierarchical computing time and the processor access memory time; the third determination submodule is used to determine the network configuration parameters based on the network communication time.

[0206] In some embodiments, the first determination submodule includes a first determination unit and a second determination unit, wherein the first determination unit is used to determine the hierarchical computing time corresponding to the hierarchical parameters of the target model under the combination of different types of parallel strategies based on a preset parallel strategy set and the processor parameters; the second determination unit is used to determine the processor access memory time based on the hierarchical parameters of the target model under the combination of different types of parallel strategies and the processor parameters.

[0207] In some embodiments, the second determination unit includes an acquisition subunit and a determination subunit, wherein the acquisition subunit is used to acquire the number of layer parameters, activation values ​​and weight parameters corresponding to the target model under the combination of different types of parallel strategies; and the determination subunit is used to determine the processor access memory time based on the number of layer parameters, activation values ​​and weight parameters corresponding to the target model under the combination of different types of parallel strategies and the processor parameters.

[0208] In some embodiments, the third determination submodule includes a third determination unit, a fourth determination unit and a fifth determination unit, wherein the third determination unit is used to determine at least one preset network topology based on the network communication time, and a communication algorithm corresponding to each preset network topology; the fourth determination unit is used to determine the degree of overlap between calculation and communication, and the cross-node communication volume based on the communication algorithm corresponding to each preset network topology; the fifth determination unit is used to determine the network parameter configuration based on the degree of overlap and the cross-node communication volume.

[0209] In some embodiments, the third determination module 540 includes a fourth determination submodule, a fifth determination submodule and a sixth determination submodule, wherein the fourth determination submodule is used to determine the network communication time, the degree of overlap of calculation and communication based on the network configuration parameters; the fifth determination submodule is used to determine the computing power corresponding to the network communication based on the hierarchical parameters of the target model and the network configuration parameters; the sixth determination submodule is used to determine the processor parameters when the training efficiency of the target model meets the second efficiency threshold based on the network communication time, the degree of overlap and the computing power corresponding to the network communication.

[0210] In some embodiments, the fourth determination submodule includes a sixth determination unit and a seventh determination unit, wherein the sixth determination unit is used to determine the intra-machine interconnection calibration parameters and inter-machine network parameters of the network based on the network configuration parameters; the seventh determination unit is used to determine the corresponding network communication time, the degree of overlap of calculation and communication under a combination of different types of parallel strategies based on the intra-machine interconnection calibration parameters and inter-machine network parameters.

[0211] In some embodiments, the fifth determination submodule includes an eighth determination unit and a ninth determination unit, wherein the eighth determination unit is used to determine the execution network nodes corresponding to the hierarchical parameters of the target model under the combination of different types of parallel strategies based on the network configuration parameters; and the ninth determination unit is used to determine the computing power corresponding to the network communication based on the execution network nodes corresponding to the hierarchical parameters of the target model under the combination of different types of parallel strategies.

[0212] In some embodiments, the model simulation training device also includes an output module for outputting at least one network configuration parameter and / or processor parameter used to train the target model, so as to enable determination of a target network configuration parameter and a target processor parameter for training the target model, or to enable deployment of a training environment for training the target model based on the target network configuration parameter and the target processor parameter.

[0213] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0214] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable an electronic device (which can be a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0215] Correspondingly, an embodiment of the present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the model training simulation method provided in the above embodiment are implemented.

[0216] Correspondingly, an embodiment of the present application provides an electronic device, Figure 6 A hardware entity diagram of an electronic device provided in an embodiment of the present application, such as Figure 6As shown, the hardware entity of the device 600 includes: a memory 601 and a processor 602, wherein the memory 601 stores a computer program that can be run on the processor 602, and when the processor 602 executes the program, the steps in the model training simulation method provided in the above embodiment are implemented.

[0217] The memory 601 is configured to store instructions and applications executable by the processor 602, and can also cache data to be processed or processed by the processor 602 and various modules in the electronic device 600 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0218] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0219] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0220] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0221] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0222] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0223] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0224] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0225] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable an electronic device (which can be a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0226] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0227] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0228] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0229] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model training simulation method, the method comprising: Obtain the target model to be simulated and trained; Determining processor parameters or network configuration parameters used to train the target model; Determining the network configuration parameters when determining based on the processor parameters and the hierarchical parameters of the target model that the training efficiency of the target model meets a first efficiency threshold; or, When it is determined based on the network configuration parameters and the hierarchical parameters of the target model that the training efficiency of the target model meets a second efficiency threshold, the processor parameters are determined.

2. The method according to claim 1, wherein determining the network configuration parameters when the training efficiency of the target model meets a first efficiency threshold based on the processor parameters and the hierarchical parameters of the target model comprises: Determine the hierarchical computing time and processor memory access time of the target model based on the processor parameters and the hierarchical parameters of the target model; Determining the network communication time when it is determined based on the layered computing time and the processor access memory time that the training efficiency of the target model meets the first efficiency threshold; The network configuration parameters are determined based on the network communication time.

3. The method according to claim 2, wherein determining the hierarchical computing time and the processor memory access time of the target model based on the processor parameters and the hierarchical parameters of the target model comprises: Based on a preset parallel strategy set and the processor parameters, determining the hierarchical computing time corresponding to the hierarchical parameters of the target model under different types of parallel strategies; The processor memory access time is determined based on the hierarchical parameters of the target model under the combination of different types of parallel strategies and the processor parameters.

4. The method according to claim 3, wherein the step of determining the processor memory access time based on the hierarchical parameters of the target model under the combination of different types of parallel strategies and the processor parameters comprises: Obtain the number of layer parameters, activation values ​​and weight parameters corresponding to the target model under the combination of different types of parallel strategies; The processor memory access time is determined based on the number of layer parameters, activation values ​​and weight parameters corresponding to the target model under the combination of different types of parallel strategies and the processor parameters.

5. The method according to claim 2, wherein determining the network parameter configuration based on the network communication time comprises: Determine at least one preset network topology based on the network communication time, and a communication algorithm corresponding to each of the preset network topologies; Determine the degree of overlap between computing and communication, and the amount of cross-node communication based on the communication algorithm corresponding to each of the preset network topologies; The network parameter configuration is determined based on the overlap degree and the cross-node communication volume.

6. The method of claim 1, wherein determining the processor parameter when determining that the training efficiency of the target model meets a second efficiency threshold based on the network configuration parameter and the hierarchical parameter of the target model comprises: Determining network communication time, and a degree of overlap of computation and communication based on the network configuration parameters; Determining a computing power corresponding to network communication based on the hierarchical parameters of the target model and the network configuration parameters; Based on the network communication time, the overlap degree and the computing power corresponding to the network communication, determine the processor parameters when the training efficiency of the target model meets the second efficiency threshold.

7. The method of claim 6, wherein determining the network communication time, the degree of overlap of computing and communication based on the network configuration parameters comprises: Determine network intra-machine interconnection calibration parameters and inter-machine network parameters based on the network configuration parameters; Based on the intra-machine interconnection calibration parameters and inter-machine network parameters, the corresponding network communication time and the degree of overlap of calculation and communication are determined under the combination of different types of parallel strategies.

8. The method according to claim 6, wherein determining the computing power corresponding to the network communication based on the hierarchical parameters of the target model and the network configuration parameters comprises: Determine, based on the network configuration parameters, the execution network nodes corresponding to the hierarchical parameters of the target model under the combination of different types of parallel strategies; The computing power corresponding to the network communication is determined based on the corresponding execution network nodes under the combination of different types of parallel strategies based on the hierarchical parameters of the target model.

9. The method according to any one of claims 1 to 8, further comprising: Output at least one network configuration parameter and / or processor parameter used to train the target model, so that a target network configuration parameter and a target processor parameter for training the target model can be determined, or so that a training environment for training the target model can be deployed based on the target network configuration parameter and the target processor parameter.

10. An electronic device comprises a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor executes to obtain a target model to be simulated and trained; determine the processor parameters or network configuration parameters used to train the target model; determine the network configuration parameters when it is determined based on the processor parameters and the hierarchical parameters of the target model that the training efficiency of the target model meets a first efficiency threshold; or determine the processor parameters when it is determined based on the network configuration parameters and the hierarchical parameters of the target model that the training efficiency of the target model meets a second efficiency threshold.

Images