Edge cloud-oriented distributed model training time evaluation method and device

Abstract

The invention discloses an edge cloud-oriented distributed model training time evaluation method and device. The method comprises the following steps: dividing a deep learning model processing load flow into a data movement stage, a model calculation stage and a communication stage; key features of a distributed cluster and model features of different deep learning models are collected, a data representative value of each feature is confirmed, load time overhead of three stages is measured through cross combination of the data representative values, and a combined feature experiment sample is constructed; modeling is carried out on the load time overhead of each stage, and a load time evaluation model of the deep learning model training task is obtained; constructing an error function, and solving parameters of the load time evaluation model by using a non-negative least square method by taking minimization of the error function as a target; and substituting the parameters of the load time evaluation model, and evaluating the load time overhead of the received task by using the updated load time evaluation model. According to the method, the isomerism problem of a deep learning model is solved through general hierarchical modeling.

Description

Technical Field

[0001] The present invention belongs to the field of edge computing technology, and specifically relates to a distributed model training time evaluation method and device for edge cloud. Background Art

[0002] With the advent of the era of large models, artificial intelligence services based on large generative models such as audio and video have emerged, and deep learning tasks such as model training and reasoning have gradually become the main load in various distributed computing clusters today. However, considering that these deep learning tasks require high resources and take a long time, the allocation of tasks and the scheduling of resources are crucial. In order to solve this problem automatically and intelligently, it is very necessary to understand and predict the performance of the load before task deployment and resource allocation.

[0003] Existing load performance (performance here is a general term, and in this article it specifically refers to load completion time) prediction technologies are often oriented towards traditional audio and video or general machine learning tasks, and lack solutions that are suitable for deep learning distributed training scenarios. To achieve load performance prediction of heterogeneous deep learning models at different cluster scales, the following challenges need to be addressed:

[0004] 1. Heterogeneity of deep learning models: On edge cloud platforms, different types of deep learning models often need to be deployed to provide artificial intelligence services. These models involve multiple modalities and structures, such as the Vision Transformer (ViT) model in visual tasks and the BERT model in natural language processing. Edge clouds need to run these diverse models in a limited resource environment, and these models differ in computing levels, composition structures, and resource requirements, resulting in significant heterogeneity of the models. Due to this heterogeneity, existing load performance prediction methods are often only applicable to specific types of models and are difficult to generalize to other model types, which poses a challenge to the unified performance optimization of edge cloud platforms;

[0005] 2. Diversity of distributed cluster scale: The resources of edge cloud platforms are significantly heterogeneous. Different nodes have different hardware performance, network conditions, and storage resources, which leads to diversity in cluster scale and capabilities. In deep learning tasks, the scale of distributed clusters is a key factor affecting load performance. However, due to the complexity or high cost of the model, existing methods can usually only build performance models based on clusters of specific, limited scales, which makes it difficult to adapt to distributed cluster environments of varying sizes in edge cloud platforms. This diversity of resources and cluster scales makes load performance prediction and optimization more complicated;

[0006] 3. Complexity of deep learning load performance: The training load of deep learning models often consists of multiple parts. The influencing factors and performance models of each part often follow different paradigms. Existing methods focus on modeling the computational process in model training and lack consideration of the complete load process. Summary of the invention

[0007] In view of the above problems, the present invention proposes a distributed model training time evaluation method and device for edge cloud, which can predict the training load performance of deep learning models on distributed clusters and can quickly migrate and generalize to different model structures and various cluster sizes. In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0008] A distributed model training time evaluation method for edge cloud includes the following steps:

[0009] S1, divides the process of deep learning model processing task load into data movement stage, model calculation stage and communication stage based on load time overhead;

[0010] S2, collect the key features of the distributed cluster and the model features of different deep learning models, use the quartile method to confirm the data representative value of each model feature based on the collected features, cross-combine the obtained data representative values ​​to measure the load time overhead of the data movement stage, model calculation stage and communication stage, and construct a combined feature experimental sample;

[0011] S3, based on the data processing process of each stage in the data movement stage, the model calculation stage and the communication stage and the model structure of the deep learning model, the load time overhead of each stage is modeled to obtain the load time evaluation model of the deep learning model training task;

[0012] S4, constructing an error function according to the load time evaluation model obtained in step S3 and the combined characteristic experimental sample obtained in step S2, and solving the parameters of the load time evaluation model by using the non-negative least squares method with the goal of minimizing the error function;

[0013] S5, substituting the parameters of the load time evaluation model obtained in step S4 into the load time evaluation model, and using the updated load time evaluation model to evaluate the load time overhead of the received task.

[0014] The step S2 comprises the following steps:

[0015] S2.1, collect key features of distributed clusters and model features of different deep learning models, the key features include the number of nodes and the number of GPUs, and the model features include batch size, image segmentation block size, hidden layer dimension of the model, dimension of the feature subspace of the self-attention mechanism, and the number of heads;

[0016] S2.2, constructing a load parameter measurement sample for each deep learning model based on all feature data obtained in step S2.1;

[0017] S2.3, using the quartile method to confirm at least two data representative values ​​of each feature in the load parameter measurement sample obtained in step S2.2;

[0018] S2.4, cross-combining the data representative values ​​obtained in step S2.3 to measure the load time overhead of each stage under each combination;

[0019] S2.5, construct a combined feature experimental sample based on the data obtained in step S2.2 and step S2.4.

[0020] In step S3, the expression of the load time evaluation model of the deep learning model training task is:

[0021] T total =T data +T compute +T communication ;

[0022] Where, T data represents the load time overhead of the data movement phase; T compute represents the load time overhead of the model calculation phase, T communication represents the load time overhead of the communication phase;

[0023] The load time overhead T of the data movement phase data The modeling formula is:

[0024]

[0025] In the formula, batch_size represents the input batch size, nodes represents the number of edge devices that process the load, i.e., the task, and θ represents the effective total bandwidth of the PCIe bus;

[0026] The load time overhead of the communication phase is T communication The modeling formula is:

[0027]

[0028] In the formula, dim represents the hidden layer dimension of the deep learning model, and θ3 represents the total network bandwidth of the edge device when processing the load;

[0029] The load time cost of the model calculation phase is T compute The modeling formula is:

[0030]

[0031] In the formula, Represents the load calculation time of the i-th network layer in the deep learning model.

[0032] The network layer includes a Rerrange layer, a Linear layer, a Dropout layer, a LayerNorm layer, an Attention module and / or a Softmax layer. The load calculation time of the Rerrange layer is T compute，Reg Indicates that the corresponding calculation formula is:

[0033]

[0034] Where N represents the size of the input tensor of the Rerrange layer, i.e., the number of elements, b represents the size of each element, θ1 represents the total GPU computing power of the edge device that processes the load, and GPUS represents the number of GPUs on each edge device.

[0035] The load calculation time of the Linear layer is T compute，Lin Indicates that the corresponding calculation formula is:

[0036]

[0037] Where dim1 represents the dimension of the first hidden layer of the linear layer, and dim2 represents the dimension of the second hidden layer of the linear layer;

[0038] The load calculation time of the Dropout layer is T compute，Dro Indicates that the corresponding calculation formula is:

[0039]

[0040] The load calculation time of the LayerNorm layer is T compute，Lay Indicates that the corresponding calculation formula is:

[0041]

[0042] The load calculation time of the Softmax layer is T compute，Sof Indicates that the corresponding calculation formula is:

[0043]

[0044] In the formula, patch_size represents the image segmentation block size;

[0045] The load calculation time of the Attention module is T compute，Att Indicates that the corresponding calculation formula is:

[0046]

[0047] In the formula, heads represents the number of attention heads, and dim_head represents the dimension of the feature subspace of the self-attention mechanism, that is, the dimension of each head.

[0048] The data movement phase refers to the transfer of data from CPU memory to GPU memory during load processing, the model calculation phase refers to the process of loading the deep learning model to process the load, and the communication phase refers to the synchronization of the parameters of the deep learning model of each edge device in the distributed cluster.

[0049] In step S4, the expression of the error function is:

[0050] min x≥0 ||f(A,x)-b|| 2 :

[0051] Where x represents the parameter matrix constructed by all load evaluation model parameters, A represents the first feature matrix constructed based on the key features and model features in the combined feature experimental samples, b represents the second feature matrix constructed based on the load time overhead of the mobile phase, model calculation phase, and communication phase in the combined feature experimental samples, and f(·) represents the load time evaluation model.

[0052] A distributed model training time evaluation device for edge cloud, comprising:

[0053] Experimental sample collection module: used to collect key features of edge devices in distributed clusters and model features of different deep learning models on edge devices, use the quartile method to confirm the data representative value of each model feature based on the collected features, cross-combine the obtained data representative values ​​to measure the load time overhead of each process when the deep learning model processes tasks, and construct combined feature experimental samples; the process of deep learning model processing tasks includes data movement stage, model calculation stage and communication stage;

[0054] Model building module: used to model the load time overhead of each stage based on the data processing process in each stage of the data movement stage, model calculation stage and communication stage and the model structure of the deep learning model, and then build a load time evaluation model for the deep learning model training task;

[0055] Model parameter calculation module: constructs an error function based on the load time evaluation model obtained by the model construction module and the combined characteristic experimental samples obtained by the experimental sample collection module, and uses the non-negative least squares method to solve the parameters of the load time evaluation model with the goal of minimizing the error function;

[0056] Load evaluation module: used to substitute the parameters obtained by the model parameter calculation module into the load time evaluation model obtained by the model construction module, and use the updated load time evaluation model to evaluate the load time overhead of the received task.

[0057] Beneficial effects of the present invention:

[0058] 1. A complete set of deep learning model training load splitting methods is proposed, including data movement, communication overhead, and model calculation overhead, to cover the complete load process, so as to establish an accurate performance prediction model for each load process through structural observation and mathematical modeling of deep learning model training load, so that the overall load performance prediction accuracy reaches more than 75%;

[0059] 2. By characterizing the cluster scale, we extract key features that reflect the cluster scale, such as the number of nodes and the number of GPUs, and add them to the load performance prediction, thereby generalizing the modeling to clusters of different scales and solving the problem of the diversity of distributed cluster scales;

[0060] 3. Through general hierarchical modeling, various deep learning models can be combined with finer-grained computing layers, thereby generalizing the modeling to various heterogeneous models and solving the heterogeneity problem of deep learning models;

[0061] 4. Provide a complete set of data-driven theories and implementation methods that can accurately and efficiently predict the training time of different deep learning model loads under distributed clusters of different sizes, provide insights and guidance for individual users who need to select resources or controllers of distributed computing clusters, and optimize their decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] Ring-All-Reduce algorithm: is an efficient communication method for gradient synchronization in distributed deep learning. It organizes all nodes (or GPUs) into a ring topology, and the gradient data is transmitted and aggregated between nodes in the form of blocks. Each node receives part of the gradient data from the adjacent node and sends part of its own data to the next node at the same time. After multiple iterations, each node can get the complete gradient value of all nodes to update the parameters. Ring-All-Reduce reduces communication overhead and improves the efficiency of distributed training by avoiding global communication.

[0066] Least Squares Method: It is an optimization technique used to fit model parameters in regression analysis. The basic idea is to find the optimal parameter value by minimizing the sum of squares of the errors between the observed data and the model predictions. Specifically, given a set of observed data and a model, the least squares method adjusts the model parameters so that the deviation between the model predictions and the actual observed values ​​is as small as possible, thereby obtaining the best fit. This method is widely used for parameter estimation of linear regression and nonlinear regression models.

[0067] A distributed model training time evaluation method for edge cloud, such as Figure 1 As shown, the following steps are included:

[0068] S1, divides the process of deep learning model processing task load into data movement stage, model calculation stage and communication stage based on load time overhead;

[0069] This application decomposes the performance overhead during the distributed deep learning workload into the following three main parts according to the processing flow:

[0070] 1. Data movement: Since the computing speed of the GPU is always higher than the speed at which data is transferred from the CPU memory to the GPU memory, data transfer becomes a potential bottleneck, which may hinder the full utilization of GPU computing resources and thus affect training efficiency;

[0071] 2. Model computation overhead: After the deep learning model is loaded, it will perform forward propagation and calculate the output layer by layer, calculate the loss function, and perform back propagation to calculate the gradient of each layer. This process involves a large number of matrix operations and complex computational processes;

[0072] 3. Communication consumption: In a distributed training environment, the parameters of the deep learning model on each node must be synchronized to ensure that all nodes perform calculations based on the latest global parameters and update the parameters of the neural network.

[0073] For the above three parts, the embodiments of the present application respectively use data movement time, load calculation time and communication time to evaluate the load time overhead of each stage. According to the total load time overhead, the performance of each deep learning model in processing the load can be evaluated.

[0074] S2, collects key features of distributed clusters and model features of different deep learning models, uses the quartile method to confirm the data representative value of each model feature based on the collected features, cross-combines the obtained data representative values ​​to measure the load time overhead of the data movement stage, model calculation stage, and communication stage, and constructs a combined feature experimental sample, including the following steps:

[0075] S2.1, collect key features of distributed clusters and model features of different deep learning models;

[0076] The distributed cluster refers to a system composed of several edge devices, i.e., edge nodes, which can communicate with each other. The key features described in this application include the number of nodes and the number of GPUs (GPUS) per node. The number of nodes refers to the maximum number of edge devices in the system that process a task when a certain edge device receives a task, hereinafter referred to as a computing node. The key features of the distributed cluster and the model features of different types of deep learning models can be obtained as experimental metadata before the experiment is carried out.

[0077] The different deep learning models include a vision ViT (Vision Transformer) model, a natural language Bert (Bidirectional Encoder Representations from Transformers) model, a GPT-2 model, and a LLama-3B model. Model features include batch size (batch_size), image segmentation block size (patch_size), hidden layer dimension of the model (dim), dimension of the self-attention mechanism feature subspace (dim_head), and number of heads (heads), and may also include the number of data loading processes (num_workers).

[0078] S2.2, constructing a load parameter measurement sample for each deep learning model based on the feature data obtained in step S2.1;

[0079] Taking the ViT model as an example, the load parameter measurement sample is: {nodes=4, batch_size=256, patch_size=8, dim=256, dim_head=32, heads=8, num_workers=8, GPUS=8}, indicating that the number of nodes is 4, 256 data items are processed in each batch, the image segmentation block size is 8, the hidden layer dimension is 256, the dimension of the self-attention mechanism feature subspace is 32, the number of heads is 8, the number of data loading processes is 8, and the number of GPUs is 8.

[0080] S2.3, using the quartile method to confirm at least two data representative values ​​of each feature in the load parameter measurement sample obtained in step S2.2;

[0081] Since deep learning training experiments often require high hardware resources and consume a large amount of time, it is necessary to complete multiple sets of repeated experiments at the lowest possible cost to collect enough feature and performance indicator samples to support subsequent modeling. The purpose of this step is to set multiple representative and high-information value ranges for each set of features.

[0082] In order to make the gap between feature samples sufficient and representative, this application adopts the quartile method to select the representative value of the feature, that is, for any feature (take batch size batch_size as an example here), the 1 / 4, 2 / 3 quantiles of all batch_size feature data in the load parameter measurement sample are respectively used as three data representative values. Preferably, it is also possible to combine the reasonable value range of each feature data, such as the reasonable value range of the batch_size feature is [1, 1024], calculate the 1 / 4, 2 / 3 quantiles and round the data in combination with the common habits in this field, and obtain the data value of batch_size [256, 512, 768]. The selection method of other model features is similar to this method, so that 3 data representative values ​​can be set for each of the 7 features. In this embodiment, the data representative values ​​of other features are as follows: nodes is [1, 2, 4], patch_size is [4, 8, 16], dim is [256, 512, 768], heads is [4, 8, 16] (make sure dim is divisible), dim_head changes with heads to satisfy dim = dim_head*heads, num_workers is [2, 4, 8], and GPUS is [2, 4, 8].

[0083] Since modeling requires certain data samples to determine key parameters, we construct sample experiments (i.e., deep learning training tasks. This article takes Vision Transformer as an example because it is the current flagship model in the field of vision and is also the basic model of large models in the field of vision) to collect load time overhead samples at each stage of the experiment. In order to collect the performance of deep learning loads under different key features, it is necessary to repeat different experimental settings many times. However, it should be noted that since this method uses a relatively simple mathematical model, it does not require a large number of sample points to determine the basic performance model. It should be noted that, except for dim, which usually satisfies the constraint that dim = dim_head*heads, these features are independent of each other. Therefore, except for dim_head, which changes with heads to meet the above constraints, other features can be independently cross-matched to collect performance indicators through repeated experiments. Repeating the experiment is required. Experiment.

[0084] S2.4, cross-combining the data values ​​obtained in step S2.3 to measure the load time overhead of each stage under each combination;

[0085] The load time overhead can be obtained by monitoring various indicators in the deep learning model training and reasoning process through professional tools such as TensorBoard, NVIDIA Nsight and PyTorchProfiler (NVIDIA Nsight is used in this application). Hereinafter, this embodiment refers to the load time overhead of the three stages as data movement time, load calculation time and communication time respectively.

[0086] S2.5, constructing a combined feature experimental sample based on the data obtained in step S2.2 and step S2.4;

[0087] For example, the combined feature experiment sample is {nodes=4, batch_size=256, patch_size=8, dim=256, dim_head=32, heads=8, num_workers=8, GPUS=8, time_trans=0.15, time_forward=2.39, time_backward=3.72, time_optim_step=2.60, time_communication=2.52, layers_time={"Linear":0.02, "Rerrange":0.07, ...}). Among them, time_trans is the time for data to be transferred from CPU memory to GPU memory, that is, data movement time, time_forward is the forward propagation time, time_backward is the backward propagation time, time_optim_step is the time for the optimizer to update model parameters, layers_time is the layer load calculation time, Linear is the linear layer load calculation time, and Rerrange is the Rerrange layer load calculation time. Among them, time_forward, time_backward, time_optim_step and layers_time are the load calculation time, and time_communication is the communication time.

[0088] The above example shows that under 4 computing nodes, each batch has 256 data items, the image segmentation block size is 8, the hidden layer dimension is 256, the dimension of the self-attention mechanism feature subspace is 32, the number of heads is 8, the number of data loading processes is 8, the number of GPUs is 8, the time to read data from CPU storage to GPU is 0.15 seconds, the forward propagation time is 2.39 seconds, the backpropagation time is 3.72 seconds, the optimizer updates the model parameters in 2.60 seconds, the communication time is 2.52 seconds, the load calculation time of each layer such as the Linear layer is 0.02 seconds, the load calculation time of the Rerrange layer is 0.07 seconds, etc.

[0089] S3, based on the data processing process of each stage in the data movement stage, the model calculation stage and the communication stage and the model structure of the deep learning model, the load time overhead of each stage is modeled to obtain the load time evaluation model of the deep learning model training task;

[0090] During the training and reasoning process of deep learning models, data needs to be transferred from CPU memory to GPU memory for calculation. The movement of data has an important impact on the overall performance. Assuming the size of the mini-batch is batch_size, and the effective total bandwidth of the PCIe bus (Peripheral Component Interconnect Express, mainly used to connect the CPU and GPU, as well as other high-speed peripheral devices) is set to θ2 (bytes / second), the calculation formula for data movement time is:

[0091]

[0092] In distributed deep learning, in order to accelerate model training, a data parallel strategy is usually adopted, that is, the training data is distributed to multiple nodes, and each node maintains a complete copy of the model. To ensure the consistency of the models of each node, the parameters need to be updated synchronously in each iteration, which involves a cross-node communication process.

[0093] The Ring-All-Reduce algorithm in Pytorch efficiently aggregates gradients between multiple nodes to synchronize data through a ring topology. In this topology, each node communicates only with its adjacent nodes. During the All-Reduce operation, data is transmitted along a ring path, and each node performs partial accumulation calculations while receiving and sending data, ultimately achieving a global reduction effect. Assuming that the total network bandwidth of the computing node when processing the load is θ3 bytes / second, the time modeling of the communication process, that is, the calculation formula for the communication time, is:

[0094]

[0095] In the formula, nodes represents the number of computing nodes, and dim represents the hidden layer dimension of the deep learning model.

[0096] In the evaluation model calculation overhead part, the embodiment of the present application divides the deep learning model into a group of general layers, and then decomposes the deep learning load into basic general components according to the layer modeling method, and models these components through mathematical methods, and finally recombines them into the original model. The load calculation time for the entire deep learning model can be approximated as the load calculation time of each component, that is, the sum of the load calculation time of each network layer of the deep learning model. Therefore, the calculation formula for the load calculation time of the deep learning model processing task is:

[0097]

[0098] In the formula, Represents the load calculation time of the i-th network layer in the deep learning model.

[0099] Each deep learning model is composed of several neural network layers, including Rerrange layer, Linear layer, Dropout layer, LayerNorm layer, Attention module, Softmax layer, etc. Specifically, the Rerrange layer operation involves dimensional transformation, transposition, reshape, etc. of tensors. This operation does not involve numerical calculations, but only changes the storage layout of data. The calculation formula for the GPU load calculation time of this layer is:

[0100]

[0101] Where N is the size of the input tensor of the Rerrange layer, i.e., the number of elements, b is the size of each element, θ1 is the total GPU computing power when the computing node processes the load, and GPUS is the number of GPUs on each computing node.

[0102] The Linear layer performs a linear transformation, assuming that the input matrix X is of size (batch_size, dim1) and the weight matrix W is of size (dim1, dim2). In general, dim1 = dim2. The calculation formula for the load calculation time of this layer is:

[0103]

[0104] Where dim1 represents the dimension of the first hidden layer of the linear layer, and dim2 represents the dimension of the second hidden layer of the linear layer.

[0105] The Dropout layer randomly sets some neurons to zero during training. For each element, a random number needs to be generated for comparison and then multiplication. The calculation formula for the load calculation time of this layer is:

[0106]

[0107] The LayerNorm layer normalizes the input features, which involves calculating the mean and variance and then normalizing them. Assuming that the size of each normalization unit is dim2, the calculation formula for the load calculation time of this layer is:

[0108]

[0109] The Attention module is used to process sequence data. It generates attention scores by calculating query Q, key K, value V, and calculates the output based on it. Assuming that the sequence length is patch_size, the model dimension is dim, the number of attention heads is heads, and the dimension of each head is dim_head, the calculation formula for the load calculation time of this layer is:

[0110]

[0111] Where, T compute，Sof Indicates the time consumption of Softmax layer calculation.

[0112] The Softmax layer is used to perform exponential operations on the received input and normalize it to obtain a probability distribution. The calculation formula for the load calculation time of this layer is:

[0113]

[0114] Combined with the modeling of the performance of each part of computing performance, data movement and communication overhead in step S1, the computing time of each mini-batch (that is, the load time evaluation model of deep learning training tasks) is obtained as follows:

[0115] T total =T data +T compute +T communication ; (10)

[0116] It should be noted that all θ in the above modeling x The parameters are all load time evaluation model parameters, including three parameters θ1, θ2, and θ3. They all need to be fitted by the least squares method to represent the impact of different device hardware and runtime environments. After changing the above environmental context, that is, when the distributed cluster involved changes, that is, when the deep learning training load is not run in the same cluster, retraining is required, but only retraining is required to use it. Specifically, it involves changes in the three parameters θ1, θ2, and θ3. It is necessary to rerun the experiment to obtain a set of experimental samples and then reapply the above modeling to obtain the corresponding θ x value.

[0117] S4, constructing an error function according to the load time evaluation model obtained in step S3 and the combined characteristic experimental sample obtained in step S2, and solving the parameters of the load time evaluation model by using non-negative least squares (NNLS) method with the goal of minimizing the error function;

[0118] The error function of the non-negative least squares problem can be expressed as:

[0119] min x≥0 ||f(A,x)-b|| 2 ; (11)

[0120] In the formula, x represents the parameter matrix constructed by all load time evaluation model parameters, A represents the first feature matrix constructed based on the key features and model features in the combined feature experimental sample, b represents the second feature matrix constructed based on the load time overhead of the mobile phase, model calculation phase and communication phase in the combined feature experimental sample, and f(·) represents the load time evaluation model, that is, formula (10). In Python, the optimize.nnls function of the SciPy library can be used to implement the calculation of non-negative least squares method. NNLS adjusts the model parameters by minimizing the objective function, that is, the sum of the squares of the difference between the predicted value f(A, x) and the actual value b, to ensure the rationality and practicality of the model output.

[0121] S5, substituting the parameters of the load time evaluation model obtained in step S4 into the load time evaluation model, and using the updated load time evaluation model to evaluate the load time overhead of the received task;

[0122] In order to effectively predict the performance of different deep learning models in processing loads, this application adopts different modeling strategies to deal with different types of overhead. For data movement and communication consumption, by analyzing their inherent patterns, models are established to capture the performance characteristics of data transmission and communication processes, which helps to identify potential performance bottlenecks and guide optimization strategies. For model computing overhead, a hierarchical modeling approach is adopted to split the computing process into multiple reusable general layers and model each layer independently, which enables us to accurately evaluate the computing performance and resource consumption of each layer. By combining the models of these layers, we can predict and optimize the overall model performance and generalize the effect to different models and cluster sizes.

[0123] The embodiment of the present application also provides a distributed model training time evaluation device for edge cloud, including:

[0124] Experimental sample collection module: used to collect key features of edge devices in distributed clusters and model features of different deep learning models on edge devices, use the quartile method to confirm the data representative value of each model feature based on the collected features, cross-combine the obtained data representative values ​​to measure the load time overhead of each process when the deep learning model processes tasks, and construct combined feature experimental samples; the process of deep learning model processing tasks includes data movement stage, model calculation stage and communication stage;

[0125] Model building module: used to model the load time overhead of each stage based on the data processing process in each stage of the data movement stage, model calculation stage and communication stage and the model structure of the deep learning model, and then build a load time evaluation model for the deep learning model training task;

[0126] Model parameter calculation module: constructs an error function based on the load time evaluation model obtained by the model construction module and the combined characteristic experimental samples obtained by the experimental sample collection module, and uses the non-negative least squares method to solve the parameters of the load time evaluation model with the goal of minimizing the error function;

[0127] Load evaluation module: used to substitute the parameters obtained by the model parameter calculation module into the load time evaluation model obtained by the model construction module, and use the updated load time evaluation model to evaluate the load time overhead of the received task.

[0128] An embodiment of the present application also provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the above-mentioned distributed model training time evaluation method for edge cloud are executed.

[0129] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned distributed model training time evaluation method for edge cloud are executed. Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, the embodiment of the above-mentioned distributed model training time evaluation method for edge cloud can be executed.

[0130] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A distributed model training time evaluation method for edge cloud, characterized in that: The steps include: S1, divides the process of deep learning model processing task load into data movement stage, model calculation stage and communication stage based on load time overhead; S2, collect the key features of the distributed cluster and the model features of different deep learning models, use the quartile method to confirm the data representative value of each model feature based on the collected features, cross-combine the obtained data representative values ​​to measure the load time overhead of the data movement stage, model calculation stage and communication stage, and construct a combined feature experimental sample; S3, based on the data processing process of each stage in the data movement stage, the model calculation stage and the communication stage and the model structure of the deep learning model, the load time overhead of each stage is modeled to obtain the load time evaluation model of the deep learning model training task; S4, constructing an error function according to the load time evaluation model obtained in step S3 and the combined characteristic experimental sample obtained in step S2, and solving the parameters of the load time evaluation model by using the non-negative least squares method with the goal of minimizing the error function; S5, substituting the parameters of the load time evaluation model obtained in step S4 into the load time evaluation model, and using the updated load time evaluation model to evaluate the load time overhead of the received task.

2. The distributed model training time evaluation method for edge cloud according to claim 1 is characterized in that: The step S2 comprises the following steps: S2.1, collect key features of distributed clusters and model features of different deep learning models, the key features include the number of nodes and the number of GPUs, and the model features include batch size, image segmentation block size, hidden layer dimension of the model, dimension of the feature subspace of the self-attention mechanism, and the number of heads; S2.2, constructing a load parameter measurement sample for each deep learning model based on all feature data obtained in step S2.1; S2.3, using the quartile method to confirm at least two data representative values ​​of each feature in the load parameter measurement sample obtained in step S2.2; S2.4, cross-combining the data representative values ​​obtained in step S2.3 to measure the load time overhead of each stage under each combination; S2.5, construct a combined feature experimental sample based on the data obtained in step S2.2 and step S2.

4.

3. The distributed model training time evaluation method for edge cloud according to claim 1 is characterized in that: In step S3, the expression of the load time evaluation model of the deep learning model training task is: T total =T data +T compute +T communication ; Where, T data represents the load time overhead of the data movement phase; T compute represents the load time overhead of the model calculation phase, T communication represents the load time overhead of the communication phase; The load time overhead T of the data movement phase data The modeling formula is: In the formula, batch_size represents the input batch size, nodes represents the number of edge devices that process the load, i.e., the task, and θ2 represents the effective total bandwidth of the PCIe bus; The load time overhead of the communication phase is T communication The modeling formula is: In the formula, dim represents the hidden layer dimension of the deep learning model, and θ3 represents the total network bandwidth of the edge device when processing the load; The load time cost of the model calculation phase is T compute The modeling formula is: In the formula, Represents the load calculation time of the i-th network layer in the deep learning model.

4. The distributed model training time evaluation method for edge cloud according to claim 3 is characterized in that: The network layer includes a Rerrange layer, a Linear layer, a Dropout layer, a LayerNorm layer, an Attention module and / or a Softmax layer. The load calculation time of the Rerrange layer is T compute,Reg Indicates that the corresponding calculation formula is: Where N represents the size of the input tensor of the Rerrange layer, i.e., the number of elements, b represents the size of each element, θ1 represents the total GPU computing power of the edge device that processes the load, and GPUS represents the number of GPUs on each edge device. The load calculation time of the Linear layer is T compute,Lin Indicates that the corresponding calculation formula is: Where dim1 represents the dimension of the first hidden layer of the linear layer, and dim2 represents the dimension of the second hidden layer of the linear layer; The load calculation time of the Dropout layer is T compute,Dro Indicates that the corresponding calculation formula is: The load calculation time of the LayerNorm layer is T compute,Lay Indicates that the corresponding calculation formula is: The load calculation time of the Softmax layer is T compute,Sof Indicates that the corresponding calculation formula is: In the formula, patch_size represents the image segmentation block size; The load calculation time of the Attention module is T compute,Att Indicates that the corresponding calculation formula is: In the formula, heads represents the number of attention heads, and dim_head represents the dimension of the feature subspace of the self-attention mechanism, that is, the dimension of each head.

5. The distributed model training time evaluation method for edge cloud according to claim 1 is characterized in that: The data movement phase refers to the transfer of data from CPU memory to GPU memory during load processing, the model calculation phase refers to the process of loading the deep learning model to process the load, and the communication phase refers to the synchronization of the parameters of the deep learning model of each edge device in the distributed cluster.

6. The distributed model training time evaluation method for edge cloud according to claim 1 is characterized in that: In step S4, the expression of the error function is: min x≥0 ||f(A,x)-b|| 2 ; Where x represents the parameter matrix constructed by all load evaluation model parameters, A represents the first feature matrix constructed based on the key features and model features in the combined feature experimental samples, b represents the second feature matrix constructed based on the load time overhead of the mobile phase, model calculation phase, and communication phase in the combined feature experimental samples, and f(·) represents the load time evaluation model.

7. A distributed model training time evaluation device for edge cloud, characterized in that: include: Experimental sample collection module: used to collect key features of edge devices in distributed clusters and model features of different deep learning models on edge devices, use the quartile method to confirm the data representative value of each model feature based on the collected features, cross-combine the obtained data representative values ​​to measure the load time overhead of each process when the deep learning model processes tasks, and construct a combined feature experimental sample; The process of deep learning model processing tasks includes data movement phase, model calculation phase and communication phase; Model building module: used to model the load time overhead of each stage based on the data processing process in each stage of the data movement stage, model calculation stage and communication stage and the model structure of the deep learning model, and then build a load time evaluation model for the deep learning model training task; Model parameter calculation module: constructs an error function based on the load time evaluation model obtained by the model construction module and the combined characteristic experimental samples obtained by the experimental sample collection module, and uses the non-negative least squares method to solve the parameters of the load time evaluation model with the goal of minimizing the error function; Load evaluation module: used to substitute the parameters obtained by the model parameter calculation module into the load time evaluation model obtained by the model construction module, and use the updated load time evaluation model to evaluate the load time overhead of the received task.

Images