A batch scheduling system and method based on interference level and communication cost

Abstract

The present invention discloses a batch scheduling system and method based on interference level and communication cost, including a batch scheduling module and a GPU scheduling optimization module. The batch scheduling module schedules the pods in the podgroup and determines whether there are enough pods in the podgroup to be successfully scheduled. If the requirements are met, these pods are bound to the corresponding nodes; otherwise, all the pods in the podgroup are rolled back and wait for rescheduling. The GPU scheduling optimization module calculates the average performance degradation and interference level of all tasks in the podgroup on the GPU; calculates the communication cost through the model synchronization cost and updated model data between GPUs; takes the weighted sum of the interference level and communication cost as the fitness and continuously updates the scheduling policy using the shuffled frog leaping algorithm to obtain the final scheduling policy. The present invention realizes batch scheduling in a plug-in manner, avoiding the cache coherence problem caused by introducing other schedulers; combines the interference level and communication cost, and uses the shuffled frog leaping algorithm to find the global optimal solution, which can effectively improve the efficiency of distributed training tasks.

Description

Technical Field

[0001] The present invention belongs to the container scheduling method in the field of cloud native, and specifically relates to a batch scheduling system and method based on interference level and communication cost. Background Art

[0002] The development of deep learning has brought new solutions to traditional problems in the computer field. As a hot topic in academia and a new opportunity in the industry, deep learning has received high attention and extensive support. At present, it has been applied to various fields and achieved remarkable success, including image recognition, object detection, and natural language processing, etc. The breakthrough results in the field of deep learning have stimulated people's interest in deep learning, making it possible to solve complex tasks in various fields through models. The large-scale parallel processing ability of GPUs is the main reason for the recent success in training deep learning models, and hardware manufacturers are constantly improving GPUs to accelerate deep learning tasks. However, at the same time, with the disruptive trend of big data, the scale of training data sets has increased significantly, and people's requirements for models are also getting higher and higher. Due to the increase in model complexity, the speed of single-card training can no longer meet people's needs. To make full use of computing resources, distributed training has received more and more attention. To better support distributed training, building a deep learning cloud platform for data processing and model training, as the infrastructure for algorithm training and network model optimization, is of great significance.

[0003] Traditional deep learning training methods are that individuals or teams purchase hardware devices, configure corresponding deep learning environments on computers or servers, and directly conduct training in the completed environments. This method has very cumbersome steps, with high thresholds in terms of both economy and experience, and it is difficult to promote it to all walks of life. For universities, laboratory teams often have a certain scale of server clusters, but students need to configure the environments vividly, which is difficult to get started, and multiple environments are likely to affect each other's work. In recent years, more and more cloud service providers have developed deep learning platforms, providing more flexible deep learning systems or tools, sharing GPUs in cloud environment clusters, and supporting the concurrent execution of multiple deep learning tasks. Deep learning training systems rely on container orchestration platforms (such as Kubernetes) to allocate the required computing resources and manage the life cycle of training tasks. The platform uses one or more containers to run training tasks and starts deep learning training processes in the containers. Training tasks use distributed frameworks (such as TensorFlow, Pytorch) to make multiple GPUs work in parallel to effectively process large training sets.

[0004] Currently, Kubernetes is the most popular container orchestration tool. However, its default scheduler only supports the allocation of whole GPU cards, which may lead to low resource utilization. In addition, the default scheduler does not support batch scheduling of multiple tasks and cannot meet the requirement of running distributed tasks together. Summary of the Invention

[0005] Object of the Invention: The object of the present invention is to provide a batch scheduling system that supports batch scheduling of multiple tasks and has high efficiency for distributed training tasks; another object of the present invention is to provide a batch scheduling method.

[0006] Technical Solution: The batch scheduling system described in the present invention includes a batch scheduling module, which is used to sort the pods in the scheduling queue so that pods in the same group are located in the same area of the scheduling queue; take out a pod from the head of the scheduling queue, select all nodes that meet the resource requirements from the node list according to the resources applied for by the user for the pod, and score all nodes that meet the resource requirements according to the scoring strategy to obtain the highest-scoring node; perform pre-scheduling processing on the highest-scoring node to pre-occupy the node resources; determine whether there are enough pods in the podgroup that are scheduled successfully. If the requirements are met, bind these pods to the corresponding nodes; if the requirements are not met, roll back all pods in the podgroup and wait for re-scheduling; among them, batch scheduling is carried out in units of podgroup, and a pod is a container group composed of one or more containers sharing resources such as network, CPU, and GPU; a GPU scheduling optimization module, which is used to perform fine-grained scheduling on GPU resources; select a task from the podgroup, where a pod is a task, and one pod is a task; respectively perform vector representation on the selected task and n tasks running on the GPU to obtain n + 1 task vectors; input the n + 1 task vectors into a multi-layer perceptron to obtain the average performance degradation of the selected task on the GPU; re-select the tasks in the podgroup and repeat the above operations until the average performance degradation of all tasks in the podgroup on the GPU is obtained, and then calculate the interference level; define the model synchronization cost between GPUs as msc ij ; construct a GPU topology tree according to the connection method between GPUs and calculate msc according to the weights of the GPU topology tree ij ; define the updated model data as muds ij , calculate the communication cost using the synchronization cost and the updated model data, and sum the weighted communication costs to obtain the weighted sum of communication costs; use the interference level and the weighted sum of communication costs as fitness, and then continuously update the scheduling strategy using the shuffled frog leaping algorithm to obtain the final scheduling strategy.

[0007] Further, in the batch scheduling module,

[0008] Sort according to the priority of the pod group to which the pod belongs. If the priorities are the same, then sort according to the initialization timestamp of the pod group; for pods that do not specify a pod group, the scheduler divides the pod into the default pod group; filter all nodes in the cluster according to the resources required by the pod, and select the nodes that meet the resource requirements; score the nodes according to the scoring strategy, select the node with the highest score, and perform pre-scheduling processing on the node with the highest score to pre-occupy the node resources; determine whether there are enough pods scheduled successfully in the pod group. If the requirements are met before the expiration time set in the pod group, then bind these pods to the corresponding nodes; if the requirements are not met within this time, then roll back all pods in the pod group, release the pre-occupied resources, and wait for re-scheduling; among them, the expiration time is set according to the task requirements.

[0009] Further, in the GPU scheduling optimization module, the calculation process of the interference level is as follows:

[0010] Define the interference-aware multi-layer perceptron:

[0011] α 1 = φ 1 (T 1 ,T 2 ,...,T n+1 )

[0012] ......

[0014]

[0015]

[0016] Among them, W 1 ……W L-1 is the weight matrix, and L is the number of layers of the multi-layer perceptron; is the transpose of the weight matrix; b 1 ……b L is the bias vector; g 2 ……g L is the Relu activation function; h is the output layer weight; h T is the transpose of the output layer weight; α 1 ……α L-1 is the input of the middle layer of the multi-layer perceptron; φ 1 ……φ L is the output of the middle layer of the multi-layer perceptron, T 1 ……T n+1 is the task vector; σ is the mapping method; is the predicted value of the performance degradation vector;

[0017] During the training process of the multi-layer perceptron, the weight matrix W is updated through the gradient descent algorithm to minimize the predicted value of the performance degradation vector and the target value z of the performance degradation vector t The mean square loss between them, and regularization is used to prevent overfitting. The loss function L is as follows:

[0018]

[0019] where λ is the regularization weight; w t is the training example weight; m is the dimension of the task vector;

[0020] Input n + 1 task vectors into the multi-layer perceptron to obtain the predicted value of the performance degradation vector The predicted value of this performance degradation vector has n + 1 elements, corresponding to the performance degradation of n + 1 tasks respectively, and then the average performance degradation SD is calculated through the following formula i :

[0021]

[0022] According to the average performance degradation of all subtasks in distributed training, and the interference level I(M) is obtained according to the following calculation formula:

[0023]

[0024] where m is the number of tasks in the podgroup.

[0025] Furthermore, in the GPU scheduling optimization module, the calculation process of the communication cost is as follows:

[0026] Define the model synchronization cost between GPU i and GPU j as msc ij , and define the updated model data as muds ij ;

[0027] Obtain the connection relationship between GPUs and establish a GPU topology tree according to this connection relationship. Calculate the model synchronization cost msc according to the weights of the GPU topology tree ij ; According to the model synchronization cost msc ij and the updated model data muds ij Calculate the communication cost C(M), and the calculation formula is:

[0028]

[0029] Among them, n is the number of GPUs, and i and j are the corresponding numbers of the GPUs.

[0030] Furthermore, in the GPU scheduling optimization module, the process of continuously updating the scheduling policy using the Shuffled Frog Leaping Algorithm is as follows:

[0031] Define the objective formula as: αI(M) + βC(M); where α and β are weights;

[0032] Set the convergence value z and the maximum number of iterations g according to the task requirements, where the default value of g is 16;

[0033] Set the number of frogs to be the same as the number of tasks to be scheduled; calculate the fitness value of each frog through the objective formula, and sort each frog from high to low according to the fitness value to obtain the sorted population;

[0034] Divide the sorted population into m meme groups, and mark the frog with the best fitness value in each meme group as F b , and mark the frog with the best fitness value in the sorted population as F g ;

[0035] Perform local update by moving each frog in the meme group towards the frog with the best fitness value in the meme group. The update formula is:

[0036]

[0037] In the formula, r is a random number between 0 and 1, D is the distance the frog moves, F i is the fitness value of the frog numbered i in the meme group, D max is the maximum distance the frog is allowed to move, D min is the minimum distance the frog is allowed to move;

[0038] If the new position after the frog jumps has a better fitness value, update the fitness value; otherwise, let the frog jump towards the frog with the best fitness value in the sorted population to perform local update. The update formula is:

[0039]

[0040] If the new position after the frog jumps has a better fitness value, update the fitness value; otherwise, generate 1 new frog at a random position to replace the original frog, and calculate the fitness value of the new frog;

[0041] After performing local update operations on all frogs, update F by re - mixing and sorting all frogs and dividing them into meme groups b and F g, and perform the local update operation again; repeat the above process of the local update operation until the fitness values of all frogs are less than z or the maximum number of hybrid iterations g is reached.

[0042] The batch scheduling method described in the present invention includes:

[0043] (1) Sort the pods in the scheduling queue so that pods in the same group are located in the same area of the scheduling queue; take out a pod from the head of the scheduling queue, select all nodes that meet the resource requirements from the node list according to the resources requested by the user for the pod, and score all nodes that meet the resource requirements according to the scoring strategy to obtain the highest-scoring node; perform pre-scheduling processing on the highest-scoring node to pre-occupy the node resources; determine whether there are enough pods in the podgroup that are successfully scheduled. If the requirements are met, bind these pods to the corresponding nodes; if the requirements are not met, roll back all pods in the podgroup and wait for re-scheduling; among them, batch scheduling is based on the podgroup, and a pod is a container group composed of one or more containers sharing resources such as network, CPU, GPU, etc.

[0044] (2) Perform fine-grained scheduling on GPU resources; select a task from the podgroup, where a pod is a task, and one pod is a task; respectively perform vector representation on the selected task and n tasks running on the GPU to obtain n + 1 task vectors; input the n + 1 task vectors into a multi-layer perceptron to obtain the average performance degradation of the selected task on the GPU; re-select tasks in the podgroup and repeat the above operations until the average performance degradation of all tasks in the podgroup on the GPU is obtained, and then calculate the interference level; define the model synchronization cost between GPUs as msc ij ; construct a GPU topology tree according to the connection method between GPUs and calculate msc according to the weights of the GPU topology tree ij ; define the updated model data as muds ij , calculate the communication cost using the synchronization cost and the updated model data, and sum the weighted communication costs to obtain the weighted sum of communication costs; use the interference level and the weighted sum of communication costs as the fitness, and then continuously update the scheduling strategy using the shuffled frog leaping algorithm to obtain the final scheduling strategy.

[0045] Further, in step (1),

[0046] Sort according to the priority of the pod group to which the pod belongs. If the priorities are the same, then sort according to the initialization timestamp of the pod group. For pods that do not specify a pod group, the scheduler divides the pods into the default pod group. Filter all nodes in the cluster according to the resources required by the pods, and select the nodes that meet the resource requirements. Score the nodes according to the scoring strategy, select the node with the highest score, and perform pre-scheduling processing on the node with the highest score to pre-occupy the node resources. Determine whether there are enough pods scheduled successfully in the pod group. If the requirements are met before the expiration time set in the pod group, then bind these pods to the corresponding nodes. If the requirements are not met within this time, then roll back all the pods in the pod group, release the pre-occupied resources, and wait for re-scheduling. Among them, the expiration time is set according to the task requirements.

[0047] Further, in step (2), the calculation process of the interference level is as follows:

[0048] Define the interference-aware multi-layer perceptron:

[0049] α 1 = φ 1 (T 1 ,T 2 ,...,T n+1 )

[0050] ......

[0052]

[0053]

[0054] Among them, W 1 ……W L-1 is the weight matrix, and L is the number of layers of the multi-layer perceptron; is the transpose of the weight matrix; b 1 ……b L is the bias vector; g 2 ……g L is the Relu activation function; h is the weight of the output layer; h T is the transpose of the weight of the output layer; α 1 ……α L-1 is the input of the middle layer of the multi-layer perceptron; φ 1 ……φ L is the output of the middle layer of the multi-layer perceptron, T 1 ……T n+1 is the task vector; σ is the mapping method; is the predicted value of the performance degradation vector;

[0055] During the training process of the multi-layer perceptron, the weight matrix W is updated through the gradient descent algorithm to minimize the predicted value of the performance degradation vector and the target value z of the performance degradation vector t The mean square loss between them, and regularization is used to prevent overfitting. The loss function L is as follows:

[0056]

[0057] where λ is the regularization weight; w t is the training example weight; m is the dimension of the task vector;

[0058] Input n + 1 task vectors into the multi-layer perceptron to obtain the predicted value of the performance degradation vector The predicted value of this performance degradation vector has n + 1 elements, corresponding to the performance degradation of n + 1 tasks respectively, and then the average performance degradation SD is calculated through the following formula i :

[0059]

[0060] According to the average performance degradation of all subtasks in distributed training, and the interference level I(M) is obtained according to the following calculation formula:

[0061]

[0062] where m is the number of tasks in the podgroup.

[0063] Furthermore, in step (2), the calculation process of the communication cost is as follows:

[0064] Define the model synchronization cost between GPU i and GPU j as msc ij , and define the updated model data as muds ij ;

[0065] Obtain the connection relationship between GPUs and establish a GPU topology tree according to this connection relationship. Calculate the model synchronization cost msc according to the weights of the GPU topology tree ij ;

[0066] According to the model synchronization cost msc ij and the updated model data muds ij Calculate the communication cost C(M), and the calculation formula is:

[0067]

[0068] Among them, n is the number of GPUs, and i and j are the corresponding numbers of the GPUs.

[0069] Furthermore, in step (2), the process of continuously updating the scheduling policy using the Shuffled Frog Leaping Algorithm is as follows:

[0070] Define the objective formula as: αI(M) + βC(M); where α and β are weights;

[0071] Set the convergence value z and the maximum number of iterations g according to the task requirements, where the default value of g is 16;

[0072] Set the number of frogs to be the same as the number of tasks to be scheduled; calculate the fitness value of each frog through the objective formula, and sort each frog from high to low according to the fitness value to obtain the sorted population;

[0073] Divide the sorted population into m meme groups, and mark the frog with the best fitness value in each meme group as F b , and mark the frog with the best fitness value in the sorted population as F g ;

[0074] Perform local update by moving each frog in the meme group towards the frog with the best fitness value in the meme group. The update formula is:

[0075]

[0076] In the formula, r is a random number between 0 and 1, D is the distance the frog moves, F i is the fitness value of the frog numbered i in the meme group, D max is the maximum distance the frog is allowed to move, D min is the minimum distance the frog is allowed to move;

[0077] If the new position after the frog jumps has a better fitness value, update the fitness value; otherwise, let the frog jump towards the frog with the best fitness value in the sorted population to perform local update. The update formula is:

[0078]

[0079] If the new position after the frog jumps has a better fitness value, update the fitness value; otherwise, generate 1 new frog at a random position to replace the original frog and calculate the fitness value of the new frog;

[0080] After performing local update operations on all the frogs, update F by re - mixing and sorting all the frogs and dividing them into meme groups b and F g, and perform the local update operation again; repeat the above process of local update operation until the fitness values of all frogs are less than z or the maximum number of hybrid iterations g is reached.

[0081] Beneficial effects: Compared with the prior art, the significant advantages of the present invention are as follows: 1. Support batch scheduling of multiple tasks: Batch scheduling is achieved through a plug-in method (each stage of batch scheduling is implemented through a plug-in), meeting the requirements of batch scheduling for multiple tasks and the co-running of distributed tasks; 2. High efficiency of distributed training tasks: Based on combining the interference level and communication cost, the hybrid frog leaping algorithm is used to find the global optimal solution, which can effectively improve the efficiency of distributed training tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 is the batch scheduling flowchart;

[0083] Figure 2 is the block diagram of GPU scheduling optimization;

[0084] Figure 3 is the flowchart of calculating the interference level;

[0085] Figure 4 is the structure diagram of the GPU topology tree;

[0086] Figure 5 is the flowchart of using the hybrid frog leaping algorithm;

[0087] Figure 6 is the comparison chart of the completion time of distributed training;

[0088] Figure 7 is the comparison chart of load balancing. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0089] The following describes the specific embodiments of the present invention with reference to the accompanying drawings.

[0090] In this embodiment, the Golang 1.15.5 programming language is adopted, and JetBrains Goland 2021.2.4 x64 is used for development. The cluster configuration is 4 computers with 8-core 2.6GHz CPUs, 16G of memory, and 4 Tesla V100 graphics cards, and the operating system is Centos7; the AlexNet model is used for the distributed training task model, and the PASCAL VOC2012 dataset is used for the dataset.

[0091] Embodiment 1

[0092] The batch scheduling system described includes a batch scheduling module and a GPU scheduling optimization module.

[0093] As Figure 1As shown, the batch scheduling module is used to sort the pods in the scheduling queue so that pods in the same group are located in the same area of the scheduling queue; take out a pod from the head of the scheduling queue, select all nodes that meet the resource requirements from the node list according to the resources requested by the user for the pod, and score all nodes that meet the resource requirements according to the scoring strategy to obtain the node with the highest score; perform pre-scheduling processing on the node with the highest score to pre-occupy the node resources; determine whether there are enough pods in the podgroup that are successfully scheduled. If the requirements are met, bind these pods to the corresponding nodes; if the requirements are not met, roll back all pods in the podgroup and wait for re-scheduling; among them, batch scheduling is based on podgroups, and a pod is a container group composed of one or more containers that share resources such as network, CPU, and GPU.

[0094] In the prefilter stage, sort the pods in the scheduling queue according to the priority of the podgroup to which the pod belongs. If the priorities are the same, sort them according to the initialization timestamp of the podgroup; for pods without a specified podgroup, the scheduler divides the pod into the default podgroup with a priority of 0.

[0095] Get all nodes that can accept pod scheduling from the node tree in the scheduling queue cache, filter all nodes in the cluster according to the resources required by the pod (such as CPU, memory, etc.) to select nodes that meet the resource requirements; send this node information to the extended scheduler, and the extended scheduler scores the nodes according to the scoring strategy, selects the node with the highest score, and records the selected GPU and the requested GPU memory in the annotation field of the pod; perform pre-scheduling processing on the node with the highest score to pre-occupy the node resources to prevent the resources from being occupied by other pods.

[0096] In the permit stage, determine whether there are enough pods in the podgroup that are successfully scheduled. If the requirements are met before the expiration time set in the podgroup, bind these pods to the corresponding nodes; if the requirements are not met within this time, roll back all pods in the podgroup (move them into the backoff queue), release the pre-occupied resources, and wait for re-scheduling; among them, the expiration time is set according to the task requirements.

[0097] Such as Figure 2As shown, the GPU scheduling optimization module is used to perform fine-grained scheduling of GPU resources; select a task from the podgroup, where a pod is a task and one pod represents one task; respectively perform vector representation on the selected task and the n tasks running on the GPU to obtain n+1 task vectors; input the n+1 task vectors into a multi-layer perceptron to obtain the average performance degradation of the selected task on the GPU; re-select tasks in the podgroup and repeat the above operations until the average performance degradation of all tasks in the podgroup on the GPU is obtained, and then calculate the interference level; define the model synchronization cost between GPUs as msc ij ; construct a GPU topology tree according to the connection method between GPUs and calculate msc based on the weights of the GPU topology tree ij ; define the updated model data as muds ij , calculate the communication cost using the synchronization cost and the updated model data, and sum the weighted communication costs to obtain the weighted sum of communication costs; use the interference level and the weighted sum of communication costs as fitness, and then continuously update the scheduling strategy using the shuffled frog leaping algorithm to obtain the final scheduling strategy.

[0098] As Figure 3 shown, the calculation process of the interference level is as follows:

[0099] The execution efficiency of distributed training tasks may be affected by interference caused by resource competition among tasks co-executing on the GPU; the resources competed by these tasks include Streaming Multiprocessors, memory resources, and interconnection networks; therefore, the present invention uses SM efficiency to characterize SM resources, uses memory utilization and throughput to characterize memory resources, and uses global download throughput and global storage throughput to characterize interconnection network resources; each distributed training task can be expressed as: where, tE SM represents SM efficiency, tU L1 represents GPU L1 cache utilization rate, tTHP L1 represents GPU L1 cache throughput, tU L2 represents GPU L2 cache utilization rate, tTHP L2 represents GPU L2 cache throughput, tU DRAM represents GPU memory utilization rate, tTHP DRAM represents GPU memory throughput, tTHP L represents global load throughput, tTHP S represents global storage throughput, batch_size represents batch size, and sub_dsize represents sub-dataset size.

[0100] Define a multi-layer perceptron for interference perception:

[0101] α 1 = φ 1 (T 1 , T 2 ,..., T n+1 )

[0102] ......

[0104]

[0105]

[0106] where W 1 ……W L-1 is the weight matrix, and L is the number of layers of the multi-layer perceptron; is the transpose of the weight matrix; b 1 ……b L is the bias vector; g 2 ……g L is the Relu activation function; h is the weight of the output layer; h T is the transpose of the weight of the output layer; α 1 ……α L-1 is the input of the middle layer of the multi-layer perceptron; φ 1 ……φ L is the output of the middle layer of the multi-layer perceptron, T 1 ……T n+1 is the task vector; σ is the mapping method; is the predicted value of the performance degradation vector.

[0107] During the training process of the multi-layer perceptron, the weight matrix W is updated by the gradient descent algorithm to minimize the mean square loss between the predicted value of the performance degradation vector and the target value z of the performance degradation vector t , and regularization is used to prevent overfitting. The loss function L is:

[0108]

[0109] where λ is the regularization weight; w t is the weight of the training example; m is the dimension of the task vector.

[0110] Input n + 1 task vectors into the multi-layer perceptron to obtain the predicted value of the performance degradation vector The predicted value of this performance degradation vector There are n + 1 elements, corresponding to the performance degradation of n + 1 tasks respectively, and the average performance degradation SD is calculated through the following formula i :

[0111]

[0112] According to the average performance degradation of all subtasks in distributed training, and the interference level I(M) is obtained according to the following calculation formula

[0113]

[0114] where m is the number of tasks in the podgroup

[0115] The calculation process of the communication cost is as follows

[0116] For distributed training, after each iterative calculation, model synchronization is required. Therefore, the execution efficiency of distributed training tasks is also affected by the communication cost between GPUs; among them, the communication cost is determined by the connection method between GPUs and the data of model updates

[0117] Define the model synchronization cost between GPU i and GPU j as msc ij , and define the updated model data as muds ij .

[0118] Obtain the connection relationship between GPUs (obtain the connection relationship between GPUs through nvml library functions), as Figure 4 shown. According to this connection relationship, establish a GPU topology tree, and calculate the model synchronization cost msc according to the weights of the GPU topology tree ij (the value of msc ij is the sum of the path weights between GPU i and GPU j ); According to the model synchronization cost msc ij and the updated model data muds ij , calculate the communication cost C(M), and the calculation formula is

[0119]

[0120] where n is the number of GPUs, and i and j are the corresponding numbers of GPUs

[0121] As Figure 5 shown, the process of continuously updating the scheduling strategy using the shuffled frog leaping algorithm is

[0122] The present invention focuses on minimizing interference and communication costs between tasks when multiple subtasks are running in parallel during distributed training. Therefore, the objective formula is defined as: αI(M) + βC(M); where α and β are weights.

[0123] Set the convergence value z and the maximum number of iterations g according to the task requirements, where the default value of g is 16.

[0124] Set the number of frogs to be the same as the number of tasks to be scheduled; randomly generate N frogs, where N is the number of subtasks of the distributed training task; calculate the fitness value of each frog through the objective formula, and sort each frog from high to low according to the fitness value to obtain the sorted population.

[0125] Divide the sorted population into m meme groups. The first frog is assigned to the first meme group, the second frog is assigned to the second meme group, the m-th frog is assigned to the m-th meme group, the (m + 1)-th frog is assigned to the first meme group, and so on until all frogs in the sorted population are assigned; mark the frog with the best fitness value in each meme group as F b , and mark the frog with the best fitness value in the sorted population as F g .

[0126] Perform local update by moving each frog in the meme group towards the frog with the best fitness value in the meme group. The update formula is:

[0127]

[0128] In the formula, r is a random number between 0 and 1, D is the distance the frog moves, F i is the fitness value of the frog numbered i in the meme group, D max is the maximum distance the frog is allowed to move, D min is the minimum distance the frog is allowed to move.

[0129] If the new position after the frog jumps has a better fitness value, update the fitness value (replace the original fitness value with the new fitness value), otherwise let the frog jump towards the frog with the best fitness value in the sorted population to perform local update. The update formula is:

[0130]

[0131] If the new position after the frog jumps has a better fitness value, update the fitness value (replace the original fitness value with the new fitness value), otherwise generate 1 new frog at a random position to replace the original frog and calculate the fitness value of the new frog.

[0132] After performing local update operations on all frogs, update F by re-mixing and sorting all frogs and dividing them into meme groups.b and F g and perform a local update operation again; repeat the above local update operation process until the fitness values of all frogs are less than z or the maximum number of hybrid iterations g is reached.

[0133] The performance analysis of Embodiment 1 is described below with reference to the accompanying drawings:

[0134] Select a working node as the task scheduling node and taint all other working nodes (taints to prevent pods from being scheduled to these nodes). When 9 tasks in the same podgroup are scheduled, the cluster resources can only meet the needs of 5 tasks. It is found that the default scheduler will still schedule those five tasks, and after introducing the batch scheduling plugin, all tasks will not be scheduled.

[0135] Cancel the taints of all working nodes, and use the default scheduling policy and the GPU optimization scheduling policy proposed by the present invention to schedule distributed tasks respectively. Adjust the number of parallel tasks and conduct multiple experiments to compare the execution efficiency of tasks under the two scheduling policies. As Figure 6 shown, the GPU scheduling optimization policy proposed by the present invention can effectively improve the execution efficiency of distributed training tasks.

[0136] Use the resource utilization rate to represent the node load situation, and use the standard deviation of the resource utilization rates between nodes to represent the load balancing degree between nodes. Use U CPU 、U MEM 、U GPU 、U GMEM to represent the CPU utilization rate, memory utilization rate, GPU utilization rate, and video memory utilization rate respectively, and use the following formula to calculate the load situation of node i:

[0137]

[0138] The average load situation of all nodes is:

[0139]

[0140] where n is the number of nodes.

[0141] Furthermore, the standard deviation of the resource utilization rates between nodes is obtained as:

[0142]

[0143] Since the larger the standard deviation, the greater the difference in the load situation between nodes, that is, the worse the load balancing, the load balancing degree is defined as:

[0144] LoadBalancing = 1 - Load std

[0145] As Figure 7 shown, the scheduling optimization strategy of the present invention can effectively improve the load balancing degree of the cluster.

[0146] Example 2

[0147] The batch scheduling method described above includes the following steps:

[0148] (1) As Figure 1 shown, sort the pods in the scheduling queue so that pods in the same group are located in the same area of the scheduling queue; take out a pod from the head of the scheduling queue, select all nodes that meet the resource requirements from the node list according to the resources requested by the user for the pod, and score all nodes that meet the resource requirements according to the scoring strategy to obtain the highest-scoring node; perform pre-scheduling processing on the highest-scoring node to pre-occupy the node resources; determine whether there are enough pods in the podgroup that are successfully scheduled. If the requirements are met, bind these pods to the corresponding nodes; if the requirements are not met, roll back all pods in the podgroup and wait for re-scheduling; among them, batch scheduling is based on podgroup, and a pod is a container group composed of one or more containers that share resources such as network, CPU, and GPU.

[0149] In the prefilter stage, sort the pods in the scheduling queue according to the priority of the podgroup to which the pod belongs. If the priorities are the same, then sort according to the initialization timestamp of the podgroup; for pods without a specified podgroup, the scheduler divides the pod into the default podgroup with a priority of 0.

[0150] Get all nodes that can accept pod scheduling from the node tree in the scheduling queue cache, filter all nodes in the cluster according to the resources required by the pod (such as CPU, memory, etc.) to select nodes that meet the resource requirements; send this node information to the extended scheduler, and the extended scheduler scores the nodes according to the scoring strategy, selects the node with the highest score, and records the selected GPU and the requested GPU memory in the annotation field of the pod; perform pre-scheduling processing on the highest-scoring node to pre-occupy the node resources to prevent the resources from being occupied by other pods.

[0151] In the permit stage, it is determined whether there are enough pods in the podgroup that are successfully scheduled. If the requirements are met before the expiration time set in the podgroup, these pods are bound to the corresponding nodes; if the requirements are not met within this time, all pods in the podgroup are rolled back (moved into the backoff queue), the preoccupied resources are released, and waiting for rescheduling; among them, the expiration time is set according to the task requirements.

[0152] (2) As Figure 2 shown, fine-grained scheduling of GPU resources is performed; a task is selected from the podgroup, where a pod is a task, and one pod is a task; the selected task and the n tasks running on the GPU are respectively vectorized to obtain n + 1 task vectors; the n + 1 task vectors are input into a multi-layer perceptron to obtain the average performance degradation of the selected task on the GPU; tasks in the podgroup are reselected, and the above operations are repeated until the average performance degradation of all tasks in the podgroup on the GPU is obtained, and then the interference level is calculated; the model synchronization cost between GPUs is defined as msc ij ; a GPU topology tree is constructed according to the connection method between GPUs, and msc is calculated according to the weights of the GPU topology tree ij ; the updated model data is defined as muds ij , the communication cost is calculated using the synchronization cost and the updated model data, and the weighted sum of the communication costs is obtained by weighting and summing the communication costs; the interference level and the weighted sum of the communication costs are used as fitness, and then the shuffled frog leaping algorithm is used to continuously update the scheduling strategy to obtain the final scheduling strategy.

[0153] As Figure 3 shown, the calculation process of the interference level is as follows:

[0154] The execution efficiency of distributed training tasks may be affected by the interference caused by resource competition among tasks co-executing on the GPU; the resources competing for these tasks include Streaming Multiprocessors, memory resources, and interconnection networks; therefore, the present invention uses SM efficiency to characterize SM resources, uses memory utilization and throughput to characterize memory resources, and uses global download throughput and global storage throughput to characterize interconnection network resources; each distributed training task can be expressed as: where, tE SM represents SM efficiency, tU L1 represents the GPU L1 cache utilization rate, tTHP L1 represents the GPU L1 cache throughput, tU L2Indicates the GPU L2 cache utilization rate, tTHP L2 Indicates the GPU L2 cache throughput, tU DRAM Indicates the GPU memory utilization rate, tTHP DRAM Indicates the GPU memory throughput, tTHP L Indicates the global load throughput, tTHP S Indicates the global store throughput, batch_size represents the batch size, and sub_dsize represents the sub - dataset size.

[0155] Define the interference - aware multi - layer perceptron:

[0156] α 1 = φ 1 (T 1 ,T 2 ,...,T n+1 )

[0157] ......

[0159]

[0160]

[0161] Among them, W 1 ……W L-1 is the weight matrix, and L is the number of layers of the multi - layer perceptron; is the transpose of the weight matrix; b 1 ……b L is the bias vector; g 2 ……g L is the Relu activation function; h is the output layer weight; h T is the transpose of the output layer weight; α 1 ……α L-1 is the input of the middle layer of the multi - layer perceptron; φ 1 ……φ L is the output of the middle layer of the multi - layer perceptron, T 1 ……T n+1 is the task vector; σ is the mapping method; is the predicted value of the performance degradation vector.

[0162] During the training process of the multi - layer perceptron, update the weight matrix W through the gradient descent algorithm to minimize the mean - square loss between the predicted value of the performance degradation vector and the target value z t of the performance degradation vector, and use regularization to prevent overfitting. The loss function L is:

[0163]

[0164] Among them, λ is the regularization weight; w t is the training example weight; m is the dimension of the task vector.

[0165] Input n + 1 task vectors into the multi-layer perceptron to obtain the predicted value of the performance degradation vector The predicted value of this performance degradation vector has n + 1 elements, corresponding to the performance degradation of n + 1 tasks respectively, and then the average performance degradation SD is calculated through the following formula i :

[0166]

[0167] According to the average performance degradation of all subtasks in distributed training, and obtain the interference level I(M) according to the following calculation formula:

[0168]

[0169] Among them, m is the number of tasks in the podgroup.

[0170] The calculation process of the communication cost is as follows:

[0171] For distributed training, after each iterative calculation, model synchronization is required. Therefore, the execution efficiency of distributed training tasks is also affected by the communication cost between GPUs; among them, the communication cost is determined by the connection method between GPUs and the data of model updates.

[0172] Define the model synchronization cost between GPU i and GPU j as msc ij , define the updated model data as muds ij .

[0173] Obtain the connection relationship between GPUs (obtain the connection relationship between GPUs through nvml library functions), as Figure 4 shown, establish a GPU topology tree according to this connection relationship, and calculate the model synchronization cost msc according to the weights of the GPU topology tree ij (the value of msc ij is the sum of the path weights between GPU i and GPU j ); calculate the communication cost C(M) according to the model synchronization cost msc ij and the updated model data muds ij , and the calculation formula is:

[0174]

[0175] Among them, n is the number of GPUs, and i and j are the corresponding numbers of the GPUs.

[0176] As Figure 5 shown, the process of continuously updating the scheduling strategy using the shuffled frog leaping algorithm is as follows:

[0177] The present invention focuses on minimizing the interference and communication cost between tasks when multiple subtasks are running in parallel during distributed training. Therefore, the objective formula is defined as: αI(M) + βC(M); where α and β are weights.

[0178] Set the convergence value z and the maximum number of iterations g according to the task requirements, where the default value of g is 16.

[0179] Set the number of frogs to be the same as the number of tasks to be scheduled; randomly generate N frogs, where N is the number of subtasks of the distributed training task; calculate the fitness value of each frog through the objective formula, and sort each frog from high to low according to the fitness value to obtain the sorted population.

[0180] Divide the sorted population into m meme groups. The first frog is assigned to the first meme group, the second frog is assigned to the second meme group, the m-th frog is assigned to the m-th meme group, the (m + 1)-th frog is assigned to the first meme group, and so on until all the frogs in the sorted population are assigned; mark the frog with the best fitness value in each meme group as F b , and mark the frog with the best fitness value in the sorted population as F g .

[0181] Perform local update by moving each frog in the meme group towards the frog with the best fitness value in the meme group. The update formula is:

[0182]

[0183] In the formula, r is a random number between 0 and 1, D is the distance the frog moves, F i is the fitness value of the frog numbered i in the meme group, D max is the maximum distance the frog is allowed to move, D min is the minimum distance the frog is allowed to move.

[0184] If the new position after the frog jumps has a better fitness value, update the fitness value (replace the original fitness value with the new fitness value), otherwise let the frog jump towards the frog with the best fitness value in the sorted population to perform local update. The update formula is:

[0185]

[0186] If the new position after the frog jumps has a better fitness value, update the fitness value (replace the original fitness value with the new fitness value); otherwise, generate a new frog at a random position to replace the original frog and calculate the fitness value of the new frog.

[0187] After performing the local update operation on all frogs, update F by re - mixing and sorting all frogs and dividing them into meme groups. b and F g , and perform the local update operation again; repeat the above process of local update operation until the fitness values of all frogs are less than z or the maximum number of hybrid iterations g is reached.

[0188] The following conducts a performance analysis on Embodiment 2 in conjunction with the attached drawings:

[0189] Select a working node as the task scheduling node and taint all other working nodes (taints, to prevent pods from being scheduled to these nodes). When 9 tasks in the same podgroup are scheduled, the cluster resources can only meet the needs of 5 tasks. It is found that the default scheduler will still schedule those five tasks, while after introducing the batch scheduling plugin, all tasks will not be scheduled.

[0190] Cancel the taints of all working nodes and schedule the distributed tasks using the default scheduling policy and the GPU - optimized scheduling policy proposed in the present invention respectively. Adjust the number of parallel tasks and conduct multiple experiments to compare the execution efficiency of tasks under the two scheduling policies. As Figure 6 shown, the GPU scheduling optimization policy proposed in the present invention can effectively improve the execution efficiency of distributed training tasks.

[0191] Use the resource utilization rate to represent the node load situation, and use the standard deviation of the resource utilization rates among nodes to represent the degree of load balance among nodes. Use U CPU 、U MEM 、U GPU 、U GMEM to represent the CPU utilization rate, memory utilization rate, GPU utilization rate, and video memory utilization rate respectively. Use the following formula to calculate the load situation of node i:

[0192]

[0193] The average load situation of all nodes is:

[0194]

[0195] where n is the number of nodes.

[0196] Furthermore, the standard deviation of the resource utilization rates among nodes is obtained as:

[0197]

[0198] Since the larger the standard deviation, the greater the difference in the load conditions between nodes, that is, the worse the load balance, the degree of load balance is defined as:

[0199] LoadBalancing = 1 - Load std

[0200] As Figure 7 shown, the scheduling optimization strategy of the present invention can effectively improve the degree of load balance of the cluster.

Claims

1. A batch scheduling system based on interference level and communication cost, characterized in that, it includes a batch scheduling module and a GPU scheduling optimization module, The batch scheduling module is used to sort the pods in the scheduling queue so that pods in the same group are located in the same area of the scheduling queue; take out a pod from the head of the scheduling queue, and according to the resources requested by the user for the pod, select all nodes that meet the resource requirements from the node list, and score all nodes that meet the resource requirements according to the scoring strategy to obtain the highest-scoring node; Perform pre-scheduling processing on the highest-scoring node to pre-occupy node resources; judge whether there are enough pods in the podgroup that are successfully scheduled. If the requirements are met, bind these pods to the corresponding nodes; if the requirements are not met, roll back all pods in the podgroup and wait for re-scheduling; among them, batch scheduling is based on podgroup, and a pod is a container group that includes one or more containers sharing network, CPU, and GPU resources; The GPU scheduling optimization module is used to perform fine-grained scheduling of GPU resources; select a task from the podgroup, where a pod is a task and one pod represents one task; respectively perform vector representation on the selected task and the n tasks running on the GPU to obtain n + 1 task vectors; input the n + 1 task vectors into a multi-layer perceptron to obtain the average performance degradation of the selected task on the GPU; re-select a task from the podgroup and repeat the above operations until the average performance degradation of all tasks in the podgroup on the GPU is obtained, and then calculate the interference level; define the model synchronization cost between GPUs as msc ij ; construct a GPU topology tree according to the connection method between GPUs and calculate msc based on the weights of the GPU topology tree ij ; define the updated model data as muds ij , calculate the communication cost using the synchronization cost and the updated model data, and sum the weighted communication costs to obtain the weighted sum of communication costs; use the interference level and the weighted sum of communication costs as the fitness, and then continuously update the scheduling strategy using the shuffled frog leaping algorithm to obtain the final scheduling strategy; In the GPU scheduling optimization module, the calculation process of the interference level is as follows: Define an interference-aware multi-layer perceptron: Among them, W 1 ……W L-1 is the weight matrix, and L is the number of layers of the multi-layer perceptron; is the transpose of the weight matrix; b 1 ……b L is the bias vector; g 2 ……g L is the Relu activation function; h is the weight of the output layer; h T is the transpose of the weight of the output layer; α 1 ……α L-1 is the input of the intermediate layer of the multi-layer perceptron; φ 1 ……φ L is the output of the intermediate layer of the multi-layer perceptron, T 1 ……T n+1 is the task vector; σ is the mapping method; is the predicted value of the performance degradation vector; During the training process of the multi-layer perceptron, the weight matrix W is updated through the gradient descent algorithm to minimize the predicted value of the performance degradation vector and the target value z of the performance degradation vector t The mean square loss between them, and regularization is used to prevent overfitting. The loss function L is as follows: where λ is the regularization weight; w t is the training example weight; m is the dimension of the task vector; Input n + 1 task vectors into a multi-layer perceptron to obtain the predicted values of the performance degradation vectors The predicted values of the performance degradation vectors have n + 1 elements, corresponding to the performance degradations of n + 1 tasks respectively, and then calculate the average performance degradation SD through the following formula i : According to the average performance degradation of all subtasks in distributed training, and obtain the interference level I(M) according to the following calculation formula: where m is the number of tasks in the podgroup; In the GPU scheduling optimization module, the calculation process of the communication cost is as follows: Define GPU i With GPU j The model synchronization cost between them is msc ij Define the updated model data as muds ij ; Obtain the connection relationship between GPUs and establish a GPU topology tree based on this connection relationship. Calculate the model synchronization cost msc according to the weight calculation model of the GPU topology tree ij ; According to the model synchronization cost msc ij and the updated model data muds ij the communication cost C(M) is calculated, and the calculation formula is: where n is the number of GPUs, and i and j are the corresponding numbers of GPUs.

2. The batch scheduling system based on interference level and communication cost according to claim 1, characterized in that: In the batch scheduling module, Sort according to the priority of the podgroup to which the pod belongs. If the priorities are the same, sort according to the initialization timestamp of the podgroup; for pods without a specified podgroup, the scheduler divides the pod into the default podgroup; Filter all nodes in the cluster according to the resources required by the pod, and select nodes that meet the resource requirements; Score the nodes according to the scoring strategy, select the node with the highest score, perform pre-scheduling processing on the highest-scoring node, and pre-occupy node resources; Judge whether there are enough pods in the podgroup that are successfully scheduled. If the requirements are met before the expiration time set in the podgroup, bind these pods to the corresponding nodes; If the requirements are not met within this time, roll back all pods in the podgroup, release the pre-occupied resources, and wait for re-scheduling; among them, the expiration time is set according to the task requirements.

3. The batch scheduling system based on interference level and communication cost according to claim 1, characterized in that: In the GPU scheduling optimization module, the process of continuously updating the scheduling strategy using the shuffled frog leaping algorithm is as follows: Define the objective formula as: αI(M)+βC(M); where α and β are weights; Set the convergence value z and the maximum number of iterations g according to the task requirements, where the default value of g is 16; Set the number of frogs to be the same as the number of tasks to be scheduled; Calculate the fitness value of each frog through the target formula, and sort each frog from high to low according to the fitness value to obtain the sorted population; Divide the sorted population into m meme groups, and mark the frog with the best fitness value in each meme group as F b , and mark the frog with the best fitness value in the sorted population as F g ; Perform local update by moving each frog in the meme group towards the frog with the best fitness value in the meme group. The update formula is: where r is a random number between 0 and 1, D is the distance the frog moves, and F i is the fitness value of the frog numbered i in the meme group, D max is the maximum distance the frog is allowed to move, D min is the minimum distance the frog is allowed to move; If the new position after the frog jumps has a better fitness value, update the fitness value; otherwise, let the frog jump towards the frog with the best fitness value in the sorted population to perform local update. The update formula is: If the new position after the frog jumps has a better fitness value, update the fitness value; otherwise, generate a new frog at a random position to replace the original frog, and calculate the fitness value of the new frog; After performing local update operations on all frogs, update F by re - mixing and sorting all frogs and dividing them into meme groups b and F g , and perform local update operations again; repeat the above process of local update operations until the fitness values of all frogs are less than z or the maximum mixing iteration number g is reached.

4. A batch scheduling method based on interference level and communication cost, including: (1) Sort the pods in the scheduling queue so that pods in the same group are located in the same area of the scheduling queue; Take out a pod from the head of the scheduling queue. According to the resources requested by the user for the pod, select all nodes that meet the resource requirements from the node list, and score all nodes that meet the resource requirements according to the scoring strategy to obtain the highest-scoring node; Perform pre-scheduling processing on the highest-scoring node to pre-occupy node resources; judge whether there are enough pods in the podgroup that are successfully scheduled. If the requirements are met, bind these pods to the corresponding nodes; if the requirements are not met, roll back all pods in the podgroup and wait for re-scheduling; among them, batch scheduling is carried out in units of podgroup, and a pod is a container group that includes one or more containers sharing network, CPU, and GPU resources; (2) Perform fine-grained scheduling on GPU resources; select a task from the podgroup, where a pod is a task and one pod represents one task; respectively perform vector representation on the selected task and the n tasks running on the GPU to obtain n + 1 task vectors; input the n + 1 task vectors into a multi-layer perceptron to obtain the average performance degradation of the selected task on the GPU; re-select a task in the podgroup and repeat the above operations until the average performance degradation of all tasks in the podgroup on the GPU is obtained, and then calculate the interference level; define the model synchronization cost between GPUs as msc ij ; construct a GPU topology tree according to the connection method between GPUs and calculate msc based on the weights of the GPU topology tree ij ; define the updated model data as muds ij , calculate the communication cost using the synchronization cost and the updated model data, and sum the weighted communication costs to obtain the weighted sum of communication costs; use the interference level and the weighted sum of communication costs as fitness, and then continuously update the scheduling strategy using the shuffled frog leaping algorithm to obtain the final scheduling strategy; In step (2), the calculation process of the interference level is: Define an interference-aware multi-layer perceptron: Among them, W 1 ……W L-1 is the weight matrix, and L is the number of layers of the multi-layer perceptron; is the transpose of the weight matrix; b 1 ……b L is the bias vector; g 2 ……g L is the Relu activation function; h is the weight of the output layer; h T is the transpose of the weight of the output layer; α 1 ……α L-1 is the input of the middle layer of the multi-layer perceptron; φ 1 ……φ L is the output of the middle layer of the multi-layer perceptron, T 1 ……T n+1 is the task vector; σ is the mapping method; is the predicted value of the performance degradation vector; During the training process of the multi-layer perceptron, the weight matrix W is updated through the gradient descent algorithm to minimize the predicted value of the performance degradation vector and the target value z of the performance degradation vector t The mean square loss between them is calculated, and regularization is used to prevent overfitting. The loss function L is as follows: where λ is the regularization weight; w t is the training example weight; m is the dimension of the task vector; Input n+1 task vectors into a multi-layer perceptron to obtain the predicted values of the performance degradation vectors The predicted values of the performance degradation vectors have n+1 elements, corresponding to the performance degradations of n+1 tasks respectively, and then the average performance degradation SD is calculated through the following formula i : According to the average performance degradation of all subtasks in distributed training, and obtain the interference level I(M) according to the following calculation formula: where m is the number of tasks in the podgroup; In step (2), the calculation process of the communication cost is: Define GPU i With GPU j The model synchronization cost between them is msc ij Define the updated model data as muds ij ; Obtain the connection relationship between GPUs and establish a GPU topology tree according to this connection relationship, and calculate the model synchronization cost msc according to the weight calculation model of the GPU topology tree ij ; According to the model synchronization cost msc ij and the updated model data muds ij the communication cost C(M) is calculated, and the calculation formula is: where n is the number of GPUs, and i and j are the corresponding numbers of GPUs.

5. The batch scheduling method based on interference level and communication cost according to claim 4, characterized in that: In step (1), Sort according to the priority of the podgroup to which the pod belongs. If the priorities are the same, then sort according to the initialization timestamp of the podgroup; for pods without a specified podgroup, the scheduler divides the pod into the default podgroup; Filter all nodes in the cluster according to the resources required by the pod, and select the nodes that meet the resource requirements; Score the nodes according to the scoring strategy, select the node with the highest score, perform pre-scheduling processing on the highest-scoring node, and pre-occupy node resources; Judge whether there are enough pods in the podgroup that are successfully scheduled. If the requirements are met before the expiration time set in the podgroup, bind these pods to the corresponding nodes; If the requirements are not met within this time, roll back all pods in the podgroup, release the pre-occupied resources, and wait for re-scheduling; among them, the expiration time is set according to the task requirements.

6. The batch scheduling method based on interference level and communication cost according to claim 4, characterized in that: In step (2), the process of continuously updating the scheduling strategy using the Shuffled Frog Leaping Algorithm is as follows: Define the objective formula as: αI(M) + βC(M); where α and β are weight values; Set the convergence value z and the maximum number of iterations g according to the task requirements, where the default value of g is 16; Set the number of frogs to be the same as the number of tasks to be scheduled; Calculate the fitness value of each frog through the objective formula, and sort each frog from high to low according to the fitness value to obtain the sorted population; Divide the sorted population into m meme groups, and mark the frog with the best fitness value in each meme group as F b , and mark the frog with the best fitness value in the sorted population as F g ; Perform local update by making each frog in the meme group jump towards the frog with the best fitness value in the meme group, and the update formula is: where r is a random number between 0 and 1, D is the distance the frog moves, and F i is the fitness value of the frog numbered i in the meme group, D max is the maximum distance the frog is allowed to move, D min is the minimum distance the frog is allowed to move; If the new position after the frog jumps has a better fitness value, update the fitness value; otherwise, make the frog jump towards the frog with the best fitness value in the sorted population to perform local update, and the update formula is: If the new position after the frog jumps has a better fitness value, update the fitness value; otherwise, generate a new frog at a random position to replace the original frog, and calculate the fitness value of the new frog; After performing local update operations on all frogs, update F by re - sorting all frogs and dividing them into meme groups through mixing operations b and F g , and then perform local update operations again; repeat the above process of local update operations until the fitness values of all frogs are less than z or the maximum mixing iteration number g is reached.

Images