Grid load balancing scheduling method based on time-series partitioning and model hybrid parallel optimization
By adopting the method of time-sequence segmentation and model hybrid parallel optimization in large-scale computing grids, the problem of inefficient task scheduling is solved, load balancing and efficient task processing are achieved, and the grid service quality is significantly improved.
Patent Information
- Application Number
- CN202210685414.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-17
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-06-17
AI Technical Summary
The prior art has low task scheduling efficiency in large-scale computing grids, especially when the data volume is large, memory reading difficulty affects the speed of iterative training, and the execution efficiency of the scheduling algorithm is not high, so it cannot effectively realize load balancing.
Using the grid load balancing scheduling method based on time-series segmentation and model mixed parallel optimization, by segmenting historical data into data blocks in parallel processing, using parallelized two-dimensional long and short-term memory network training, predicting the workload of computing nodes, and based on the minimum work The scheduling is completed with the principle of minimum load matching.
It significantly improves scheduling efficiency, realizes load balancing among nodes, speeds up task request processing, ensures a good user experience, and is better than the traditional Min-Min scheduling methods and existing scheduling strategies.
Smart Images

Figure CN114968596B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of task scheduling of computer software and large-scale computing grids and relates to a grid load balancing scheduling method based on time series segmentation and model hybrid parallel optimization. Background Art
[0002] Grid computing is a type of distributed computing. It studies how to divide a problem that requires a huge amount of computing power to solve into many small parts, and then divide these parts among many computers and processors for processing, and finally combine these calculation results to get the final result. With the rapid development of the Internet, grid computing has become a common computing model for processing computationally intensive problems. Large-scale computing grids use the Internet to organize computing resources scattered in different geographical locations into a virtual supercomputer. When high-tech fields or cutting-edge technology research face large-scale computing problems, this virtual supercomputer can provide them with the required computing speed and storage capacity. In recent years, grid computing has been widely used in genomics, molecular dynamics, quantum chemistry, and other scientific research fields.
[0003] As the key to high performance, high availability, low cost and avoiding single point failure of large-scale computing grids, task scheduling has received extensive attention from grid researchers. Efficient task scheduling is crucial to the operation efficiency of large-scale distributed heterogeneous computing grids. A good scheduling method should have the characteristics of high efficiency and low latency. In the inventor's early patent application CNCN201910442178.4, the inventor proposed a large-scale computing grid sub-node workload prediction model based on a two-dimensional long short-term memory neural network, such as Figure 1 As shown, in this application, all data is read into the memory at one time, and the status values of the total time and date present this mesh structure. The disadvantage is that when the amount of data is large, the memory cannot be read, and the conduction value has little effect. This approach will make the acquisition of the status value of subsequent iterative training slow, and thus the overall iteration speed will be slow; further, in CNCN201910442178.4, the inventor combined this model to generate a scheduling algorithm for load balancing. Figure 2 As shown in the figure, however, the huge amount of training data, complex model structure, and unique time characteristics of different computing nodes make the execution efficiency of this algorithm not high in actual scheduling. Summary of the invention
[0004] The technical problem to be solved by the present invention is to overcome the above-mentioned defects in the prior art and provide a grid load balancing scheduling method based on time series segmentation and model hybrid parallel optimization.
[0005] A grid load balancing scheduling method based on time series segmentation and model hybrid parallel optimization includes the following steps: S100: historical data is segmented into different data blocks, and the segmented data blocks are distributed to different processors; S200: for each single processor, parallelized two-dimensional long short-term memory network cell training is used; S300: error values calculated by different processors are collected, the average value is obtained, and the intra-cell matrix of the previous moment is updated; S400: iterative training is performed to obtain a two-dimensional long short-term memory network that can predict the workload of each computing node in the grid; S500: using the obtained two-dimensional long short-term memory network, according to the frequency of each computing node, the workload of each sub-node is predicted in parallel; S600: according to the principle of matching the minimum workload with the minimum load, the binary of the computing node and the job is determined and the scheduling is completed.
[0006] Further, the step S100 includes: S101: determine that the training block length in the date direction is D+1, and the load in the date direction is required to be arranged in sequence D days before the current time point, and D+1 is used to generate the initial state value; S102: determine that the training block length in the time direction is T+2, and the load in the time direction is required to be arranged in sequence T hours before the current time point, and T+2 is used to generate the initial state value; S103: divide the data into (T+2)*(D+1)-2 data blocks. The step S200 includes: determine the processing data volume and combination method of a single processor, and the training formula of any parameter in the two-dimensional long short-term memory network cell structure of the child node is shown in formula (1), wherein W is any parameter in the two-dimensional long short-term memory network cell structure, α is the learning rate, ΔW is the partial derivative of the loss function with respect to any parameter W, the superscript represents the time, and the subscript coordinate represents the matrix position;
[0007]
[0008] (1) The calculation method of ΔW is as follows. Where LOSS represents the loss function, f represents an arbitrary function, and n is the total amount of data;
[0009]
[0010] Divide (2) into k segments:
[0011]
[0012] Formula (3) is abstracted as follows: where l represents the data set of a single processor, k is the number of processors;
[0013]
[0014] Pick The original formula (2) is transformed into:
[0015]
[0016] At this time, the loss calculation method is converted into each processor processing m copies The amount of data per processor is The combination method is the average value of the sum of the calculation results of each processor.
[0017] Optionally, the step S300 includes: taking the forget gate matrix of a two-dimensional long short-term memory network as an example, the partial derivative of a single value in the matrix is calculated as follows:
[0018]
[0019] Step S500 includes: S501: collecting the job set JS and the computing node set NS within the scheduling interval. Different computing nodes have different computing frequencies, and the frequencies are normalized; initializing the properties of the grid computing nodes, specifically including: minimum CPU core number limit - MinCL (NS j ), Maximum CPU core limit - MaxCL (NS j ), Available cores ACN (NS j ), runtime limit WTL (NS j ), deploy application software set AV (NS j ). The properties of the initialization job set include: the number of cores required - CN (JS i ), estimated maximum running time-WT(JS i ), required software APP (JS i ); S502: Using the parallelized two-dimensional long short-term memory network cell structure for each node ACN (NS j ) for prediction. The step S502 includes: inputting the previous T moments and D dates of each node into a vector in order, which can be expressed as the following formula (7): where NWV j represents the input load vector, NW j Indicates the load at the corresponding moment. ACN(NS j ) is calculated as in formula (8). At this point, the number of available cores of each computing node at the future moment has been predicted, and the grid resources will be further reasonably allocated based on this;
[0020]
[0021]
[0022] Preferably, the step S600 includes: S601: according to the principle of matching the job with the smallest workload with the computing node with the smallest load, a tuple of the computing node and the job is obtained, and the conditions for matching the job with the computing node are as follows;
[0023]
[0024] S602: Combine the node and the job into a tuple (JS i , NS i ) Complete the scheduling, deploy the corresponding job to the computing node, and update the job computing node set and the job set; S603: Determine whether the job set is empty. If not, repeatedly execute step S601 and step S602.
[0025] The present invention provides a grid load balancing scheduling method based on time segmentation and model hybrid parallel optimization, which is of great significance for improving grid service quality, accelerating the processing speed of task requests, ensuring a good user experience, and better realizing load balancing.
[0026] Aiming at the constraint dependency between jobs and computing nodes in large-scale computing grids, the imbalance of computing sub-node workloads, and the low execution efficiency of existing task scheduling strategies, the present invention proposes a grid load balancing scheduling method based on time series segmentation and model hybrid parallel optimization. This method can effectively improve scheduling efficiency, achieve load balancing between nodes, improve grid service quality, speed up the processing speed of task requests, and ensure good user experience. Experimental results show that this method is significantly superior to traditional Min-Min scheduling methods and existing scheduling methods in terms of load balancing and execution efficiency.
[0027] The scheduling simulation was carried out by simulating the scheduling center of the Chinese Academy of Sciences. The time interval was 600 seconds and the number of computing nodes was 46. CUDA was used to parallel optimize the training and runtime prediction of the two-dimensional long short-term memory network. The acceleration efficiency and prediction results were compared as follows: Figure 8 As for the overall scheduling algorithm, we compared Min-Min and the existing load optimization scheduling methods. Our proposed algorithm is 12.27% better than the Min-Min algorithm and 93.24% better than the existing scheduling strategy. Fig. 9 The results show that this method is significantly better than the algorithm in balancing the load and improving the efficiency of the algorithm. It can effectively improve the efficiency of the algorithm and achieve load balancing between nodes. It has a good effect on improving the quality of grid services, speeding up the processing speed of task requests, and ensuring a good user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 The data reading method of the prior art;
[0029] Figure 2 It is a two-dimensional long short-term memory network cell structure in the prior art;
[0030] Figure 3 is a flow chart of the scheduling method of the present invention;
[0031] Figure 4 It is the overall execution flow chart;
[0032] Figure 5 The parallel data reading method of the present invention;
[0033] Figure 6 It is a two-dimensional long short-term memory network cell structure;
[0034] Figure 7 For the parallelism of the model in back-propagation;
[0035] Figure 8 A hybrid parallel prediction architecture for grid computing node data and models;
[0036] Fig. 9 The pseudo code of the algorithm for executing step 5 is shown below:
[0037] Fig.10 Comparison of speedup and prediction results for parallelized 2D LSTM networks;
[0038] Fig.11 This is a comparison chart of the average load between this scheduling algorithm, Min-Min, and the original scheduling strategy. DETAILED DESCRIPTION
[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, so that the above-mentioned and other objects, features and advantages of the present invention will be clearer. The same reference numerals in all the drawings indicate the same parts. The drawings are not deliberately drawn to scale, and the focus is on illustrating the subject matter of the present invention.
[0040] Aiming at the constraint dependency between jobs and computing nodes in large-scale computing grids, the imbalance of computing sub-node workloads, and the low execution efficiency of existing task scheduling strategies, the present invention proposes a grid load balancing scheduling method based on time series segmentation and model hybrid parallel optimization. This method can effectively improve scheduling efficiency, achieve load balancing between nodes, improve grid service quality, speed up the processing speed of task requests, and ensure good user experience. Experimental results show that this method is significantly superior to traditional Min-Min scheduling methods and existing scheduling methods in terms of load balancing and execution efficiency.
[0041] See also Figure 3 , Figure 4 The scheduling method of the present invention comprises the following steps:
[0042] S100: dividing the historical data into different data blocks, and distributing the divided data blocks to different processors.
[0043] Different sub-nodes in large-scale computing grids have different date and time characteristics. The time series historical data of a single sub-node is segmented according to the characteristics. The original data is read in as follows: Figure 1 As shown. The characteristic is that all data is read into the memory at one time, and the state values of the total time and date present this mesh structure. The disadvantage is that when the amount of data is large, the memory cannot be read in, and the conduction value has little effect, this approach will make the acquisition of the state value of subsequent iterative training slow, and thus the overall iteration speed will be slow.
[0044] The present invention performs segmentation training by observing the characteristics of different sub-nodes on the date and time axis. Specifically:
[0045] S101: Determine that the training block length in the date direction is D+1 (the date direction requires the loads arranged in order D days before the current time point, and D+1 is used to generate the initial state value);
[0046] S102: Determine that the training block length in the time direction is T+2 (the time direction requires the loads arranged in sequence T hours before the current time point, and T+2 is used to generate the initial state value).
[0047] S103: Divide the data into (T+2)*(D+1)-2 data blocks. The improved data reading method is as follows Figure 5 As shown. It represents the data block after segmentation. Usually D = 7, T = 24, and the size of the training block is (7+1)*(24+2)-2. That is to say, using n (7+1)*(24+2)-2 blocks to train a sub-node can obtain a neural network that can predict the load of the sub-node. Different sub-nodes have different date and time features, so the values of T and D are uncertain.
[0048] S200: For each single processor, use parallelized two-bit long short-term memory network cell training.
[0049] After splitting the historical data, we obtain the time series data blocks for parallel training of each child node. First, we determine the processing data volume and combination method of a single processor.
[0050] The general training formula for any parameter in the 2D LSTM network cell structure of the child node is shown in formula (1). Where W is any parameter in the 2D LSTM network cell structure, α is the learning rate, ΔW is the partial derivative of the loss function with respect to any parameter W, the superscript represents the time, and the subscript coordinate represents the matrix position.
[0051]
[0052] (1) The calculation method of ΔW is as follows: Where LOSS represents the loss function, f represents an arbitrary function, and n represents the total amount of data.
[0053]
[0054] Divide (2) into k segments:
[0055]
[0056] Formula (3) is abstracted as follows: Where l represents the data set of a single processor and k is the number of processors.
[0057]
[0058] We take The original formula (2) is transformed into:
[0059]
[0060] At this time, the loss calculation method is converted into each processor processing m copies The amount of data per processor is The combination method is the average value of the sum of the calculation results of each processor.
[0061] S300: Collect the error values calculated by different processors, obtain the average value, and update the intracellular matrix at the previous moment.
[0062] Generally speaking, computers take much longer to calculate multiplication and division than addition and subtraction, and the partial derivatives in the training of 2D LSTM networks involve a large number of multiplications. Time series data parallelism is far from enough, so we propose parallelism in the training of 2D LSTM cell structures. Figure 6 As shown. Taking the forget gate matrix of a two-dimensional long short-term memory network as an example, the partial derivative of a single value in the matrix is calculated as follows:
[0063]
[0064] We will parallelize the partial derivative calculation of the matrix in the 2D LSTM network cell structure to speed up its training, such as Figure 7 As shown, we execute the blocks on the same line of parameters in the structure in parallel.
[0065] S400: Perform iterative training to obtain a two-dimensional long short-term memory network that can predict the workload of each computing node in the grid.
[0066] After completing the above step S300, we obtain the data blocks used for training of each computing node and the parallelized two-dimensional long short-term memory network cell structure. We nest the two parallel methods for training. If the iterative training is not completed, this step is repeatedly executed. If it is completed, the next step is executed.
[0067] S500: Using the obtained two-dimensional long short-term memory network, according to the frequency of each computing node, the workload of each sub-node is predicted in parallel.
[0068] See also Figure 8 After completing the iterative training, we have obtained the two-dimensional long short-term memory network cell structure for workload prediction of each node in a large-scale computing grid. We will use these networks to predict the parallel workload of each computing node, thereby promoting the rational allocation of computing resources.
[0069] S501: Collect the job set JS and computing node set NS within the scheduling interval. Different computing nodes have different computing frequencies, so we normalize the frequencies (actually, we change the units, change the measurement numbers, and divide by a fixed value). Initialize the properties of the grid computing nodes. Specifically, it includes: Minimum CPU core limit - MinCL (NS j ), Maximum CPU core limit - MaxCL (NS j ), Available cores ACN (NS j ), runtime limit WTL (NS j ), deploy application software set AV (NS j ). The properties of the initialization job set include: the number of cores required - CN (JS i ), estimated maximum running time-WT(JS i ), required software APP (JS i ).
[0070] S502: Using the parallelized two-dimensional long short-term memory network cell structure for each node's ACN (NS j ) for prediction. Specifically: the previous T moments and D dates of each node are sequentially input into a vector, which can be expressed as the following formula (7): j represents the input load vector, NW j Indicates the load at the corresponding moment. ACN(NS j ) is calculated as shown in formula (8). At this point, we have predicted the number of available cores of each computing node at the future moment, and we will further reasonably allocate grid resources based on this.
[0071]
[0072]
[0073] S600: According to the principle of matching the minimum workload with the minimum load, determine the binary pair of computing node and job and complete the scheduling.
[0074] After completing the parallel computing node resource prediction, the load imbalance of large-scale computing grids requires a reasonable combination of computing nodes and jobs, which specifically includes the following steps:
[0075] S601: According to the principle that the job with the smallest workload matches the computing node with the smallest load, a pair of computing nodes and jobs is obtained. The conditions for matching a job with a computing node are as follows.
[0076]
[0077] S602: Combine the node and the job into a tuple (JS i , NS i ) completes the scheduling, deploys the corresponding jobs to the computing nodes, and updates the job computing node set and job set. The update method is as follows.
[0078]
[0079] S603: Determine whether the job set is empty. If not, repeatedly execute step S601 and step S602. If it is empty, then complete the scheduling of the time interval and end. The load balancing scheduling algorithm for parallel workload prediction shown in step S600 is as follows: Fig. 9 shown.
[0080] Aiming at the problems of constraint dependency between tasks and computing nodes, unbalanced workload of computing subnodes, and low efficiency of existing task scheduling strategies under large-scale computing grids, this paper proposes a grid load balancing scheduling method based on time segmentation and model hybrid parallel optimization. We simulated the scheduling center of the Chinese Academy of Sciences, with a time interval of 600 seconds and 46 computing nodes. We used CUDA to perform parallel optimization of the training and runtime prediction of the two-dimensional long short-term memory network. The acceleration efficiency and prediction results were compared as shown in the figure. Fig.10 As for the overall scheduling algorithm, we compared Min-Min and the existing load optimization scheduling methods. Our proposed algorithm is 12.27% better than the Min-Min algorithm and 93.24% better than the existing scheduling strategy. Fig.11 As shown. The results show that this method is significantly better than the algorithm in balancing the load and improving the efficiency of the algorithm. It can effectively improve the efficiency of the algorithm and achieve load balancing between nodes. It has a good effect on improving the quality of grid services, speeding up the processing speed of task requests, and ensuring a good user experience.
[0081] Many specific details are described in the above description to facilitate a full understanding of the present invention. However, the above description is only a preferred embodiment of the present invention. The present invention can be implemented in many other ways different from those described herein, so the present invention is not limited to the specific implementation disclosed above. At the same time, any person familiar with the art can make many possible changes and modifications to the technical solution of the present invention using the methods and technical contents disclosed above without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes. Any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.
Claims
1. A grid load balancing scheduling method based on time series segmentation and model hybrid parallel optimization, The method is characterized in that it comprises the following steps: S100: dividing the historical data into different data blocks, and distributing the divided data blocks to different processors; S200: for each single processor, use parallelized 2D LSTM network cell training; S300: Collect the error values calculated by different processors, obtain the average value, and update the intracellular matrix at the previous moment; S400: performing iterative training to obtain a two-dimensional long short-term memory network capable of predicting the workload of each computing node in the grid; S500: using the obtained two-dimensional long short-term memory network, predicting the workload of each sub-node in parallel according to the frequency of each computing node; S600: According to the principle of matching the minimum workload with the minimum load, determine the binary pair of computing node and job and complete the scheduling; The step S100 includes: S101: Determine that the training block length in the date direction is D+1. The date direction requires the loads arranged in order D days before the current time point. D+1 is used to generate the initial state value. S102: Determine that the training block length in the time direction is T+2. The time direction requires the loads arranged in sequence T hours before the current time point. T+2 is used to generate the initial state value. S103: Divide the data into (T+2)*(D+1)-2 data blocks; The step S200 includes: Determine the amount of data processed by a single processor and the combination method. The general training formula of any parameter in the two-dimensional long short-term memory network cell structure of the child node is shown in formula (1), where W is any parameter in the two-dimensional long short-term memory network cell structure, α is the learning rate, ΔW is the partial derivative of the loss function with respect to any parameter W, the superscript represents the time, and the subscript coordinate represents the matrix position; (1) The calculation method of ΔW is as follows, where LOSS represents the loss function, f represents an arbitrary function, and n is the total amount of data; Divide (2) into k segments: Formula (3) is abstracted as follows, where l represents the data set of a single processor and k is the number of processors; Pick The original formula (2) is transformed into: At this time, the loss calculation method is converted into each processor processing m copies The amount of data per processor is The combination method is the average value of the sum of the calculation results of each processor.
2. The grid load balancing scheduling method based on time series segmentation and model hybrid parallel optimization according to claim 1 is characterized in that: The step S300 includes: Taking the forget gate matrix of a two-dimensional long short-term memory network as an example, the partial derivative of a single value in the matrix is calculated as follows:
3. The grid load balancing scheduling method based on time series segmentation and model hybrid parallel optimization according to claim 2 is characterized in that: The step S400 includes: performing training in a nested manner in parallel.
4. The grid load balancing scheduling method based on time segmentation and model hybrid parallel optimization according to claim 3 is characterized in that: The step S500 includes: S501: Collect the job set JS and computing node set NS within the scheduling interval. Different computing nodes have different computing frequencies, so the frequencies are normalized. Initialize the properties of the grid computing nodes, including: minimum CPU core count limit - MinCL (NS j ), Maximum CPU core limit - MaxCL (NS j ), Available cores ACN (NS j ), runtime limit WTL (NS j ), deploy application software set AV (NS j ); The properties of the initialization job set include: the number of cores required - CN (JS i ), estimated maximum running time-WT(JS i ), required software APP (JS i ); S502: Using the parallelized two-dimensional long short-term memory network cell structure for each node's ACN (NS j ) to make predictions.
5. The grid load balancing scheduling method based on time segmentation and model hybrid parallel optimization according to claim 4 is characterized in that: The step S502 includes: inputting the previous T moments and D dates of each node into a vector in order, which can be expressed as the following formula (7): where NWV j represents the input load vector, NW j Indicates the load at the corresponding moment, ACN(NS j ) is calculated as in formula (8). At this point, the number of available cores of each computing node at the future moment has been predicted, and the grid resources will be further reasonably allocated based on this; 6. The grid load balancing scheduling method based on time series segmentation and model hybrid parallel optimization according to claim 5 is characterized in that: The step S600 includes: S601: According to the principle that the job with the smallest workload matches the computing node with the smallest load, a tuple of the computing node and the job is obtained. The conditions for matching the job with the computing node are as follows; S602: Combine the node and the job into a tuple (JS i , NS i ) Complete the scheduling, deploy the corresponding jobs to the computing nodes, and update the job computing node set and job set; S603: Determine whether the job set is empty. If not, repeatedly execute step S601 and step S602.
7. The grid load balancing scheduling method based on time series segmentation and model hybrid parallel optimization according to claim 6 is characterized in that: The update method is as follows:
8. The grid load balancing scheduling method based on time segmentation and model hybrid parallel optimization according to any one of claims 2 to 7, characterized in that: D=7, T=24.
