A cluster job waiting time prediction method based on FELSTM model

Abstract

The present invention discloses a cluster job waiting time prediction method based on a FELSTM model, and relates to the field of data analysis technology. The present invention comprises the following steps: collecting historical job data from a target computer system and normalizing the historical job data; dividing the normalized historical job data into an input sequence and an output sequence, wherein the input sequence is data used to predict the output sequence, and the output sequence is a target sequence that the model expects to output; dividing the divided input sequence and output sequence into a training data set, a validation set, and a test set; constructing a FELSTM model through a deep learning framework, setting the loss function to SmoothL1Loss, and using the Adam optimizer for parameter optimization; performing hyperparameter optimization on the FELSTM model so that the loss function of the model converges stably and reaches a minimum value during training; and after training, verifying the model performance and evaluating the prediction accuracy on the test set.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, specifically to a method for predicting cluster job waiting time based on the FELSTM model. Background Technology

[0002] In busy computing clusters, the scheduling efficiency of the job scheduling system significantly impacts the overall cluster utilization. As high-performance computing plays an increasingly important role in scientific research, the difficulty in matching computing resources with ever-growing computing demands is becoming increasingly prominent. Therefore, how to effectively and fully utilize the limited resources in high-performance computing clusters and implement more intelligent job scheduling is a necessary and worthy subject of in-depth research.

[0003] The foundation of intelligent cluster scheduling lies in predicting the waiting time required for cluster jobs to queue (referred to as cluster job waiting time). While the Load Sharing Facility (LSF) cluster management software provides a function to predict job waiting time, this requires users to specify the upper limit of each job's runtime or an estimated runtime in advance. Therefore, some researchers have begun to study the prediction of job runtime to indirectly predict cluster job waiting time. However, since the runtime prediction can only be performed on specific jobs, indirectly predicting job waiting time by predicting job runtime has limitations in practicality and flexibility. Job waiting time is related to multiple factors, including the status of existing jobs in the cluster, the user's job type, and usage habits, making it difficult to effectively predict using traditional statistical analysis or machine learning methods. Summary of the Invention

[0004] The purpose of this invention is to provide a cluster job waiting time prediction method based on the FELSTM model to solve the problems mentioned in the background art.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] A cluster job waiting time prediction method based on the FELSTM model, the method includes the following steps:

[0007] Step S100. Collect historical job data from the target computer system and normalize the historical job data;

[0008] Step S200. Divide the normalized historical job data into an input sequence and an output sequence. The input sequence is the data used to predict the output sequence, and the output sequence is the target sequence that the model expects to output. Then, divide the input sequence and output sequence into a training dataset, a validation set, and a test set.

[0009] Step S300. Construct a FELSTM model using a deep learning framework, set the loss function to SmoothL1Loss, and optimize the parameters using the Adam optimizer;

[0010] Step S400. Optimize the hyperparameters of the FELSTM model so that the loss function converges stably and reaches its minimum value during training; after training, verify the model performance and evaluate the prediction accuracy on the test set.

[0011] Historical job data includes job ID, number of resources requested by the user, number of available resources by the user, number of resources waiting by the user, number of jobs waiting by the user, job execution time, user CPU usage time, number of available resources in the queue, number of resources waiting in the queue, and job waiting time.

[0012] The job ID refers to the unique identifier assigned by the system when a job is submitted; the number of user-requested resources refers to the number of CPU cores requested by the user when a job is submitted; the number of user-idle resources refers to the maximum number of cores available in the general computing queue when a job is submitted, minus the number of cores already used by the user; the number of user-waiting resources refers to the total number of cores the user is waiting to run when a job is submitted; the number of user-waiting jobs refers to the number of jobs the user is waiting to run when a job is submitted; the user job running time refers to the total running time of jobs already run by the user when a job is submitted, in seconds; the user CPU usage time refers to the total CPU time the user has already run jobs when a job is submitted, in seconds; the number of queue-idle resources refers to the number of cores available in the queue when a job is submitted; the number of queue-waiting resources refers to the total number of cores waiting to run in the queue when a job is submitted; and the job waiting time refers to the time required for a job to run from submission, in seconds.

[0013] In step S100, the specific process of normalizing historical job data is as follows:

[0014] For historical task data, the Min-Max method is used to normalize the historical task data. The calculation formula is as follows:

[0015] y'=(yi-y_min) / (y_max-y_min),

[0016] Where y' represents the normalized historical task data, yi represents the historical task data in the original sample, y_max represents the maximum value of the corresponding historical task data in the original sample, and y_min represents the minimum value of the corresponding historical task data in the original sample.

[0017] Step S200 includes:

[0018] S201. Organize the normalized historical task data into a historical task dataset D, where D = {d1, d2, ..., dn}, where d1 represents the feature vector corresponding to the first historical task, d2 represents the feature vector corresponding to the second historical task, and so on, with dn representing the feature vector corresponding to the nth historical task, and n representing the data number of the historical task, which is a positive integer; extract the task waiting time from the historical task dataset D and form a task waiting time set W, where W = {w1, w2, ..., wn}, similarly, w1 represents the task waiting time corresponding to the first historical task, w2 represents the task waiting time corresponding to the second historical task, and wn represents the task waiting time corresponding to the nth historical task;

[0019] S202. Define the input step size H, representing the number of consecutive jobs contained in each input sequence; divide the input sequence and the output sequence, where the following operations are performed on the input sequence:

[0020] For each job i, from H to n, generate an input sequence Xi, where Xi = [di-H+1,di-H+2,...,di]. Set the waiting time of the current job to -1, and the corresponding input sequence Xi = [di-H+1,di-H+2,...,di-1,di,-1].

[0021] The following operation applies to the output sequence: extract the waiting time of the current job from the job waiting time set W as the output sequence Yi, and Yi = wi;

[0022] Summarize all input sequences and output sequences to form the input sequence set X and the output sequence set Y, respectively, where the input sequence set X = [XH, XH+1, ..., Xn] and the output sequence set Y = [YH, YH+1, ..., Yn].

[0023] S203. Define the total number of samples in the input sequence set X and the output sequence set Y as N, and N = n - H + 1; divide the dataset according to the following proportions: training set N_train is 80%, validation set N_val is 10%, and test set N_test is 10%; the specific number of partitions is: N_train = ⌊0.8 × N⌋, N_val = ⌊0.1 × N⌋, N_test = N - N_train - N_val, where ⌊⌋ represents rounding down; according to the calculated number of partitions, divide the input sequence set X and the output sequence set Y into training set, validation set and test set.

[0024] The FELSTM model structure in step S300 includes an input layer, a feature encoding layer, a multi-layer LSTM, a fully connected layer, and an output layer.

[0025] The input layer receives an input sequence Xi, and the dimension of the input sequence Xi is (N, H, F), where F represents the feature dimension; the feature encoding layer encodes the input sequence Xi to expand its dimension; the multi-layer LSTM constructs a Long Short-Term Memory (LSTM) network, and the state update process of the LSTM is as follows:

[0026] Input gate: i t =σ(W i ·[h t-1 ,x t ]+b i ), where i t σ represents the activation value of the input gate, determining the effect of the current input on the cell state; σ represents the sigmoid activation function, mapping the input to between 0 and 1; W i The weight matrix of the input gate is responsible for performing a linear transformation between the previous hidden state and the current input; [h t-1 ,x t ] indicates the connection to the hidden state h of the previous time step. t-1 and the input x at the current time step t The vector, b, is represented as a combined input; i This represents the bias vector of the input gate, used to adjust the activation value of the input gate and increase the flexibility of the model.

[0027] Forgotten Gate: f t =σ(W f ·[h t-1 ,x t ]+b f ), where f t The activation value of the forget gate controls the degree to which the cell state from the previous time step is retained in the current state, and its value ranges from 0 to 1; W f The weight matrix of the forget gate is responsible for performing a linear transformation between the previous hidden state and the current input; b f This represents the bias vector of the forget gate, used to adjust the activation value of the forget gate and enhance the flexibility of the model.

[0028] Output gate: o t =σ(W o ·[h t-1 ,x t ]+b o ), where o t W represents the activation value of the output gate, controlling the degree of influence of the current cell state on the current hidden state; its value ranges from 0 to 1. o The weight matrix of the output gate is responsible for performing a linear transformation between the previous hidden state and the current input; b o This represents the bias vector of the output gate, used to adjust the activation value of the output gate and enhance the flexibility of the model.

[0029] Cell state: C ~ t =tanh(W C ·[h t-1 ,x t ]+b C ), C t =f t ·C t-1 +i t ·C ~ t ; where C ~ t Represents the candidate cell state at the current time step; tanh represents the hyperbolic tangent activation function, which maps the input to between -1 and 1, providing a non-linear transformation; W C The weight matrix representing the cell state is responsible for performing a linear transformation between the previous hidden state and the current input; b C A bias vector representing the cell state, used to adjust the calculation of candidate cell states; C t It represents the cell state at the current time step, taking into account the influence of the previous state and the current input;

[0030] The fully connected layer passes the output of the multi-layer LSTM to the fully connected layer to generate the final output; the output layer directly uses the output of the fully connected layer as the final prediction value.

[0031] The feature encoding layer encodes the input sequence Xi to expand its dimension, specifically including:

[0032] Z t =tanh(W z ·x t +b z ), where Z t W represents the output after feature encoding. z Let b represent the weight matrix. z Indicates the bias term;

[0033] The fully connected layer passes the output of the multi-layer LSTM to the fully connected layer to generate the final output, specifically including:

[0034] y pred =W out ·h T +b out , where y pred W represents the final predicted output. out h represents the weight matrix of the output layer. T This represents the hidden state at the last time step, containing information about the entire input sequence; b out This represents the bias vector of the output layer.

[0035] Fully connected layers can meet the requirements of feature encoding layers, but using only fully connected layers also has problems. When processing input data, fully connected layers obtain output data by multiplying the weight matrix by the input data matrix and adding a bias term. However, during training, the sizes of the weight matrix and the bias term constantly change, which leads to the output data having a non-zero mean and an unrestricted range. This slows down the convergence speed of the optimization algorithm and makes it prone to gradient explosion. To solve this problem, a hyperbolic tangent (tanh) activation function is added after the fully connected layer to keep the output data having a zero mean and limiting its value range to [-1, 1], thereby accelerating the convergence speed of the optimization algorithm and preventing gradient explosion.

[0036] Step S300 sets the loss function to SmoothL1Loss, which includes the following:

[0037] For each job i, calculate the predicted value y. pred The loss L between the actual value and the true value i The true value is the output sequence Yi; the specific calculation formula is as follows:

[0038] ,

[0039] And the total loss is the average of all operation losses, L. total .

[0040] Step S300 uses the Adam optimizer to optimize parameters, specifically including:

[0041] In the implementation of the Adam optimizer, the update formulas for the first and second moments are as follows:

[0042] First-order moment update: m t =β1m t-1 +(1-β1)∇L total , where m t β1 represents the first moment at the current time step; β1 represents the decay rate of the first moment, typically taken as 0.9; m t-1 ∇L represents the first moment of the previous time step; total This represents the gradient of the current loss function with respect to the parameters;

[0043] Second moment update: v t =β2v t-1 +(1-β2)(∇L total ) 2 Among them, v t β2 represents the second moment at the current time step; β2 represents the decay rate of the second moment, typically taken as 0.999; v t-1 This represents the second moment of the previous time step;

[0044] The parameter update formula is:

[0045] θ new =θ old -(αm t ) / [(v t )^(1 / 2)+ϵ], where θ new This represents the updated parameter value, θ. old This represents the parameter values ​​before the update, α represents the learning rate, and ϵ represents a constant.

[0046] The specific content of hyperparameter optimization of the FELSTM model in step S400 includes:

[0047] Define the hyperparameters as input stride H, learning rate α, and number of hidden layer nodes N. h LSTM layer number N L and the number of feature encoding nodes N f The grid search method was chosen for hyperparameter optimization, and for each set of parameter combinations (H, α, N)... h N L N f Perform the following operations: Build and train a FELSTM model using different combinations of hyperparameters, and record the loss function L during each training process. total The value;

[0048] For each combination of hyperparameters, calculate the average loss L on the validation set. avg Choose one that results in an average loss L on the validation set. avg The smallest combination of hyperparameters is used as the final hyperparameters.

[0049] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention introduces a feature encoding layer, which combines a fully connected layer with a hyperbolic tangent (tanh) activation function, aiming to efficiently extract features from the input data. This design not only accelerates the convergence process of the optimization algorithm but also effectively avoids the gradient explosion problem. Based on this, the feature encoding layer is combined with a Long Short-Term Memory (LSTM) network to develop the FELSTM model. When this model is used to predict the waiting time of clustered jobs, it not only avoids the data "overwhelming" problem caused by the low feature dimension of clustered job data but also improves the efficiency of the training process and the accuracy of the prediction results. Attached Figure Description

[0050] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0051] Figure 1This is a schematic diagram of the process of a cluster job waiting time prediction method based on the FELSTM model according to the present invention.

[0052] Figure 2 This is a schematic diagram of the data collection results for a cluster job waiting time prediction method based on the FELSTM model according to the present invention.

[0053] Figure 3 This is a data normalization diagram illustrating a cluster job waiting time prediction method based on the FELSTM model according to the present invention.

[0054] Figure 4 This is a schematic diagram of the input and output sequences of a cluster job waiting time prediction method based on the FELSTM model according to the present invention.

[0055] Figure 5 This is a diagram showing the overall structure of the FELSTM model for a cluster job waiting time prediction method based on the FELSTM model according to the present invention.

[0056] Figure 6 This is a diagram showing the hyperparameter values ​​of a cluster job waiting time prediction method based on the FELSTM model according to the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Please see Figure 1 The present invention provides the following technical solution:

[0059] A cluster job waiting time prediction method based on the FELSTM model, the method includes the following steps:

[0060] Step S100. Collect historical job data from the target computer system and normalize the historical job data;

[0061] Step S200. Divide the normalized historical job data into an input sequence and an output sequence. The input sequence is the data used to predict the output sequence, and the output sequence is the target sequence that the model expects to output. Then, divide the input sequence and output sequence into a training dataset, a validation set, and a test set.

[0062] Step S300. Construct a FELSTM model using a deep learning framework, set the loss function to SmoothL1Loss, and optimize the parameters using the Adam optimizer;

[0063] Step S400. Optimize the hyperparameters of the FELSTM model so that the loss function converges stably and reaches its minimum value during training; after training, verify the model performance and evaluate the prediction accuracy on the test set.

[0064] Historical job data includes job ID, number of resources requested by the user, number of available resources by the user, number of resources waiting by the user, number of jobs waiting by the user, job execution time, user CPU usage time, number of available resources in the queue, number of resources waiting in the queue, and job waiting time.

[0065] The job ID refers to the unique identifier assigned by the system when a job is submitted; the number of user-requested resources refers to the number of CPU cores requested by the user when a job is submitted; the number of user-idle resources refers to the maximum number of cores available in the general computing queue when a job is submitted, minus the number of cores already used by the user; the number of user-waiting resources refers to the total number of cores the user is waiting to run when a job is submitted; the number of user-waiting jobs refers to the number of jobs the user is waiting to run when a job is submitted; the user job running time refers to the total running time of jobs already run by the user when a job is submitted, in seconds; the user CPU usage time refers to the total CPU time the user has already run jobs when a job is submitted, in seconds; the number of queue-idle resources refers to the number of cores available in the queue when a job is submitted; the number of queue-waiting resources refers to the total number of cores waiting to run in the queue when a job is submitted; and the job waiting time refers to the time required for a job to run from submission, in seconds.

[0066] In step S100, the specific process of normalizing historical job data is as follows:

[0067] For historical task data, the Min-Max method is used to normalize the historical task data. The calculation formula is as follows:

[0068] y'=(yi-y_min) / (y_max-y_min),

[0069] Where y' represents the normalized historical task data, yi represents the historical task data in the original sample, y_max represents the maximum value of the corresponding historical task data in the original sample, and y_min represents the minimum value of the corresponding historical task data in the original sample.

[0070] Step S200 includes:

[0071] S201. Organize the normalized historical task data into a historical task dataset D, where D = {d1, d2, ..., dn}, where d1 represents the feature vector corresponding to the first historical task, d2 represents the feature vector corresponding to the second historical task, and so on, with dn representing the feature vector corresponding to the nth historical task, and n representing the data number of the historical task, which is a positive integer; extract the task waiting time from the historical task dataset D and form a task waiting time set W, where W = {w1, w2, ..., wn}, similarly, w1 represents the task waiting time corresponding to the first historical task, w2 represents the task waiting time corresponding to the second historical task, and wn represents the task waiting time corresponding to the nth historical task;

[0072] S202. Define the input step size H, representing the number of consecutive jobs contained in each input sequence; divide the input sequence and the output sequence, where the following operations are performed on the input sequence:

[0073] For each job i, from H to n, generate an input sequence Xi, where Xi = [di-H+1,di-H+2,...,di]. Set the waiting time of the current job to -1, and the corresponding input sequence Xi = [di-H+1,di-H+2,...,di-1,di,-1].

[0074] The following operation applies to the output sequence: extract the waiting time of the current job from the job waiting time set W as the output sequence Yi, and Yi = wi;

[0075] Summarize all input sequences and output sequences to form the input sequence set X and the output sequence set Y, respectively, where the input sequence set X = [XH, XH+1, ..., Xn] and the output sequence set Y = [YH, YH+1, ..., Yn].

[0076] S203. Define the total number of samples in the input sequence set X and the output sequence set Y as N, and N = n - H + 1; divide the dataset according to the following proportions: training set N_train is 80%, validation set N_val is 10%, and test set N_test is 10%; the specific number of partitions is: N_train = ⌊0.8 × N⌋, N_val = ⌊0.1 × N⌋, N_test = N - N_train - N_val, where ⌊⌋ represents rounding down; according to the calculated number of partitions, divide the input sequence set X and the output sequence set Y into training set, validation set and test set.

[0077] The FELSTM model structure in step S300 includes an input layer, a feature encoding layer, a multi-layer LSTM, a fully connected layer, and an output layer.

[0078] The input layer receives an input sequence Xi, and the dimension of the input sequence Xi is (N, H, F), where F represents the feature dimension; the feature encoding layer encodes the input sequence Xi to expand its dimension; the multi-layer LSTM constructs a Long Short-Term Memory (LSTM) network, and the state update process of the LSTM is as follows:

[0079] Input gate: i t =σ(W i ·[h t-1 ,x t ]+b i ), where i t σ represents the activation value of the input gate, determining the effect of the current input on the cell state; σ represents the sigmoid activation function, mapping the input to between 0 and 1; W i The weight matrix of the input gate is responsible for performing a linear transformation between the previous hidden state and the current input; [h t-1 ,x t ] indicates the connection to the hidden state h of the previous time step. t-1 and the input x at the current time step t The vector, b, is represented as a combined input; i This represents the bias vector of the input gate, used to adjust the activation value of the input gate and increase the flexibility of the model.

[0080] Forgotten Gate: f t =σ(W f ·[h t-1 ,x t ]+b f ), where f t The activation value of the forget gate controls the degree to which the cell state from the previous time step is retained in the current state, and its value ranges from 0 to 1; W f The weight matrix of the forget gate is responsible for performing a linear transformation between the previous hidden state and the current input; b f This represents the bias vector of the forget gate, used to adjust the activation value of the forget gate and enhance the flexibility of the model.

[0081] Output gate: o t =σ(W o ·[h t-1 ,x t ]+b o ), where o t W represents the activation value of the output gate, controlling the degree of influence of the current cell state on the current hidden state; its value ranges from 0 to 1. o The weight matrix of the output gate is responsible for performing a linear transformation between the previous hidden state and the current input; b o This represents the bias vector of the output gate, used to adjust the activation value of the output gate and enhance the flexibility of the model.

[0082] Cell state: C ~ t =tanh(W C ·[h t-1 ,x t ]+b C ), C t =f t ·C t-1 +i t ·C ~ t ; where C ~ t Represents the candidate cell state at the current time step; tanh represents the hyperbolic tangent activation function, which maps the input to between -1 and 1, providing a non-linear transformation; W C The weight matrix representing the cell state is responsible for performing a linear transformation between the previous hidden state and the current input; b C A bias vector representing the cell state, used to adjust the calculation of candidate cell states; C t It represents the cell state at the current time step, taking into account the influence of the previous state and the current input;

[0083] The fully connected layer passes the output of the multi-layer LSTM to the fully connected layer to generate the final output; the output layer directly uses the output of the fully connected layer as the final prediction value.

[0084] The feature encoding layer encodes the input sequence Xi to expand its dimension, specifically including:

[0085] Z t =tanh(W z ·x t +b z ), where Z t W represents the output after feature encoding. z Let b represent the weight matrix. z Indicates the bias term;

[0086] The fully connected layer passes the output of the multi-layer LSTM to the fully connected layer to generate the final output, specifically including:

[0087] y pred =W out ·h T +b out , where y pred W represents the final predicted output. out h represents the weight matrix of the output layer. T This represents the hidden state at the last time step, containing information about the entire input sequence; b out This represents the bias vector of the output layer.

[0088] Fully connected layers can meet the requirements of feature encoding layers, but using only fully connected layers also has problems. When processing input data, fully connected layers obtain output data by multiplying the weight matrix by the input data matrix and adding a bias term. However, during training, the sizes of the weight matrix and the bias term constantly change, which leads to the output data having a non-zero mean and an unrestricted range. This slows down the convergence speed of the optimization algorithm and makes it prone to gradient explosion. To solve this problem, a hyperbolic tangent (tanh) activation function is added after the fully connected layer to keep the output data having a zero mean and limiting its value range to [-1, 1], thereby accelerating the convergence speed of the optimization algorithm and preventing gradient explosion.

[0089] Step S300 sets the loss function to SmoothL1Loss, which includes the following:

[0090] For each job i, calculate the predicted value y. pred The loss L between the actual value and the true value i The true value is the output sequence Yi; the specific calculation formula is as follows:

[0091] ;

[0092] And the total loss is the average of all operation losses, L. total .

[0093] Step S300 uses the Adam optimizer to optimize parameters, specifically including:

[0094] In the implementation of the Adam optimizer, the update formulas for the first and second moments are as follows:

[0095] First-order moment update: m t =β1m t-1 +(1-β1)∇L total , where m t β1 represents the first moment at the current time step; β1 represents the decay rate of the first moment, typically taken as 0.9; m t-1 ∇L represents the first moment of the previous time step; total This represents the gradient of the current loss function with respect to the parameters;

[0096] Second moment update: v t =β2v t-1 +(1-β2)(∇L total ) 2 Among them, v t β2 represents the second moment at the current time step; β2 represents the decay rate of the second moment, typically taken as 0.999; v t-1 This represents the second moment of the previous time step;

[0097] The parameter update formula is:

[0098] θ new =θ old -(αm t ) / [(v t )^(1 / 2)+ϵ], where θ new This represents the updated parameter value, θ. old This represents the parameter values ​​before the update, α represents the learning rate, and ϵ represents a constant.

[0099] The specific content of hyperparameter optimization of the FELSTM model in step S400 includes:

[0100] Define the hyperparameters as input stride H, learning rate α, and number of hidden layer nodes N. h LSTM layer number N L and the number of feature encoding nodes N f The grid search method was chosen for hyperparameter optimization, and for each set of parameter combinations (H, α, N)... h N L N f Perform the following operations: Build and train a FELSTM model using different combinations of hyperparameters, and record the loss function L during each training process. total The value;

[0101] For each combination of hyperparameters, calculate the average loss L on the validation set. avg Choose one that results in an average loss L on the validation set. avg The smallest combination of hyperparameters is used as the final hyperparameters.

[0102] In this embodiment, job data from the general computing queue was collected from January 1, 2022 to April 30, 2023. The original dataset contains 2,987,315 job records.

[0103] First, the 2,987,315 job records were cleaned to remove jobs that exited abnormally during the waiting period and those with conditional dependencies, resulting in 1,918,041 valid job records. Then, for each job record, data was collected across the following 10 dimensions: Job ID, number of resources requested by the user (CPU cores), number of idle resources by the user, number of resources the user is waiting for, number of jobs the user is waiting for, user job execution time, user CPU usage time, number of idle resources in the queue, number of resources waiting in the queue, and job waiting time. Figure 2 As shown.

[0104] To analyze data with different dimensions, the Min-Max method was used to normalize the dataset, and the results are as follows: Figure 3 As shown.

[0105] The input and output sequences are then divided into training, validation, and test sets, with proportions of 80%, 10%, and 10%, respectively. A schematic diagram illustrating the results of data partitioning of the input and output sequences when parameter H is 7 is shown below. Figure 4 As shown, each row in the input sequence result diagram serves as the input to the FELSTM model, and its corresponding target output is the corresponding row in the output sequence result diagram.

[0106] On an NVIDIA V100 GPU, PyTorch was used to build the FELSTM model, and the model was trained and tested. The overall structure diagram of the FELSTM model is shown below. Figure 5 As shown. When constructing the FELSTM model, the SmoothL1Loss loss function is used, and the Adam optimizer is used to optimize the model parameters;

[0107] The model's hyperparameters were optimized and trained. The FELSTM model required optimization of five hyperparameters: input stride, learning rate, number of hidden layer nodes, number of LSTM layers, and number of feature encoding nodes. A grid search method was used to optimize these hyperparameters. The goal was to achieve stable convergence of the model's loss function during training and minimize the loss value. The final hyperparameters used in the FELSTM model are as follows: Figure 6 As shown, the FELSTM model for cluster job waiting time is trained.

[0108] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0109] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cluster job waiting time prediction method based on the FELSTM model, characterized in that: The method includes the following steps: Step S100. Collect historical job data from the target computer system and normalize the historical job data; Step S200. Divide the normalized historical job data into an input sequence and an output sequence. The input sequence is the data used to predict the output sequence, and the output sequence is the target sequence that the model expects to output. Then, divide the input sequence and output sequence into a training dataset, a validation set, and a test set. Step S300. Construct a FELSTM model using a deep learning framework, set the loss function to SmoothL1Loss, and optimize the parameters using the Adam optimizer; The FELSTM model structure in step S300 includes an input layer, a feature encoding layer, a multi-layer LSTM, a fully connected layer, and an output layer. The input layer receives an input sequence Xi, and the dimension of the input sequence Xi is (N, H, F), where F represents the feature dimension; the feature encoding layer encodes the input sequence Xi to expand its dimension; the multi-layer LSTM constructs a Long Short-Term Memory (LSTM) network, and the state update process of the LSTM is as follows: Input gate: i t =σ(W i ·[h t-1 ,x t ]+b i ), where i t σ represents the activation value of the input gate; W represents the activation function of the sigmoid function. i Represents the weight matrix of the input gate; [h t-1 ,x t ] indicates the connection to the hidden state h of the previous time step. t-1 and the input x at the current time step t The vector, b, is represented as a combined input; i This represents the bias vector of the input gate; Forgotten Gate: f t =σ(W f ·[h t-1 ,x t ]+b f ), where f t The activation value of the Forgotten Gate; W f The weight matrix of the forget gate is responsible for performing a linear transformation between the previous hidden state and the current input; b f The bias vector representing the forget gate; Output gate: o t =σ(W o ·[h t-1 ,x t ]+b o ), where o t W represents the activation value of the output gate. o The weight matrix of the output gate is responsible for performing a linear transformation between the previous hidden state and the current input; b o This represents the bias vector of the output gate, used to adjust the activation value of the output gate and enhance the flexibility of the model. Cell state: C ~ t =tanh(W C ·[h t-1 ,x t ]+b C ), C t =f t ·C t-1 +i t ·C ~ t ; where C ~ t Represents the candidate cell state at the current time step; tanh represents the hyperbolic tangent activation function; W C The weight matrix representing cell states; b C A bias vector representing the cell state; C t This indicates the cell state at the current time step; Hidden state: h t =ot·tanh(C t ), where h t This represents the hidden state at the current time step; The fully connected layer passes the output of the multi-layer LSTM to the fully connected layer to generate the final output; the output layer directly uses the output of the fully connected layer as the final prediction value. Step S400. Optimize the hyperparameters of the FELSTM model so that the loss function converges stably and reaches its minimum value during training; after training, verify the model performance and evaluate the prediction accuracy on the test set.

2. The cluster job waiting time prediction method based on the FELSTM model according to claim 1, characterized in that: The historical job data includes job ID, number of resources requested by the user, number of idle resources by the user, number of resources waiting by the user, number of jobs waiting by the user, user job execution time, user CPU usage time, number of idle resources in the queue, number of resources waiting in the queue, and job waiting time. The job ID refers to the unique identifier assigned by the system when a job is submitted; the number of user-requested resources refers to the number of CPU cores requested by the user when a job is submitted; the number of user-idle resources refers to the maximum number of cores available in the general computing queue when a job is submitted, minus the number of cores already used by the user; the number of user-waiting resources refers to the total number of cores the user is waiting to run when a job is submitted; the number of user-waiting jobs refers to the number of jobs the user is waiting to run when a job is submitted; the user job running time refers to the total running time of jobs already run by the user when a job is submitted, in seconds; the user CPU usage time refers to the total CPU time the user has already run jobs when a job is submitted, in seconds; the number of queue-idle resources refers to the number of cores available in the queue when a job is submitted; the number of queue-waiting resources refers to the total number of cores waiting to run in the queue when a job is submitted; and the job waiting time refers to the time required for a job to run from submission, in seconds.

3. The cluster job waiting time prediction method based on the FELSTM model according to claim 1, characterized in that: In step S100, the specific process of normalizing historical job data is as follows: For historical task data, the Min-Max method is used to normalize the historical task data. The calculation formula is as follows: y'=(yi-y_min) / (y_max-y_min), Where y' represents the normalized historical task data, yi represents the historical task data in the original sample, y_max represents the maximum value of the corresponding historical task data in the original sample, and y_min represents the minimum value of the corresponding historical task data in the original sample.

4. The cluster job waiting time prediction method based on the FELSTM model according to claim 2, characterized in that: Step S200 includes: S201. Organize the normalized historical task data into a historical task dataset D, where D = {d1, d2, ..., dn}, where d1 represents the feature vector corresponding to the first historical task, d2 represents the feature vector corresponding to the second historical task, and so on, with dn representing the feature vector corresponding to the nth historical task, and n representing the data number of the historical task, which is a positive integer; extract the task waiting time from the historical task dataset D and form a task waiting time set W, where W = {w1, w2, ..., wn}, similarly, w1 represents the task waiting time corresponding to the first historical task, w2 represents the task waiting time corresponding to the second historical task, and wn represents the task waiting time corresponding to the nth historical task; S202. Define the input step size H, representing the number of consecutive jobs contained in each input sequence; divide the input sequence and the output sequence, where the following operations are performed on the input sequence: For each job i, from H to n, generate an input sequence Xi, where Xi = [di-H+1,di-H+2,...,di]. Set the waiting time of the current job to -1, and the corresponding input sequence Xi = [di-H+1,di-H+2,...,di-1,di,-1]. The following operation applies to the output sequence: extract the waiting time of the current job from the job waiting time set W as the output sequence Yi, and Yi = wi; Summarize all input sequences and output sequences to form the input sequence set X and the output sequence set Y, respectively, where the input sequence set X = [XH, XH+1, ..., Xn] and the output sequence set Y = [YH, YH+1, ..., Yn]. S203. Define the total number of samples in the input sequence set X and the output sequence set Y as N, and N = n - H + 1; divide the dataset according to the following proportions: training set N_train is 80%, validation set N_val is 10%, and test set N_test is 10%; the specific number of partitions is: N_train = ⌊0.8 × N⌋, N_val = ⌊0.1 × N⌋, N_test = N - N_train - N_val, where ⌊⌋ represents rounding down; according to the calculated number of partitions, divide the input sequence set X and the output sequence set Y into training set, validation set and test set.

5. The cluster job waiting time prediction method based on the FELSTM model according to claim 1, characterized in that: The feature encoding layer encodes the input sequence Xi to expand its dimension, specifically including: Z t =tanh(W z ·x t +b z ), where Z t W represents the output after feature encoding. z Let b represent the weight matrix. z Indicates the bias term; The fully connected layer passes the output of the multi-layer LSTM to the fully connected layer to generate the final output, specifically including: y pred =W out ·h T +b out , where y pred W represents the final predicted output. out h represents the weight matrix of the output layer. T This represents the hidden state at the last time step, containing information about the entire input sequence; b out This represents the bias vector of the output layer.

6. The cluster job waiting time prediction method based on the FELSTM model according to claim 1, characterized in that: The step S300, which sets the loss function to SmoothL1Loss, specifically includes: For each job i, calculate the predicted value y. pred The loss L between the actual value and the true value i The true value is the output sequence Yi; the specific calculation formula is as follows: ， And the total loss is the average of all operation losses, L. total .

7. The cluster job waiting time prediction method based on the FELSTM model according to claim 1, characterized in that: Step S300 uses the Adam optimizer to optimize parameters, specifically including: In the implementation of the Adam optimizer, the update formulas for the first and second moments are as follows: First-order moment update: m t =β1m t-1 +(1-β1)∇L total , where m t β1 represents the first moment at the current time step; m represents the decay rate of the first moment; t-1 ∇L represents the first moment of the previous time step; total This represents the gradient of the current loss function with respect to the parameters; Second moment update: v t =β2v t-1 +(1-β2)(∇L total ) 2 Among them, v t β2 represents the second moment at the current time step; v represents the decay rate of the second moment; t-1 This represents the second moment of the previous time step; The parameter update formula is: θ new =θ old -(αm t ) / [(v t )^(1 / 2)+ϵ], where θ new This represents the updated parameter value, θ. old This represents the parameter values ​​before the update, α represents the learning rate, and ϵ represents a constant.

8. The cluster job waiting time prediction method based on the FELSTM model according to claim 1, characterized in that: The specific content of hyperparameter optimization of the FELSTM model in step S400 includes: Define the hyperparameters as input stride H, learning rate α, and number of hidden layer nodes N. h LSTM layer number N L and the number of feature encoding nodes N f The grid search method was chosen for hyperparameter optimization, and for each set of parameter combinations (H, α, N)... h N L N f Perform the following operations: Build and train a FELSTM model using different combinations of hyperparameters, and record the loss function L during each training process. total The value; For each combination of hyperparameters, calculate the average loss L on the validation set. avg Choose one that results in an average loss L on the validation set. avg The smallest combination of hyperparameters is used as the final hyperparameters.

Images