Parallel greedy genetic algorithm for job scheduling in cluster environment

Abstract

The invention discloses a parallel greedy genetic algorithm for cluster job scheduling, which comprises the following steps that: a main scheduler firstly receives jobs and initially arranges the jobs by using a backfill strategy, then encodes the jobs into DAG chromosomes, generates a plurality of initial individuals according to a greedy heuristic form, performs selection, crossover, variation and elitist retention on populations by the genetic algorithm, and iteratively optimizes a scheduling scheme; under the master-slave parallel architecture, a master process is responsible for job preprocessing and population generation, and slave processes complete genetic computation in parallel and update a job queue in real time. The method considers completion time, load balance and resource utilization rate, and can significantly improve cluster scheduling efficiency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computers, and particularly relates to a parallel greedy genetic algorithm for job scheduling in a cluster environment. BACKGROUND

[0002] Currently, in scientific research and large-scale data processing applications, a large amount of data needs to be processed and high-performance computing resources need to be used; in actual business scenarios, jobs submitted by users have different processor / node requirements and time requirements, and traditional scheduling algorithms may not be able to quickly adapt to these dynamic changes; with the wide application of current high-performance computing resources, the job scheduling problem in the cluster environment is increasingly concerned, and researchers have developed various algorithms to solve this problem, among which the genetic algorithm is widely used due to its good search ability and adaptability; however, the traditional genetic algorithm has the disadvantages of slow convergence speed and easy falling into local optimum when processing large-scale job scheduling. SUMMARY

[0003] The technical problem to be solved by the application is to provide a parallel genetic algorithm combined with a greedy algorithm to quickly generate an initial population and accelerate the convergence process, thereby improving the efficiency and performance of job scheduling.

[0004] The technical scheme of the application is as follows:

[0005] A parallel greedy genetic algorithm for job scheduling in a cluster environment comprises the following steps:

[0006] Step 1, accepting job requests: the main scheduler receives job requests from users, each job J i contains the following information: required processor or node number P i , estimated running time T i , task dependency information D i and job submission time R i ; the request queue is defined as: J={J1, J2,..., J n};

[0007] Step 2, backfill algorithm pre-scheduling: a backfill algorithm is used, for each cluster node pool C k , the following constraint is used: let the current idle time slice be W k (t), when J i is completed within W k (t) and does not block the previous job, it is allowed to backfill:

[0008]

[0009] Step 3, Job encoding as DAG chromosome: Encode the job tasks as a directed acyclic graph (DAG) and represent the DAG as a chromosome, where each node represents a job task and each edge represents a dependency between tasks;

[0010] Step 4, Generate initial population using greedy algorithm, including the following steps:

[0011] Step 4.1, Select the first independent task: Build a set of independent tasks and select one task v1 from it as the starting point of the chromosome;

[0012] Step 4.2, Find the largest compatible task subset: Based on priority, find the largest set of tasks that are compatible with the selected tasks;

[0013] Step 4.3, Assign tasks to nodes: Assign the found task set to a node and update the remaining capacity and available time of the node;

[0014] Step 4.4, Check if there are still schedulable tasks: When there are no ready tasks that can be immediately scheduled or all nodes are full, exit, otherwise go back to step 4.2;

[0015] Step 4.5, Repeat the above process to form the initial population: By using different initial selection sequences multiple times, construct the initial population;

[0016] Step 5, Apply genetic algorithm on the initial population, including the following steps:

[0017] Step 5.1, Selection operation: Select parent chromosomes according to the fitness function, the higher the fitness of the chromosome, the greater the probability of being selected;

[0018] Step 5.2, Crossover operation: Cross the selected parent chromosomes, combine their partial genes to form new child chromosomes, using single-point crossover or double-point crossover;

[0019] Step 5.3, Mutation operation: Mutate the newly generated chromosomes by swapping the positions of tasks in the chromosome or changing the execution order of tasks to increase the diversity of the population;

[0020] Step 5.4, Fitness evaluation: Evaluate the fitness of the new chromosomes to determine whether they meet the scheduling requirements;

[0021] Step 5.5, Update the population: According to the fitness evaluation results, select the new chromosome with the highest fitness to replace the corresponding individual in the original population to form the next generation population: P t+1 ←elitism(P t ∪P new ),

[0022] where Pt+1 For the next generation population, P t is the current population, P new is the newborn offspring population, and elitism(·) is the elite retention function. In the merged pool, the top N offspring are sorted from high to low according to fitness, and the top N offspring are selected to form the next generation, ensuring that the best genes are not lost and the population size is constant.

[0023] Step 6: Update the job request queue: Send the latest scheduling solution to the cluster for execution and update the job request queue to reflect the current status.

[0024] Step 7: Use a master-slave architecture to achieve parallelization: the master process is responsible for receiving job requests, executing the backfill algorithm, building the DAG, and generating the initial population. The slave process is responsible for executing the selection, crossover, mutation, and fitness calculation processes in parallel to accelerate algorithm execution. The parallel acceleration yield is estimated as follows:

[0025]

[0026] Among them, S p is the speedup ratio, that is, the speed increase of the algorithm relative to serial execution when using p parallel processing units; T1 is the serial execution time, that is, the total time required to fully run the algorithm or task on a single processing unit; T p is the parallel execution time, that is, the total time required to complete the same work on p parallel processing units.

[0027] Furthermore, the step 3 is specifically: modeling the job task as a DAG graph:

[0028] Graph G = (V, E), where V is the set of tasks and E is the set of dependencies;

[0029] Each chromosome χ is represented as a topologically sorted sequence:

[0030]

[0031] Where χ is the chromosome, [v1,v2,...,v m ] is the gene sequence, that is, the task node list arranged in sequence. The smaller the subscript, the earlier it is scheduled; v i is the i-th node, v j is the j-th node, m is the total number of nodes, and E is the set of dependent edges.

[0032] Furthermore, the expression of the independent task set in step 4.1 is:

[0033]

[0034] Among them, V indepFor the independent task set, i.e. the ready task set, these tasks have no predecessors and can be immediately scheduled for execution; v is a single task node, V is the full set of task nodes, (u, v) E is a dependency edge, u is a potential predecessor task, and E is the edge set;

[0035] The step 4.2 comprises the following steps:

[0036] Step 4.2.1, define a greedy score function:

[0037]

[0038] Wherein, p(v) is a resource density score, reflecting the number of processors occupied per unit time, the larger the value, the higher the resource consumption per unit time of the task; T v is the execution time, i.e. the estimated running time when the task v exclusively occupies the required resources; v is the number of required processors or nodes, i.e. the amount of computing resources that the task v occupies at a time when it runs;

[0039] Step 4.2.2, select a subset of tasks compatible with the current task from the ready tasks, so that:

[0040]

[0041] Wherein, S is the optimal compatible task set, S' is the candidate subset, V ready is the ready task set and;

[0042] The step 4.3 is specifically: assign S to node C k , and update its available capacity Cap k and the next available time Avail k :

[0043]

[0044] The expression of the initial population in the step 4.5 is:

[0045] P0={χ1,χ2,...,χ M},

[0046] Wherein, P0 is the initial population, i.e. the full set of individuals that the genetic algorithm has at the 0th generation; χ M is the Mth chromosome in the initial population, and M is the population size, i.e. the number of chromosomes in the initial population.

[0047] Further, the step 5.1 is specifically: adopt a roulette or tournament strategy to select parent chromosomes, with a high probability for those with high fitness:

[0048]

[0049] makespan(χ)=max v∈V (start(v)+T v )-min v∈V start(v),

[0050] where P(χ i ) is the selection probability, χ i is the chromosome, i.e. the ith candidate scheduling scheme in the population; f(χ j ) is the fitness function, which is used to score the pros and cons of χ i , the greater the value, the better the scheme; P is the current population, makespan(χ) is the makespan; min v∈V start(v) is the earliest start time, start(v) is the task start time, max v∈V (start(v)+T v ) is the latest end time, T v is the task execution time, and V is the task set;

[0051] The step 5.3 is specifically: when , then swap(v i ,v j ) is allowed;

[0052] The step 5.4 includes:

[0053] The comprehensive scheduling objective function is defined as the fitness:

[0054]

[0055] The makespan is:

[0056] makespan(χ)=max v∈V (start(v)+T v )-min v∈V start(v),

[0057] The load imbalance is:

[0058]

[0059] The resource utilization rate is:

[0060]

[0061] where f(χ) is the fitness value, the greater f(χ), the better the scheme; α is the makespan weight, β is the load balancing weight, γ is the resource utilization rate weight, k is the node index, is the maximum node load, Load k Load

[0062] Further, the step 6 is specifically: distributing the optimal scheduling solution output by the genetic algorithm to each node for execution, then removing the scheduled job from the queue, and continuing to accept new jobs:

[0063]

[0064] wherein χ * is the optimal chromosome, that is, the chromosome with the highest fitness value f(χ) in the current population P t χ is the candidate chromosome, P t is the t-th generation population, and f(χ) is the fitness function value.

[0065] Advantages of the present application:

[0066] 1. Improve resource utilization: the present application introduces a backfill scheduling algorithm to reasonably fill idle time slices without blocking previous jobs, thereby significantly improving the overall CPU or node utilization efficiency of the cluster;

[0067] 2. Meet task dependency: the present application encodes jobs as DAG (Directed Acyclic Graph) structures to ensure that the dependency relationship between tasks is accurately expressed and reasonably scheduled, and is suitable for complex process jobs;

[0068] 3. Optimize scheduling quality: the present application generates a high-quality initial population through greedy heuristic, which helps to guide the genetic algorithm to quickly converge and reduce the interference of inefficient solutions;

[0069] 4. Enhance population diversity: the present application introduces crossover and mutation operations in the genetic algorithm, combined with an elite reservation mechanism, to ensure that the optimal solution is not lost and to enhance the search space coverage capability;

[0070] 5. Support parallel acceleration: the present application uses a master-slave structure to execute genetic operators in parallel, so that the computationally intensive scheduling optimization process can be efficiently run in a multi-core or distributed environment, significantly shortening the scheduling solution time;

[0071] 6. Consider multiple objectives for optimization: the fitness function of the present application can comprehensively consider job completion time, node load balancing degree and resource utilization, so that the scheduling result can achieve a relatively optimal balance in multiple dimensions;

[0072] 7. Strong adaptability: the algorithm framework of the application is universal, can be applied to various types of task scheduling scenes such as HPC, high-performance cloud computing, edge computing platform, and has good expansibility and practicability. BRIEF DESCRIPTION OF DRAWINGS

[0073] Fig. 1 is a basic flow chart of a parallel greedy genetic algorithm for job scheduling in a cluster environment.

[0074] Fig. 2 is a principle diagram of a parallel greedy genetic algorithm for job scheduling in a cluster environment. DETAILED DESCRIPTION

[0075] As shown in Figs. 1-2 , a parallel greedy genetic algorithm for job scheduling in a cluster environment comprises the following steps:

[0076] Step 1, accepting job request: the main scheduler receives a job request from a user, each job J i contains the following information: required processor or node number P i , estimated running time T i , task dependency information D i and job submission time R i ; the request queue is defined as: J={J1,J2,...,J n};

[0077] Step 2, backfill algorithm pre-scheduling: using backfill algorithm, for each cluster node pool C k , the following constraint is adopted: let the current idle time slice be W k (t), when J i is completed within W k (t) and does not block the previous job, it is allowed to backfill:

[0078]

[0079] Step 3, job coding as DAG chromosome: coding the job task as a directed acyclic graph DAG, and taking the DAG as a chromosome, wherein each node represents a job task, and each edge represents the dependency relationship between tasks;

[0080] Step 4, using greedy algorithm to generate initial population, comprising the following steps:

[0081] Step 4.1, selecting the first independent task: constructing an independent task set and selecting a task v1 from the set as the starting point of the chromosome;

[0082] Step 4.2, Finding the largest compatible subset of tasks: Based on the priority, find the largest set of tasks that are compatible with the selected tasks;

[0083] Step 4.3, Assigning tasks to nodes: Assign the found task set to a node and update the remaining capacity and available time of the node;

[0084] Step 4.4, Checking if there are still schedulable tasks: When there are no ready tasks that can be immediately scheduled or all nodes are full, exit, otherwise go back to step 4.2;

[0085] Step 4.5, Repeating the above process to form the initial population: By using different initial selection orders multiple times, construct the initial population;

[0086] Step 5, Applying genetic algorithm on the initial population, including the following steps:

[0087] Step 5.1, Selection operation: Select parent chromosomes according to the fitness function, the higher the fitness, the higher the probability of being selected;

[0088] Step 5.2, Crossover operation: Cross the selected parent chromosomes, combine their partial genes to form new child chromosomes, using single-point crossover or double-point crossover;

[0089] Step 5.3, Mutation operation: Mutate the newly generated chromosomes by swapping the positions of tasks in the chromosomes or changing the execution order of tasks to increase the diversity of the population;

[0090] Step 5.4, Fitness evaluation: Evaluate the fitness of the new chromosomes to determine whether they meet the scheduling requirements;

[0091] Step 5.5, Update the population: According to the fitness evaluation results, select the new chromosomes with the highest fitness to replace the corresponding individuals in the original population to form the next generation population: P t+1 ←elitism(P t ∪P new ),

[0092] where P t+1 is the next generation population, P t is the current population, P new is the new child population, and elitism(·) is the elite retention function. The first N individuals are selected from the merged pool in descending order of fitness to form the next generation, ensuring that the optimal genes are not lost and the population size is constant;

[0093] Step 6, Update the job request queue: Send the latest scheduling solution to the cluster for execution and update the job request queue to reflect the current state;

[0094] Step 7, parallelization is achieved using master-slave architecture: the master process is responsible for receiving job requests, executing backfill algorithms, building DAGs, and generating initial populations, and the slave process is responsible for parallel execution of selection, crossover, mutation, and fitness calculation processes to speed up algorithm execution; the parallel speedup rate is estimated as follows:

[0095]

[0096] where S p is the speedup ratio, i.e., the speedup of the algorithm relative to serial execution when using p parallel processing units; T1 is the serial execution time, i.e., the total time required to run the algorithm or task completely on a single processing unit; T p is the parallel execution time, i.e., the total time required to complete the same work on p parallel processing units.

[0097] Preferably, step 3 is specifically: modeling the job task as a DAG graph:

[0098] Graph G = (V, E), where V is the task set and E is the dependency relationship set;

[0099] Each chromosome χ is represented as a topological sorting sequence:

[0100]

[0101] where χ is the chromosome, [v1, v2,..., v m ] is the gene sequence, i.e., the list of tasks nodes arranged in order, and the smaller the subscript, the earlier the task is scheduled; v i is the i-th node, v j is the j-th node, m is the total number of nodes, and E is the set of dependency edges.

[0102] Preferably, the expression of the independent task set in step 4.1 is:

[0103]

[0104] where V indep is the set of independent tasks, i.e., the set of ready tasks, which have no predecessors and can be immediately scheduled for execution; v is a single task node, V is the full set of task nodes, (u, v) ∈ E is a dependency edge, u is a potential predecessor task, and E is the edge set;

[0105] Step 4.2 includes the following steps:

[0106] Step 4.2.1, define a greedy scoring function:

[0107]

[0108] wherein p(v) is the resource density score, reflecting the number of processors occupied per unit time, the larger the value, the higher the resource consumption per unit time of the task; T v is the execution duration, i.e. the estimated running time when the task v exclusively occupies the required resources; v is the number of required processors or nodes, i.e. the amount of computing resources occupied at one time when the task v runs;

[0109] Step 4.2.2, select a subset of tasks compatible with the current task from the ready tasks, such that:

[0110]

[0111] wherein S is the optimal compatible task set, S' is the candidate subset, V ready is the ready task set and;

[0112] The step 4.3 is specifically: assign S to the node C k , and update its available capacity Cap k and the next available time Avail k :

[0113]

[0114] The expression of the initial population in the step 4.5 is:

[0115] P0={χ1,χ2,...,χ M},

[0116] wherein P0 is the initial population, i.e. the entire individual set possessed by the genetic algorithm in the 0th generation; χ M is the Mth chromosome in the initial population, and M is the population size, i.e. the number of chromosomes in the initial population.

[0117] Preferably, the step 5.1 is specifically: adopt roulette or tournament strategy to select parent chromosomes, and the higher the fitness, the greater the probability:

[0118]

[0119] makespan(χ)=max v∈V (start(v)+T v )-min v∈V start(v),

[0120] wherein P(χ i ) is the selection probability, χ i is the chromosome, i.e. the ith candidate scheduling scheme in the population; f(χ j ) is the fitness function, used to measure χ iThe better the score, the larger the value, indicating that the scheme is better; P is the current population, makespan (χ) is the completion time; min v∈V start(v) is the earliest start time, start(v) is the task start time, max v∈V (start(v)+T v ) is the latest end time, T v is the task execution time, and V is the task set.

[0121] The step 5.3 is specifically: when , then allow: swap(v i ,v j );

[0122] The step 5.4 includes:

[0123] Define the comprehensive scheduling objective function as the fitness:

[0124]

[0125] Completion time:

[0126] makespan(χ)=max v∈V (start(v)+T v )-min v∈V start(v),

[0127] Load imbalance:

[0128]

[0129] Resource utilization:

[0130]

[0131] Wherein, f(χ) is the fitness value, the larger f(χ) is, the better the scheme is; α is the completion time weight, β is the load balancing weight, γ is the resource utilization weight, k is the node index, is the maximum node load, is the minimum node load, Load k is the load of node k, when imbalance=0, all node loads are the same, and the load distribution is the most balanced; when imbalance>0.3, the load distribution is uneven.

[0132] Preferably, the step 6 is specifically: the optimal scheduling solution output by the genetic algorithm is distributed to each node for execution, then the scheduled job is removed from the queue, and new jobs are continuously accepted:

[0133] Wherein, χ *is the optimal chromosome, that is, in this generation population P t The chromosome with the highest fitness value f(χ) is selected; χ is the candidate chromosome, P t is the t-th generation population, and f(χ) is the fitness function value.

[0134] In actual implementation, the master scheduler receives job requests from users. Each job request contains two main parameters: the required number of processors or nodes and the job running time. For each cluster type, a backfill algorithm is applied to assign jobs to nodes. The job tasks are then encoded into a directed acyclic graph (DAG), and the DAG is represented as a chromosome for application in the genetic algorithm. A greedy algorithm is then used to generate the initial population. The genetic algorithm is applied to the initial population, including operations such as selection, crossover, mutation, and fitness evaluation. The latest scheduling solution is then sent to the cluster and the job request queue is updated. Finally, a master-slave architecture is used to achieve parallelization.

[0135] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not used to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A parallel greedy genetic algorithm for job scheduling in a cluster environment, characterized by: The following steps are involved: Step 1: Accept job request: The main scheduler receives job request from the user. i Contains the following information: the number of processors or nodes required P i , estimated running time T i , task dependency information D i and the job submission time R i ; The request queue is defined as: J = {J1, J2, ..., J n }; Step 2: Backfill algorithm pre-scheduling: Use the backfill algorithm to pre-scheduling each cluster node pool C. k , using the following constraints: Let the current idle time slice be W k (t), when J i In W k (t) is completed and does not block the previous operation, it is allowed to backfill: Step 3: Encode the job into a DAG chromosome: Encode the job task into a directed acyclic graph (DAG) and represent the DAG as a chromosome, where each node represents a job task and each edge represents the dependency between tasks. Step 4: Generate the initial population using a greedy algorithm, including the following steps: Step 4.

1. Select the first independent task: Build an independent task set and select a task v1 from it as the starting point of the chromosome; Step 4.2: Find the largest compatible task subset: Based on the priority, find the largest task subset that is compatible with the selected tasks. Step 4.3, assign tasks to nodes: assign the found task set to a node and update the node's remaining capacity and available time; Step 4.4: Check if there are any schedulable tasks: If there are no ready tasks that can be scheduled immediately or all nodes are full, exit; otherwise, return to step 4.

2. Step 4.5: Repeat the above process to form the initial population: construct the initial population by using different initial selection orders multiple times; Step 5: Apply the genetic algorithm to the initial population, including the following steps: Step 5.1, selection operation: select the parent chromosome according to the fitness function. The chromosome with higher fitness has a greater probability of being selected. Step 5.2, crossover operation: Cross the selected parent chromosomes and combine some of their genes to form new daughter chromosomes, using single-point crossover or double-point crossover; Step 5.3, mutation operation: perform mutation operation on the newly generated chromosome to increase the diversity of the population by swapping the positions of tasks in the chromosome or changing the execution order of tasks; Step 5.4, fitness evaluation: perform fitness evaluation on the new chromosome to determine whether it meets the scheduling requirements; Step 5.5, update the population: According to the fitness evaluation results, select the new chromosome with the highest fitness to replace the corresponding individuals in the original population to form the next generation population: P t+1 ←elitism(P t ∪P new ), Among them, P t+1 For the next generation population, P t is the current population, P new is the newborn offspring population, and elitism(·) is the elite retention function. In the merged pool, the top N offspring are sorted from high to low according to fitness, and the top N offspring are selected to form the next generation, ensuring that the best genes are not lost and the population size is constant. Step 6: Update the job request queue: Send the latest scheduling solution to the cluster for execution and update the job request queue to reflect the current status. Step 7: Use a master-slave architecture to achieve parallelization: the master process is responsible for receiving job requests, executing the backfill algorithm, building the DAG, and generating the initial population. The slave process is responsible for executing the selection, crossover, mutation, and fitness calculation processes in parallel to accelerate algorithm execution. The parallel acceleration yield is estimated as follows: Among them, S p is the speedup ratio, that is, the speed increase of the algorithm relative to serial execution when using p parallel processing units; T1 is the serial execution time, that is, the total time required to fully run the algorithm or task on a single processing unit; T p is the parallel execution time, that is, the total time required to complete the same work on p parallel processing units.

2. A parallel greedy genetic algorithm for job scheduling in a cluster environment according to claim 1, characterized in that: The step 3 is specifically: modeling the job task as a DAG graph: Graph G = (V, E), where V is the set of tasks and E is the set of dependencies; Each chromosome χ is represented as a topologically sorted sequence: Where χ is the chromosome, [v1,v2,...,v m ] is the gene sequence, that is, the task node list arranged in sequence. The smaller the subscript, the earlier it is scheduled; v i is the i-th node, v j is the j-th node, m is the total number of nodes, and E is the set of dependent edges.

3. A parallel greedy genetic algorithm for job scheduling in a cluster environment according to claim 1, characterized in that: The expression of the independent task set in step 4.1 is: Among them, V indep is a set of independent tasks, i.e., ready tasks, which do not have any predecessors and can be scheduled for execution immediately; v is a single task node, V is the set of all task nodes, (u, v)∈E is a dependency edge, u is a potential predecessor task, and E is an edge set; The step 4.2 includes the following steps: Step 4.2.

1. Define the greedy scoring function: Among them, ρ(v) is the resource density score, which reflects the number of processors occupied per unit time. The larger the value, the higher the resource consumption per unit time of the task; T v is the execution time, that is, the estimated running time when task v exclusively occupies the required resources, P v is the number of processors or nodes required, that is, the amount of computing resources occupied at one time when task v is running; Step 4.2.2: Select a subset of tasks compatible with the current task from the ready tasks, such that: Among them, S is the optimal compatible task set, S′ is the candidate subset, V ready is the set of ready tasks; The step 4.3 is specifically: assign S to node C k , and update its available capacity Cap k Next available time Avail k : The expression of the initial population in step 4.5 is: P0={χ1,χ2,...,χ M }, Among them, P0 is the initial population, that is, the set of all individuals that the genetic algorithm has in the 0th generation; M is the Mth chromosome in the initial population, and M is the population size, that is, the number of chromosomes in the initial population.

4. A parallel greedy genetic algorithm for job scheduling in a cluster environment according to claim 1, characterized in that: The step 5.1 is specifically as follows: a roulette wheel or tournament strategy is used to select the parent chromosome, and the one with higher fitness has a higher probability: makespan(χ)=max v∈V (start(v)+T v )-min v∈V start(v), Among them, P(χ i ) is the selection probability, χ i is the chromosome, i.e. the i-th candidate scheduling scheme in the population; f(χ j ) is the fitness function, which is used to measure χ i The score of good and bad, the larger the value, the better the solution; P is the current population, and makespan(χ) is the completion time; min v∈V start(v) is the earliest starting time, start(v) is the task start time, max v∈V (start(v)+T v ) is the latest ending time, T v is the task execution time, V is the task set; The step 5.3 is specifically as follows: When , it is allowed to: swap(v i ,v j ); The step 5.4 includes: Define the comprehensive scheduling objective function as fitness: Completion time: makespan(χ)=max v∈V (start(v)+T v )-min v∈V start(v), Load imbalance: Resource utilization: Among them, f(χ) is the fitness value. The larger f(χ) is, the better the solution is. α is the completion time weight, β is the load balancing weight, γ is the resource utilization weight, and k is the node index. is the maximum node load, is the minimum node load, Load k is the load of node k. When imbalance = 0, all nodes have the same load and the load distribution is most balanced. When imbalance > 0.3, the load distribution is uneven.

5. A parallel greedy genetic algorithm for job scheduling in a cluster environment according to claim 1, characterized in that: Step 6 specifically includes: distributing the optimal scheduling solution output by the genetic algorithm to each node for execution, then removing the scheduled jobs from the queue and continuing to accept new jobs: Among them, χ * is the optimal chromosome, that is, in this generation population P t The chromosome with the highest fitness value f(χ) is selected; χ is the candidate chromosome, P t is the t-th generation population, and f(χ) is the fitness function value.

Images