Reliability-constrained workflow scheduling method in cloud environments

Abstract

The present invention relates to a reliability-constrained workflow scheduling method in a cloud environment, comprising the following steps: Step S1: constructing a problem model that uses triangular fuzzy numbers to represent the execution time and transmission time of tasks under the overall reliability constraint of the workflow; Step S2: improving the PSO algorithm to construct an adaptive particle swarm optimization algorithm based on a genetic algorithm; Step S3: using the adaptive particle swarm optimization algorithm based on the genetic algorithm to optimize the fuzzy completion time and fuzzy execution cost of the workflow while satisfying the overall reliability constraint of the workflow, thereby obtaining an optimal workflow scheduling solution. This method has better adaptability to workflow scheduling with reliability constraints in cloud environments where server performance fluctuations and downtime may occur.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing workflow scheduling, and in particular to a reliability-constrained workflow scheduling method in a cloud environment. Background Art

[0002] Because cloud data centers possess abundant computing and storage resources, they are often used to provide services to end users and address the problem of insufficient computing resources on user devices. Cloud computing technology can flexibly provide resources on demand based on user needs, allowing real-world applications to be executed faster than ever before. This is crucial for computationally intensive applications such as astronomy, high-energy physics, bioinformatics, and earthquake science. These complex, computationally intensive applications are composed of hundreds or thousands of interdependent tasks and are often modeled as workflows. In fact, workflow scheduling is crucial, as the quality of the scheduling results directly affects the completion time and execution cost of the workflow model. In particular, completing workflows within a reasonable time and budget remains a daunting challenge.

[0003] The goal of workflow scheduling is to select appropriate computing resources for each task in a workflow to complete the workflow while meeting user needs. Existing workflow scheduling research is mostly based on optimizing execution costs under deadline constraints or minimizing completion times under budget constraints. First, these studies fail to consider the potential for server downtime in real-world environments, which can prevent workflows from completing on schedule. Second, single-goal optimization often fails to effectively address user needs. When optimizing completion time, achieving a shorter completion time often results in higher server rental costs. When optimizing execution cost, achieving a lower cost often leads to the pursuit of cheaper but slower servers. This is due to the nature of servers: higher-performing servers are generally more expensive.

[0004] At the same time, most existing workflow scheduling problems in cloud environments are based on the assumption that the execution time of each workflow task on a specific type of virtual machine is deterministic and can be accurately calculated in advance. However, because real-world servers cannot maintain persistent performance, the actual task execution time may fluctuate, affecting its subtasks. Since real-world servers cannot always execute tasks according to a predetermined state, an efficient scheduling strategy is needed that can simultaneously optimize multiple QoS while taking into account real-world server performance and possible issues such as downtime. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a workflow scheduling method oriented to reliability constraints in a cloud environment, aiming to solve the above problems.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A workflow scheduling method for reliability constraints in a cloud environment includes the following steps:

[0008] Step S1: Construct a problem model that uses triangular fuzzy numbers to represent the execution time and transmission time of tasks under the overall reliability constraint of the workflow;

[0009] Step S2: Improve the PSO algorithm and construct an adaptive particle swarm optimization algorithm based on genetic algorithm;

[0010] Step S3: The adaptive particle swarm optimization algorithm based on the genetic algorithm optimizes the fuzzy completion time and fuzzy execution cost of the workflow under the overall reliability constraint of the workflow to obtain the optimal workflow scheduling solution.

[0011] Furthermore, the overall reliability constraints of the workflow are as follows:

[0012] The workflow is represented by a directed acyclic graph (DAG), i.e., G = <T, E, D>;

[0013] Where T represents a set of nodes, T={t1,t2,...,t n}, each node is a task; E represents a set of edges between tasks, E={e 1,2 ,e 1,3 ,...,e i,j}, indicating the control or data dependency between tasks, task t i and task t j The size of the data transferred between i,j =(t i ,t j ) represents; D={d(t1),d(t2),...,d(t n )} represents the computational workload of the task. From the above definition, we can conclude that task t i The direct predecessor task P(t i )={t k |e k,i Each workflow has a set reliability value, Rel. When the reliability of the scheduling strategy satisfies the given reliability constraint, the scheduling strategy is considered feasible.

[0014] Assume that the resources of the cloud environment are composed of m different virtual machine instance types, and R = {r1, r2, ..., r m} indicates that for resource r i use Indicates that Represents resource r iThe opening time, Represents resource r i Closing time, u i Represents resource r i The computing power of c i Represents resource r i The price per unit time, ε i Represents resource r i The failure rate of different instances varies with their computing power;

[0015] Assume that task t i Deploy to resource j Then task t i The execution time is:

[0016]

[0017] For task t i The parent task P(t i )={t p |e p,i}, parent task t p To task t i The transmission time trans(t p ,t i )for:

[0018]

[0019] Where β represents r(t p ) and r(t i ) between the bandwidth;

[0020] Considering the data dependency between tasks, that is, the child task can only be started after all parent tasks are completed, task t i The start time is defined as follows:

[0021] ST(t i )=max{max(FT(t p )+trans(t p ,t i )),Ava(r(t i ))}

[0022] Ava(r(t i )) represents the virtual machine r(t i ) Prepare to execute task t i The earliest time, FT(t p ) represents task t p The completion time of task t i The completion time is:

[0023] FT(ti )=ST(t i )+ET(t i ,r(t i ))

[0024] Therefore, the overall execution time of the workflow is:

[0025] T total =max{FT(t i )|t i ∈T}

[0026] The workflow execution cost includes the computation cost and the data transmission cost, so the execution cost is:

[0027]

[0028] Among them, c j,k Represented as resource r j Transfer 1GB of data to r k The required price, λ rj For resources j The asking price unit time, when task i and task j are scheduled on different virtual machine instances, s i,j =1, otherwise s i,j =0.

[0029] Considering the task execution failure caused by the fault, we assume that the instantaneous fault follows the Poisson distribution, and the resource r i The failure rate is ε i , then task t i In the resource j The reliability of the above execution is:

[0030]

[0031] According to the additivity of Poisson distribution, the overall reliability of the workflow is:

[0032]

[0033] Based on the above definition, the optimization scheduling problem of completion time and execution cost under the overall reliability constraint of the workflow can be formally expressed as:

[0034]

[0035] Among them, Rel is the overall reliability value of the workflow of the current scheduling scheme, σ rel is a predefined reliability constraint threshold.

[0036] Furthermore, fuzzy theory is introduced and triangular fuzzy numbers are used to represent the computing time and transmission time of the task. The membership function is u(x), where t m The predefined execution time of the task, the left and right endpoints t l and t u Indicates the range of task execution time

[0037] use The triangular fuzzy number representing the scalar τ, based on the concept of uncertainty in running time and transmission time, the completion time and execution cost of the workflow are both triangular fuzzy numbers, expressed as and The optimization problem in this paper is formally expressed as:

[0038]

[0039] For optimization goals Its value is a triangular fuzzy number, and its value is determined by the mean and variance Joint decision-making; optimizing goals The calculation method is as follows:

[0040]

[0041] For the mean and variance are the mean and standard deviation of the fuzzy set under uniform distribution and proportional distribution respectively, and the triangular fuzzy number This is a case based on proportional distribution, so the mean and variance Calculated by the following formula:

[0042]

[0043]

[0044] η is the standard deviation The weight of triangular fuzzy number The same treatment method

[0045] Furthermore, for the estimated time t, the corresponding triangular fuzzy number t m is the most likely execution time of the task, that is, the execution time of a given task on the server, t l and t u The values ​​are respectively from the interval [δ1×t m ,t m ] and [t m ,δ2×t m ] are randomly selected, where δ2>1>δ1; the trigonometric functions are defined as follows:

[0046] Addition operation of triangular fuzzy numbers:

[0047] Comparison operation of triangular fuzzy numbers: If but

[0048] Multiplication operation of triangular fuzzy numbers:

[0049] Furthermore, the adaptive particle swarm optimization algorithm based on genetic algorithm is specifically as follows:

[0050] (1) Using a two-dimensional discrete particle encoding method consisting of cloud computing resources and tasks, a particle represents a solution in the problem space. The position of particle i at time t is as follows:

[0051]

[0052] in, Indicates the virtual machine number where the first task of the i-th particle is located at time t;

[0053] (2) The goal is to optimize the total cost f of the workflow under the constraints of the overall reliability of the workflow. The total cost f includes the fuzzy execution cost of the workflow and fuzzy execution time There are two scheduling goals, which belongs to a multi-objective planning problem, so the fitness function is set as:

[0054]

[0055] Where k1 and k2 represent the weight coefficients of completion time and execution cost respectively, T one ,C one Indicates the completion time and execution cost of all tasks executed on only one server;

[0056] (3) The crossover operator and mutation operator of the genetic algorithm are introduced. When particle i is in the tth iteration, the particle update method is as follows, where and ⊙ represent the crossover operator and mutation operator respectively:

[0057]

[0058] The mutation operator is introduced into the inertia part of the traditional PSO update formula, and its update method is as follows:

[0059]

[0060] Among them, r1 is a random number (0,1), p m For a given mutation probability, when r1 <p m,but A mutation operation occurs. Will change randomly A server code of a quantile, if r1≥p m No mutation behavior occurs;

[0061] For the personal cognition part and the social cognition part, the crossover operator is introduced to update the corresponding parts of the traditional update formula. The update method is as follows:

[0062]

[0063]

[0064] Among them, these two formulas update the personal cognition part and the social cognition part respectively, r2 and r3 are random numbers (0,1), and p c For a given crossover probability, when r2 (or r3) <p c When A mutation operation occurs, C p (or C g ) will randomly select two quantiles of the particle and encode the server code between the particle quantiles and (or gBest t-1 )The server codes between the corresponding quantiles are crossed.

[0065] (4) The new inertia factor w adjustment strategy can be based on the current particle and the historical global best particle gBest t-1 The value of w is adaptively adjusted based on the difference in , and the update method of w is as follows:

[0066]

[0067]

[0068] in, Represents particles and the historical global best particle gBest t-1 The number of different quantiles between |T| represents the number of subtasks in the workflow;

[0069] The update method of personal cognitive factors and social cognitive factors adopts a linear increase and decrease strategy. The update method is as follows:

[0070]

[0071]

[0072] in, and are the initial values ​​set for parameters c1 and c2 respectively. and The final values ​​of c1 and c2.

[0073] Furthermore, the situations of the two particles to be compared are divided into the following three cases for discussion:

[0074] 1) For two particles to be compared, if both particles meet the reliability constraint, the particle with the smaller total cost is selected.

[0075]

[0076] 2) If one particle satisfies the reliability constraint and the other particle does not, the particle that satisfies the reliability constraint is selected.

[0077]

[0078] 3) If both particles do not meet the reliability constraint, the particle with greater reliability is selected because it is more likely to become a feasible solution after iteration.

[0079]

[0080] Furthermore, the step S3 is specifically as follows:

[0081] 1) Initialize the relevant parameters of the adaptive particle swarm algorithm based on genetic algorithm, such as population size PN, maximum number of iterations M ax iter , inertia factor w, and randomly generate the population;

[0082] 2) Calculate the fitness. The initial state of each particle is the individual best particle, and the particle with the smallest fitness value in the initial population is set as the global best particle;

[0083] 3) Introduce the mutation and crossover operators of the genetic algorithm to update the position of the particles themselves and calculate the fitness of the updated particles;

[0084] 4) If the fitness of the updated particle is less than the fitness of the individual best particle, the individual best particle is updated and the current particle is set as the individual best particle;

[0085] 5) At the same time, the fitness of the updated particle is compared with the fitness of the global best particle. If the fitness of the current particle is less than that of the global best particle, the global best particle is updated, the current particle is set as the global best particle, and the best fitness is updated;

[0086] 6) Check whether the algorithm iteration end condition is met. If so, the algorithm ends; otherwise, return to 3).

[0087] Compared with the prior art, the present invention has the following beneficial effects:

[0088] To address issues such as performance fluctuations and downtime that may occur during virtual machine operation, this paper designs a problem model that uses triangular fuzzy numbers to represent the execution time and transmission time of tasks under the overall reliability constraints of the workflow. It also proposes an APSOGA algorithm to optimize the completion time and execution cost of the workflow. This algorithm is based on the traditional PSO algorithm and incorporates mutation and crossover operations from the genetic algorithm to prevent the algorithm from falling into local optimality. Scheduling tests of three scheduling strategies for scientific workflows of five different specifications show that the APSOGA strategy has better adaptability to workflow scheduling with reliability constraints in cloud environments where servers may experience performance fluctuations and downtime. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 The scheduling model framework in one embodiment of the present invention

[0090] Figure 2 This is a triangular fuzzy number membership function diagram in one embodiment of the present invention.

[0091] Figure 3 is the coding particle mapping relationship in one embodiment of the present invention;

[0092] Figure 4 is a particle mutation operation in one embodiment of the present invention;

[0093] Figure 5 It is a cross operation of the individual cognitive part in one embodiment of the present invention;

[0094] Figure 6 It is a cross operation of the social cognition part in one embodiment of the present invention;

[0095] Figure 7 5 scientific workflow structure diagrams in one embodiment of the present invention. DETAILED DESCRIPTION

[0096] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0097] Please refer to Figure 1-7 The present invention provides a workflow scheduling method for reliability constraints in a cloud environment, comprising the following steps:

[0098] Step S1: Construct a problem model that uses triangular fuzzy numbers to represent the execution time and transmission time of tasks under the overall reliability constraint of the workflow;

[0099] Step S2: Improve the PSO algorithm and construct an adaptive particle swarm optimization algorithm based on genetic algorithm;

[0100] Step S3: The adaptive particle swarm optimization algorithm based on the genetic algorithm optimizes the fuzzy completion time and fuzzy execution cost of the workflow under the overall reliability constraint of the workflow to obtain the optimal workflow scheduling solution.

[0101] In this embodiment, the workflow scheduling model framework mainly consists of three parts: cloud environment resources, workflow with reliability constraints, and scheduler. Figure 1 As shown;

[0102] In this embodiment, the workflow is represented by a directed acyclic graph (DAG), that is, G = <T, E, D>. Where T represents a set of nodes, T = {t1, t2, ..., t n}, each node is a task. E represents a set of edges between tasks, E = {e 1,2 ,e 1,3 ,...,e i,j}, indicating the control or data dependency between tasks, task t i and task t j The size of the data transferred between i,j =(t i ,t j ) represents. D={d(t1),d(t2),...,d(t n )} represents the computational workload of the task. From the above definition, we can conclude that task t i The direct predecessor task P(t i )={t k |e k,i Each workflow has a set reliability Rel value. When the reliability of the scheduling strategy meets the given reliability constraint, the scheduling strategy is considered feasible.

[0103] Cloud platforms usually provide computing resources to users in the form of virtual machines. There are several characteristics of cloud platform virtual machines. (1) Once a task is assigned to a virtual machine, it will be completely executed on this virtual machine. (2) A virtual machine can only execute one task at a time and cannot execute multiple tasks simultaneously. The resources of the cloud environment proposed in this paper are composed of m different virtual machine instance types, denoted by R = {r1, r2, ..., r m} indicates that for resource r i Available Indicates that Represents resource r i The opening time, Represents resource r i Closing time, u i Represents resource r i The computing power of c iRepresents resource r i The price per unit time, ε i Represents resource r i The failure rate of different instances varies.

[0104] Assume that task t i Deploy to resource j Then task t i The execution time is:

[0105]

[0106] For task t i The parent task P(t i )={t p |e p,i}, parent task t p To task t i The transmission time trans(t p ,t i )for:

[0107]

[0108] Where β represents r(t p ) and r(t i ). Considering the data dependency between tasks, that is, the child task can only start after all parent tasks are completed, task t i The start time is defined as follows:

[0109] ST(t i )=max{max(FT(t p )+trans(t p ,t i )),Ava(r(t i ))}

[0110] Ava(r(t i )) represents the virtual machine r(t i ) Prepare to execute task t i The earliest time, FT(t p ) represents task t p The completion time of task t i The completion time is:

[0111] FT(t i )=ST(t i )+ET(t i ,r(t i ))

[0112] Therefore, the overall execution time of the workflow is:

[0113] T total =max{FT(t i )|t i ∈T}

[0114] The workflow execution cost includes the computation cost and the data transmission cost, so the execution cost is:

[0115]

[0116] Among them, c j,k Represented as resource r j Transfer 1GB of data to r k The required price, λ rj For resources j The asking price unit time, when task i and task j are scheduled on different virtual machine instances, s i,j =1, otherwise s i,j =0.

[0117] For users, whether the workflow can be completed on time according to their needs is the primary concern. Due to factors such as virtual machine crashes and software defects, tasks on virtual machines may not be completed on time. In this study, task execution failures caused by faults should be considered. Generally, transient faults follow a Poisson distribution and are caused by resource i The failure rate is ε i , then task t i In the resource j The reliability of the above execution is:

[0118]

[0119] From the formula, we can know that the reliability of a single task is determined by the execution time of the task on the virtual machine and the failure rate of the virtual machine. If the execution time of the task is longer and the machine failure rate is higher, the reliability is lower.

[0120] According to the additivity of Poisson distribution, the overall reliability of the workflow is:

[0121]

[0122] Based on the above definitions, this paper focuses on the optimization scheduling problem of completion time and execution cost under the overall reliability constraint of the workflow, which can be formally expressed as:

[0123]

[0124] Among them, Rel is the overall reliability value of the workflow of the current scheduling scheme, σ rel is a predefined reliability constraint threshold.

[0125] In this example, for workflow scheduling problems in a deterministic cloud environment, we always assume that server performance is not affected by external factors, and that the execution time and transmission time of tasks on the server can be determined in advance. However, in real-world environments, server performance fluctuations and other factors can cause task execution times to differ from the predetermined time.

[0126] Therefore, this paper introduces fuzzy theory and uses triangular fuzzy numbers to represent the computing time and transmission time of the task. The membership function u(x) is as follows Figure 2 As shown:

[0127] Among them, t m The predefined execution time of the task, the left and right endpoints t l and t u Indicates the range of variation of task execution time.

[0128] Unified use A triangular fuzzy number representing the scalar τ. Based on the concept of uncertainty in running time and transmission time, the completion time and execution cost of a workflow are both triangular fuzzy numbers, expressed as and Then the optimization problem in this paper can be formally expressed as:

[0129]

[0130] For optimization goals Its value is a triangular fuzzy number, and its value is determined by the mean and variance Palacios et al. proposed a comparison criterion to minimize the linear combination of two objective values, called the Palacios criterion. Therefore, the optimization objective The calculation method is as follows:

[0131]

[0132] For the mean and variance Lee et al. defined the mean and standard deviation of fuzzy sets under uniform distribution and proportional distribution respectively. The triangular fuzzy number in this paper is This is a case based on proportional distribution, so the mean and variance Calculated by the following formula:

[0133]

[0134]

[0135] η is the standard deviation The weight of .

[0136] For triangular fuzzy numbers The same treatment method

[0137] In this embodiment, a more practical fuzzy method is proposed to characterize the execution time and transmission time of the task. For the estimated time t, the corresponding triangular fuzzy number is t m is the most likely execution time of the task, that is, the execution time of a given task on the server, t l and t u The values ​​are respectively from the interval [δ1×t m ,t m ] and [t m ,δ2×t m ] are randomly selected, where δ2>1>δ1.

[0138] When performing workflow scheduling, some operations on triangular fuzzy numbers are required. Therefore, we need to redefine some operations on fuzzy numbers.

[0139] Addition operation of triangular fuzzy numbers. For two triangular fuzzy numbers and According to fuzzy theory

[0140] The fuzzy number addition principle defined in derives the following fuzzy number addition rule:

[0141]

[0142] Comparison operation of triangular fuzzy numbers. Inspired by the comparison principle proposed in existing research to compare the size of two fuzzy numbers, the comparison principle adopted in this paper is as follows: but

[0143] The multiplication operation of triangular fuzzy numbers is given by the formula:

[0144]

[0145] In this embodiment, the adaptive particle swarm optimization algorithm based on genetic algorithm APSOGA is specifically described from the following five parts:

[0146] (1) Question coding

[0147] Due to the massive scale of cloud computing resources, the construction of the resource pool significantly impacts the algorithm's search efficiency. To improve the algorithm's search efficiency, we need to design a reasonable encoding method that allows the algorithm to better solve the discrete optimization problem of workflow scheduling. This paper uses a two-dimensional discrete particle encoding method consisting of cloud computing resources and tasks. Each particle represents a solution in the problem space. The position of particle i at time t is shown below:

[0148]

[0149] in, Indicates the virtual machine number where the first task of the i-th particle is located at time t. Figure 3 The following figure shows the coded particles corresponding to a workflow scheduling strategy containing five subtasks. Taking Task 1 as an example, the VM number corresponding to Task 1 is 4, which means that Task 1 will be assigned to VM number 4 in the resource pool for execution.

[0150] (2) Fuzzy fitness function

[0151] In this embodiment, the goal is to optimize the total cost f of the workflow under the overall reliability constraint of the workflow. The total cost f includes the fuzzy execution cost of the workflow and fuzzy execution time This paper has two scheduling objectives, which belongs to a multi-objective planning problem, so the fitness function is set as:

[0152]

[0153] Where k1 and k2 represent the weight coefficients of completion time and execution cost respectively, T one ,C one represents the completion time and execution cost of all tasks executed on a single server. Since particles may fail to satisfy reliability constraints during iteration, it follows that candidate solutions may contain infeasible solutions that do not satisfy reliability constraints. Therefore, when comparing the total costs of two particles in the algorithm, it is important to consider whether the particles are feasible solutions. This article discusses the following three cases for the two particles to be compared.

[0154] 1) For two particles to be compared, if both particles meet the reliability constraint, the particle with the smaller total cost is selected.

[0155]

[0156] 2) If one particle satisfies the reliability constraint and the other particle does not, the particle that satisfies the reliability constraint is selected.

[0157]

[0158] 3) If both particles do not satisfy the reliability constraint, this paper selects the particle with greater reliability because it is more likely to become a feasible solution after iteration.

[0159]

[0160] (3) Particle update strategy

[0161] When searching for the optimal solution, particles need to continuously update their own speed and position. The traditional PSO method of updating its own position has the defect of premature convergence. To avoid premature convergence, APSOGA introduces the crossover operator and mutation operator of the genetic algorithm to update the corresponding parts of the update formula. When particle i is in the tth iteration, the particle update method is as follows, where and ⊙ represent the crossover operator and mutation operator respectively.

[0162]

[0163] APSOGA introduces a mutation operator into the inertia part of the traditional PSO update formula, and its update method is as follows:

[0164]

[0165] Among them, r1 is a random number (0,1), p m For a given mutation probability, when r1 <p m ,but A mutation operation occurs. Will change randomly A server code of a quantile, if r1≥p m No mutation behavior occurs. Figure 4 For Figure 3 Encoded particle mutation operation.

[0166] For the personal cognition part and the social cognition part, the crossover operator is introduced to update the corresponding parts of the traditional update formula. The update method is as follows:

[0167]

[0168]

[0169] Among them, these two formulas update the personal cognition part and the social cognition part respectively, r2 and r3 are random numbers (0,1), and p c For a given crossover probability, when r2 (or r3) <p c When A mutation operation occurs, Cp (or C g ) will randomly select two quantiles of the particle and encode the server code between the particle quantiles and (or gBest t-1 )The server codes between the corresponding quantiles are crossed. Figure 5 For Figure 3 The individual cognitive parts of the cross operation, Figure 6 For Figure 3 The social cognitive part operates crosswise.

[0170] (4) Parameter adjustment

[0171] The inertia weight factor w determines the convergence and search capabilities of the PSO. When the inertia factor w is small, the algorithm has stronger local search capabilities, while when it is large, the algorithm has stronger global search capabilities. The following formula is the inertia adjustment strategy of the traditional PSO algorithm.

[0172]

[0173] Among them, w max and w min They are the maximum and minimum values ​​of w set during initialization, iter cur and iter max is the current number of iterations and the maximum number of iterations of the algorithm.

[0174] The inertia factor w of traditional PSO is only related to the number of iterations, which cannot well meet the complexity of actual problems. This embodiment proposes a new adjustment strategy for the inertia factor w, which can adjust the inertia factor w according to the current particle physics. and the historical global best particle gBest t-1 The value of w is adaptively adjusted based on the difference in , thereby enhancing the search capability of the APSOGA algorithm. The update method of w is as follows:

[0175]

[0176]

[0177] in, Represents particles and the historical global best particle gBest t-1 The number of different quantiles between |T| represents the number of subtasks in the workflow. When the value of is small, it means that the number of percentile differences between the current particle and the historical global best particle is small. Therefore, the value of w should be reduced at this time to enhance the local search ability of the algorithm, improve the convergence effect of the algorithm, and find the optimal solution. On the contrary, the value of w should be increased to enhance the global search ability of the algorithm and expand the search space of the algorithm.

[0178] In addition, the algorithm's personal and social cognitive factors are updated using a linear increase and decrease strategy. The update method is as follows.

[0179]

[0180]

[0181] in, and are the initial values ​​set for parameters c1 and c2 respectively. and The final values ​​of c1 and c2.

[0182] (5) Mapping of particles to scheduling results

[0183] First, for the coding particle i, the mapping from the coding particle i to the scheduling result is given, as shown in Algorithm 1:

[0184] Algorithm 1: Mapping of encoded particles to scheduling results

[0185] Input: (W, R, X)

[0186] Output: F

[0187]

[0188] In addition, the main process of the APSOGA-based workflow fuzzy scheduling strategy includes the following six steps.

[0189] 1) Initialize the relevant parameters of APSOGA, such as population size PN, maximum number of iterations Max iter , inertia factor w, etc., and randomly generate the population.

[0190] 2) Calculate the fitness. The initial state of each particle is the individual best particle, and the particle with the smallest fitness value in the initial population is set as the global best particle.

[0191] 3) Introduce the mutation and crossover operators of the genetic algorithm to update the position of the particles themselves and calculate the fitness of the updated particles.

[0192] 4) If the fitness of the updated particle is less than the fitness of the individual best particle, the individual best particle is updated and the current particle is set as the individual best particle.

[0193] 5) At the same time, the fitness of the updated particle is compared with the fitness of the global best particle. If the fitness of the current particle is less than the fitness of the global best particle, the global best particle is updated, the current particle is set as the global best particle and the best fitness is updated.

[0194] 6) Check whether the algorithm iteration end condition is met. If so, the algorithm ends; otherwise, return to 3).

[0195] Example 1:

[0196] The workflow test model used in this example adopts workflows from five different fields studied by Bharathi et al.: CyberShake for earthquake science, Sipht for bioinformatics, Montage for astronomy, LIGO for gravity physics, and Epigenomics for biological genetics. These five workflows each have their own different structures. Figure 7 The structures of the five workflow samples are given. The detailed information of these workflows are stored in XML format files.

[0197] This article selects three different workflow sizes for different workflows: a micro workflow with approximately 30 tasks, a small workflow with approximately 50 tasks, and a medium workflow with approximately 100 tasks. The cloud resource pool in this article consists of six cloud servers, as shown in Table 1. Assuming the m4.16xlarge server has the highest computing power, the computation time of each task in the workflow on the m4.16xlarge server is directly obtained from the corresponding XML file. Based on the performance ratio of other servers to the m4.16xlarge server, the execution time of each task on other servers can be calculated.

[0198] Six virtual machines were selected from the Amazon EC2 cloud platform for the simulation experiment. The virtual machine resource configuration information is shown in Table 1. Amazon EC2 usually charges a price interval of 60s or 1h. i ,This paper selects 60s as the asking price interval, and the price of transmitting 1GB of data is 0.2.

[0199] Table 1 Virtual machine configuration information

[0200] Table 1 Virtual machine configuration information table

[0201]

[0202] Workflow reliability σ rel Let T be the reliability of the workflow running on the server m4.xlarge at a given time. one and C oneThe completion time and execution cost of the workflow running on a single m4.2xlarge server are shown in Table 2. To convert the task execution time and transmission time into triangular fuzzy numbers, parameters δ1 and δ2 are set to 0.85 and 1.2, respectively, and the standard deviation weight coefficient η is set to 1. The fitness function weight coefficients k1 and k2 are 0.2 and 0.3, respectively.

[0203] In order to evaluate the effectiveness of the APSOGA algorithm, this embodiment compares the APSOGA strategy with the traditional PSO strategy and the random strategy. These algorithms are usually used as comparison algorithms in existing workflow scheduling problems.

[0204] In the traditional PSO strategy, the same encoding method as APSOGA is adopted, the update method adopts the traditional update method, and the traditional PSO algorithm parameter setting refers to existing research. The population size is set to 50, the maximum number of iterations is 500, c1=1, c2=1, and w=1.

[0205] Random strategy: This method is based on a random search strategy and uses the same encoding method as APSOGA. It updates the particle encoding in a random manner. Each iteration does not affect each other. It randomly searches in the solution space of the problem, calculates the fitness of each particle, and records the optimal solution during the search process.

[0206] To test the workflow scheduling performance of the APSOGA strategy, PSO strategy, and random strategy in a cloud environment with factors such as network fluctuations and server failures, this embodiment uses three strategies to conduct 30 repeated experiments on five workflows of different scales. Tables 2, 3, and 4 respectively record the optimal fitness values ​​and average fitness values ​​of the workflow scheduling experiments of the three strategies. The fitness value represents the weighted sum of completion time and execution cost.

[0207] Table 2 shows the scheduling results of 30 repeated experiments using the APSOGA, PSO, and random strategies for five micro-workflows. As can be seen from the table, for the micro-workflows, the APSOGA strategy consistently achieves the optimal solution for both average and optimal fitness values, followed by the PSO strategy, and the random strategy performs the worst. This is because the APSOGA strategy incorporates genetic algorithm mutation and crossover operations on top of the PSO strategy, resulting in better global optimization capabilities than the PSO algorithm. The random strategy, on the other hand, performs poorly due to its low efficiency in searching for solutions in the problem space.

[0208] The scheduling results for 30 repetitions of a small workflow are shown in Table 3. The APSOGA strategy achieves the optimal solution in both optimal fitness and average fitness. Furthermore, the APSOGA strategy outperforms the traditional PSO strategy by up to 4.7% in optimal fitness, and the random strategy by 20%.

[0209] Table 2 Comparison of micro-workflow experiment effects

[0210]

[0211]

[0212] Table 4 shows the scheduling results for 30 iterations of a medium-sized workflow. The APSOGA strategy outperforms other algorithms to varying degrees. Furthermore, the performance of the random strategy for medium-sized and micro-sized workflows shows that as the number of tasks increases and the problem space expands, the performance of the random strategy deteriorates. The gap between the experimental results and those of the APSOGA strategy widens with the increasing number of tasks. In summary, the APSOGA strategy exhibits superior scheduling performance compared to other scheduling strategies for both small and large workflows.

[0213] Table 3 Comparison of small-workflow experiment effects

[0214]

[0215] The APSOGA strategy introduces the genetic algorithm's mutation and crossover operators based on the traditional PSO strategy. This allows the algorithm to avoid the premature convergence problem of the traditional PSO strategy, prevent particles from falling into local optimality, and enable particles to have better global optimization capabilities. Secondly, the APSOGA strategy adjusts the update method for the inertia factor w. The traditional PSO strategy uses a linear method to update the inertia factor w, which is only related to the current number of iterations and cannot effectively solve complex problems. The APSOGA strategy uses an adaptive adjustment method, taking into account the number of percentile differences between the current particle and the global optimal particle, improving the algorithm's search performance for w. Experimental results show that the APSOGA strategy can obtain better scheduling solutions compared to both the traditional PSO strategy and the random strategy.

[0216] Table 4 Comparison of medium-workflow experiment effects

[0217]

[0218] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. A workflow scheduling method for reliability constraints in a cloud environment, characterized by: The following steps are involved: Step S1: Construct a problem model that uses triangular fuzzy numbers to represent the execution time and transmission time of tasks under the overall reliability constraint of the workflow; Step S2: Improve the PSO algorithm and construct an adaptive particle swarm optimization algorithm based on genetic algorithm; Step S3: Adaptive particle swarm optimization based on genetic algorithm optimizes the fuzzy completion time and fuzzy execution cost of the workflow under the overall reliability constraint of the workflow to obtain the optimal workflow scheduling solution; The optimization scheduling problem of completion time and execution cost under the overall reliability constraints of the workflow can be formally expressed as: Among them, T total is the total execution time of the workflow, C total is the workflow execution cost, Rel is the overall reliability value of the workflow in the current scheduling scheme, σ rel is a predefined reliability constraint threshold; The adaptive particle swarm optimization algorithm based on genetic algorithm is as follows: (1) Using a two-dimensional discrete particle encoding method consisting of cloud computing resources and tasks, a particle represents a solution in the problem space. The position of particle i at time t is as follows: in, Indicates the virtual machine number where the first task of the i-th particle is located at time t; (2) The goal is to optimize the total cost f of the workflow under the constraints of the overall reliability of the workflow. The total cost f includes the fuzzy execution cost of the workflow and fuzzy execution time There are two scheduling goals, which belongs to a multi-objective planning problem, so the fitness function is set as: Where k1 and k2 represent the weight coefficients of completion time and execution cost respectively, T one ,C one Indicates the completion time and execution cost of all tasks executed on only one server; (3) The crossover operator and mutation operator of the genetic algorithm are introduced. When particle i is in the tth iteration, the particle update method is as follows, where and ⊙ represent the crossover operator and mutation operator respectively: The mutation operator is introduced into the inertia part of the traditional PSO update formula, and its update method is as follows: Among them, r1 is a random number (0,1), p m For a given mutation probability, when r1 <p m ,but A mutation operation occurs. Will change randomly A server code of a quantile, if r1≥p m No mutation behavior occurs; For the personal cognition part and the social cognition part, the crossover operator is introduced to update the corresponding parts of the traditional update formula. The update method is as follows: Among them, these two formulas update the personal cognition part and the social cognition part respectively, r2 and r3 are random numbers (0,1), and p c For a given crossover probability, when r2 <p c or r3 <p c When A mutation operation occurs, C p or C g The two quantiles of the particle will be randomly selected and the server code between the particle quantiles will be or gBest t-1 The server codes between the corresponding quantiles are crossed; (4) The new inertia factor w adjustment strategy can be based on the current particle and the historical global best particle gBest t-1 The value of w is adaptively adjusted based on the difference in , and the update method of w is as follows: in, Represents particles and the historical global best particle gBest t-1 The number of different quantiles between |T| represents the number of subtasks in the workflow; The specific situations of the two particles to be compared are divided into the following three cases for discussion: 1) For two particles to be compared, if both particles meet the reliability constraint, the particle with the smaller total cost is selected. 2) If one particle satisfies the reliability constraint and the other particle does not, the particle that satisfies the reliability constraint is selected. 3) If both particles do not meet the reliability constraint, the particle with greater reliability is selected because it is more likely to become a feasible solution after iteration.

2. The reliability-constrained workflow scheduling method in a cloud environment according to claim 1, characterized in that: The overall workflow execution time, workflow execution cost, and overall workflow reliability are as follows: The workflow is represented by a directed acyclic graph (DAG), i.e., G = <T, E, D>; Where T represents a set of nodes, T={t1,t2,...,t n }, each node is a task; E represents a set of edges between tasks, E={e 1,2 ,e 1,3 ,...,e i,j }, indicating the control or data dependency between tasks, task t i and task t j The size of the data transferred between i,j =(t i ,t j ) represents; D={d(t1),d(t2),...,d(t n )} represents the computational workload of the task; from the above definition, we can conclude that task t i The direct predecessor task P(t i )={t k |e k,i Each workflow has a set reliability value, Rel. When the reliability of the scheduling strategy satisfies the given reliability constraint, the scheduling strategy is considered feasible. Assume that the resources of the cloud environment are composed of m different virtual machine instance types, and R = {r1, r2, ..., r m } indicates that for resource r i use Indicates that Represents resource r i The opening time, Represents resource r i Closing time, u i Represents resource r i The computing power of c i Represents resource r i The price per unit time, ε i Represents resource r i The failure rate of different instances varies with their computing power; Assume that task t i Deploy to resource j Then task t i The execution time is: For task t i The parent task P(t i )={t p |e p,i }, parent task t p To task t i The transmission time trans(t p ,t i )for: Where β represents r(t p ) and r(t i ) between the bandwidth; Considering the data dependency between tasks, that is, the child task can only be started after all parent tasks are completed, task t i The start time is defined as follows: ST(t i )=max{max(FT(t p )+trans(t p ,t i )),Ava(r(t i ))} Ava(r(t i )) represents the virtual machine r(t i ) Prepare to execute task t i The earliest time, FT(t p ) represents task t p The completion time of task t i The completion time is: FT(t i )=ST(t i )+ET(t i ,r(t i )) Therefore, the overall execution time of the workflow is: T total =max{FT(t i )|t i ∈T} The workflow execution cost includes the computation cost and the data transmission cost, so the execution cost is: Among them, c j,k Represented as resource r j Transfer 1GB of data to r k The required price, For resources j The asking price unit time, when task i and task j are scheduled on different virtual machine instances, s i,j =1, otherwise s i,j =0; Considering the task execution failure caused by the fault, we assume that the instantaneous fault follows the Poisson distribution, and the resource r i The failure rate is ε i , then task t i In the resource j The reliability of the above execution is: According to the additivity of Poisson distribution, the overall reliability of the workflow is:

3. The reliability-constrained workflow scheduling method in a cloud environment according to claim 1, characterized in that: Fuzzy theory is introduced and triangular fuzzy numbers are used to represent the computing time and transmission time of tasks. The membership function is u(x), where t m The predefined execution time of the task, the left and right endpoints t l and t u Indicates the range of variation of task execution time; use The triangular fuzzy number representing the scalar τ, based on the concept of uncertainty in running time and transmission time, the completion time and execution cost of the workflow are both triangular fuzzy numbers, expressed as and The optimization problem in this paper is formally expressed as: For optimization goals Its value is a triangular fuzzy number, and its value is determined by the mean and variance Joint decision-making; optimizing goals The calculation method is as follows: For the mean and variance are the mean and standard deviation of the fuzzy set under uniform distribution and proportional distribution respectively, and the triangular fuzzy number This is a case based on proportional distribution, so the mean and variance Calculated by the following formula: η is the standard deviation The weight of triangular fuzzy number The same treatment method 4. The reliability-constrained workflow scheduling method in a cloud environment according to claim 1, characterized in that: For the estimated time t, the corresponding triangular fuzzy number t m is the most likely execution time of the task, that is, the execution time of a given task on the server, t l and t u The values ​​are respectively from the interval [δ1×t m ,t m ] and [t m ,δ2×t m ] are randomly selected, where δ2>1>δ1; the trigonometric functions are defined as follows: Addition operation of triangular fuzzy numbers: Comparison operation of triangular fuzzy numbers: If but Multiplication operation of triangular fuzzy numbers:

5. The reliability-constrained workflow scheduling method in a cloud environment according to claim 1, characterized in that: The update method of personal cognitive factors and social cognitive factors adopts a linear increase and decrease strategy, and the update method is as follows: in, and are the initial values ​​set for parameters c1 and c2 respectively. and The final values ​​of c1 and c2.

6. The reliability-constrained workflow scheduling method in a cloud environment according to claim 1, characterized in that: The step S3 is specifically as follows: 1) Initialize the relevant parameters of the adaptive particle swarm algorithm based on genetic algorithm, such as population size PN, maximum number of iterations Max iter , inertia factor w, and randomly generate the population; 2) Calculate the fitness. The initial state of each particle is the individual best particle, and the particle with the smallest fitness value in the initial population is set as the global best particle; 3) Introduce the mutation and crossover operators of the genetic algorithm to update the position of the particles themselves and calculate the fitness of the updated particles; 4) If the fitness of the updated particle is less than the fitness of the individual best particle, the individual best particle is updated and the current particle is set as the individual best particle; 5) At the same time, the fitness of the updated particle is compared with the fitness of the global best particle. If the fitness of the current particle is less than that of the global best particle, the global best particle is updated, the current particle is set as the global best particle, and the best fitness is updated; 6) Check whether the algorithm iteration end condition is met. If so, the algorithm ends; otherwise, return to 3).

Images