A workflow scheduling method considering security risk and cost optimization in cloud environment

Abstract

The application provides a workflow scheduling method considering security risks and cost optimization in a cloud environment, which reduces the overall security risks and execution costs of the workflow under the given deadline constraint of a user. Firstly, the cloud computing resources are defined, the workflow structure is abstracted, and a security risk calculation method is proposed. Then, a reinforcement learning model oriented to security risks and cost optimization is constructed. Next, an action selection algorithm in the learning process is determined. Finally, iterative learning is carried out based on the Q learning algorithm idea, and finally the task scheduling scheme is determined, and the overall security risks and total costs of the workflow are obtained.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of cloud computing, and particularly relates to a workflow scheduling method considering security risks and cost optimization in a cloud environment. BACKGROUND

[0002] A workflow is an abstraction and description of a set of task flows, and the tasks have certain dependency relationships. A directed acyclic graph (DAG) is the most commonly used model for constructing a workflow, wherein a vertex represents a task node to be executed, and a directed edge represents data transmission between tasks. The application of a workflow is very wide, for example, used for analyzing and monitoring a drug treatment process, used for analyzing molecular properties, etc., which greatly increases the complexity of workflow scheduling, and usually needs to rely on more powerful storage and computing resources to complete scheduling within an acceptable time. Cloud computing integrates resources such as software, hardware, data and information services together to form a super-large system resource pool to execute some complex tasks, and a user configures corresponding computing resources according to individualized needs, and can complete related tasks without additional manual interaction. Under the support of the powerful elasticity and scalability of cloud computing, the user can quickly update and remove the resources, thereby reducing the use cost of the user. As a commonly used application mode in a cloud environment, a cloud workflow usually defines related business rules according to different business needs. A user can deploy a cloud workflow on a cloud platform in a public cloud, a private cloud, a hybrid cloud and the like, and utilize the high performance and flexibility of cloud computing to realize automatic execution and management of the workflow application.

[0003] Using cloud resources needs to pay corresponding fees, and the lower the cost, the slower the task completion time is, and the main concern of cloud workflow scheduling research is mostly concentrated on the optimization of cost and time overhead. Meanwhile, there are many potential security risks in a cloud computing environment, which can cause tampering, deletion and leakage of workflow data, and seriously damage the security of user data, endanger personal privacy, business secrets and even national security, and therefore, security has become one of the important factors to be considered in cloud workflow scheduling. Therefore, the present application provides a workflow scheduling method considering security risks and cost optimization in a cloud environment. SUMMARY

[0004] The purpose of the present application is to reduce the overall security risks and execution cost of a workflow under the constraint of a deadline given by a user.

[0005] To achieve the above object, the application provides a workflow scheduling method considering security risk and cost optimization in a cloud environment. First, a security risk calculation model is constructed based on a cloud computing resource model and a cloud workflow model; then a reinforcement learning model under the cloud workflow is constructed; then an action selection algorithm is given; finally, based on the Q learning idea, iteration is carried out to select the most suitable server for each task and obtain the final allocation scheme, calculate the overall security risk and execution cost of the workflow; the basic process of the method is as shown in Figure 4

[0006] 1. The further defined technical solution of the application is:

[0007] Preferably, in the above technical solution, the following steps are included:

[0008] Step 1: Define the cloud computing resources used by the workflow. The communication link between servers is in full connectivity mode, and the transmission bandwidth between servers is equal. Different servers provide different running speeds, use prices and security levels.

[0009] Step 2: Abstract the workflow structure. Use DAG graph to represent the workflow, connect the front task with the rear task, the front and rear tasks have data dependency, and the edge has weight, i.e. the transmission data between tasks; on this basis, security demand attribute is added to each task, which represents the degree of attention to different security.

[0010] Step 3: Define the security risk model. The security risk in the workflow scheduling process is a probability problem, and the distribution of risk probability generally follows Poisson probability distribution in any given time interval, so this paper uses Poisson probability distribution model to represent the security risk probability. The risk model considers the task execution risk and data transmission risk.

[0011] Step 4: Construct the reinforcement learning model for security risk and cost optimization. First, construct the state space by using the workflow scheduling condition; then construct the action space by using the task sub deadline; finally, define the reward function of security risk and cost.

[0012] Step 5: Determine the action selection algorithm. Based on the ε-greedy greedy algorithm, select the action, control the probability of random action selection through the parameter ε, so that the agent can explore unknown actions under a certain probability. Use the task sub deadline to constrain the action set to meet the deadline constraint.

[0013] ​Step 6: Task scheduling based on Q-learning idea. The core process is to select an action according to the current state, then calculate the immediate reward generated by the action and get the next action, then update the Q table and state, repeat the process until the maximum iteration number is reached, to obtain lower security risk and execution cost.

[0014] 2. Preferably, in the above technical solution, in step 1, the cloud computing resources are defined as follows.

[0015] The cloud computing resource model is represented as a binary tuple R={V, SP}. Wherein, V={v i |1≤i≤m} represents a set of m servers, each server has different execution capabilities and the communication link between servers adopts full connectivity, pc i represents the execution capability of the i-th server, price j represents the use price of the i-th server per unit time, and bw represents the transmission bandwidth between servers. SP={sp i |1≤i≤m} represents a set of server security levels, represents the security level set of the i-th server, respectively represent the security level of confidentiality, integrity and authenticity.

[0016] 3. Preferably, in the above technical solution, in step 2, the workflow structure is abstracted as follows.

[0017] The workflow W=(T, E, SR, DT) is represented based on the DAG graph, wherein: T={t i |1≤i≤n} represents a set of workflow tasks, t i represents the i-th workflow task, data i represents the number of instructions that the i-th task needs to execute; E={e i，j |1≤i,j≤n} represents an edge set, e i，j represents the data transmission amount between tasks, task t i must be completed before task t j , the entry task is recorded as t entry , the exit task is recorded as t exit , the predecessor and successor tasks of task t i are recorded as and t i succ ; SR={sr i |1≤i≤n} represents a set of task security requirements, represents the security requirement of the i-th task, respectively represent the security requirements of confidentiality, integrity and authenticity; DT represents the deadline of the workflow, which is usually specified by the user.

[0018] task t i The execution time ET(t j , v i ) on server v j can be calculated by equation (2.1).

[0019]

[0020] The average execution time of task t i can be calculated by equation (2.2), |V| represents the number of servers.

[0021]

[0022] The data transfer time TT(t i , t j ) from task t i to task t j can be calculated by equation (2.3). When two tasks are not executed on the same server, i.e., s(t i )≠ s(t j ), the data transfer time is calculated by the amount of data e i，j transferred and the transfer bandwidth bw; when two tasks are executed on the same server, i.e., s(t i ) = s(t j ), the data does not need to be transferred between servers, so the transfer time is 0.

[0023]

[0024] The earliest start time EST(t i , v j ) of task t i on server v j depends on the maximum idle time of the server and the maximum time for all predecessor tasks to complete data transfer, which can be calculated by equation (2.4). The maximum idle time of server v j is represented by empty(v j ), and the value of empty(v entry ) is 0.

[0025]

[0026] The completion time FT of a task on a server is the sum of the earliest completion time and the execution time of the task, which can be calculated by equation (2.5). The completion time of the workflow is the completion time FT(t exit , v j).

[0027] FT(t i v j ) = EST(t i v j )+ET(t i v j (2.5)

[0028] Task t i On server v j The cost incurred during the execution of the above (cost(t)) i v j ) is the product of the unit time price and the execution time, which can be calculated by formula (2.6).

[0029] cost(t i v j )=ET(t i v j )·price j (2.6)

[0030] The total cost of workflow scheduling, cost(W), is the sum of the execution costs of all tasks, and can be calculated using formula (2.7), s(t i ) represents task t i Selected server number.

[0031]

[0032] 4. Preferably, in the above technical solution, in step 3, the security risk model is defined as follows.

[0033] Step 31, Define task execution risks. Task t i On server v j When executed on server v j Security level provided Unable to satisfy task t i security requirements Right now At that time, the probability of risk increases. The increase in differences exhibits an exponential growth trend; conversely, no risk arises, and the probability of safety risk is 0. Task t i On server v j Security requirements during execution The resulting security risks p x e (t i v j It can be calculated by equation (3.1).

[0034]

[0035] where λ is the risk factor, representing the number of external attacks on the server per unit time. A negative exponential probability model is used to ensure that the security risk probability increases as the degree of satisfaction of the security requirement of the task decreases.

[0036] Task t i is executed on server v j e The execution risk p i (t j , v e ) can be calculated by equation (3.2).

[0037]

[0038] The execution risk p e (W) of a workflow is the average of all the task execution risks, which can be calculated by equation (3.3).

[0039]

[0040] where |T| represents the number of tasks in the workflow; σ ij = 1 indicates that task t i is assigned to server v j for execution, and σ ij = 0 indicates that task t i is not assigned to server v j for execution.

[0041] Step 32, data transmission risk definition. The security risk probability of data transmission from task t i to task t j is defined as p t (t i , t j ), which can be calculated by equation (3.4). If the two tasks before and after the transmitted data are assigned to the same server for execution, the risk probability is 0.

[0042]

[0043] where s(t i ) represents the number of the selected server of task t i .

[0044] The data transmission risk p t (W) of a workflow can be calculated by equation (3.5), and |E| represents the number of data transmission edges in the workflow.

[0045]

[0046] Step 33, workflow overall security risk definition. The overall security risk p(W) of the workflow is represented as the average of the task execution risk p e (W) and the data transmission risk p t (W) can be calculated by formula (3.6).

[0047]

[0048] The goal of the present application is to minimize the overall security risk p(W) and the execution cost cost(W) of the workflow under the condition that the deadline constraint of the export task is met, as shown in formula (3.7).

[0049]

[0050] wherein rw represents the risk preference, i.e. the degree of preference of the user for optimizing the security risk, and rw∈[0~1]. The larger the rw is, the more inclined to optimize the security risk; on the contrary, the more inclined to optimize the execution cost. The user can select a suitable rw value according to different needs.

[0051] 5. Preferably, in the above technical solution, in step 4, the reinforcement learning model for the security risk and cost optimization problem is constructed, specifically as follows.

[0052] Step 41, construction of state space. For an input workflow instance, the priority and scheduling order list of the task are fixed and unchangeable, at a certain scheduling time point, the current task to be scheduled and the scheduled end task are known, while the unscheduled task has great uncertainty. Therefore, the system state can be defined by the known task scheduling situation.

[0053] The state space is defined as S={s i |i=1,2,...,n s}。n s represents the number of states in scheduling, s i represents the i-th state, each state is related to the current task to be scheduled and the scheduled end task, and can be represented as a binary tuple s i =<t cur ,s sched >,t cur represents the current task to be scheduled, s sched ={(t i ,s(t i )|t i ∈T sched} represents the scheduling result of the scheduled task, which is composed of the task and the server selected by the task, wherein T schedThis represents the set of tasks that have finished scheduling. This set is dynamically updated during the scheduling process, and is stored in T when a task completes its execution. sched Middle. s sched It is dynamic; the same task can be scheduled and executed on different servers, leading to variations in performance. sched The changes.

[0054] Before starting the learning iteration, an initial state s1 must be specified for the system to ensure that subsequent iterations can continue. In workflow scheduling, scheduling always begins with the entry task; therefore, the initial state is related to the entry task and is defined as follows: Once a task is scheduled and completed, that task will be added to T. sched To update the next state of s sched In a set, the next task in the scheduling order list will become the next state t. cur For example, the initial state is The next task is t2. Assuming task t1 was executed on server v3, the next state can be represented as s2 =<t2,{(t1,v3)}> .

[0055] Step 42: Construct the action space. The set of available actions contains all the possible actions the agent can take in a given state, denoted by A. i Represents state s i The set of available actions. Generally, for workflow scheduling problems, A... i This is a set of all servers, meaning that in each state, any server can be selected to execute the current task. However, this may lead to excessively long workflow execution times. To meet deadline constraints, task sub-deadlines (sdt) are used to apply to the action set A. i Limitations are imposed to ensure that available actions do not cause the completion time to exceed the deadline. The sub-deadlines of the task can be calculated using formula (4.1).

[0056]

[0057] Where pr(t) i ) represents task t i The priority can be calculated using formula (4.2).

[0058]

[0059] If the completion time of a task in a certain state on the server does not exceed the sub-deadline of the task, then the server is added to the set of available actions for that state, as shown in formula (4.3).

[0060] A i ={v j∈ V | FT(t cur , v j )≤ sdt(t cur )} (4.3)

[0061] Step 43: Define the reward function. In the scheduling process, for the current state, the environment will feedback the reward value generated by the action after the action is executed. The better the action is selected, the greater the reward value is. In this chapter, there are two optimization objectives, safety risk and execution cost, so the reward value includes the reward value of safety risk and the reward value of execution cost.

[0062] Safety risk includes execution risk and data transmission risk. When an action is selected and executed, the safety risk var(risk) increased by the system can be calculated by formula (4.4), denotes the number of predecessor tasks of task t i .

[0063]

[0064] For safety risk, in state s i , the reward value r1(s i , a i ) generated by action a i is the ratio of the minimum safety risk to the actual safety risk in all available actions, which can be calculated by formula (4.5), a i ∈ A i denotes the actual selected action in state s i . This means that the smaller the safety risk increased by the system after the action is executed, the greater the reward value corresponding to the safety risk.

[0065]

[0066] For execution cost, in state s i , the reward value r2(s i , a i ) generated by action a i is the ratio of the minimum cost to the actual cost in all available actions, which can be calculated by formula (4.6). This means that the lower the cost of task execution after the action is executed, the greater the reward value corresponding to the cost.

[0067]

[0068] The total reward reward(s i , a i ) generated by action a i in state s i is determined by r1 and r2 together, as shown in formula (4.7).

[0069] reward(s i a i )=rw·r1(s i a j )+(1-rw)·r2(s i a j (4.7)

[0070] Here, rw represents risk appetite. The larger the rw, the more important the reward for safety risk, and vice versa.

[0071] 6. Preferably, in the above technical solution, step 5, determining the action selection algorithm, is as follows.

[0072] In reinforcement learning, action selection generally employs two strategies: exploitation and exploration. Exploitation involves the agent selecting actions based on currently known information, i.e., choosing the action with the highest Q-value. However, this results in actions that have not yet been selected being never chosen, severely impacting the algorithm's practicality. Exploration, on the other hand, attempts to select new actions in hopes of obtaining greater long-term rewards. In this invention, an ε-greedy greedy algorithm is used to select actions, with the parameter ε controlling the probability of random action selection, allowing the agent to explore unknown actions with a certain probability.

[0073] First, calculate the set of available actions A according to formula (4.3). i Then, a random number rd (rd∈[0,1]) is generated, which determines the strategy for choosing an action. If rd<ε, from A... i Randomly select an action and return it; if rd≥ε, select A. i The action with the highest Q value is returned. The system state during workflow scheduling has significant uncertainty and cannot be pre-initialized in the Q table; it should be dynamically added during scheduling. Therefore, if state s is detected... i If the value is not in the Q table, you must first add the s value. i Add the action to the Q-table and initialize the Q-values ​​for all actions to 0. Then, use a random strategy to select from A. i Select one action to return.

[0074] 7. Preferably, in the above technical solution, in step 6, task scheduling is performed based on the Q-learning concept, as follows.

[0075] Step 61, Initialization. Initialize the deadline DT, set the risk factor λ, and the risk preference rw. Calculate the execution time of each task on each server according to formula (2.1), calculate the average execution time of each task according to formula (2.2), and calculate the sub-deadlines of all tasks according to formula (4.1).

[0076] Step 62, calculate the priority of all tasks according to formula (4.2) and sort them in descending order to get the scheduling order list of tasks, denoted as scheduleList.

[0077] Step 63, initialize the Q table. Denote the Q table as qTable, which is a set with key-value pairs as elements, where the key represents the system state and the value represents the Q value list corresponding to all actions in that state. The state in the environment is related to the current task and the scheduled situation, which is affected by the action selection, leading to unpredictable states in the environment. Therefore, qTable is initialized as an empty set, and new states are dynamically added to qTable during the learning process, Q values are initialized and updated.

[0078] Step 64, reset the scheduling environment. In order to ensure that the current scheduling result does not affect the next scheduling, the environment needs to be reset, mainly resetting the scheduling result list, completion time, execution cost, safety risk, and execution cost data.

[0079] Step 65, update the value of ε. The role of ε is to control the probability of randomly selecting an action from the available action set A i The larger the value of ε, the greater the probability of randomly selecting an action in step 5. When the agent and the environment first interact, there is less past experience to draw upon, so ε should be set to a larger value. As the learning time increases, ε should gradually decrease to ensure that each state can select the action with the maximum Q value after learning is complete. Denote the current iteration number as iter and the maximum iteration number as maxIterNum. Update ε using formula (6.1), and gradually reduce the value of ε from 1 to 0.

[0080]

[0081] Step 66, initialize the system state. Before starting learning, an initial state is specified, where the current task in the initial state is the first task in the scheduling order list, and the scheduled situation is an empty set.

[0082] Step 67, perform a round of scheduling. Traverse the scheduling list scheduleList and schedule tasks one by one. After all tasks are scheduled, if the maximum iteration number is reached, go to step 68; otherwise, go back to step 64. The process of a round of scheduling is as follows.

[0083] During the traversal of tasks, for each task t i ∈scheduleList, first generate the current state s i , and select an action ai ; then perform the action, i.e. dispatch task t i to a i corresponding server, and after the execution is completed, t i and its selected server join s sched , then generate the next state s i+1 . Then calculate the immediate reward reward(s i , a i ) generated by performing a i in the current state according to formula (4.7). If the current task is not an exit task, update Q(s i , a i ) using formula (6.2);

[0084]

[0085] wherein, a e [0, 1] is a learning rate, indicating the degree to which the new value covers the old value; g e [0, 1] is a discount factor, indicating the degree to which the current value is affected by future rewards. If g is 0, it means that only the current reward is considered, and as g increases, longer-term future rewards will be considered. If the current task is an exit task, therefore, update Q(s i , a i ) as formula (6.3), since the next state cannot be obtained at this time, there is no corresponding future reward, and only the current reward is considered when updating.

[0086] Q(s i , a i ) = Q(s i , a i ) + a · (reward(s i , a i ) - Q(s i , a i )) (6.3)

[0087] Finally, update the current state s i+1 with the next state s i , continue to dispatch the next task until all tasks are dispatched.

[0088] Step 68, calculate the overall security risk according to formula (3.6) and the total cost according to formula (2.7).

[0089] Step 69, the algorithm ends. BRIEF DESCRIPTION OF DRAWINGS

[0090] Figure 1 is a workflow instance;

[0091] Figure 2the number of instructions and security requirements of the workflow tasks;

[0092] Figure 3 the related parameters of the servers;

[0093] Figure 4 the abstract flowchart; DETAILED DESCRIPTION

[0094] Embodiments of the present application are described in detail below with reference to the attached drawings. The embodiments described below are examples of the present application and are not intended to limit the present application.

[0095] The workflow W in the example contains 8 tasks in total, where t1 is the entry task and t8 is the exit task. The workflow structure and the e i，j The number of instructions and security requirements of each task are shown in Table 1. The server set contains 3 servers, and the processing capacity, transmission bandwidth, unit price and security level of each server are shown in Table 2. Figure 1 Figure 2 Figure 3

[0096] First, the deadline DT = 30, λ = 1.0, and rw = 0.5 are initialized. The execution time of each task on each server, the average execution time, and the sub-deadline are calculated and shown in Table 1. The qTable is initialized to be empty. The maximum number of iterations is set to be 300.

[0097] Table 1 Execution time, average execution time and sub-deadline of tasks on servers

[0098]

[0099] The priority of each task is then calculated and sorted in descending order, and the scheduling order is t1, t4, t5, t3, t2, t7, t6, t8.

[0100] The iterative learning process is then started. In the first round of the learning process, the value of ε is first calculated to be 1. According to the scheduling order, the first task is t1, and thus the initial state is (t1, []). The selected action is v3, i.e., task t1 is scheduled to be executed on server v3. The next task is t4, and thus the next state that can be obtained is (t4, [(t1, v3)]). The qTable is then updated, as shown in Table 2. Since the states are dynamically added in the learning process, there are only two states in the current qTable.

[0101] Table 2 First update of qTable in the first round of the learning process

[0102] ​​​

[0103] When the qTable is updated, the current state is updated to (t4, [(t1, v3)]), and the Q table is updated according to the same operation process. Until the exit task is scheduled, the first round of learning process is completed, and the qTable at this time is shown in Table 3.

[0104] Table 3 qTable updated at the end of the first round of learning process

[0105]

[0106] After repeating the above learning process 300 times, the algorithm is completed, the completion time of the workflow is 26.35 at this time, the deadline constraint is satisfied, the overall safety risk is 0.21, and the total cost is 49.725.

[0107] The foregoing description of specific exemplary embodiments of the application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise forms disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain the principles of the application and its practical application, and to thereby enable others skilled in the art to best utilize the application and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the scope of the application be defined by the claims and their equivalents.

Claims

1.A method for workflow scheduling considering security risk and cost optimization in cloud environment, characterized in that The method comprises the following steps of: Step 1: defining the cloud computing resources used by the workflow; Step 2: abstracting the workflow structure; Step 3: defining the security risk model; Step 4: constructing a reinforcement learning model oriented to security risk and cost optimization; first, constructing a state space by using the workflow scheduling condition, then constructing an action space by using the task sub-deadline, and finally defining a reward function including the reward value of security risk and the execution cost; Step 5: determining the action selection algorithm; based on the ε-greedy greedy algorithm, the action is selected, the probability of randomly selecting the action is controlled through the parameter ε, so that the intelligent agent can explore the unknown action at a certain probability, and the action set is constrained by using the task sub-deadline to meet the deadline constraint; Step 6: task scheduling based on the Q learning idea. 2.The method of claim 1, wherein, In step 1, the cloud computing resources are defined; the communication links between servers are in a full connected mode, the transmission bandwidth between servers is equal, different servers provide different running speeds, use prices and security levels. 3.The method of claim 1, wherein, In step 2, the workflow structure is abstracted; the workflow is represented by using a DAG graph, the front task is connected with the rear task, the front and rear tasks have data dependency, and the edge has a weight, i.e. the transmission data between tasks; on this basis, a security demand attribute is added to each task, indicating the degree of attention to different security. 4.The method of claim 1, wherein, In step 3, the security risk model is defined; the security risk in the workflow scheduling process is a probability problem, the distribution of the risk probability follows the Poisson probability distribution in any given time interval, therefore, the Poisson probability distribution model is used to represent the security risk probability, and the risk model comprehensively considers the task execution risk and the data transmission risk. 5.The method of claim 1, wherein, In step 6, the task scheduling based on the Q learning idea; the core process is to select an action according to the current state, then calculate the immediate reward generated by the action and obtain the next action, then update the Q table and the state, repeat the process until the maximum iteration number is reached, so as to obtain lower security risk and execution cost.

Images