Workflow scheduling method considering security risk and cost optimization in cloud environment
By building a security risk calculation model and reinforcement learning model in a cloud computing environment, combining Q learning ideas, and optimizing workflow scheduling, the security risk and cost optimization problems of workflow scheduling in cloud computing are solved, and the security is reduced within the deadline is achieved The effect of risk and execution costs.
Patent Information
- Application Number
- CN202310518448.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-05-09
AI Technical Summary
Workflow scheduling in the cloud computing environment faces the dual challenges of security risks and cost optimization. The existing technology is difficult to effectively reduce overall security risks and execution costs, especially under the deadline constraints given by users.
Adopt security risk calculation model based on cloud computing resource model and cloud workflow model, combine reinforcement learning model and Q learning idea, select the most suitable server for each task through action selection algorithm, and optimize the overall security risk and execution cost of workflow .
While meeting the deadline constraints, the overall security risks and execution costs of the workflow are significantly reduced, providing a dynamically adjusted solution to meet the security preferences and cost optimization of different user needs.
Smart Images

Figure CN118945229A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of cloud computing, and in particular to a workflow scheduling method that considers security risks and cost optimization in a cloud environment. Background Art
[0002] Workflow is an abstraction and description of a set of task processes, which have certain dependencies between them. Directed Acyclic Graph (DAG) is the most commonly used model to build workflows. The vertices in the graph represent the task nodes to be executed, and the directed edges in the graph represent the data transmission between tasks. Workflows are widely used, such as for analyzing and monitoring drug treatment processes, for analyzing molecular properties, etc., which greatly increases the complexity of workflow scheduling, and usually requires more powerful storage and computing resources to complete scheduling within an acceptable time. Cloud computing integrates resources such as software, hardware, data, and information services to form a super-large system resource pool to perform some complex tasks. Users configure corresponding computing resources according to personalized needs, and can complete related tasks without additional human interaction. With the support of the powerful elasticity and scalability of cloud computing, users can quickly update and remove resources, reducing the user's usage cost. As a commonly used application model in cloud environments, cloud workflows usually define relevant business rules according to different business needs. Users can deploy cloud workflows on cloud platforms using public clouds, private clouds, hybrid clouds, etc., and leverage the high performance and flexibility of cloud computing to achieve automated execution and management of workflow applications.
[0003] The use of cloud resources requires corresponding fees. The lower the cost, the slower the task completion time. The main focus of cloud workflow scheduling research is also mostly on the optimization of cost and time expenditure. At the same time, there are many potential security risks in the cloud computing environment, which may lead to the tampering, deletion and leakage of workflow data, seriously undermining the security of user data, endangering personal privacy, business secrets, and even national security. Therefore, security has become one of the important factors that need to be considered in cloud workflow scheduling. To this end, this patent proposes a workflow scheduling method in a cloud environment that takes security risks and cost optimization into consideration. Summary of the invention
[0004] The purpose of the present invention is to reduce the overall security risk and execution cost of the workflow under the deadline constraints given by the user.
[0005] To achieve the above objectives, the present invention provides a workflow scheduling method that considers security risks and cost optimization in a cloud environment. First, based on the cloud computing resource model and cloud workflow model, a security risk calculation model is constructed; then a reinforcement learning model under cloud workflow is constructed; then an action selection algorithm is given; finally, based on the Q learning idea, iteration is performed to select the most suitable server for each task, obtain the final allocation plan, and calculate the overall security risk and execution cost of the workflow; the basic process of this method is as follows Figure 4 shown.
[0006] 1. The technical solution further defined in the present invention is:
[0007] Preferably, the above technical solution includes the following steps:
[0008] Step 1: Define the cloud computing resources used by the workflow. The communication link between servers is fully connected, the transmission bandwidth between servers is equal, and different servers provide different operating speeds, usage prices, and security levels.
[0009] Step 2: Abstract the workflow structure. Use a DAG graph to represent the workflow. The previous task is connected to the next task. The previous and next tasks have data dependencies. The edges have weights, that is, the data transmitted between tasks. On this basis, security requirement attributes are added to each task to indicate the degree of importance attached to different security.
[0010] Step 3: Define the security risk model. The security risk in the workflow scheduling process is a probabilistic problem. The distribution of risk probability generally follows the Poisson probability distribution within any given time interval. Therefore, this paper uses the Poisson probability distribution model to represent the security risk probability. This risk model comprehensively considers the task execution risk and data transmission risk.
[0011] Step 4: Build a reinforcement learning model for security risk and cost optimization. First, use the workflow scheduling to build the state space; then use the task sub-deadlines to build the action space; finally, define the reward function for security risk and cost.
[0012] Step 5: Determine the action selection algorithm. Select actions based on the ε-greedy algorithm, and control the probability of randomly selecting actions through the parameter ε, so that the agent can explore unknown actions with a certain probability. Use the task sub-deadline to constrain the action set to meet the deadline constraint.
[0013] Step 6: Perform task scheduling based on the Q-learning concept. The core process is to select an action to execute based on the current state, then calculate the immediate reward generated by the action and obtain the next action, then update the Q table and state, and repeat the process until the maximum number of iterations is reached to obtain lower security risks and execution costs.
[0014] 2. Preferably, in the above technical solution, in step 1, cloud computing resources are defined as follows.
[0015] The cloud computing resource model is represented by a tuple R = {V, SP}. i |1≤i≤m} represents a set of m servers, each server has different execution capabilities and the communication links between servers are fully connected. i represents the execution capability of the i-th server, price j represents the usage price per unit time of the i-th server, and bw represents the transmission bandwidth between servers. SP = {sp i |1≤i≤m} represents the security level set of the server, represents the security level set of the i-th server, Respectively represent the security levels for confidentiality, integrity and authenticity.
[0016] 3. Preferably, in the above technical solution, in step 2, the abstract workflow structure is as follows.
[0017] Based on the DAG graph, the workflow is represented by W = (T, E, SR, DT), where: T = {t i |1≤i≤n} represents the set of workflow tasks, t i represents the i-th workflow task, data i Indicates the number of instructions that the i-th task needs to execute; E = {e i,j |1≤i, j≤n} represents the edge set, e i,j represents the amount of data transfer between tasks, task t i Must be in task t j Completed before, the entry task is recorded as t entry , the export task is denoted as t exit , task t i The predecessor task and successor task of and t i succ SR = {sr i |1≤i≤n} represents the set of task safety requirements, represents the safety requirement of the ith task, They represent the security requirements of confidentiality, integrity and authenticity respectively; DT represents the deadline of the workflow, which is usually specified by the user.
[0018] Task i On the server v j The execution time ET(t i , v j ) can be calculated by formula (2.1).
[0019]
[0020] Task i The average execution time can be calculated by formula (2.2), where |V| represents the number of servers.
[0021]
[0022] Task i To task t j The data transmission time TT(t i , t j ) can be calculated by formula (2.3). When the two tasks are not executed on the same server, that is, s(t i )≠s(t j ), the data transmission time is determined by the amount of data transmitted, e i,j and transmission bandwidth bw; when two tasks are executed on the same server, s(t i )=s(t j ), data does not need to be transferred between servers, so the transfer time is 0.
[0023]
[0024] Task i On the server v j The earliest start time EST(t i , v j ) depends on the maximum idle time of the server and the maximum time for all predecessor tasks to complete data transmission, which can be calculated by formula (2.4). j The maximum idle time is expressed as empty(v j ) means, empty(v entry ) has a value of 0.
[0025]
[0026] The task completion time FT on the server is the sum of the earliest completion time and execution time of the task, which can be calculated by formula (2.5). The completion time of the workflow is the completion time FT of the export task (t exit , v j).
[0027] FT(t i , v j )=EST(t i , v j )+ET(t i , v j ) (2.5)
[0028] Task i On the server v j The cost cost(t i , v j ) is the product of the unit time price and the execution time, which can be calculated by formula (2.6).
[0029] cost(t i , v j )=ET(t i , v j )·price j (2.6)
[0030] The total cost of workflow scheduling, cost(W), is the sum of the execution costs of all tasks, which can be calculated by formula (2.7), s(t i ) represents task t i The server ID you selected.
[0031]
[0032] 4. Preferably, in the above technical solution, in step 3, a security risk model is defined as follows.
[0033] Step 31, define the task execution risk. i On the server v j When executed on the server, if the server v j The level of security provided Unable to meet the task i Security needs Right now When The increase of the difference shows an exponential growth trend; otherwise, no risk will occur and the probability of safety risk is 0. Task t i On the server v j When executing on The security risk generated x e (t i , v j ) can be calculated by formula (3.1).
[0034]
[0035] Among them, λ is the risk factor, which indicates the number of times the server is attacked by external attacks per unit time. The use of a negative exponential probability model is to ensure that the probability of security risk increases as the degree of satisfaction of task security requirements decreases.
[0036] Task i On the server v j Execution risk on e (t i , v j ) can be calculated by formula (3.2).
[0037]
[0038] Workflow execution risk p e (W) is the mean risk of all task executions and can be calculated by formula (3.3).
[0039]
[0040] Where |T| represents the number of tasks in the workflow; σ ij =1 indicates task t i Assign to server v j Execute on, σ ij =0 means task t i Not assigned to server v j Execute on.
[0041] Step 32: Data transmission risk definition. Task t i To task t j The security risk probability of data transmission is defined as p t (t i , t j ), which can be calculated by formula (3.4). If the two tasks before and after the data transmission are assigned to the same server for execution, the risk probability is 0.
[0042]
[0043] Among them, s(t i ) represents task t i The ID of the selected server.
[0044] Workflow data transfer riskp t (W) can be calculated by formula (3.5), and |E| represents the number of data transmission edges in the workflow.
[0045]
[0046] Step 33: Define the overall security risk of the workflow. The overall security risk p(W) of the workflow is expressed as the task execution risk p e (W) and data transmission risks t The average value of (W) can be calculated by formula (3.6).
[0047]
[0048] The goal of the present invention is to minimize the overall security risk p(W) and execution cost cost(W) of the workflow under the condition that the export task meets the deadline constraint, as shown in formula (3.7).
[0049]
[0050] Among them, rw represents risk preference, that is, the user's preference for optimizing security risks, rw∈[0~1]. The larger the rw, the more inclined to optimize security risks; conversely, the more inclined to optimize execution costs. Users can choose the appropriate rw value according to different needs.
[0051] 5. Preferably, in the above technical solution, in step 4, a reinforcement learning model for security risk and cost optimization problems is constructed, as follows.
[0052] Step 41, constructing the state space. For an input workflow instance, the priority and scheduling order list of tasks are fixed. At a certain scheduling time point, the current tasks to be scheduled and the tasks that have been scheduled are known, while the unscheduled tasks have great uncertainty. Therefore, the system state can be defined by the known task scheduling situation.
[0053] The state space is defined as S = {s i |i=1,2,...,n s}. s represents the number of states in the schedule, s i Represents the i-th state. Each state is related to the tasks currently waiting to be scheduled and the tasks that have been scheduled and completed, which can be represented as a two-tuple s i = <t cur ,s sched >, t cur Indicates the current task to be scheduled, s sched ={(t i , s(t i )|t i ∈T sched} represents the scheduling result of the scheduled task. The scheduling result of the task consists of the task and the server selected by the task. schedIndicates the set of tasks that have been scheduled. This set is dynamically updated during the scheduling process. When a task is completed, it will be stored in T sched In. sched It changes dynamically. Choosing different servers to schedule the same task will result in s sched changes.
[0054] Before starting the learning iteration, an initial state s1 must be specified for the system to ensure that subsequent iterations can continue. In the workflow scheduling process, scheduling always starts from the entry task, so the initial state is related to the entry task and is defined as When a task is scheduled, it will be added to T sched To update the next state s sched The next task in the scheduling order list will become the next state t cur For example, the initial state is The next task is t2. Assuming that task t1 selects server v3 for execution, the next state can be expressed as s2=<t2,{(t1,v3)}> .
[0055] Step 42: Construct the action space. The available action set contains all the optional actions of the agent in a certain state, using A i Indicates state s i The set of available actions under. In general, for workflow scheduling problems, A i is the set of all servers, which means that any server can be selected to execute the current task in each state, but this may cause the workflow execution time to be too long. In order to meet the deadline constraint, the task sub-deadline sdt is used to set the action set A. i Restrictions are imposed to ensure that the available actions do not cause the completion time to exceed the deadline as much as possible. The sub-deadline of the task can be calculated using formula (4.1).
[0056]
[0057] Where pr(t i ) represents task t i The priority can be calculated by formula (4.2).
[0058]
[0059] If the completion time of a task in a certain state on a server does not exceed the sub-deadline of the task, the server is added to the set of available actions for the state, as shown in formula (4.3).
[0060] A i = {v j∈V|FT(t cur , v j )≤sdt(t cur )} (4.3)
[0061] Step 43: Define the reward function. During the scheduling process, for the current state, after selecting an action to execute, the environment will feedback the reward value generated by executing the action. The better the action selected, the greater the reward value. In this chapter, there are two optimization objectives: security risk and execution cost. Therefore, the reward value includes the reward value of security risk and the reward value of execution cost.
[0062] Security risk includes execution risk and data transmission risk. When an action is selected for execution, the increased security risk var(risk) of the system can be calculated by formula (4.4): Represents task t i The number of predecessor tasks.
[0063]
[0064] For security risks, in state s i Next, select action a i The resulting reward value r1(s i , a i ) is the ratio of the minimum safety risk to the actual safety risk among all optional actions, which can be calculated by formula (4.5), i ∈A i Indicates state s i This means that after the action is executed, the smaller the security risk added to the system, the greater the reward value corresponding to the security risk.
[0065]
[0066] For execution cost, in state s i Next, select action a i The resulting reward value r2(s i , a i ) is the ratio of the minimum cost to the actual cost among all optional actions, which can be calculated by formula (4.6). This means that after the action is executed, the lower the cost of task execution, the greater the reward value corresponding to the cost.
[0067]
[0068] In status i Next, perform action a i The total reward(s) generated i , a i ) is determined by r1 and r2, as shown in formula (4.7).
[0069] reward(s i , a i )=rw·r1(s i , a j )+(1-rw)·r2(s i , a j ) (4.7)
[0070] Among them, rw represents risk preference. The larger the rw is, the more importance is attached to security risk rewards. Conversely, the smaller the rw is, the more importance is attached to execution cost rewards.
[0071] 6. Preferably, in the above technical solution, in step 5, an action selection algorithm is determined, as follows.
[0072] In reinforcement learning, there are generally two strategies for action selection: utilization and exploration. The utilization strategy means that the agent selects actions based on the current known information, that is, selects the action with the maximum Q value, but this will cause those actions that have not been selected to never be selected, which seriously affects the practicality of the algorithm. The exploration strategy will try to select new actions in the hope of obtaining greater long-term rewards. In the present invention, actions are selected based on the ε-greedy algorithm, and the probability of randomly selecting actions is controlled by the parameter ε, so that the agent can explore unknown actions with a certain probability.
[0073] First, calculate the available action set A according to formula (4.3): i , and then generate a random number rd (rd∈[0,1]), which determines the strategy for selecting actions. If rd<ε, from A i Randomly select an action to return; if rd ≥ ε, select A i The action with the largest Q value is returned. The system state in the workflow scheduling process has great uncertainty and cannot be initialized in the Q table in advance. It should be dynamically increased during the scheduling process. Therefore, if the state s is found i If the Q table is not in the table, first i Add the Q table and initialize the Q values corresponding to all actions to 0. At this time, the random strategy is also used to select the action from A. i Select an action to return.
[0074] 7. Preferably, in the above technical solution, in step 6, task scheduling is performed based on the Q learning idea, as follows.
[0075] Step 61, initialization. Initialize the deadline DT, set the risk factor λ, and the risk preference rw. Calculate the execution time of each task on each server according to formula (2.1), calculate the average execution time of each task according to formula (2.2), and calculate the sub-deadline of all tasks according to formula (4.1).
[0076] Step 62, calculate the priorities of all tasks according to formula (4.2) and sort them in descending order to obtain a task scheduling order list, recorded as scheduleList.
[0077] Step 63, initialize the Q table. QTable is represented by qTable, which is a set with key-value pairs as elements of the set, where the key represents the system state and the value represents the Q value list corresponding to all actions in the state. The state in the environment is related to the current task and the scheduled situation, and the scheduled situation is affected by the action selection, which makes the state in the environment unpredictable. Therefore, qTable is initialized to an empty set. During the learning process, new states are dynamically added to qTable, Q values are initialized, and Q values are updated.
[0078] Step 64, reset the scheduling environment. In order to ensure that the result of the current scheduling does not affect the next scheduling, it is necessary to reset the environment, mainly resetting the scheduling result list, completion time, execution cost, security risk, execution cost and other data.
[0079] Step 65, update the value of ε. The role of ε is to control the available action set A i The probability of randomly selecting an action in step 5 is greater. The larger the ε is, the greater the probability of randomly selecting an action in step 5. When the agent and the environment first interact, there is less past experience to draw on. At this time, ε should take a larger value. As the learning time increases, ε should gradually decrease to ensure that each state can select the action with the maximum Q value after learning is completed. Iter represents the current number of iterations, maxIterNum represents the maximum number of iterations, and formula (6.1) is used to update ε, and the value of ε gradually decreases from 1 to 0.
[0080]
[0081] Step 66, initialize the system state. Before starting learning, you need to specify an initialization state The current task in the initial state is the first task in the scheduling order list, and the scheduled state is an empty set.
[0082] Step 67, perform a round of scheduling. Traverse the schedule list scheduleList, schedule tasks in sequence, and after all tasks are scheduled, if the maximum number of iterations is reached, jump to step 68; otherwise, jump back to step 64. The process of a round of scheduling is as follows.
[0083] In the process of traversing tasks, for each task t i ∈scheduleList, first generate the current state s i , and select an action a according to step 5i ; Then execute the action, that is, task t i Dispatch to a i Execute on the corresponding server and after completion, i and the server of their choice to join sched , then generate the next state s i+1 Then, according to formula (4.7), the execution a in the current state is calculated. i The instant reward(s) generated i , a i ). If the current task is not an export task, then use formula (6.2) to update Q(s i , a i );
[0084]
[0085] Among them, α∈[0,1] is the learning rate, which indicates the degree to which the new value covers the old value; γ∈[0,1] is the discount factor, which indicates the degree to which the current value is affected by future rewards. If γ is 0, it means that only the current reward is considered. As γ increases, more distant future rewards will be considered. If the current task is an export task, Q(s) is updated using formula (6.3) i , a i ), since the next state cannot be obtained at this time, there is no corresponding future reward, and only the current reward needs to be considered during the update.
[0086] Q(s i , a i )=Q(s i , a i )+α·(reward(s i , a i )-Q(s i , a i )) (6.3)
[0087] Finally, use the next state s i+1 Update current status i , continue to schedule the next task until all tasks are scheduled.
[0088] Step 68, calculate the overall security risk according to formula (3.6) and calculate the total cost according to formula (2.7).
[0089] Step 69, the algorithm ends. BRIEF DESCRIPTION OF THE DRAWINGS
[0090] Figure 1 is a workflow instance;
[0091] Figure 2The number of instructions and security requirements for workflow tasks;
[0092] Figure 3 Related parameters for the server;
[0093] Figure 4 It is a summary flow chart; DETAILED DESCRIPTION
[0094] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.
[0095] The workflow W in the example contains 8 tasks, of which t1 is the entry task and t8 is the exit task. i,j The value of Figure 1 As shown in the figure, the number of instructions and security requirements of each task are as follows Figure 2 The server set contains 3 servers, and the processing capacity, transmission bandwidth, unit price and security level of the servers are as follows Figure 3 shown.
[0096] First, initialize the deadline DT = 30, λ = 1.0, rw = 0.5. Calculate the execution time, average execution time and sub-deadline time of each task on each server, as shown in Table 1. Initialize qTable to be empty. The maximum number of iterations is set to 300.
[0097] Table 1. Task execution time, average execution time and sub-deadline time on the server
[0098]
[0099] Then calculate the priorities of the 8 tasks and sort them in descending order, and the scheduling order is t1, t4, t5, t3, t2, t7, t6, t8.
[0100] Then the iterative learning process begins. In the first round of learning, the value of ε is calculated to be 1. According to the scheduling order, the first task is t1, so the initial state is (t1, []), and the selected action is v3, that is, task t1 is scheduled to server v3 for execution. The next task is t4, so the next state can be obtained as (t4, [(t1, v3)]). Then update qTable, as shown in Table 2. Since the state is dynamically added during the learning process, there are only two states in the current qTable.
[0101] Table 2 The first update of qTable during the first round of learning
[0102]
[0103] When the qTable is updated, the current state is updated to (t4, [(t1, v3)]), and the Q table is updated according to the same operation process. After the export task scheduling is completed, the first round of learning process ends, and the qTable at this time is shown in Table 3.
[0104] Table 3 Updated qTable at the end of the first round of learning process
[0105]
[0106] After repeating the above learning process 300 times, the algorithm ends. At this time, the completion time of the workflow is 26.35, which meets the deadline constraint. The calculated overall security risk is 0.21 and the total cost is 49.725.
[0107] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.
Claims
1. A workflow scheduling method considering security risks and cost optimization in a cloud environment, characterized in that Scheduling is performed under a deadline constraint given by a user to reduce the overall security risk and execution cost of the workflow. The method includes the following steps. Step 1: Define the cloud computing resources used by the workflow; Step 2: Abstract the workflow structure; Step 3: Define the security risk model; Step 4: Build a reinforcement learning model for security risk and cost optimization; Step 5: Determine the action selection algorithm; Step 6: Perform task scheduling based on Q-learning idea.
2. According to claim 1, a workflow scheduling method considering security risks and cost optimization in a cloud environment is characterized in that: In step 1, cloud computing resources are defined; the communication links between servers are fully connected, the transmission bandwidth between servers is equal, and different servers provide different operating speeds, usage prices and security levels.
3. According to claim 1, a workflow scheduling method considering security risks and cost optimization in a cloud environment is characterized in that: In step 2, the workflow structure is abstracted; the workflow is represented by a DAG graph, where the previous task is connected to the next task, and the previous and next tasks have data dependencies, with weights on the edges, i.e., the data transmitted between tasks; on this basis, security requirement attributes are added to each task to indicate the degree of importance attached to different security.
4. According to claim 1, a workflow scheduling method considering security risks and cost optimization in a cloud environment is characterized in that: In step 3, the security risk model is defined; the security risk in the workflow scheduling process is a probabilistic problem, and the distribution of risk probability generally follows the Poisson probability distribution within any given time interval, so the Poisson probability distribution model is used to represent the security risk probability. This risk model comprehensively considers the task execution risk and data transmission risk.
5. According to claim 1, a workflow scheduling method considering security risks and cost optimization in a cloud environment is characterized in that: In step 4, a reinforcement learning model for security risk and cost optimization is constructed. First, the state space is constructed using the workflow scheduling situation. Then, the action space is constructed using the task sub-deadlines. Finally, the reward function for security risk and cost is defined.
6. The workflow scheduling method considering security risks and cost optimization in a cloud environment according to claim 1, characterized in that: In step 5, determine the action selection algorithm; select actions based on the ε-greedy algorithm, and use the parameter ε to control the probability of randomly selecting actions, so that the agent can explore unknown actions with a certain probability. Use the task sub-deadline to constrain the action set to meet the deadline constraint.
7. The workflow scheduling method considering security risks and cost optimization in a cloud environment according to claim 1, characterized in that: In step 6, task scheduling is performed based on the Q-learning concept; the core process is to select an action to execute based on the current state, then calculate the immediate reward generated by the action and obtain the next action, then update the Q table and state, and repeat the process until the maximum number of iterations is reached to obtain lower security risks and execution costs.
Citation Information
Patent Citations
Electrical equipment maintenance decision generation method based on reinforcement learning
CN113627533A
Workflow scheduling method considering security risk in cloud environment
CN114625496A
Workflow scheduling method based on reinforcement learning in cloud edge environment
CN114942799A
Faster fitted q-iteration using zero-suppressed decision diagram
US20230076608A1
