Hybrid cloud workflow task scheduling method based on topology analysis and privacy quantization

Abstract

This invention provides a hybrid cloud workflow task scheduling method based on topology analysis and privacy quantification. It calculates task allocation time, data transmission time, backup time, encryption time, and decryption time for a second set of tasks to be allocated to the public cloud, calculates the total cost of each particle, and outputs the optimal scheduling scheme with the goal of minimizing the total cost. The second set of tasks to be allocated to the public cloud includes tasks with encryption level allocation and proactive fault-tolerant backup. Highly sensitive tasks are allocated to the private cloud through privacy cost estimation. Based on breadth-first search to determine node levels and betweenness centrality to identify critical task nodes, proactive backup is implemented to cut off cascading fault propagation links. This invention obtains a globally optimal task scheduling scheme while satisfying privacy and reliability constraints, balancing resource consumption and data security, improving the reliability of task scheduling in a hybrid cloud environment, and reducing overall scheduling costs.

Description

Technical Field

[0001] This application relates to the field of cloud computing resource scheduling technology, and in particular to a workflow task scheduling method for multi-objective optimization of cost, privacy and reliability for scientific workflows in a hybrid cloud system. Background Technology

[0002] With the advent of the big data era, modern scientific research (such as bioinformatics and astronomy) increasingly relies on large-scale scientific workflows. Workflows typically consist of thousands of tasks with complex dependencies, formally represented as a directed acyclic graph (DAG). Hybrid cloud environments, combining the data security of private clouds with the elastic scalability of public clouds, have become the preferred platform for executing such data-intensive workflows.

[0003] However, workflow scheduling in a hybrid cloud environment faces significant challenges. Existing scheduling strategies often employ coarse-grained partitioning, strictly restricting sensitive tasks to resource-constrained private clouds, making these private clouds bottlenecks. Alternatively, complex encryption mechanisms are introduced in public clouds, but the computational overhead and cross-cloud bandwidth costs associated with encryption result in high execution costs. Furthermore, existing fault-tolerance mechanisms, such as active replication, typically treat failures as independent events, ignoring the impact of DAG topology on fault propagation. In workflows with strict dependencies, failures at critical nodes often propagate downstream along dependency edges, triggering cascading failures that paralyze the entire workflow and reduce system reliability. Moreover, since workflow scheduling is a nondeterministic multinomial (NP-hard) problem, and with the introduction of task privacy constraints and system reliability constraints requiring sensitive tasks to match nodes with specific security levels, its feasible solution space becomes severely fragmented. Traditional particle swarm optimization algorithms are prone to getting trapped in local optima in such constrained spaces, leading to persistently high overall execution costs for the resulting scheduling schemes. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention provides a hybrid cloud workflow task scheduling method based on topology analysis and privacy quantization. This method can effectively escape local optima and minimize the global cost of workflow scheduling while strictly meeting privacy and reliability constraints.

[0005] The hybrid cloud workflow task scheduling method based on topology analysis and privacy quantization of the present invention calculates the task allocation time, data transmission time, task backup time, task encryption time and task decryption time for the second set of tasks to be allocated to the public cloud, and uses the particle swarm optimization algorithm to calculate the current total cost of particles. The current total cost of particles is compared with the historical total cost of particles, and the scheduling scheme with the minimum total cost of particles is output.

[0006] The second set of tasks to be assigned to the public cloud is the first set of tasks to be assigned to the public cloud after the encryption level allocation and proactive fault-tolerant backup.

[0007] Furthermore, the first set of tasks to be allocated to the public cloud is the initial set of tasks sorted according to their privacy sensitivity, with tasks of high privacy sensitivity pre-allocated to private cloud virtual machines, until the private cloud resources reach saturation and the remaining task set is completed.

[0008] Furthermore, the initial workflow task set is sorted according to its privacy sensitivity, and tasks with high privacy sensitivity are pre-assigned to private cloud virtual machines, including:

[0009] S21: For each task in the initial workflow task set Calculate privacy sensitivity According to the formula:

[0010] ,

[0011] in, The pre-defined task inherent privacy factor, i.e., the privacy protection level; denominator Representative task The amount of computation required to perform the task; Indicates task The set of all direct successor tasks in the workflow Indicates task Transmitted to a specific subsequent task The amount of data.

[0012] S22: Use the randomized quicksort algorithm, according to... The values ​​are sorted in descending order to obtain an ordered task set. ;

[0013] S23: Traversal The tasks with the highest priority are pre-assigned to private cloud virtual machines in sequence until the private cloud resources are saturated.

[0014] Further, step S22 includes:

[0015] S221: Obtain a collection of several Array of unsorted tasks And the starting index of the sorting operation that needs to be performed. and end index bit ;

[0016] Compare the size relationship between the start index bit and the end index bit:

[0017] like This indicates that there are at least two tasks within the current interval that need to be sorted, so step S222 is executed.

[0018] like If the number of tasks in the current interval is 1 or 0, the interval is considered to be in a naturally ordered state, and the sorting operation of the current branch is terminated.

[0019] S222: Using a random function, at the starting index position With end index bit Generate a random integer between the given values ​​and denote it as the random index. ;

[0020] S223: Sort the array to be sorted In the random index position The task at the office With the end index bit The task at the office To swap positions;

[0021] S224: Set the first pointer Second pointer , put the first pointer Initialize to This pointer is used to dynamically record the right boundary of sub-intervals that are less than or equal to the reference element; the second pointer... Initialize to This pointer is used to traverse the current sorted interval from left to right;

[0022] S225: Control the second pointer From the starting index bit Gradually increase the traversal to The task of extracting the current pointer position during each traversal. Compare with the benchmark task:

[0023] like If the value is less than or equal to the baseline task, then the first pointer will be... Increment the value by 1 and set the task With the task To swap positions;

[0024] like If the value is greater than the reference element, no data exchange operation is performed, and the second pointer... Continue iterating backwards.

[0025] S226: At the second pointer After completing the traversal, move the task corresponding to the next position of the first pointer. With the end index bit The baseline task at the location Perform position swapping and record the new position index of the baseline task. The array With index The physical boundary is divided into two independent sub-intervals:

[0026] Left sub-interval All tasks within this interval are less than or equal to the baseline task;

[0027] right subinterval All tasks within this range are greater than the baseline task;

[0028] S227: Boundary index calculated based on step S226 :

[0029] Using the left sub-interval as the new processing target, set the starting index to The ending index is set to Jump back and execute step S221;

[0030] Using the right sub-interval as the new processing target, set the starting index to The ending index is set to Jump back and execute step S221;

[0031] When all branches of the recursive call satisfy the termination condition of step S221, it means that the entire array... After all sorting operations are completed, the final sorted task array is output. .

[0032] Furthermore, the encryption level allocation includes setting an encryption level for the first set of tasks to be allocated to the public cloud, including:

[0033] Through different encryption algorithms With encryption level The correspondence between them is used to select different levels of encryption algorithms for different tasks;

[0034] Data vulnerability in computational tasks:

[0035] ,

[0036] in, This indicates data vulnerability, which physically refers to the vulnerability of data transfer between tasks. Maximum number of plaintexts required to perform a brute-force attack on an encrypted network With the selected encryption level The number of plaintexts required to process the corresponding encryption algorithm ;

[0037] like If the condition is not met, the allocation will be redistributed until it is met;

[0038] in, This indicates the exposure risk threshold.

[0039] Preferably, the redistribution is carried out using a random allocation method;

[0040] Furthermore, the proactive fault-tolerant backup includes performing structure-aware proactive fault-tolerant backup on the first set of tasks to be allocated to the public cloud, including:

[0041] S41: Use a breadth-first search algorithm to traverse the first set of tasks to be assigned to the public cloud in a hierarchical manner. Using the DAG graph, establish the hierarchical structure of all tasks, and traverse the levels where the tasks are located to obtain the hierarchical structure of all tasks.

[0042] S42: Traverse each level from high to low according to the level output in step S41, and calculate the betweenness centrality of each node in each level.

[0043] Furthermore, nodes The betweenness centrality formula is:

[0044] ,

[0045] in, Represents a node betweenness centrality, Represents node pairs The total number of shortest paths between them. Represents node pairs Passing through the node The number of shortest paths;

[0046] S43: Add a backup copy for the task with the highest centrality value of the number of intermediaries at the same level;

[0047] Furthermore, including if reliability gain rate Greater than the preset expected reliability gain If so, continue with the task backup. Less than the preset expected reliability gain If the backup benefit is already small, then terminate active backup.

[0048] ,

[0049] ,

[0050] ,

[0051] ,

[0052] In the formula, These represent the new overall reliability achieved by the task after adding the backup. This indicates the original basic reliability of the task before the backup was added. Indicates task The base reliability probability of successful execution on a single virtual machine. This represents the probability of a single instance failing. This represents the constant failure rate of different virtual machines per unit of time. This refers to the specific virtual machine instance that hosts the task and its replicas. Indicates the task Assigned to Execution time, Indicates task The computational load, For CPU single-core computing power, For virtual machines The number of CPU cores;

[0053] Furthermore, the total cost is calculated as follows:

[0054] ,

[0055] ,

[0056] ,

[0057] ,

[0058] ,

[0059] ,

[0060] in, Indicates the total execution time. express The unit encryption time of the encryption algorithm. The amount of data transmitted. Use security level The task encryption time required to perform encryption. Represented as from task Transfer data to task The amount of data, Representative task The allocated virtual machine, and This represents the environment in which the virtual machine resides, namely a public cloud environment and a private cloud environment. This indicates the unit price of execution cost. This indicates the unit price of bandwidth cost. Indicates the task Assigned to Execution time, Indicates task The computational load, For CPU single-core computing power, For virtual machines The number of CPU cores; Indicates from the task Transfer data to task Data transmission time, This represents the cost charged by the public cloud provider for inter-cloud transmission per unit of data. This indicates the cost of transmission within the cloud.

[0061] Compared with the prior art, the beneficial effects of the present invention include:

[0062] This invention proposes a "privacy sensitivity" index through the Privacy Cost Estimation and Calculation of Privacy (PCEA). This index transforms abstract privacy requirements into computable data, which is then used to sort and pre-allocate tasks. This approach prioritizes the operation of highly sensitive tasks in the private cloud while avoiding resource bottlenecks in the private cloud caused by coarse-grained partitioning, such as restricting all sensitive tasks to the private cloud. This achieves a balance between privacy protection and resource load.

[0063] The structure-aware dynamic backup strategy of this invention differs from traditional independent fault handling. This strategy utilizes betweenness centrality to identify key "bridging" task nodes in the DAG graph and implements preventative backups. This not only effectively cuts off the propagation links of cascading faults, significantly improving system reliability, but also avoids resource waste caused by excessive backups through a reliability gain threshold judgment mechanism.

[0064] The hierarchical encryption of this invention takes into account both computational time overhead and bandwidth costs for cross-cloud and intra-cloud transmission, filling the gap in existing scheduling strategies that introduce additional overhead for enhanced security.

[0065] This invention employs an improved particle swarm optimization for global optimization. By introducing privacy vulnerability constraint checks during initialization and iteration, it can effectively handle the discontinuous solution space segmented by constraints, avoid the algorithm getting trapped in local optima, and thus quickly obtain a task mapping scheme that balances low cost, high reliability, and strong privacy protection. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of an embodiment of the hybrid cloud workflow task scheduling method based on topology analysis and privacy quantification in this application. Detailed Implementation

[0067] To make the objectives, technical solutions, features, and advantages of this application clearer and to enable those skilled in the art to better understand the technical solutions of this application, the following detailed description of this application is provided in conjunction with the accompanying drawings and embodiments.

[0068] The numbering of each step in this patent is merely a marker for ease of description and reference, and should not be construed as performing the patent's solution in sequence or otherwise restrictive.

[0069] This invention is based on a hybrid cloud system, establishes a workflow model based on a directed acyclic graph (DAG), defines nodes as tasks and edges as data dependencies, constructs a hybrid resource pool including public and private clouds, sets the computing performance of virtual machines, on-demand billing models, and cross-cloud / intra-cloud data transmission bandwidth cost models, and introduces a Poisson distribution model for transient failures to calculate the reliability of tasks and the system.

[0070] This embodiment includes a workflow scheduling method that balances privacy and reliability in a hybrid cloud environment, comprising:

[0071] S1. Obtain task data from the workflow and preprocess it to obtain the initial task set of the workflow.

[0072] In the hybrid cloud scheduling platform, initialize the hybrid cloud resource pool and define the private cloud. He Gongyouyun Configure virtual machine properties, including the number of CPU cores. Single-core processing power Unit time cost Configure network parameters, including cloud bandwidth. and cross-cloud bandwidth Load privacy and security parameters, and set the encryption algorithm set. and the security level corresponding to the encryption algorithms in this set. and unit data encryption time .

[0073] By parsing user-submitted workflow XML files, such as scientific workflows like Montage and Cybershake, task nodes are extracted to obtain the initial task set for the workflow.

[0074] Get task node set and the set of dependent edges Get the amount of data required to execute each task. Data transmission volume on the edge ,in, Let i represent each task in the workflow, i = 1, 2, ..., n, where n is the total number of tasks.

[0075] Preferably, the initial task set of the workflow can be further standardized to eliminate the influence of units.

[0076] S2. Sort the initial workflow task set according to the level of privacy sensitivity, and pre-allocate the tasks with high privacy sensitivity to the private cloud virtual machine until the private cloud resources are saturated. The remaining task set is the first task set to be allocated to the public cloud.

[0077] To address the trade-off between privacy protection and resource consumption, this embodiment employs privacy cost estimation for the tasks in the first task set to be allocated to the public cloud.

[0078] S21: For each task in the initial workflow task set Calculate privacy sensitivity According to the formula:

[0079] ,

[0080] in, The preset task inherent privacy factor, i.e., the privacy protection level; Representative task The amount of computation required to perform the task; Indicates task The set of all direct successor tasks in the workflow Indicates task Transmitted to a specific subsequent task The data volume reflects the proportion of data involved in the subsequent transmission of the task relative to its own computational volume; the privacy sensitivity quantifies the proportion of encryption cost in privacy tasks. The higher the proportion, the greater the encryption transmission overhead caused by executing the task in the public cloud, and the greater the cost reduction benefit when executing it in the private cloud.

[0081] S22: Use the randomized quicksort algorithm, according to... The values ​​are sorted in descending order to obtain an ordered task set. .

[0082] S221: Obtain a collection of several Array of unsorted tasks And the starting index of the sorting operation that needs to be performed. and end index bit .

[0083] Compare the size relationship between the start index bit and the end index bit:

[0084] like This indicates that there are at least two tasks within the current interval that need to be sorted, so step S222 is executed.

[0085] like If the number of tasks in the current interval is 1 or 0, the interval is considered to be in a naturally ordered state, and the sorting operation of the current branch is terminated.

[0086] S222: Using a random number generation module or random function, at the starting index position With end index bit Generate a random integer between the given values ​​and denote it as the random index. The purpose of introducing a random index here is to eliminate the influence of the initial distribution pattern of the input data on sorting performance and to prevent the algorithm from degrading under extreme data conditions.

[0087] S223: Sort the array to be sorted In the random index position The task at the office With the end index bit The task at the office Perform a position swap. After the swap, extract the current position at the end index. The task at this location serves as the baseline task for dividing the current interval.

[0088] S224: Set the first pointer Second pointer Wherein: the first pointer Initialize to This pointer is used to dynamically record the right boundary of sub-intervals that are less than or equal to the reference element; the second pointer... Initialize to This pointer is used to traverse the current sorted interval from left to right.

[0089] S225: Control the second pointer From the starting index bit Gradually increase the traversal to The task during each traversal is to extract the current pointer position. Compare with the benchmark task:

[0090] like If the value is less than or equal to the baseline task, then the first pointer will be... Increment the value by 1 and set the task With the task To swap positions;

[0091] like If the value is greater than the reference element, no data exchange operation is performed, and the second pointer... Continue iterating backwards.

[0092] S226: At the second pointer After completing the traversal, move the task corresponding to the next position of the first pointer. With the end index bit The baseline task at the location Perform a position swap. After the swap, record the new location index of the baseline task. At this time, the array With index The physical boundary is divided into two independent sub-intervals:

[0093] Left sub-interval All tasks within this interval are less than or equal to the baseline task;

[0094] right subinterval All tasks within this interval are greater than the baseline task.

[0095] S227: Boundary index calculated based on step S226 :

[0096] Using the left sub-interval as the new processing target, set the starting index to The ending index is set to Jump back and execute step S221;

[0097] Similarly, taking the right sub-interval as the new processing target, the starting index is set to... The ending index is set to Jump back and execute step S221;

[0098] When all branches of the recursive call satisfy the termination condition of step S221, it means that the entire array... After all sorting operations are completed, the final sorted task array is output. .

[0099] S23: Traversal The tasks with the highest priority are pre-assigned to private cloud virtual machines in sequence until the private cloud resources are saturated. The remaining tasks are marked as the first set of tasks to be assigned to the public cloud for further processing.

[0100] This step sorts the tasks and, within the limitations of private cloud capacity, prioritizes mapping highly privacy-sensitive tasks to the private cloud, while allocating the remaining tasks to the public cloud, thereby reducing the risk of privacy leaks during the preprocessing stage.

[0101] Furthermore, a privacy cost estimation is proposed based on task ranking, transforming abstract privacy requirements into calculable metrics.

[0102] This invention proposes the concept of "privacy sensitivity," which comprehensively considers the amount of data transmitted, the amount of computation, and the privacy factor of a task, and quantifies the degree to which a task consumes privacy protection resources.

[0103] S3. Set the encryption level for the first set of tasks to be assigned to the public cloud. Calculate the encryption time.

[0104] Furthermore, through different encryption algorithms With encryption level The correspondence between them is used to select different levels of encryption algorithms for different tasks.

[0105] Different encryption algorithms are selected, each with its own performance speed and security level, as shown in Table 1.

[0106] The hierarchical encryption design reflects a typical trade-off between security and performance; stronger privacy protection leads to higher computational overhead, resulting in slower transmission speeds.

[0107] Table 1 Performance, speed, and encryption level of different encryption algorithms

[0108]

[0109] Table 1 shows that IDEA, previously used in secure communication tools like PGP, employs a 128-bit key. Its complex design and slow encryption / decryption speeds, however, offer strong resistance to cracking and are suitable for scenarios with extremely high privacy requirements. DES, a classic symmetric encryption algorithm, uses only a 56-bit valid key, has moderate speed, and is vulnerable to brute-force attacks. Rijndael, a symmetric encryption standard adopted by the US federal government, is fast, simple in structure, and highly efficient in both hardware and software implementation, balancing security and efficiency. Blowfish is extremely fast, simple to implement, and has no patent restrictions, making it suitable for scenarios with high performance requirements but moderate security needs. RC4, a stream cipher algorithm, is extremely simple in design and extremely fast. However, it has serious vulnerabilities and is only suitable for scenarios with no security requirements and only simple obfuscation.

[0110] Privacy constraints refer to the level of protection required for the data itself; a higher level indicates a greater necessity for privacy protection. Data vulnerability refers to the degree to which privacy risks may arise; lower data vulnerability means a smaller risk of exposure, making the task itself relatively safe, and thus eliminating the need for a virtual machine with high privacy protection. For example, a task with low privacy constraints may have high data vulnerability; because the task itself has low privacy constraints, a highly privacy-conscious virtual machine is unnecessary. Conversely, a task with high privacy constraints and high data vulnerability requires a highly privacy-conscious virtual machine.

[0111] The calculation formula for data vulnerability in computational tasks is as follows:

[0112] ,

[0113] in, This indicates data vulnerability, which physically refers to the vulnerability of data transfer between tasks. Maximum number of plaintexts required to perform a brute-force attack on an encrypted network With the selected encryption level The number of plaintexts required to process the corresponding encryption algorithm .

[0114] The purpose of this definition is to build a security barrier for tasks that must be executed in the public cloud, given the limited capacity of the private cloud. By applying tiered encryption to tasks assigned to the public cloud, the system can avoid the high time and computational overhead of indiscriminately using the highest level of encryption.

[0115] Utilizing data vulnerability to precisely quantify data at specific transmission volumes and encryption level Exposure risk below, and based on the exposure risk threshold As a hard constraint, the first constraint is shown in the formula:

[0116] ,

[0117] If the task allocation scheme leads to If the conditions are not met, tasks will be reassigned until the constraints are satisfied. Tasks that satisfy the first constraint formula will be assigned to the public cloud, thus ensuring that the overall privacy risk of the workflow remains strictly within the safety red line.

[0118] Preferably, the present invention adopts a random allocation method for the redistribution. Since the tasks with high privacy constraints have been allocated to the private cloud in the early stage, the remaining tasks to be allocated have small constraints, and the probability of obtaining a feasible solution by random allocation is high.

[0119] For tasks assigned to the public cloud, hierarchical encryption is implemented, and the encryption and decryption time costs are calculated to assess cost and achieve a balance between flexible scheduling and execution efficiency of hybrid cloud resources.

[0120] Continue processing the first set of tasks to be assigned to the public cloud.

[0121] S4. Perform structure-aware proactive fault-tolerant backup on the first task set to be assigned to the public cloud, and calculate the backup time.

[0122] To prevent cascading failures of the entire workflow due to task failures, the structure-aware proactive fault-tolerant backup specifically includes:

[0123] S41: Use a breadth-first search algorithm to traverse the first set of tasks to be assigned to the public cloud in a hierarchical manner. Using a DAG graph, establish the hierarchical structure of all tasks, and traverse the levels where the tasks are located to obtain the hierarchical structure of all tasks.

[0124] Since the closer a level is to the root node, the more downstream task nodes it has, proactive backups prioritize task nodes at lower levels. Therefore, this traversal process is used first to obtain the hierarchical structure of all tasks.

[0125] S411: Initialize an empty queue Used to temporarily store nodes to be processed, and also creates an empty set. To remove duplicates, the system records visited nodes, avoids repeated traversals, and defines an empty sequence. The final traversal result used for storage.

[0126] S412: Determine the starting node for traversal Enqueue s, denoted as and will Adding to the visited set is represented as: .

[0127] S413: Inspection If the value is empty, execute S414; otherwise, the traversal is complete, and jump to S417.

[0128] S414: Will The head node of the queue is popped and marked as the current working node. ,Will Append to the result sequence The end of.

[0129] S415: Edge Set Based Search for For all directed edges starting from the origin, obtain The set of out-degree adjacent nodes.

[0130] S416: Traverse each node in S415 :like If so, mark it as visited. And push them into the queue, represented as Otherwise, skip that node. After processing, return to execute S413.

[0131] S417: Output the final traversal sequence with the level-order traversal visit order. .

[0132] Based on the traversal sequence Update the corresponding successor node of each node in the sequence. The hierarchy is determined by this, thus deriving the hierarchy of all nodes. The formula is:

[0133] ,

[0134] in, Indicates the successor node Topological hierarchy; Represents a node The set of all direct predecessor nodes in a directed acyclic graph; Represents a node Predecessor node hierarchy The maximum value. This formula sets the current node's level to the maximum level value of all its parent nodes plus the maximum level value. This ensures that the hierarchy of subsequent tasks strictly lags behind all their dependent tasks, thereby dividing the entire workflow into clear execution batches from top to bottom using breadth-first search. This lays a structured foundation for subsequent hierarchical calculation of task betweenness centrality and proactive fault-tolerant replicas.

[0135] S42: Traverse each level from high to low according to the level output in step S41, and calculate the betweenness centrality of each node in each level.

[0136] node The betweenness centrality formula is:

[0137] ,

[0138] in, Represents a node betweenness centrality, Represents node pairs The total number of shortest paths between them. Represents node pairs Passing through the node The number of shortest paths. The shortest paths are calculated using a full-source shortest path algorithm, such as Floyd-Warshall or multiple Dijkstra's algorithms. These algorithms are not innovative in this invention and can be implemented using existing technologies, so they will not be elaborated further.

[0139] S43: Add a backup copy to the task with the highest centrality value of the number of intermediaries at the same level.

[0140] Nodes with high betweenness centrality are often bridges connecting different subtask groups. If they fail, they will block the main workflow path. Therefore, nodes with high betweenness centrality need to have higher reliability.

[0141] Faults are mainly divided into transient faults and permanent faults. Permanent faults are generally caused by hardware problems and are unpredictable, while transient faults are predictable. This patent focuses on transient faults in virtual machines.

[0142] In a DAG-based workflow, transient faults occur following a Poisson distribution. Over a period of time... The reliability of internal events is expressed as:

[0143] ,

[0144] in, This represents the constant failure rate of a virtual machine per unit of time.

[0145] The task Assigned to The task allocation time is given by the following formula:

[0146] ,

[0147] in, Indicates the task Assigned to Execution time, Indicates task The computational load, For CPU single-core computing power, For virtual machines The number of CPU cores.

[0148] In virtual machine Tasks executed on The reliability is given by the following formula:

[0149] ,

[0150] in, This represents the constant failure rate of different virtual machines per unit of time. This indicates the specific virtual machine instance that hosts the task and its copies for execution.

[0151] This invention employs a proactive backup approach to achieve fault tolerance. Proactive backup strategies are particularly suitable for hybrid clouds because they can reduce the long recovery delays associated with cross-cloud recovery processes, thereby improving execution stability.

[0152] Because backed-up tasks can be added to the candidate set during the active backup process described above, there may be more than one backup. Given that each task has a specified number of backup copies, and each copy is also called an instance, Defined as The number of replicas. If at least one instance completes execution without failure, then the task... If it is considered to be successfully executable, then its reliability is:

[0153] ,

[0154] in, Indicates task The base reliability probability of successful execution on a single virtual machine. This represents the probability of a single instance failing.

[0155] The formula above allows us to determine the overall fault tolerance reliability of a task under an active fault-tolerance mechanism, where multiple replicas execute in parallel, and only one replica succeeds. Furthermore, calculations are performed using the aforementioned formula before implementing the proactive backup process. .

[0156] S44: Reliability gain rate obtained after adding a backup copy to the computation task The formula is:

[0157] ,

[0158] In the formula, These represent the new overall reliability achieved by the task after adding the backup. This indicates the original basic reliability of the task before the backup was added.

[0159] If reliability gain rate Greater than the preset expected reliability gain Then continue according to the hierarchical sequence. Perform a task backup, and then continue backing up the next task at the same level. Less than the preset expected reliability gain If the backup benefit is already small, the active backup process should be terminated.

[0160] This invention establishes a rigorous input-output quantification mechanism by calculating the relative improvement brought by adding backups. A secondary cost-effectiveness filter is performed using high-between-centrality nodes selected in the preliminary steps, ensuring that resources are used only for the most critical tasks. Even for core bridging tasks in the topology, if their original reliability is already extremely high, the security improvement from adding extra replicas is negligible, and the system will decisively abandon backup operations to save costs. Only when a new replica can fill a significant reliability vulnerability will expensive virtual machine computing and data transmission resources be truly invested, preventing the waste caused by blindly stacking replicas and effectively achieving a balance between high reliability and low cost.

[0161] This step addresses the cascading failure problem by proposing a backup strategy that combines betweenness centrality and node hierarchy. Leveraging the topological characteristics of the workflow's DAG structure embodied in node hierarchy and betweenness centrality, a breadth-first search is used to determine the topological hierarchy of tasks. The betweenness centrality of each node in the workflow is calculated to identify critical bridging tasks connecting different task clusters. Replicas are proactively configured for critical tasks with high betweenness centrality, prioritizing them according to their hierarchical order. However, a reliability gain threshold mechanism is further employed, performing backup operations only when the reliability improvement from adding backups is significant, thus avoiding excessive resource consumption.

[0162] In this invention, "particle" represents a specific task-to-virtual machine scheduling mapping scheme. The position of the particle... , representing the solution vector of the current scheduling scheme, where each dimension corresponds to a task, and the specific value of the dimension represents the virtual machine index to which the task is assigned. The particle's velocity. , represents the step size and direction of the particle's position update in the solution space, and its physical meaning is the tendency and probability of the particle to change the current task-virtual machine mapping relationship in the next iteration.

[0163] S5. For the second set of tasks to be assigned to the public cloud, calculate the task allocation time, data transmission time, backup time, encryption time, and decryption time. Use the particle swarm optimization algorithm to calculate the current total cost of each particle. Compare the current total cost with the historical total costs of particles, and output the scheduling scheme with the minimum total cost. The specific process is as follows: Figure 1 As shown.

[0164] Enter the second set of tasks to be assigned to the public cloud. This second set of tasks to be assigned to the public cloud is the first set of tasks to be assigned to the public cloud after the encryption level allocation and active backup.

[0165] S51: Set population size and number of iterations ,initial It is 0.

[0166] Each particle Represented as a length of A vector, where n is the total number of tasks in the workflow. The vector of order n... The value of the bit represents the task. Allocated virtual machine index Each particle's position vector corresponds to a mapping task-virtual machine scheduling scheme.

[0167] S52: Initialize the particles and randomly generate their positions. and speed .

[0168] S53: For each particle, calculate the total cost of the current particle using the task allocation time, data transmission time, task backup time, task encryption time, and task decryption time.

[0169] Total cost The fitness function is used in particle swarm optimization (PSO) algorithms. The value of the fitness function in PSO typically corresponds to the output of the objective function.

[0170] This patent guides particles to continuously converge towards high-scoring regions by comparing their fitness values, ultimately finding the global optimal solution.

[0171] S531: Calculate task allocation time;

[0172] Virtual machines in a cloud environment are represented as Each virtual machine is associated with a specific type that defines its functionality and cost. In the widely adopted "pay-as-you-go" pricing model, users pay based on the duration of virtual machine usage. Therefore, tasks... Assigned to The task allocation time is given by the following formula:

[0173]

[0174] in, Indicates the task Assigned to Execution time, Indicates task The computational load, For CPU single-core computing power, For virtual machines The number of CPU cores.

[0175] S532: Calculate data transmission time;

[0176] Data transmission within a private cloud is treated as data transmission within the user's system, ignoring the transmission cost. Let... This represents the cost charged by the public cloud provider for inter-cloud transmission per unit of data. This represents the cost of cloud-based transmission. (From the task...) Transfer data to task Data transmission time Represented as:

[0177] ,

[0178] in, Represented as from task Transfer data to task The amount of data, Representative task The allocated virtual machine, and This represents the environment in which the virtual machine resides, namely a public cloud environment and a private cloud environment, with symbols... It indicates an AND relationship.

[0179] S533: Calculate the task encryption time and task decryption time;

[0180] Encryption time is a linear function of data size and the chosen security level. This depends on the amount of data transmitted. Use security level Encryption time required for the task Represented as:

[0181] ,

[0182] in express The unit encryption time of the encryption algorithm.

[0183] Task decryption time is expressed as Since the encryption time is close to the encryption time, this invention sets the decryption time to be equal to the encryption time. The only difference between encryption and decryption is that encryption is performed on the preceding virtual machine, while decryption is performed on the following virtual machine.

[0184] In the proposed hybrid cloud model, tasks without priority dependencies are initially assigned to virtual machines for execution. Once the preceding task is completed, data is transferred to the virtual machine responsible for the subsequent task. Encryption and decryption are performed if privacy is required. Unlike simple fault-tolerance mechanisms, tasks requiring backup are copied to other virtual machines during scheduling; importantly, sensitive data in the backup tasks must also be encrypted during transmission.

[0185] Therefore, backup tasks are subject to the same privacy protection protocols. Furthermore, virtual machines incur idle time costs during data transfer.

[0186] Therefore, the typical processing time for a task includes task allocation time, data transmission time, task encryption time, and task decryption time, as shown below:

[0187] ,

[0188] Furthermore, considering the backup fault tolerance mechanism, the total execution time is the sum of the regular processing time and the task backup time:

[0189] ,

[0190] The cost model uses a pay-as-you-go approach, charging users based on the duration of cloud system usage. The computational cost of a workflow is the sum of the products of the execution time of all task replicas and the unit execution cost of the corresponding virtual machine. The total cost is the sum of execution cost and bandwidth cost.

[0191] ,

[0192] in, This indicates the unit price of execution cost. This indicates the unit price of bandwidth cost.

[0193] S534: By comparing the total cost of the current particles Compared with the historical best scheduling scheme of this individual particle Total cost and the historical optimal scheduling scheme for all particles globally. Total cost This determines the current position of the particle. The specific rules are as follows:

[0194] If the current Less than Then set the current particle's position to ;

[0195] If the current Less than Then set the current particle's position to .

[0196] Then, the velocities of all particles are updated:

[0197] ,

[0198] Update location:

[0199] ,

[0200] In the formula, and These represent the current particle's position vector and update velocity in the solution space, respectively; and and represent the position and velocity generated after this iteration, respectively; Inertial weights are used to maintain a balance between global exploration and local development; and Learning factors representing individual cognition and social experience, respectively, are combined with random numbers within the interval and Together, they introduce necessary randomness into the particle's optimization process and control the step size as it approaches the optimal solution; This refers to the optimal scheduling position of the individual with the highest fitness (i.e., the lowest total execution cost) found by the particle itself in historical iterations, while This represents the globally optimal scheduling scheme found so far for the entire particle swarm.

[0201] S54: Determine the current iteration number Has the maximum number of iterations been reached? If the maximum number of iterations has not been reached, then the number of iterations... Increment by 1, return to step S52, update the particle swarm, and perform the next round of cost evaluation and optimization; if the maximum number of iterations has been reached, stop the iteration and output the mapping scheme corresponding to the globally optimal particle recorded at this time.

[0202] Based on this optimal solution, a specific list of scheduling instructions is generated to guide the hybrid cloud system in executing workflow tasks.

[0203] This step is based on cost-optimized scheduling using an improved particle swarm optimization (PSO) algorithm. While satisfying the aforementioned privacy constraints and backup strategies, the PSO algorithm searches for the optimal scheduling scheme in a discontinuous solution space. Each particle is encoded as a mapping vector from task to virtual machine. The fitness function is defined as the total execution cost. During the iteration process, the velocity and position update formulas guide the population towards the global optimum, ultimately outputting a workflow task scheduling scheme that balances low cost, high reliability, and strong privacy protection.

[0204] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A hybrid cloud workflow task scheduling method based on topology analysis and privacy quantization, which calculates task allocation time, data transmission time, task backup time, task encryption time, and task decryption time for a second set of tasks to be assigned to the public cloud, uses a particle swarm optimization algorithm to calculate the current total cost of particles, compares the current total cost of particles with the historical total cost of particles, and outputs the scheduling scheme with the minimum total cost of particles, characterized in that... The second set of tasks to be assigned to the public cloud is the first set of tasks to be assigned to the public cloud after encryption level allocation and proactive fault-tolerant backup.

2. The method according to claim 1, characterized in that, The first set of tasks to be allocated to the public cloud is formed by sorting the initial task set according to the level of privacy sensitivity, pre-allocating tasks with high privacy sensitivity to private cloud virtual machines, and continuing until the private cloud resources reach saturation and the remaining task set is completed.

3. The method according to claim 2, characterized in that, The process of sorting the initial workflow task set according to its privacy sensitivity, and pre-allocating tasks with high privacy sensitivity to private cloud virtual machines, includes: S21: For each task in the initial workflow task set Calculate privacy sensitivity , ， in, The pre-defined task inherent privacy factor, i.e., the privacy protection level; denominator Representative task The amount of computation required to execute; Indicates task The set of all direct successor tasks in the workflow Indicates task Transmitted to a specific subsequent task The amount of data; S22: Use the randomized quicksort algorithm, according to... The values ​​are sorted in descending order to obtain an ordered task set. ; S23: Traversal The tasks with the highest priority are pre-assigned to private cloud virtual machines in sequence until the private cloud resources are saturated.

4. The method according to claim 1, characterized in that, Step S22 includes: S221: Obtain a collection of several Array of unsorted tasks And the starting index of the sorting operation that needs to be performed. and end index bit ; Compare the size relationship between the start index bit and the end index bit: like This indicates that there are at least two tasks within the current interval that need to be sorted, so step S222 is executed. like If the number of tasks in the current interval is 1 or 0, the interval is considered to be in a naturally ordered state, and the sorting operation of the current branch is terminated. S222: Using a random function, at the starting index position With end index bit Generate a random integer between the given values ​​and denote it as the random index. ; S223: Sort the array to be sorted In the random index position The task at the office With the end index bit The task at the office To swap positions; S224: Set the first pointer Second pointer , put the first pointer Initialize to This pointer is used to dynamically record the right boundary of sub-intervals that are less than or equal to the reference element; the second pointer... Initialize to This pointer is used to traverse the current sorting interval from left to right; S225: Control the second pointer From the starting index bit Gradually increasing traversal to The task of extracting the current pointer position during each traversal. Compare with the benchmark task: like If the value is less than or equal to the baseline task, then the first pointer will be... Increment the value by 1 and set the task With the task To swap positions; like If the value is greater than the reference element, no data exchange operation is performed, and the second pointer... Continue iterating backwards; S226: At the second pointer After completing the traversal, move the task corresponding to the next position of the first pointer. With the end index bit The baseline task at the location Perform position swapping and record the new position index of the baseline task. The array With index The physical boundary is divided into two independent sub-intervals: Left sub-interval All tasks within this interval are less than or equal to the baseline task; right subinterval All tasks within this range are greater than the baseline task; S227: Boundary index calculated based on step S226 : Using the left sub-interval as the new processing target, set the starting index to The ending index is set to Jump back and execute step S221; Using the right sub-interval as the new processing target, set the starting index to The ending index is set to Jump back and execute step S221; When all branches of the recursive call satisfy the termination condition of step S221, it means that the entire array... After all sorting operations are completed, the final sorted task array is output. .

5. The method according to claim 1, characterized in that, The encryption level allocation includes setting an encryption level for the first set of tasks to be allocated to the public cloud, including: Through different encryption algorithms With encryption level The correspondence between them is used to select different levels of encryption algorithms for different tasks; Data vulnerability in computational tasks: ， in, This indicates data vulnerability, which physically refers to the vulnerability of data transfer between tasks. Maximum number of plaintexts required to perform a brute-force attack on an encrypted network With the selected encryption level The number of plaintexts required to process the corresponding encryption algorithm ; like If the condition is not met, the allocation will be redistributed until it is met; in, This indicates the exposure risk threshold.

6. The method according to claim 5, characterized in that, The redistribution is carried out using a random allocation method.

7. The method according to claim 1, characterized in that, The proactive fault-tolerant backup includes performing structure-aware proactive fault-tolerant backup on the first set of tasks to be allocated to the public cloud, including: S41: Use a breadth-first search algorithm to traverse the first set of tasks to be assigned to the public cloud in a hierarchical manner. Using the DAG graph, establish the hierarchical structure of all tasks, and traverse the levels where the tasks are located to obtain the hierarchical structure of all tasks. S42: Traverse each level from high to low according to the level output in step S41, and calculate the betweenness centrality of each node in each level. S43: Add a backup copy to the task with the highest centrality value of the number of intermediaries at the same level.

8. The method according to claim 1, characterized in that, Step S42 includes: node The formula for betweenness centrality is: ， in, Represents a node betweenness centrality, Represents node pairs The total number of shortest paths between them. Represents node pairs Passing through the node The number of shortest paths.

9. The method according to claim 8, characterized in that, Step S43 includes if reliability gain rate Greater than the preset expected reliability gain If so, continue with the task backup. Less than the preset expected reliability gain If so, then the active backup will be terminated; ， ， ， ， in, These represent the new overall reliability achieved by the task after adding the backup. This indicates the original basic reliability of the task before the backup was added. Indicates task The base reliability probability of successful execution on a single virtual machine. This represents the probability of a single instance failing. This represents the constant failure rate of different virtual machines per unit of time. This refers to the specific virtual machine instance that hosts the task and its replicas. Indicates the task Assigned to Execution time, Indicates task The computational load, For CPU single-core computing power, For virtual machines The number of CPU cores.

10. The method according to claim 1, characterized in that, The total cost is calculated as follows: ， ， ， ， ， ， in, Indicates the total execution time. express The unit encryption time of the encryption algorithm. The amount of data transmitted. Use security level The task encryption time required to perform encryption. Represented as from task Transfer data to task The amount of data, Representative task The assigned virtual machine, and This represents the environment in which the virtual machine resides, namely a public cloud environment and a private cloud environment, with symbols... Indicates an AND relationship. This indicates the unit price of execution cost. This indicates the unit price of bandwidth cost. Indicates the task Assigned to Execution time, Indicates task The computational load, For CPU single-core computing power, For virtual machines The number of CPU cores; Indicates from the task Transfer data to task Data transmission time, This represents the cost charged by the public cloud provider for inter-cloud transmission per unit of data. This indicates the cost of transmission within the cloud.

Images