A Latency-Optimized Scientific Workflow Data Layout Method for Hybrid Cloud Environment

Abstract

The invention discloses a time-delay-optimized scientific workflow data layout method in a hybrid cloud environment, which considers the characteristics of data layout in a hybrid cloud environment, combines the dependencies between scientific workflow data, and considers the bandwidth between cloud data centers and private cloud data. The influence of factors such as the number of centers and capacity on the transmission delay; firstly, pre-process it to improve the execution efficiency of the data layout strategy in the later stage; by introducing the crossover operator and mutation operator of the genetic algorithm, avoid the particle swarm optimization algorithm. The problem of premature convergence can improve the diversity of population evolution, effectively compress the data transmission delay, and effectively reduce the data transmission delay of scientific workflow in the hybrid cloud environment. The invention improves the execution efficiency of the data layout strategy and optimizes the transmission time delay of the scientific workflow data layout.

Description

technical field [0001] The invention relates to a scientific workflow data layout method in the field of parallel and distributed high-performance computing, in particular to a time delay optimization-oriented scientific workflow data layout method in a hybrid cloud environment. Background technique [0002] Scientific workflow systems are data-intensive applications that have been widely used in scientific research fields such as astronomy, high-energy physics, and bioinformatics. Scientific workflow applications are data-driven, and there are complex data dependencies between computing task nodes, and the size of the processed data sets can usually reach TB or even PB levels. These datasets include existing original input datasets, as well as intermediate and final datasets generated during processing and analysis. Due to the complex nature of scientific workflow application structure and large data volume, it has strict requirements on the computing power and data storag...

AI Technical Summary

Problems solved by technology

(2) For security reasons, private data is designated to be stored in a specific private cloud data center. Due to the limited capacity of the private cloud data center, it needs to be transmitted across data centers. Considering the impact of bandwidth factors and optimizing data transmission delay is a challenge for scientific workflow data layout

Therefore, the current research work has not yet formed a complete and effective solution to the delay-optimized scientific workflow data layout problem in the hybrid cloud environment.

Images