A delay-oriented scientific workflow data layout method in hybrid cloud environment

Abstract

The invention discloses a scientific workflow data layout method oriented to time delay optimization in a hybrid cloud environment, which considers the characteristics of data layout under the hybridcloud environment, combines the dependency relationship among scientific workflow data, and considers the influence of factors such as the bandwidth between cloud data centers, the number and capacityof private cloud data centers and the like on transmission time delay. Firstly, preprocessing is carried out to improve the efficiency of the data layout strategy in the later period. By introducingcrossover operator and mutation operator of genetic algorithm, the problem of premature convergence of particle swarm optimization algorithm is avoided, the diversity of population evolution is improved, the data transmission delay is effectively reduced, and the data transmission delay of scientific workflow in hybrid cloud environment is effectively reduced. The invention improves the executionefficiency of the data layout strategy and optimizes the transmission delay of the scientific workflow data layout.

Description

technical field [0001] The invention relates to a scientific workflow data layout method in the field of parallel and distributed high-performance computing, in particular to a time delay optimization-oriented scientific workflow data layout method in a hybrid cloud environment. Background technique [0002] Scientific workflow systems are data-intensive applications that have been widely used in scientific research fields such as astronomy, high-energy physics, and bioinformatics. Scientific workflow applications are data-driven, and there are complex data dependencies between computing task nodes, and the size of the processed data sets can usually reach TB or even PB levels. These datasets include existing original input datasets, as well as intermediate and final datasets generated during processing and analysis. Due to the complex nature of scientific workflow application structure and large data volume, it has strict requirements on the computing power and data storag...

AI Technical Summary

Problems solved by technology

(2) For security reasons, private data is designated to be stored in a specific private cloud data center. Due to the limited capacity of the private cloud data center, it needs to be transmitted across data centers. Considering the impact of bandwidth factors and optimizing data transmission delay is a challenge for scientific workflow data layout

Therefore, the current research work has not yet formed a complete and effective solution to the delay-optimized scientific workflow data layout problem in the hybrid cloud environment.

Images