A Multi-data Center Data Layout Optimization Method Based on Computational Correlation

Abstract

The invention discloses an optimization method for data layout of multi-data centres based on calculating relevancy. The optimization method comprises the steps of: generating an access associated matrix of an aggregation of computing sets and data sets according to a situation of executing calculation and procession of the data sets; obtaining, by means of calculation, the calculating relevancy between any two data sets, generating a corresponding calculating relevancy matrix; calculating a basic capacity of each data centre, defining a layout associated matrix, and laying out the data sets according to the calculating relevancy. According to the optimization method for data layout of multi-data centres based on calculating relevancy disclosed by the invention, by means of establishing the access associated matrix and the layout matrix, specific mathematical expression of the calculating relevancy is provided; according to the established calculating relevancy matrix, data layout is realized by a method of a low complexity; moreover, new data and intermediate data are dynamically laid out to a proper data centre, so that data scheduling cross the data centres can be effectively reduced, and an access performance of a system is improved.

Description

technical field [0001] The invention relates to the field of distributed data storage and management, in particular to a multi-data center data layout optimization method based on calculation correlation. Background technique [0002] The data explosion has brought the information society into the era of big data. Big data has a huge amount of data, contains rich and diverse information, and brings huge economic and social benefits, but also faces severe technical challenges. The "3V" characteristics of big data (that is, large capacity, fast update speed, and various types) make traditional database management unable to meet the requirements of big data storage and management. Cloud computing is currently a very important distributed network computing platform. , is regarded as a very effective storage, management and analysis platform for big data, and is a cost-effective solution for big data management and analysis. [0003] However, in the distributed multi-data cente...

AI Technical Summary

Problems solved by technology

However, the data volume of big data is already very large, and unreasonable copy methods will further bring huge storage space overhead. Some data set copies that are not commonly used will cause too much unnecessary redundancy in the system, seriously affecting storage efficiency and space utilization. Rate

[0006] Comprehensive analysis shows that the current multi-data center data layout optimization methods have the following defects: a single static layout method cannot achieve data layout optimization, a single dynamic layout method is too complex, and the existing methods lack the correlation between data to specifically and dynamic description, the data scheduling problem generated by distributed data-intensive computing is powerless or has little effect, and the implementation complexity of the data layout optimization method is relatively high

Images