Data storage system, metadatabase synchronization method and data cross-domain calculation method

Abstract

The invention discloses a data storage system, a metadatabase synchronization method and a data cross-domain calculation method. The data storage system comprises at least two data centers, wherein each data center comprises an access layer, a compiling layer, a computing layer, a storage layer and a bottom container cloud platform; The access layer is used for providing a unified data access interface; The compiling node is used for inquiring the metadatabase according to the received SQL statement, generating a matched execution plan and distributing the job task to the computing node for execution according to the execution plan; The computing nodes are used for acquiring data from data nodes of a data center where the computing nodes are located for computing according to the operationtasks and sending computing results to the computing nodes serving as summarizing nodes; The data node is used for storing data; And the bottom container cloud platform is used for carrying out containerization management on all services of the same data center. According to the technical scheme provided by the embodiment of the invention, the input cost is saved while the cross-domain computingservice is provided with high quality, and the data compliance requirement is met.

Description

technical field [0001] Embodiments of the present invention relate to information processing technologies, and in particular to a data storage system, metadata database synchronization, and data cross-domain computing methods. Background technique [0002] With the increasing amount of data and the need for business expansion, more and more enterprises have begun to deploy their own data centers. Because of the needs of some special industries, sometimes it is necessary to combine the data of multiple enterprises or organizations for joint computing, that is to say, it is necessary to solve the problem of data joint computing across data centers. Each data center is equivalent to a domain. The network within the domain is fast, but the network between domains is much slower and unstable than the network within the domain. Therefore, if a large amount of network overhead is generated during joint computing, There will be a relatively large performance problem. [0003] At p...

AI Technical Summary

Problems solved by technology

Each data center is equivalent to a domain. The network within the domain is fast, but the network between domains is much slower and unstable than the network within the domain. Therefore, if a large amount of network overhead is generated during joint computing, There will be a relatively large performance problem

[0003] At present, there are strongly consistent and scalable global distributed databases on the market. The above-mentioned global distributed databases have two main defects. One is that it requires high-cost investment, and the other is that it does not meet the data compliance requirements. Regulatory requirements, that is, the requirement that data in a data center cannot be outflowed to other data centers

[0004] The reason for the first defect is that in order to meet the performance requirements of available scenarios, it is necessary to reduce the delay between data centers to a very low standard, which inevitably requires relatively high investment and investment in the network between data centers. Optimization; the reason for the second defect is that, from a business perspective, the data centers of the same company may not be used when performing cross-data center calculations, because the possibility of all data centers using the same database at the same time is relatively small , and the existing global distributed databases are all calculated nearby through multiple copies, that is, when the data is written, it will be written to other data centers, so this method does not meet the data compliance requirements

Images