Multicenter data task processing method, multicenter data task processing device and multicenter data task processing system

Abstract

The invention discloses a multicenter data task processing method, a multicenter data task processing device and a multicenter data task processing system. The processing method comprises the steps: A, collecting data task information; B, generating a data task model graph according to the data task information, cutting the data task model graph, namely grouping all nodes in the data task model graph, wherein each node of the data task model graph is one data task; and C, allocating data tasks of different groups into corresponding data centers. By means of the multicenter data task processing method, the device and the system, through the establishment of a model reflecting the correlations among data tasks, multicenter task allocation is performed based on the model, and thus the task allocation is visual, accurate and balanced. Transfer and redundancy of large amount of data are avoided, data volume transmitted among each center is small, therefore pressure of network load is reduced, overall data processing efficiency is improved, and loads in all the data centers are basically the same.

Description

technical field [0001] The invention relates to a business support technology, in particular to a multi-center data task processing method, device and system. Background technique [0002] For security reasons, enterprises usually establish two or more data processing centers to achieve disaster recovery. In daily life, task allocation is used to distribute the load among data processing centers. Each data processing center handles different tasks and stores different data, especially complex intermediate data that is not directly oriented to applications. However, due to the complex relationship between data, often a data processing needs to rely on multiple data sources, which is very likely to be the result of data processed by another data center. In this way, the data needs to be exported from a certain center, then transmitted through the network, and then imported to A computing center is required, and this process consumes a lot of time, which greatly reduces the ov...

AI Technical Summary

Problems solved by technology

[0011] 1. The existing dynamic allocation algorithm is suitable for calculations that are irrelevant to data or not related to massive data, but not suitable for calculation scenarios with a huge amount of data, which will cause a large amount of data migration and data redundancy;

[0012] 2. The existing fixed allocation method is usually divided by geographical application, and is usually based on long-term statistical information. It is difficult to ensure optimal allocation, which will lead to redundant data storage in multiple places, large data transmission volume, repeated calculations, and inability to A series of issues such as forming a unified view;

[0013] 3. The method of merging based on the client is not suitable for calculations with a huge amount of data;

[0014] 4. The distribution and parallel processing of data calculation based on a single center is not suitable for data calculation across data centers because it does not consider data synchronization and data migration between data centers

Images