Batch processing method and system supporting multi-master distributed data processing

Abstract

The invention relates to the technical field of information data processing, and in particular relates to a batch processing method and a system which support multi-master distributed data processing. The batch processing system is internally provided with a batch processing frame which mainly comprises a batch processing engine program, a batch processing managing and monitoring program, a batch processing flow control file and a batch processing batch-step defining file, wherein the flow control file and the batch-step defining file are two configuring files, when the batch processing engine is started, the flow control file can be traversed so as to obtain a batch-step code which needs to be processed, script information needing to be executed can be obtained from the batch-step defining file according to the batch processing code, and then the script is executed to complete batch steps. Compared with the prior art, the batch processing method and system has great breakthroughs and advantages, solves many shortcomings and defects in the prior art, and has high concurrency, high availability and high expandability.

Description

[Technical field] [0001] The invention involves the field of information data processing technology, which is specifically a batch processing method and system that supports multi -host distributed data processing. [Background technique] [0002] With the increasingly vigorous and mature of the securities trading market, the amount of transaction data is also increasing, and it has higher requirements for the stability and processing efficiency of the transaction and processing system.After each trading period of each trading day, the trading system began to liquidate the transaction that day, and then handled the complex non -real -time transaction business.This type of post -processing process is usually called batch processing.To put it simply, batch processing is to group different tasks, that is, in batches, and then perform these tasks in a certain order. [0003] Depending on the complexity of the business, the complexity of the batch processing system is not the same.The ...

AI Technical Summary

Problems solved by technology

Second, the exception handling ability is poor

Third, the coupling of each batch of steps is high, and the scalability is not good

It limits the time point, but it may not guarantee the actual execution sequence, so it is not stable and reliable enough. In the existing market, there is still no batch processing system with high processing efficiency, safety, stability and reliability to meet the increasingly abundant data volume.

Images