Online small-batch data processing method and device, equipment and storage medium

Abstract

The invention provides an online small-batch data processing method and device, equipment and a storage medium, which can be applied to the financial field or other fields. The method comprises the following steps: generating a next-day operation instance table according to a next-day schedule before daily switching, and writing the next-day operation instance table into a cache module, wherein the job instance table comprises specified execution time of a corresponding operation task; and sending a scheduling updating instruction to an executor, so that the executor obtains the corresponding operation task from the operation instance table of the cache module and executes the operation task according to the appointed execution time of the corresponding operation task. According to the specification, the dispatching pressure of a dispatching center in a service scene between online and batch is relieved.

Description

Technical field

[0001] This specification relates to the field of large data processing, in particular, involving a small batch data processing method, device, device, and storage medium. Background technique

[0002] Most of the business in the financial business system can be implemented online (ie real time) or batch. Common pairs of guests can be completed online by online transactions, customers are relatively high in real-time requirements for trading results, such as online transfer. Bulk business is mainly for large-scale complex data processing, real-time requirements, such as day-end account clearing, and ending.

[0003] In addition, there are some services between the two, which handles these services easily make the scheduling center of the financial business system. Therefore, how to mitigate the scheduling pressure under the business scene of online and bulk, has become the technical problem to be solved urgently. Inventive content

[0004] The object of this spec...

Images