Data processing method and system, electronic equipment and computer program product

Abstract

The embodiment of the invention discloses a data processing method and system, electronic equipment and a computer program product. The method comprises the following steps: when a log reading event aiming at target data is monitored, determining whether a permission of sending a first time log exists or not, wherein the permission is given in sequence according to a moving track of the target data moving between at least two servers; if yes, obtaining at least one log of the target data, the timestamp of which is greater than or equal to the first time; and sending the at least one log to a reader. The embodiment of the invention provides a permission sending mechanism, that is to say, the condition of the log corresponding to the target data is recorded in the log file of each server when the target data is transferred among a plurality of servers. When any server wants to send the log of the target data to a reading party (such as a data log subscribing party / consumer), the server needs to have authority, so that the log of the target data is sent to the reading party according to a time sequence without chaos.

Description

technical field [0001] The present application belongs to the technical field of databases, and in particular relates to a data processing method, system, electronic equipment and computer program product. Background technique [0002] After the data in the database changes, many scenarios require real-time subscription table data changes, such as synchronization to message queues, message communication between applications, and real-time computing scenarios. CDC (change data capture, change data capture) capability is a commonly used function of the database. The core idea of ​​CDC is to monitor and capture changes in the database (including insertion, update, deletion, etc. of data or data tables), record these changes in the order in which they occur, and write them into the message middleware for other services to perform Subscribe and consume. A file that records data, a data table, or a change in a partition of a data table can be called a log. [0003] Similar to H...

AI Technical Summary

Problems solved by technology

When collecting the logs corresponding to the partition data from different servers, if the logs of each server are simply collected concurrently, the logs corresponding to the same partition data will not be received by the downstream subscriber / consumer device in chronological order. out of order

Images