Database synchronization data processing method, storage medium and device

Abstract

The application provides a database synchronization data processing method, a storage medium and equipment. The method comprises the following steps: a database analyzes transaction data, and saves breakpoint information into each piece of analyzed transaction data; the breakpoint information is obtained from the transaction data, and a to-be-synchronized data table set is found according to the breakpoint information and an LSN segment; changed operation information of a target data row is found in a CD table through the breakpoint information and the to-be-synchronized data table set, and synchronization is performed. Through the method, the inconsistency problem caused by separate storage of the breakpoint and the data is avoided, the order of the event data of multiple CD tables is ensured, and the unique sequence number at the time of processing is recorded in the breakpoint information, so that the breakpoint can be submitted when any row is processed.

Description

TECHNICAL FIELD

[0001] The present application relates to database technology, and in particular, to a method for processing database synchronization data, a storage medium and an apparatus. BACKGROUND

[0002] The current database is divided into three stages when using data synchronization tools for real-time data synchronization, the first stage is to initialize the loading of inventory data, and the basic point of data synchronization is obtained; the second stage is to synchronize the incremental data based on the synchronization basic point established by the initialization data loading; the third stage is to periodically compare and verify the source data and target data of data synchronization to confirm that no data is lost in the data synchronization process. The second stage and the third stage will be in a long-time parallel state.

[0003] In the second stage of incremental data synchronization, the incremental data is obtained by analyzing the database log, so as to realize real-time data synchronization, which is a common real-time replication technology. This technology parses the source database online log or archive log to obtain the addition, deletion and modification changes of data, and then converts these changes into a specific message format of the synchronization software internally, and sends them to the target data synchronization software through the private transmission protocol of the data synchronization software. Finally, the target synchronization software restores the transaction log obtained to the SQL statement supported by the target database and executes it on the target database to realize real-time data synchronization and maintain the data consistency of the source and target databases.

[0004] DB2 SQL replication generates an intermediate table (CD table) for each source table that starts to capture data, which is used to store the data changes of the source table. In the above-mentioned second stage of incremental data synchronization, the incremental data needs to be read from each CD table in turn, and the data is written into the source file buffer after being parsed. The traditional scheme needs an additional database table to record the breakpoint of each capture table, and the data is committed after being written into the file buffer. SUMMARY

[0005] An object of the present application is to avoid the problem that the breakpoint is inconsistent with the actual data.

[0006] A further object of the present application is to make the breakpoint in order to ensure the orderliness of the event data of multiple CD tables.

[0007] A further object of the present application is that the number of changed data contained in the breakpoint is controllable.

[0008] In particular, the present application provides a method for processing database synchronization data, which comprises:

[0009] The database parses the transaction data and saves the breakpoint information into each parsed transaction data;

[0010] Obtaining breakpoint information in transaction data, and finding a set of data tables to be synchronized according to the breakpoint information and the LSN segment;

[0011] Finding changed target data row change operation information in the CD table through the breakpoint information and the set of data tables to be synchronized, and performing synchronization.

[0012] Optionally, the breakpoint information comprises an LSN segment start value, an LSN segment end value, and a changed target data row serial number.

[0013] The step of finding the set of data tables to be synchronized according to the breakpoint information and the LSN segment comprises:

[0014] Obtaining an LSN maximum value in the LSN segment;

[0015] Judging whether the LSN maximum value is empty;

[0016] In a case where the LSN maximum value is not empty, judging whether the LSN segment start value in the breakpoint information is empty;

[0017] In a case where the LSN segment start value is not empty, judging whether the LSN segment start value is less than the LSN maximum value;

[0018] In a case where the LSN segment start value is less than the LSN maximum value, judging whether the LSN segment end value in the breakpoint information is empty;

[0019] In a case where the LSN segment end value is not empty, querying log information in the LSN segment start value to the LSN segment end value to obtain the set of data tables to be synchronized.

[0020] If yes, the step of finding changed target data row change operation information in the CD table through the breakpoint information and the set of data tables to be synchronized, and performing synchronization comprises:

[0021] Iterating the set of data tables to be synchronized;

[0022] Processing changed data in the set of data tables to be synchronized one by one.

[0023] Optionally, the breakpoint information further comprises a source table unique identifier.

[0024] The step of processing changed data in the set of data tables to be synchronized one by one comprises:

[0025] Judging whether a source table unique identifier of a data table to be synchronized in the set of data tables to be synchronized is greater than or equal to the source table unique identifier in the breakpoint information;

[0026] If the source table unique identifier of the data table to be synchronized is greater than or equal to the source table unique identifier in the breakpoint information, querying and processing changed data in the data table to be synchronized;

[0027] Set the changed target data row sequence number in the breakpoint information to null.

[0028] Optionally, the breakpoint information comprises: an LSN segment start value, an LSN segment end value, and a changed target data row sequence number.

[0029] The step of querying and processing the changed data in the set of data tables to be synchronized comprises:

[0030] Querying the data in the CD table located in the interval from the LSN segment start value to the LSN segment end value in the breakpoint information, and generating a changed data table.

[0031] Traversing the data in the changed data table.

[0032] Analyzing the data rows in the changed data table one by one, whose sequence numbers are greater than the changed target data row sequence number in the breakpoint information.

[0033] Assigning the sequence number of the analyzed data row to the changed target data row sequence number in the breakpoint information.

[0034] Adding the analyzed data to the parsed row data array.

[0035] After the traversal ends, submitting the remaining data in the parsed row data array to the queue and performing the data change operation.

[0036] Optionally, the step of adding the analyzed data to the parsed row data array further comprises:

[0037] In the case that the LSN segment changes or the size of the parsed row data array is equal to the preset transaction shard size, submitting the parsed row data array to the queue and performing the data change operation.

[0038] Emptying the parsed row data array.

[0039] Optionally, the step of judging whether the LSN segment start value in the breakpoint information is null further comprises: in the case that the LSN segment start value is null, assigning the LSN maximum value to the LSN segment start value.

[0040] The step of judging whether the LSN segment end value in the breakpoint information is null further comprises: in the case that the LSN segment end value is null, assigning the LSN maximum value to the LSN segment end value.

[0041] Optionally, the breakpoint information further comprises: a source table unique identifier.

[0042] The step of synchronizing the changed target data row by querying and processing the changed data in the set of data tables to be synchronized further comprises:

[0043] Set the LSN segment end value in the breakpoint information to the LSN segment start value;

[0044] Set the LSN segment end value to null;

[0045] Set the source table unique identifier in the breakpoint information to a set value;

[0046] Clean up the data in the CD table.

[0047] According to another aspect of the present application, there is also provided a machine readable storage medium having stored thereon a machine executable program which, when executed by a processor, implements any of the above database synchronization data processing methods.

[0048] According to still another aspect of the present application, there is also provided a computer device comprising a memory, a processor, and a machine executable program stored on the memory and running on the processor, and wherein the processor implements any of the above database synchronization data processing methods when executing the machine executable program.

[0049] The database synchronization data processing method of the present application, by saving the breakpoint in the parsed transaction data in a preset format, when synchronizing data, finds the data table set requiring synchronization operation by comparing the breakpoint data with the LSN, and then finds the specific change operation of the data according to the CD table corresponding to the data table set and executes it, so as to realize data synchronization. By this method, the inconsistency problem caused by separate storage of the breakpoint and the data is avoided.

[0050] Further, in the breakpoint data in the preset format, the LSN segment start value, the LSN segment end value, the source table unique ID and the changed unique sequence number corresponding CD table processed by the breakpoint are stored, which record the position processed by the breakpoint, so as to realize the purpose of correctly finding the position of the synchronization data.

[0051] The above and other objects, advantages and features of the present application will become more apparent from the following detailed description of some embodiments thereof, when taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0052] Some specific embodiments of the present application will be described in detail below with reference to the accompanying drawings, in an exemplary and non-limiting manner. The same reference numerals in the drawings denote the same or similar components or parts. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0053] Figure 1 is a flowchart of a to-be-synchronized data table set of a database synchronization data processing method according to an embodiment of the present application;

[0054] Figure 2is a flowchart of a change operation corresponding to an acquisition change table of a database synchronization data processing method according to an embodiment of the application;

[0055] Figure 3 is a flowchart of a database synchronization data processing method according to an embodiment of the application;

[0056] Figure 4 is a schematic diagram of a machine readable storage medium in a database synchronization data processing method according to an embodiment of the application; and

[0057] Figure 5 is a schematic diagram of a computer device in a database synchronization data processing method according to an embodiment of the application. DETAILED DESCRIPTION

[0058] Those skilled in the art should understand that the embodiments described below are only a part of the embodiments of the present application, rather than all the embodiments of the present application, and are intended to explain the technical principles of the present application, rather than to limit the protection scope of the present application. Based on the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should fall within the protection scope of the present application.

[0059] The current database is divided into three stages when using a data synchronization tool to perform real-time data synchronization. The first stage is to initialize the loading of inventory data to obtain the basis point of data synchronization. The second stage is to perform incremental data synchronization based on the synchronization basis point established by the initialization data loading. The third stage is to periodically compare and verify the source data and target data of data synchronization to confirm that no data is lost in the data synchronization process. The second stage and the third stage will be in a long-time parallel state.

[0060] In the second stage of incremental data synchronization, the incremental data is obtained by analyzing the database log, so as to realize real-time data synchronization. This technology obtains the addition, deletion and modification changes of data by analyzing the online log or archive log of the source database, converts these changes into a specific message format of the synchronization software in a transaction unit, sends them to the target data synchronization software through the private transmission protocol of the data synchronization software, and finally the target synchronization software restores the obtained transaction log into the SQL statement supported by the target database and executes it on the target database, to realize real-time data synchronization and maintain the data consistency of the source and target databases.

[0061] One existing approach is that DB2 SQL replication generates an intermediate table (CD table) for each source table that opens capture data, for storing data changes of the source table. In the above-mentioned second stage incremental data synchronization, incremental data needs to be read from each CD table in sequence, and after data parsing, the data is written into the source file buffer. The traditional scheme needs an additional database table to record the breakpoint of each capture table, and after the data is written into the file buffer, the data is committed.

[0062] However, since the data is written into the local file buffer, and the breakpoint is recorded in the additional database table, because they are two different storages, without introducing distributed transactions and other technologies, whether the breakpoint is stored first and then the data is written, or the data is written first and then the breakpoint is saved, in the case of failure in the post-processing step, the data and the breakpoint may be inconsistent.

[0063] To solve the above problems, a database synchronization data processing method is provided.

[0064] Figure 1 FIG. 1 is a flowchart of a process of acquiring a set of data tables to be synchronized according to an embodiment of the database synchronization data processing method. The flowchart includes the following steps.

[0065] In step S101, the breakpoint is parsed to obtain breakpoint information. The breakpoint information includes: a unique ID (tableObjectId) of a source table, a starting value (fromLsn) of a LSN (Log sequence number) segment, an ending value (toLsn) of the LSN segment, and a unique sequence number (intentSeq) of a change. The LSN is used to identify the position of a specific log file record in a log file. The last step uses last to represent this breakpoint information.

[0066] In DB2, the uniqueness of a table is determined by TBSPACEID (table space ID) and TABLEID (table ID). Since the TABLEID field type in the SYSCAT.TABLES view is SMALLINT (-32768-32767), TBSPACEID can be concatenated to the high bit. Therefore, one optional generation method of tableObjectId is as follows:

[0067] tableObjectId = tbSpaceId * 65536 + tableId

[0068] Those skilled in the art can select a suitable generation method according to actual needs.

[0069] In step S102, maxLsn is obtained. That is, the latest log file sequence number in the LSN segment is obtained.

[0070] Step S103, judging whether maxLsn is available. This step includes: if maxLsn is not available, it is displayed as empty, indicating that no change data has been generated, and step S102 is executed after a time interval is set.

[0071] Step S104, judging whether last.fromLsn is available in the case that the result of step S103 is yes.

[0072] Step S105, last.fromLsn=maxLsn in the case that the result of step S104 is no. This step includes: in the case that the result of step S104 is no, indicating that the program has not recorded breakpoint information, maxLsn is assigned to last.fromLsn.

[0073] Step S106, judging whether last.fromLsn<maxLsn in the case that the result of step S104 is yes or step S105 is executed. This step includes: when last.fromLsn is available or after being assigned, it is judged whether it is less than maxLsn, if no, indicating that the latest data has been read, step S102 is executed after a time interval is set.

[0074] Step S107, judging whether last.toLsn is available in the case that the result of step S106 is yes.

[0075] Step S108, last.toLsn=maxLsn in the case that the result of step S107 is no. This step includes: if last.toLsn is empty, it indicates that the program has not recorded breakpoint information or step S115 has been executed, and maxLsn is assigned to last.toLsn.

[0076] Step S109, obtaining a set of to-be-synchronized data tables in the case that the result of step S107 is yes or step S108 is executed. This step includes: in the case that the result of step S107 is yes or step S108 is executed, it indicates that there is a data table that needs data change, and the data table that needs data change from last.fromLsn to last.toLsn is queried and a set of to-be-synchronized data tables is generated. In an example of the method, the set of to-be-synchronized data tables does not include the data table corresponding to last.fromLsn itself and includes the data table corresponding to last.toLsn itself, and whether to include can be selected according to actual conditions by those skilled in the art.

[0077] Step S110, looping through the set of to-be-synchronized data tables.

[0078] Step S111, judging whether tabeObjectid is greater than or equal to Last.tableObjectid. In this step, if tabeObjectid is less than Last.tableObjectid, it indicates that the latest data has been read, and then the other data tables are traversed in sequence.

[0079] Step S112, querying the processing table change data.

[0080] Step S113, last.intentSeq=NULL. After the change data of the data table is processed, last.intentSeq is reset.

[0081] Step S114, judging whether the loop ends. This step includes judging whether the loop traversal ends, and if not, the data table set is traversed in sequence.

[0082] Step S115, in the case that the result of step S114 is yes, last.fromLsn=Last.tolsn, last.toLsn=NULL, last.tableObjectld=-1. This step includes that in the case that the result of step S114 is yes, i.e. the loop processing of the data table set to be synchronized ends, the breakpoint information is rolled forward, i.e. Last.tolsn is assigned to last.fromLsn, last.toLsn is reset, and last.tableObjectld is set to -1.

[0083] Step S116, cleaning the CD table data. This step includes cleaning the CD table data that has been processed in the database context, so as to prevent the CD table data from excessive expansion.

[0084] In this method, the breakpoint information is stored in the parsed transaction data, so as to avoid the inconsistency problem caused by separate storage of data and breakpoint; the fromLsn and tolsn stored in the breakpoint information can find the LSN interval corresponding to the data to be changed; the tableObjectId stored in the breakpoint information can find the table being processed when the breakpoint is restored; the intentSeq stored in the breakpoint information can find the data change row in the CD table when the breakpoint is restored. Thus, the position of the change data is accurately found, and the change operation is performed.

[0085] Figure 2 is a flowchart of acquiring a change operation corresponding to a change table according to an embodiment of the database synchronization data processing method of the present application. Figure 2 The flowchart shown can be an optional execution manner of step S110, and the flowchart can include:

[0086] Step S201, obtaining CD table information corresponding to the data table.

[0087] Step S202, traversing the row data in a loop.

[0088] Step S203, judging whether last.intentSeq is less than intentSeq. This step includes: if last.intentSeq is greater than intentSeq, indicating that the row data has been updated, then continue to execute step S202 to traverse the subsequent row data.

[0089] Step S204, parsing the row data.

[0090] Step S205, judging whether LSN changes or dbmsDataArr.size >= the size of the number of shards. Wherein dbmsDataArr represents the parsed row data array, used to store the row data ready to be executed.

[0091] Step S206, in the case of yes in step S205, submitting the queue and emptying dbmsDataArr. This step includes: when LSN changes or the size of the parsed row data array dbmsDataArr.size >= the size of the number of transaction shards, then submit to the queue for execution, and empty dbmsDataArr.

[0092] Step S207, in the case of no in step S205, last.intentSeq = intentSeq, dbmsDataArr.add. This step includes: when LSN does not change or the size of the parsed row data array dbmsDataArr.size is less than the size of the number of transaction shards, the unique sequence number (intentSeq) of the parsed row data is assigned to last.intentSeq, which is used to record the position reached by the breakpoint, and then the parsed row data is added to dbmsDataArr.

[0093] Step S208, judging whether the loop is ended. This step includes: if the loop is not ended, then continue to traverse other row data.

[0094] Step S209, in the case of yes in step S208, submitting the remaining data in dbmsDataArr to the queue. After the loop is ended, the remaining data in dbmsDataArr is submitted to the queue to start the change operation.

[0095] Through the method, the CD table information corresponding to each table in the data table set can be obtained in a cycle, and the change information in the CD table is added to the dbmsDataArr and sent to the queue for processing, so that the change operation of data is realized, and the position reached by the breakpoint is recorded through the intentSeq, so that the purpose of submitting the breakpoint at any row can be achieved.

[0096] Figure 3 is a flowchart of a database synchronization data processing method according to an embodiment of the present application; the flowchart can include:

[0097] In step S301, the database parses transaction data, and saves breakpoint information into each piece of parsed transaction data. The breakpoint information includes: event prefix identifier, source table unique ID (tableObjectId), LSN (Log sequence number) segment starting value (fromLsn), LSN segment ending value (toLsn), and changed unique sequence number (intentSeq).

[0098] In step S302, the breakpoint information is obtained from the transaction data, and the data table set to be synchronized is found according to the breakpoint information and the LSN segment. This step includes: obtaining the maximum LSN value (maxLsn) in the LSN segment; judging whether the maximum LSN value is empty; in the case where the maximum LSN value is not empty, judging whether the LSN segment starting value in the breakpoint information is empty; in the case where the LSN segment starting value is not empty, judging whether the LSN segment starting value is less than the maximum LSN value; in the case where the LSN segment starting value is less than the maximum LSN value, judging whether the LSN segment ending value in the breakpoint information is empty; in the case where the LSN segment ending value is not empty, querying the log information from the LSN segment starting value to the LSN segment ending value, and obtaining the data table set to be synchronized. Through this step, the data table set to be synchronized can be accurately found according to the breakpoint information.

[0099] In step S303, the change operation information of the changed target data row is found in the CD table through the breakpoint information and the data table set to be synchronized, and is synchronized. This step includes: traversing the data table set to be synchronized; judging whether the source table unique identifier of the data table to be synchronized in the data table set to be synchronized is greater than or equal to the source table unique identifier in the breakpoint information; if yes, querying and processing the changed data in the data table to be synchronized; setting the sequence number of the changed target data row in the breakpoint information to be empty.

[0100] The step of querying and processing the changed data in the to-be-synchronized data table comprises: querying the data in the CD table in the interval from the LSN segment start value to the LSN segment end value in the breakpoint information, and generating a changed data table; traversing the data in the changed data table; sequentially analyzing the data rows in the changed data table whose data row sequence numbers are greater than the changed target data row sequence number in the breakpoint information; assigning the sequence number of the analyzed data row to the changed target data row sequence number in the breakpoint information; adding the analyzed data to the parsed row data array; and after the traversal ends, submitting the remaining data in the parsed row data array to a queue and performing a data change operation.

[0101] By the method, the inconsistency caused by separate storage of the breakpoint and the data is avoided, the orderliness of the event data of the multiple CD tables is ensured, and the processing position of the breakpoint is recorded through the intentSeq, so that the breakpoint can be submitted when any row is processed.

[0102] The embodiment also provides a machine readable storage medium and a computer device. Figure 4 FIG. 1 is a schematic diagram of a machine readable storage medium 401 according to an embodiment of the present application, Figure 5 FIG. 2 is a schematic diagram of a computer device 503 according to an embodiment of the present application.

[0103] The machine readable storage medium 401 has a machine executable program 402 stored thereon, and the machine executable program 402 is executed by a processor to implement the database synchronization data processing method of any of the above embodiments.

[0104] The computer device 503 can include a memory 501, a processor 502, and a machine executable program 402 stored in the memory 501 and running on the processor 502, and the processor 502 implements the database synchronization data processing method of any of the above embodiments when executing the machine executable program 402.

[0105] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, such as the loop traversing the to-be-synchronized data table set, can be embodied in any machine readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- based system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination thereof.

[0106] For the description of this embodiment, the machine-readable storage medium 401 can be any device that can contain, store, communicate, propagate or transport the program for use by or in connection with the instruction execution system, apparatus or device. More specific examples (non-exhaustive list) of the machine-readable storage medium 401 include the following: an electrical connection having one or more wires (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the machine-readable storage medium 401 can even be paper or other suitable medium on which the program can be printed, as the program can be electronically obtained, for example by optical scanning of the paper or other medium, followed by electronic conversion into a useable form, and then stored in a computer memory.

[0107] It should be understood that parts of the application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, a number of steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system.

[0108] The computer device 503 can be, for example, a server, a desktop computer, a notebook computer, a tablet computer or a smartphone. In some examples, the computer device 503 can be a cloud computing node. The computer device 503 can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer device 503 can be implemented in a distributed cloud computing environment in which remote processing devices are linked through a communication network. In a distributed cloud computing environment, program modules can be located in local or remote computer system storage media including storage devices.

[0109] The computer device 503 can include a processor 502 suitable for executing stored instructions, a memory 501 providing temporary storage space for the operation of the instructions during operation. The processor 502 can be a single-core processor, a multi-core processor, a computing cluster or any number of other configurations. The memory 501 can include random access memory (RAM), read-only memory, flash memory or any other suitable storage system.

[0110] The processor 502 can be connected through the system interconnect (e.g., PCI, PCI-Express, etc.) to an I / O interface (input / output interface) adapted to connect the computer device 503 to one or more I / O devices (input / output devices). The I / O devices can include, for example, a keyboard and a pointing device, where the pointing device can include a touchpad or a touchscreen, etc. The I / O devices can be built-in components of the computer device 503, or can be devices externally connected to the computer device.

[0111] The processor 502 can also be linked through the system interconnect to a display interface adapted to connect the computer device 503 to a display device. The display device can include a display screen that is a built-in component of the computer device 503. The display device can also include a computer monitor, a television, or a projector, etc. that is externally connected to the computer device 503. In addition, a network interface controller (NIC) can be adapted to connect the computer device 503 to a network through the system interconnect. In some embodiments, the NIC can use any suitable interface or protocol (such as Internet Small Computer System Interface, etc.) to transfer data. The network can be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, etc. Remote devices can be connected to the computer device through the network.

[0112] The flowcharts provided in this embodiment are not intended to indicate that the operations of the methods should be executed in any particular order, or that all of the operations of the methods should be included in every implementation. In addition, the methods can include additional operations. Additional changes can be made to the above-described methods within the scope of the technical ideas provided by the methods of this embodiment.

[0113] At this point, those skilled in the art will recognize that the present application has been described herein with reference to a number of illustrative embodiments and implementations, and that the generic principles and features can be employed to impart the spirit of the application, and with the scope of the application being accorded the broadest scope of which it is entitled to under the law.

Claims

1. A method for processing database synchronization data, comprising: The database parses transaction data and saves breakpoint information to each parsed transaction data entry; The breakpoint information is obtained from the transaction data, and the set of data tables to be synchronized is found based on the breakpoint information and the LSN segment. By using the breakpoint information and the set of data tables to be synchronized, the change operation information of the changed target data row in the CD table is found and synchronized. The step of searching for the change operation information of the changed target data row in the CD table using the breakpoint information and the set of data tables to be synchronized, and then synchronizing it, includes: traversing the set of data tables to be synchronized; and processing the changed data in the set of data tables to be synchronized one by one. The breakpoint information also includes: a unique identifier for the source table; The step of processing the changed data in the set of data tables to be synchronized one by one includes: determining whether the unique identifier of the source table of the data table to be synchronized in the set of data tables to be synchronized is greater than or equal to the unique identifier of the source table in the breakpoint information; if the unique identifier of the source table of the data table to be synchronized is greater than or equal to the unique identifier of the source table in the breakpoint information, then querying and processing the changed data in the data table to be synchronized; and setting the sequence number of the changed target data row in the breakpoint information to null.

2. The method for processing database synchronization data according to claim 1, wherein, The breakpoint information includes: LSN segment start value, LSN segment end value, and the sequence number of the changed target data row; The step of finding the set of data tables to be synchronized based on the breakpoint information and LSN segments includes: Obtain the maximum LSN value in the LSN segment; Determine whether the maximum value of the LSN is empty; If the maximum value of LSN is not empty, determine whether the starting value of the LSN segment in the breakpoint information is empty; If the starting value of the LSN segment is not empty, determine whether the starting value of the LSN segment is less than the maximum value of the LSN. If the starting value of the LSN segment is less than the maximum value of the LSN, determine whether the ending value of the LSN segment in the breakpoint information is empty; If the end value of the LSN segment is not empty, query the log information from the start value of the LSN segment to the end value of the LSN segment to obtain the set of data tables to be synchronized.

3. The method for processing database synchronization data according to claim 1, wherein, The breakpoint information includes: LSN segment start value, LSN segment end value, and the sequence number of the changed target data row; The steps of querying and processing changed data in the set of data tables to be synchronized include: Query the data in the CD table that is located between the start value and the end value of the LSN segment in the breakpoint information, and generate a change data table; Iterate through the data in the changed data table; Analyze each data row in the changed data table whose data row sequence number is greater than the changed target data row sequence number in the breakpoint information; Assign the sequence number of the parsed data row to the sequence number of the changed target data row in the breakpoint information; Add the parsed data to the parsed row data array; After the traversal is complete, the remaining data in the parsed row data array is submitted to the queue and a data modification operation is performed.

4. The method for processing database synchronization data according to claim 3, wherein, The step of adding the parsed data to the parsed row data array also includes: If the LSN segment changes or the size of the parsed row data array is equal to the preset number of transaction shards, the parsed row data array is submitted to the queue and a data change operation is performed. Clear the parsed row data array.

5. The method for processing database synchronization data according to claim 2, wherein, After the step of determining whether the starting value of the LSN segment in the breakpoint information is empty, the method further includes: if the starting value of the LSN segment is empty, assigning the maximum value of the LSN to the starting value of the LSN segment; The step of determining whether the LSN segment end value in the breakpoint information is empty further includes: if the LSN segment end value is empty, assigning the maximum value of LSN to the LSN segment end value.

6. The method for processing database synchronization data according to claim 2, wherein, The step of searching for the change operation information of the modified target data row in the CD table using the breakpoint information and the set of data tables to be synchronized, and then synchronizing it, includes: Assign the end value of the LSN segment in the breakpoint information to the start value of the LSN segment; Set the end value of the LSN segment to empty; The unique identifier of the source table in the breakpoint information is set to a set value; Clean up the data in the CD table.

7. A machine-readable storage medium having a machine-executable program stored thereon, wherein the machine-executable program, when executed by a processor, implements the database synchronization data processing method according to any one of claims 1 to 6.

8. A computer device comprising a memory, a processor, and a machine-executable program stored in the memory and running on the processor, wherein the processor, when executing the machine-executable program, implements the database synchronization data processing method according to any one of claims 1 to 6.

Images