A CDC database log synchronization method, device, equipment and medium

Abstract

The embodiment of the specification discloses a CDC database log synchronization method, which is applied to the technical field of database synchronization and is used for solving the problems of low transmission efficiency and poor real-time performance of current CDC data, and the method comprises the following steps: loading CDC database logs in a source database into memory; performing block processing on the CDC database logs in the memory to obtain a block information file and CDC database log file blocks; transmitting the block information file to a current target database; if the transmission of the block information file is completed, transmitting each CDC database log file block to the current target database; and checking each CDC database log file block in the current target database based on the block information file to determine whether to retransmit the CDC database log file block.

Description

Technical Field

[0001] This specification relates to the field of data synchronization technology, and in particular to a method, apparatus, device and medium for synchronizing CDC database logs. Background Technology

[0002] Change Data Capture (CDC) is a data processing technique used to capture changed data in a data source and generate corresponding change data records. With the continuous increase in user data volume and demands, more and more platforms, including transactional databases and analytical databases, contain a single database system. When data in a database changes due to operations such as insertion, deletion, and updates, it is necessary to synchronize the data with other databases that also use that data. This ensures data synchronization and consistency across the platform and avoids processing errors caused by data inconsistencies.

[0003] Currently, when processing CDC database logs using existing CDC database synchronization systems, although CDC technology can synchronize data changes to the target system almost in real time, data synchronization may be affected by factors such as network latency and system load in certain situations, leading to a reduction in real-time performance. In complex data environments, data synchronization may face various challenges, such as data conflicts and data loss. These problems may lead to damage to data integrity during data synchronization transmission, and the inability to determine integrity results in poor data synchronization performance. Furthermore, although the database logs corresponding to CDC technology have achieved data dimensionality reduction, for large amounts of data in complex systems, batch data synchronization based on CDC database logs may suffer from low data transmission and synchronization efficiency. Summary of the Invention

[0004] To address the aforementioned technical problems, this specification provides one or more embodiments of a CDC database log synchronization method, apparatus, device, and medium.

[0005] One or more embodiments of this specification employ the following technical solutions:

[0006] This specification provides one or more embodiments of a method for synchronizing CDC database logs, the method including:

[0007] The CDC database logs from the source database are then loaded into memory.

[0008] The CDC database log in memory is segmented into blocks to obtain the segmentation information file and the CDC database log file blocks;

[0009] The block information file is transferred to the current target database. If the block information file transfer is complete, each block of the CDC database log file is transferred to the current target database.

[0010] The CDC database log file blocks in the current target database are verified based on the block information file to determine whether to retransmit the CDC database log file blocks.

[0011] Optionally, in one or more embodiments of this specification, the CDC database log in memory is segmented into blocks to obtain a segmentation information file and CDC database log file blocks, specifically including:

[0012] Obtain the data source corresponding to each changed data in the CDC database log, and extract sensitive data in the CDC database log based on the data source;

[0013] The CDC database log is divided into blocks according to the preset file block configuration information to obtain multiple CDC database log file blocks.

[0014] Determine whether each of the CDC database log file blocks contains the sensitive data, and determine the permission information of each of the CDC database log file blocks based on the type of the sensitive data;

[0015] Obtain the basic block information of each CDC database log file block, and summarize the basic block information and permission information of each CDC database log file block to generate a block information file of the CDC database log; wherein, the basic block information includes at least: the file block size and the file block data volume of the CDC database log file block.

[0016] Optionally, in one or more embodiments of this specification, extracting sensitive data from the CDC database logs based on the data source specifically includes:

[0017] Based on the data source, determine the data type corresponding to each of the changed data, and then determine the suspected sensitive data in the CDC database log based on the preset sensitive data type;

[0018] Obtain the regular expressions of sensitive data from each of the data sources, and judge the suspected sensitive data one by one based on the regular expressions of the sensitive data.

[0019] If the suspected sensitive data is determined to be sensitive data, then the sensitive data is extracted and recorded.

[0020] Optionally, in one or more embodiments of this specification, before transferring each of the CDC database log file blocks to the current target database, the method further includes:

[0021] Based on whether the CDC database log file block contains sensitive data, the CDC database log file is divided into a first CDC database log file block and a second CDC database log file block.

[0022] The role information corresponding to each of the first CDC database log file blocks is determined based on the permission information of each first CDC database log file block.

[0023] The first CDC database log file block is encrypted based on the role information and the public key of the source database to obtain an encrypted file block;

[0024] The network channel corresponding to the source database and the current target database is determined, and each second CDC database log file block and the encrypted file block are transmitted in parallel to the current target database based on the network channel.

[0025] Optionally, in one or more embodiments of this specification, verifying each CDC database log file block in the current target database based on the block information file to determine whether to retransmit the CDC database log file block specifically includes:

[0026] Integrity verification is performed on each block of the CDC database log file in the current target database based on the block information file;

[0027] If it is determined based on the block information file that the CDC database log file block is consistent with the block information in the block information file, then the CDC database log file block is determined to be complete.

[0028] If it is determined based on the block information file that the CDC database log file block is inconsistent with the block information in the block information file, then the CDC database log file block is retransmitted.

[0029] Optionally, in one or more embodiments of this specification, after verifying each CDC database log file block in the current target database based on the block information file to determine whether to retransmit the CDC database log file block, the method further includes:

[0030] Based on the block information file, determine whether each block of the CDC database log file corresponding to the CDC database log has been transmitted.

[0031] If so, the partitioning order of each CDC database log file block is determined based on the partitioning information file;

[0032] Based on the division order, the blocks of each CDC database log file are concatenated to form a CDC database log file to be stored.

[0033] Optionally, in one or more embodiments of this specification, after concatenating the CDC database log file blocks based on the partitioning order to synthesize and store the CDC database log file to be stored, the method further includes:

[0034] If it is determined that the current target database has already stored the CDC database log file to be stored, then based on the id information of the CDC database log file, a file block deletion instruction for the source database and a file block deletion instruction for the current target database are generated;

[0035] Send corresponding file block deletion instructions to the source database and the current target database respectively to delete each CDC database log file block corresponding to the CDC database log.

[0036] This specification provides one or more embodiments of a CDC database log synchronization device, the device comprising:

[0037] The loading unit is used to load the CDC database logs from the source database into memory;

[0038] The segmentation unit is used to segment the CDC database log in memory into blocks and obtain the segmentation information file and CDC database log file blocks.

[0039] The transmission unit is used to transmit the block information file to the current target database. If the transmission of the block information file is completed, each block of the CDC database log file is transmitted to the current target database.

[0040] The verification unit is used to verify each of the CDC database log file blocks in the current target database based on the block information file, so as to determine whether to retransmit the CDC database log file blocks.

[0041] This specification provides one or more embodiments of a CDC database log synchronization device, comprising:

[0042] At least one processor; and,

[0043] A memory communicatively connected to the at least one processor; wherein,

[0044] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0045] The CDC database logs from the source database are then loaded into memory.

[0046] The CDC database log in memory is segmented into blocks to obtain the segmentation information file and the CDC database log file blocks;

[0047] The block information file is transferred to the current target database. If the block information file transfer is complete, each block of the CDC database log file is transferred to the current target database.

[0048] The CDC database log file blocks in the current target database are verified based on the block information file to determine whether to retransmit the CDC database log file blocks.

[0049] This specification provides one or more embodiments of a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0050] The CDC database logs from the source database are then loaded into memory.

[0051] The CDC database log in memory is segmented into blocks to obtain the segmentation information file and the CDC database log file blocks;

[0052] The block information file is transferred to the current target database. If the block information file transfer is complete, each block of the CDC database log file is transferred to the current target database.

[0053] The CDC database log file blocks in the current target database are verified based on the block information file to determine whether to retransmit the CDC database log file blocks.

[0054] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0055] By segmenting the CDC database logs, subsequent synchronization is performed in small CDC database log file blocks. This reduces the possibility of data congestion and delays caused by network latency, system load, and other factors during data synchronization, thus ensuring the reliability and stability of data transmission. Before transmitting the CDC database logs, the segmented information file is transmitted to the current target database. Integrity verification of each CDC database log file block is then performed based on this segmented information file, ensuring the integrity of the data synchronization process and preventing data corruption or loss during transmission that could lead to unreliable data synchronization. The file block deletion command removes the synchronized CDC database log file blocks from both the source and target databases. This ensures efficient use of database storage space and prevents the erroneous reuse of processed CDC database log file blocks in subsequent operations. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0057] Figure 1 A schematic flowchart illustrating a CDC database log synchronization method provided in this embodiment of the specification;

[0058] Figure 2 This is a schematic diagram of the internal structure of a CDC database log synchronization device provided in an embodiment of this specification;

[0059] Figure 3 This is a schematic diagram of the internal structure of a CDC database log synchronization device provided in an embodiment of this specification;

[0060] Figure 4 This is a schematic diagram of the internal structure of a non-volatile storage medium provided in the embodiments of this specification. Detailed Implementation

[0061] This specification provides a method, apparatus, device, and medium for synchronizing CDC database logs.

[0062] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0063] like Figure 1 As shown in the diagram, this specification provides a flowchart illustrating a method for synchronizing CDC database logs. Figure 1 As can be seen, one or more embodiments of this specification provide a method for synchronizing CDC database logs, the method including:

[0064] S101: Obtain the CDC database logs from the source database and load the CDC database logs into memory.

[0065] To ensure the normal operation of the system by synchronizing data in the database, this embodiment first obtains the CDC database log from the source database (the database from which the synchronization data to be transmitted) and loads the CDC database log into memory for subsequent processing. It should be noted that the CDC database log is a log file that records database change information.

[0066] S102: Perform block processing on the CDC database log in the memory, and obtain the block information file and CDC database log file blocks.

[0067] To enhance the reliability and stability of data transmission while improving its efficiency, this embodiment of the specification performs block processing on the CDC database log in memory obtained in step S101, thereby obtaining the block information file and CDC database log file blocks corresponding to the CDC database log. It can be understood that by segmenting the CDC database log, subsequent transmission and synchronization of the CDC database log in the form of small CDC database log file blocks reduces the possibility of data congestion and stalling caused by factors such as network latency and system load during data synchronization, thus ensuring the reliability and stability of data transmission during data synchronization.

[0068] Specifically, in one or more embodiments of this specification, the CDC database log in memory is segmented into blocks to obtain segmentation information files and CDC database log file blocks, which specifically includes the following process:

[0069] First, because files in the database may contain sensitive information, and different databases correspond to different administrators and users, to avoid the leakage of sensitive information during data synchronization, this embodiment obtains the data source corresponding to each changed data in the CDC database log, and then extracts the sensitive data in the CDC database log based on the data source. Then, to improve data transmission efficiency and reduce data lag, the CDC database log is divided into blocks according to preset file block configuration information, resulting in multiple CDC database log file blocks. Next, it is determined whether each CDC database log file block contains sensitive data, and then the permission information of each CDC database log file block is determined based on the type of sensitive data. Then, the basic block information of each CDC database log file block is obtained, and the basic block information and permission information of each CDC database log file block are summarized to generate a CDC database log block information file. It should be noted that the basic block information includes at least: the file block size and the file block data volume of the CDC database log file block.

[0070] Furthermore, in one or more embodiments of this specification, the process of extracting sensitive data from the CDC database log based on the data source specifically includes the following implementation steps: First, the data type corresponding to each changed data is determined according to the data source, thereby identifying suspected sensitive data in the CDC database log based on a preset sensitive data type. Then, the regular expression of the sensitive data in each data source is obtained, and the suspected sensitive data is compared and judged one by one according to the regular expression of the sensitive data. If it is determined that the suspected sensitive data matches the regular expression of the sensitive data, the suspected sensitive data is extracted and recorded as sensitive data in the CDC database log.

[0071] S103: Transmit the block information file to the current target database. If the transmission of the block information file is complete, then transmit each block of the CDC database log file to the current target database.

[0072] After completing the block processing of the CDC database logs in the source database based on the above step S102, in order to facilitate the verification of the integrity of the CDC database log file blocks during data transmission, in this embodiment of the specification, the block information file is first transmitted to the current target database before transmitting the CDC database logs. If the block information file transmission is complete, then each CDC database log file block is simultaneously transmitted to the current target database for data synchronization. It should be noted that the current target database is the target database that needs to receive CDC database logs for data synchronization.

[0073] Furthermore, in one or more embodiments of this specification, before transferring each CDC database log file block to the current target database, in order to ensure the reliability of data transmission and avoid the leakage of sensitive information during data transmission, the method further includes the following process:

[0074] First, the CDC database log file is divided into first CDC database log file blocks containing sensitive data and second CDC database log file blocks not containing sensitive data, based on whether the CDC database log file blocks contain sensitive data. Then, role information corresponding to each first CDC database log file block is determined based on its permission information. Next, the first CDC database log file blocks are encrypted using the role information and the source database's public key to obtain encrypted file blocks. Finally, the network channel corresponding to the source database and the current target database is determined, and the second CDC database log file blocks and the encrypted file blocks are transmitted in parallel to the current target database through the network channel. This parallel transmission of CDC database log file blocks improves the transmission efficiency of data requiring synchronization.

[0075] S104: Verify each CDC database log file block in the current target database based on the block information file to determine whether to retransmit the CDC database log file block.

[0076] Based on the above-described S103 process of transmitting the block information file and CDC database log file block to the current target file block, this embodiment of the specification verifies each of the CDC database log file blocks in the current target database through the block information file to determine the integrity of the data and whether the CDC database log file block needs to be retransmitted. By verifying the integrity of the file blocks, it is convenient to retransmit incomplete or missing CDC database log file blocks, ensuring the integrity rate of the data and effectively preventing data corruption or data loss during data transmission.

[0077] Specifically, in one or more embodiments of this specification, verifying each CDC database log file block in the current target database based on the chunk information file to determine whether to retransmit the CDC database log file block includes the following steps: performing an integrity verification on each of the CDC database log file blocks in the current target database based on the chunk information file. If it is determined according to the chunk information file that the CDC database log file block is consistent with the chunk information file, then it can be determined that the CDC database log file block is complete. If it is determined according to the chunk information file that the CDC database log file block is inconsistent with the chunk information file, then the CDC database log file block is retransmitted.

[0078] Furthermore, in one or more embodiments of this specification, after verifying each CDC database log file block in the current target database based on the block information file to determine whether to retransmit the CDC database log file block, the method further includes: based on the block information file received in the current target database during the above process, determining whether each CDC database log file block corresponding to the CDC database log has been transmitted. If the transmission has been completed, then the partitioning order of each CDC database log file block is determined according to the obtained block information file. This facilitates the splicing of each CDC database log file block according to the partitioning order to synthesize and store the CDC database log file to be stored.

[0079] Furthermore, in one or more embodiments of this specification, in order to ensure that the storage space of the current target database and the source database can be reasonably utilized, after concatenating the blocks of each CDC database log file according to the partitioning order to synthesize the CDC database log file to be stored, the method further includes the following process:

[0080] If it is determined that the target database already stores the CDC database log file to be stored, then based on the CDC database log file ID information, file block deletion instructions for both the source database and the target database are generated. The corresponding file block deletion instructions are then sent to both the source and target databases to delete the respective CDC database log file blocks. Deleting the synchronized CDC database log file blocks in both the source and target databases via file block deletion instructions ensures efficient use of database storage space and prevents the erroneous reuse of processed CDC database log file blocks in subsequent operations, thus improving database reliability.

[0081] In a specific application scenario described in this manual, the above method is implemented in the CDC database log synchronization scenario through the following steps:

[0082] Step S1: Connect to the source database, then read the CDC database log files in the source database, and load the data from these CDC database log files into memory for processing.

[0083] Step S2: Generate file block information based on preset file block configuration information, such as file block size. Perform block or composite processing on the CDC database logs loaded into memory in step S1, writing each file block to a separate file. Record relevant information for each file block in another file for subsequent file block integrity verification.

[0084] Step S3: Transfer the file block information file. The file block information file generated in S2 is transferred to the target database through an appropriate network channel.

[0085] Step S4: Transfer file blocks. After the file block information file from step S3 is transferred to the target library, all the segmented file block files are transferred to the target library using an appropriate parallel transfer method. The order of the file blocks is irrelevant during this process.

[0086] Step S5: Check the integrity of the file block file. After the file blocks are transferred to the target library, check each file block using the file block information file transferred in the previous step S3. If any file block is found to be incomplete or missing, retransmit it.

[0087] Step S6: Synthesize CDC database logs. After all file blocks have been successfully transferred and checked, use these file blocks and file block information files to synthesize CDC database log files in the target database.

[0088] Step S7: Send a command to the source library to delete all file blocks. After ensuring that the target library has successfully received and processed all file blocks, send a command to the source library to delete all transmitted file blocks, thereby ensuring that the storage space of the source library is not occupied by useless data blocks.

[0089] Step S8: Send a command to the target library to delete all file blocks. Simultaneously, send a command to the target library to delete all received and processed file blocks. This ensures that the target library's storage space is used efficiently and prevents processed data blocks from being incorrectly reused in subsequent operations.

[0090] like Figure 2 As shown in the figure, this specification provides a schematic diagram of the internal structure of a CDC database log synchronization device in an embodiment. Figure 2 As can be seen from one or more embodiments of this specification, a CDC database log synchronization device includes:

[0091] Loading unit 201 is used to load the CDC database logs from the source database into memory;

[0092] Segmentation unit 202 is used to segment the CDC database log in memory into blocks and obtain segmentation information file and CDC database log file blocks;

[0093] Transmission unit 203 is used to transmit the block information file to the current target database, and if the transmission of the block information file is completed, to transmit each block of the CDC database log file to the current target database;

[0094] The verification unit 204 is used to verify each of the CDC database log file blocks in the current target database based on the block information file, so as to determine whether to retransmit the CDC database log file blocks.

[0095] like Figure 3 As shown in the figure, this specification provides a schematic diagram of the internal structure of a CDC database log synchronization device in an embodiment. Figure 3 As can be seen from one or more embodiments of this specification, a CDC database log synchronization device includes:

[0096] At least one processor; and,

[0097] A memory communicatively connected to the at least one processor; wherein,

[0098] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0099] The CDC database logs from the source database are then loaded into memory.

[0100] The CDC database log in memory is segmented into blocks to obtain the segmentation information file and the CDC database log file blocks;

[0101] The block information file is transferred to the current target database. If the block information file transfer is complete, each block of the CDC database log file is transferred to the current target database.

[0102] The CDC database log file blocks in the current target database are verified based on the block information file to determine whether to retransmit the CDC database log file blocks.

[0103] like Figure 4 As shown in the figure, this specification provides a schematic diagram of the internal structure of a non-volatile storage medium in an embodiment. Figure 4 As can be seen, one or more embodiments of this specification provide a non-volatile storage medium storing computer-executable instructions 401, which are capable of:

[0104] The CDC database logs from the source database are then loaded into memory.

[0105] The CDC database log in memory is segmented into blocks to obtain the segmentation information file and the CDC database log file blocks;

[0106] The block information file is transferred to the current target database. If the block information file transfer is complete, each block of the CDC database log file is transferred to the current target database.

[0107] The CDC database log file blocks in the current target database are verified based on the block information file to determine whether to retransmit the CDC database log file blocks.

[0108] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0109] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0110] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for synchronizing CDC database logs, characterized in that, The method includes: Obtain the CDC database logs from the source database and load the CDC database logs into memory; The CDC database log in memory is segmented into blocks to obtain the segmentation information file and the CDC database log file blocks; The block information file is transferred to the current target database. If the block information file transfer is complete, each block of the CDC database log file is transferred to the current target database. Based on the block information file, each block of the CDC database log file in the current target database is verified to determine whether to retransmit the CDC database log file block. The CDC database log in memory is segmented into blocks to obtain segmentation information files and CDC database log file blocks, specifically including: Obtain the data source corresponding to each changed data in the CDC database log, and extract sensitive data in the CDC database log based on the data source; The CDC database log is divided into blocks according to the preset file block configuration information to obtain multiple CDC database log file blocks. Determine whether each of the CDC database log file blocks contains the sensitive data, and determine the permission information of each of the CDC database log file blocks based on the type of the sensitive data; Obtain the basic block information of each CDC database log file block, and summarize the basic block information and permission information of each CDC database log file block to generate a block information file of the CDC database log; wherein, the basic block information includes at least: the file block size and the file block data volume of the CDC database log file block.

2. The method for synchronizing CDC database logs according to claim 1, characterized in that, Sensitive data is extracted from the CDC database logs based on the aforementioned data source, specifically including: Based on the data source, determine the data type corresponding to each of the changed data, and then determine the suspected sensitive data in the CDC database log based on the preset sensitive data type; Obtain the regular expressions of sensitive data from each of the data sources, and judge the suspected sensitive data one by one based on the regular expressions of the sensitive data. If the suspected sensitive data is determined to be sensitive data, then the sensitive data is extracted and recorded.

3. The method for synchronizing CDC database logs according to claim 1, characterized in that, Before transferring each of the CDC database log file blocks to the current target database, the method further includes: Based on whether the CDC database log file block contains sensitive data, the CDC database log file is divided into a first CDC database log file block and a second CDC database log file block. The role information corresponding to each of the first CDC database log file blocks is determined based on the permission information of each first CDC database log file block. The first CDC database log file block is encrypted based on the role information and the public key of the source database to obtain an encrypted file block; The network channel corresponding to the source database and the current target database is determined, and each second CDC database log file block and the encrypted file block are transmitted in parallel to the current target database based on the network channel.

4. The method for synchronizing CDC database logs according to claim 1, characterized in that, Based on the segmentation information file, each CDC database log file block in the current target database is verified to determine whether to retransmit the CDC database log file block. Specifically, this includes: Integrity verification is performed on each block of the CDC database log file in the current target database based on the block information file; If it is determined based on the block information file that the CDC database log file block is consistent with the block information in the block information file, then the CDC database log file block is determined to be complete. If it is determined based on the block information file that the CDC database log file block is inconsistent with the block information in the block information file, then the CDC database log file block is retransmitted.

5. The method for synchronizing CDC database logs according to claim 1, characterized in that, After verifying each CDC database log file block in the current target database based on the block information file to determine whether to retransmit the CDC database log file block, the method further includes: Based on the block information file, determine whether each block of the CDC database log file corresponding to the CDC database log has been transmitted. If so, the partitioning order of each CDC database log file block is determined based on the partitioning information file; Based on the division order, the blocks of each CDC database log file are concatenated to form a CDC database log file to be stored.

6. The method for synchronizing CDC database logs according to claim 5, characterized in that, After concatenating the CDC database log file blocks according to the aforementioned partitioning order to synthesize a CDC database log file for storage, the method further includes: If it is determined that the current target database has already stored the CDC database log file to be stored, then based on the id information of the CDC database log file, a file block deletion instruction for the source database and a file block deletion instruction for the current target database are generated; Send corresponding file block deletion instructions to the source database and the current target database respectively to delete each CDC database log file block corresponding to the CDC database log.

7. A synchronization device for CDC database logs, characterized in that, The device includes: The loading unit is used to obtain the CDC database logs from the source database and load the CDC database logs into memory; The segmentation unit is used to segment the CDC database log in memory into blocks and obtain the segmentation information file and CDC database log file blocks. The transmission unit is used to transmit the block information file to the current target database. If the transmission of the block information file is completed, each block of the CDC database log file is transmitted to the current target database. The verification unit is used to verify each of the CDC database log file blocks in the current target database based on the block information file, so as to determine whether to retransmit the CDC database log file blocks. The CDC database log in memory is segmented into blocks to obtain segmentation information files and CDC database log file blocks, specifically including: Obtain the data source corresponding to each changed data in the CDC database log, and extract sensitive data in the CDC database log based on the data source; The CDC database log is divided into blocks according to the preset file block configuration information to obtain multiple CDC database log file blocks. Determine whether each of the CDC database log file blocks contains the sensitive data, and determine the permission information of each of the CDC database log file blocks based on the type of the sensitive data; Obtain the basic block information of each CDC database log file block, and summarize the basic block information and permission information of each CDC database log file block to generate a block information file of the CDC database log; wherein, the basic block information includes at least: the file block size and the file block data volume of the CDC database log file block.

8. A synchronization device for CDC database logs, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Obtain the CDC database logs from the source database and load the CDC database logs into memory; The CDC database log in memory is segmented into blocks to obtain the segmentation information file and the CDC database log file blocks; The block information file is transferred to the current target database. If the block information file transfer is complete, each block of the CDC database log file is transferred to the current target database. Based on the block information file, each block of the CDC database log file in the current target database is verified to determine whether to retransmit the CDC database log file block. The CDC database log in memory is segmented into blocks to obtain segmentation information files and CDC database log file blocks, specifically including: Obtain the data source corresponding to each changed data in the CDC database log, and extract sensitive data in the CDC database log based on the data source; The CDC database log is divided into blocks according to the preset file block configuration information to obtain multiple CDC database log file blocks. Determine whether each of the CDC database log file blocks contains the sensitive data, and determine the permission information of each of the CDC database log file blocks based on the type of the sensitive data; Obtain the basic block information of each CDC database log file block, and summarize the basic block information and permission information of each CDC database log file block to generate a block information file of the CDC database log; wherein, the basic block information includes at least: the file block size and the file block data volume of the CDC database log file block.

9. A non-volatile storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are capable of: Obtain the CDC database logs from the source database and load the CDC database logs into memory; The CDC database log in memory is segmented into blocks to obtain the segmentation information file and the CDC database log file blocks; The block information file is transferred to the current target database. If the block information file transfer is complete, each block of the CDC database log file is transferred to the current target database. Based on the block information file, each block of the CDC database log file in the current target database is verified to determine whether to retransmit the CDC database log file block. The CDC database log in memory is segmented into blocks to obtain segmentation information files and CDC database log file blocks, specifically including: Obtain the data source corresponding to each changed data in the CDC database log, and extract sensitive data in the CDC database log based on the data source; The CDC database log is divided into blocks according to the preset file block configuration information to obtain multiple CDC database log file blocks. Determine whether each of the CDC database log file blocks contains the sensitive data, and determine the permission information of each of the CDC database log file blocks based on the type of the sensitive data; Obtain the basic block information of each CDC database log file block, and summarize the basic block information and permission information of each CDC database log file block to generate a block information file of the CDC database log; wherein, the basic block information includes at least: the file block size and the file block data volume of the CDC database log file block.

Images