Method and computer program for tracking change data capture log history or CDC log history (tracking change data capture log history)

Abstract

To provide a method that includes obtaining a first snapshot of a source system and deriving a set of key-value pairs reflecting the first snapshot.SOLUTION: A mirror operation of the source system is performed to obtain CDC change operations. The method obtains a first CDC log as a first sequence of key-value pairs. A second snapshot of the source system is obtained and a set of key-value pairs is derived, which reflects the second snapshot. The first sequence of key-value pairs is compared with the set of key-value pairs to derive corrective CDC operations, and captured as a set of key-value pairs. The corrective CDC operations represent corrections to be performed with respect to the first sequence of key-value pairs. A second CDC log is obtained as a second sequence of key-value pairs. The corrective CDC operations ensure that the second sequence of key-value pairs are coherent with the set of key-value pairs.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates generally to a computerized method and computer program product for tracking change data capture (CDC) log history, and more particularly to a method that relies on modified CDC operations to generate a consistent CDC log. Summary of the Invention [Problem to be solved by the invention]

[0002] Most corporate data is stored in relational data warehouses, where it is typically updated, processed, and queried to generate actionable information about, for example, a company's operations. Data lakes allow data from many different sources to be combined so that additional value can be extracted from such data. For example, combining weather data and supply chain data can lead to predictions about potential risks in those supply chains. Therefore, it is interesting to copy relevant data from multiple different sources to many different storage and processing systems, ideally in real time. In a hybrid cloud, such systems would run both on the company's private cloud and on one or more cloud vendors' public clouds. For example, a company's sales data may be stored in a transactional system located on the company's premises and also copied to a public cloud where analytical processing can generate sales recommendations.

[0003] In change data capture (CDC) systems, only data that has actually changed in a source system is updated in a target system. A key challenge in these systems is identifying which portions of a dataset have changed. In relational database systems, this can be achieved efficiently by examining transaction logs.

[0004] Typically, a CDC system first performs an initial refresh / snapshot of the table to a messaging system such as a Kafka topic (or MQ queue), and then all subsequent changes are read from the change log and propagated to the topic. All changes, including the initial refresh, are stored as separate messages. The CDC system can ensure coherence between these two independent operations by noting the operation for which the first refresh was performed and ensuring that all operations performed after that refresh are accurately captured. Reading this topic then allows a replica of the source database to be created in the target system. Data in the target database is defined as coherent with respect to the data in the source database when the state in the target system represents some valid state of the source system. In contrast to uniqueness, which is a requirement for all databases, coherence pertains more specifically to replica databases. [Means for solving the problem]

[0005] According to a first aspect, the present invention is embodied as a method for tracking change data capture log history, or CDC log history. First, a first snapshot of a source system is taken, and a set of key-value pairs S1 reflecting the first snapshot is derived. Next, a mirror operation of the source system is performed, and CDC change operations are correspondingly obtained. The CDC change operations represent changes to be performed on the set of key-value pairs S1. Such operations result in a change data capture log history, or CDC log history. M Then the first CDC log is captured as a first sequence of key-value pairs S A and they are obtained as set S1 and set S MAlso, a second snapshot of the source system is taken (after the first snapshot is taken), and a set of key-value pairs S2 is derived, which reflects the second snapshot. A is then compared with the set of key-value pairs S2 to derive a modifying CDC operation, which is captured as a set of key-value pairs S3. A Finally, the second CDC log represents a second sequence of key-value pairs, S B and it is obtained as a sequence S A and set S3. The modify CDC operation creates a second sequence of key-value pairs S B is coherent with the set of key-value pairs S2 as a whole.

[0006] Preferably, the method further comprises interpreting the second sequence of key-value pairs to modify the current state of the target system so as to reach a target state that is coherent with the state of the source system at the time the second snapshot is taken.

[0007] According to another aspect, the present invention is embodied as a computer program product for tracking CDC log history, the computer program product including a computer-readable storage medium having program instructions embodied thereon, the program instructions being executable by a processing means to cause the steps of the method to be performed.

[0008] A computer controlled method and a computer program product embodying the present invention will now be described, by way of non-limiting example, and with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0009] In the accompanying drawings, like reference characters refer to identical or functionally similar elements throughout the individual figures, and together with the following detailed description, which is incorporated into and forms a part of this specification, serve to further illustrate various embodiments and explain various principles and all advantages of the present disclosure.

[0010] [Figure 1] 1 illustrates a schematic representation of a CDC system interacting with both a source database system and a target database system, as in an embodiment.

[0011] [Figure 2] FIG. 1 is a diagram showing a CDC log that is typically acquired by a conventional method.

[0012] [Figure 3] FIG. 1 illustrates how a coherent CDC log can be generated according to an embodiment.

[0013] [Figure 4A] FIG. 10 is an additional diagram illustrating a method for generating a coherent CDC log that is compatible with sorted compaction of the CDC log, as in an embodiment. [Figure 4B] FIG. 10 is an additional diagram illustrating a method for generating a coherent CDC log that is compatible with sorted compaction of the CDC log, as in an embodiment.

[0014] [Figure 5] 1 is a flowchart illustrating high-level steps of a method for tracking CDC log history, according to an embodiment.

[0015] [Figure 6] FIG. 1 is a schematic representation of a general-purpose computerized system suitable for implementing one or more method steps as associated with embodiments of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0016] The accompanying drawings show simplified representations of devices or portions thereof, as may be relevant to the embodiments. Like or functionally similar elements in the drawings are assigned the same reference numerals unless otherwise specified. Detailed Description of Embodiments of the Invention

[0017] The first part of the sequence of messages in the topic corresponds to a full snapshot (corresponding to an operation sometimes called a "refresh" or "load"), and the subsequent replication operations are called mirror operations. The CDC system ensures that mirroring starts with the correct operation after the snapshot operation is complete, and thus such an operation can be considered a single, unified operation: a snapshot-mirror operation.

[0018] The second time a snapshot mirror operation is performed on a table, the topic it writes to must be empty. The mirror operation is guaranteed to start after the snapshot is complete, but there are no guarantees of coherence between data that is refreshed multiple times. For example, if a row exists in the table in the first refresh but not the second, it will still exist in the Kafka topic and will never be deleted.

[0019] Therefore, any resulting target table will be incoherent with respect to the source table. This is a consequence of the fact that data in a relational database can be updated in two ways: by performing an operation or by performing a full snapshot. Meanwhile, topics such as Kafka topics will still represent such operations in a similar manner. Note that doing so is a common practice in production systems. For example, databases are periodically backed up and restored on different machines for upgrades, maintenance, etc. The above problem is therefore not merely a theoretical one, but one that enterprise systems must account for. Some database systems, e.g., Microsoft® SQL, do not allow snapshot operations on tables in capture mode, as do some other operations, such as truncate operations.

[0020] This problem is illustrated by a simple example in Figure 1. Suppose a snapshot is taken at time t1. As a result, rows corresponding to keys 1 and 2 (row A, row B) are added. Between times t1 and t2, mirroring occurs, which changes the value of the row for key 1 and adds a new row corresponding to key 3. Then, at time t3, a new snapshot occurs, which causes any data found in the source database to be entered into the topic. In this example, it has keys 3 and 4; that is, keys 1 and 2 no longer exist in the source. However, if created from the values ​​found in the topic, the target database would not be coherent with respect to the source database. In fact, there are no DELETE operations in the log because the rows corresponding to keys 1 and 2 were never explicitly deleted from the system.

[0021] In effect, each snapshot of a table creates a new version of that table, which must be reflected in both the (Kafka) representation of that table and any tables created in the target database that are derived from that representation.

[0022] One solution known in the art is to simply delete the old CDC log and start again. This has the advantage of simplicity, but it also means that any downstream systems that read the log -Recognize that old CDC logs have been deleted, Accurately update any downstream systems, e.g., databases, elastic search indexes, - Switch to the new CDC log, and -Reread all data, It is necessary to do so.

[0023] The inventors have realized and concluded that the last point is particularly problematic, for example, when the source and target systems are separated by a wide area network (WAN, i.e., a network with high latency and low bandwidth) or when the target system does not support the transactional aspect of switching to a new CDC log after new data has been read. For example, when the source system is on an on-premise system but the target system is on a public cloud, the entire table needs to be transferred over the WAN. Now, if the table is very large, which is often the case in practice, this can be very lengthy.

[0024] As described, an approach often used in the art is to simply delete the old CDC log on the target and start again. While this approach is conceptually simple, downstream systems reading the log must specifically re-read all the data, which can be quite lengthy in some cases. The present invention solves this problem by creating a coherent CDC log from a new snapshot and an existing CDC log. For example, a new snapshot is performed on an existing topic that is already in mirror mode and already contains data, leaving the data at the target coherent with respect to the source, as described in more detail below.

[0025] The following description is structured as follows: First, general embodiments and advanced variants are described (Section 1). The next section addresses more specific embodiments and technical implementation details (Sections 2 and 3). Note that the method and its variants are collectively referred to as "the method." All references Sj refer to method steps in the flowchart of FIG. 5, and references S x refers to a set or sequence of key-value pairs, and the reference numbers refer to physical parts or components of the system 1. 1. General embodiments and advanced variations

[0026] With reference to Figures 1, 2, 3, and 5, an aspect of the present invention will first be described, which relates to a method for tracking CDC log history. This method may typically be performed by CDC system 20 or any system authorized to interact with source system 10 and, possibly, as in embodiments described later, with target system 30 to update the target system. This CDC system could also form part of source system 10 or target system 30. The CDC system could, for example, run on any physical or virtual machine. Note that for completeness, several target systems could be included, but for simplicity, Figure 1 shows only one such target system 30.

[0027] According to the method, a first snapshot of the source system 10 is taken in step S10. A set of key-value pairs S1 is then derived, with reference to FIG. 3. The set S1 reflects the taken first snapshot. A mirror operation of the source system 10 is then performed in step S20. CDC change operations are obtained accordingly, such operations representing changes to be performed with respect to the set of key-value pairs S1, taking into account the mirror operation. As well as set S1, the CDC change operations are performed on the set of key-value pairs S2. M Accordingly, the first CDC log contains a first sequence of key-value pairs, S A The first sequence of key-value pairs S can be obtained as A is set S1 and set S M Contains both key-value pairs.

[0028] A second snapshot of the source system 10 is taken in step S40, for example as a load operation, and a set S2 of key-value pairs is derived accordingly, such that set S2 reflects the second snapshot.

[0029] The first sequence S of key-value pairs A is then compared S50 with a set of key-value pairs S2 to derive a modifying CDC operation, which is captured as a set of key-value pairs S3. A modifying CDC operation is defined as a sequence of key-value pairs S A It indicates that something is being done with respect to

[0030] Finally, the second CDC log contains a second sequence of key-value pairs, S B and a second sequence of key-value pairs S B is a sequence S A A second sequence of key-value pairs S contains key-value pairs from both set S1 and set S2. B The modified CDC operation is derived in such a way as to ensure that S is overall coherent with the set of key-value pairs S2.

[0031] It should be noted that the values ​​of the above-named key-value pairs may comprise any data or data set, preferably structured data. Such values ​​usually correspond to database rows, i.e., rows in the source system 10. The term "key-value pair" should be understood in a broad sense in this document; that is, it refers to any association of data (value) with a corresponding identifier (key). If necessary, the method may further generate a unique key if it is missing in the source system 10. In this way, a suitable key is always available in the sets S1, S2, S3, and the sequence S. A Or sequence S B or combination thereof. Such a unique key may be generated, for example, by hashing the contents of the corresponding value.

[0032] For example, a first snapshot may be taken at a first time t1, and the mirror operation may be performed during a time span spanning from the first time t1 to a second time t2, with the second time t2 occurring after time t1, as assumed in FIG. 3. A second snapshot would typically be taken at a third time t3, which occurs after t2. Note that this approach is also useful if the second snapshot is taken at a time t3 that does not necessarily occur after t2 but occurs after t1. In all cases, the target system 30 may ultimately reach a target state that is coherent with the state of the source system 10 at time t3.

[0033] After the mirror operation S20, the CDC system is in a mirrored state, ready for a subsequent load operation. However, a subsequent load operation S40 may potentially cause the CDC system 20 to load a state that corresponds to a different state, i.e., a state that is incoherent with the state of the source system after the mirror operation S20. For example, the source system 10 may have reverted to a backup state in the meantime, which may not match the state of the system 10 immediately after the mirror operation S20.

[0034] However, thanks to the corrective operation, a second sequence of key-value pairs S50 is captured as key-value pairs S3. B is coherent with the second snapshot corresponding to key-value pair S2, i.e., the second sequence can be interpreted to cause the target system 30 to reach a state reflecting the second snapshot, i.e., a state corresponding to the state of the source system 10 at time t3.

[0035] Therefore, the method generates a coherent CDC log S Bcan be created from the new snapshot S2 and the existing CDC log S1 by generating a series of corrective CDC operations. Such corrective operations lead the target system 30, after processing, to be in the same state as if it had read the new snapshot S2. Furthermore, since only CDC messages corresponding to changed values ​​(e.g., rows) are added, the difference between the new snapshot S2 and the first CDC log S1 will be small (which is the most frequent case in practice), and a coherent CDC log S1 will be created. B The amount of data that must be processed by interpreting the new snapshot S2 is much smaller (potentially by several orders of magnitude) than if the target system 30 had to read the entire new snapshot S2 as in the conventional approach. Also, the target system 30 does not need to know that it needs to take any particular action on the target or switch topic.

[0036] Another advantage of the proposed method is that it allows different types of operations to be interleaved while still generating a consistent CDC log. That is, a database table as a whole can be modified in two different ways: row operations (e.g., insert, update, etc.) and table operations (e.g., refresh, truncate, etc.), as known per se. A conventional CDC log tracking the first type of operation cannot be semantically combined directly with changes made from the second type, as this would result in a lower degree of data coherence. However, the present method allows the two types of operations triggered above to be interleaved while still generating a consistent CDC log. This is achieved by modifying the old CDC log state to a coherent CDC log state, a mechanism that is preferably performed incrementally and is therefore also referred to in this document as "morphing."

[0037] 5, the method may further cause the second sequence of key-value pairs to be interpreted (e.g., by the CDC system 20 or the target system 30) S70 to modify the current state of the target system 30. This then causes the second sequence S B enables the target system 30 to reach a target state that is coherent with the state of the source system 10 at (i.e., from) the time the second snapshot was taken, due to the corrective operations involved.

[0038] Preferably, a second sequence of key-value pairs S B is obtained as an ordered sequence S60, and the set of key-value pairs S1 is obtained as a set of key-value pairs S M preceded by a set of key-value pairs S M It precedes the set of key-value pairs S3 as shown in Figure 3. Similarly, the first sequence of key-value pairs S A When formatting S30 the first CDC log, Λ can be obtained as an ordered sequence, which allows the comparison to be performed in linear time.

[0039] In an embodiment, the corrective CDC operations are captured incrementally S50, i.e., each operation reflects one change at a time. Thus, the second CDC log may potentially include one or more of each (or several) of the following operations: DELETE, INSERT, and UPDATE. Each of these operations is captured as a key-value pair. In practice, however, the corrective CDC operations will typically comprise multiple DELETE, INSERT, and UPDATE operations.

[0040] As described above, all values ​​of the key-value pairs that cause the above typically correspond to database rows in the source system 10. Thus, in an embodiment, if a given database row indexed in the first CDC log is not reflected in the second snapshot, then in step S50 (i.e., the first sequence S A The comparison performed between set S1 and set S2 may result in a corrective CDC operation being a DELETE operation for a given database row. Similarly, if a given database row indexed in the first CDC log is reflected in the second snapshot, but a non-key field of the row changes, then comparison S50 may result in a corrective CDC operation being a corresponding UPDATE operation to update the non-key field. Also, if a given database row (as reflected in the second snapshot) is not indexed in the first CDC log, then comparison S50 may result in one of the corrective CDC operations being an INSERT operation for the given row. However, if a given database row (indexed in the first CDC log) is reflected identically in the second snapshot, then comparison S50 does not generate any corrective CDC operations for the row.

[0041] Any suitable algorithm may be contemplated to derive the modified CDC operation S50. Furthermore, this algorithm preferably includes a first sequence S A and set S2. Thus, in an embodiment, step S50 further comprises selecting the most appropriate algorithm for deriving the modified CDC operation. A and S2.

[0042] Interestingly, our approach is compatible with the sorted compaction of CDC logs detailed in Section 2. In addition, our approach can still be performed by a CDC system configured to split data into different partitions. In that case, database rows need to be mapped according to the different partitions in the CDC system and based on key-value pairs.

[0043] According to another aspect, the present invention may be embodied as a computer program product. The computer program product comprises a computer-readable storage medium having program instructions embodied therein. Such program instructions may be executed, for example, by the processing means 105 of the CDC system 20. They may alternatively be executed on one or more appropriately connected physical machines, or by virtual machines, for example, in a cloud environment, if desired. In all cases, such instructions cause the processing means to perform steps such as those described above. Additional discussion of computer program products and computerized systems is provided in Section 3.

[0044] The above embodiments are briefly described with reference to the accompanying drawings and may accommodate several variants. Several combinations of the above features may be envisaged. Examples are given in the following section 2. Specific Embodiments

[0045] This section describes an embodiment that allows a coherent CDC log to be created based on the new snapshot and the existing CDC log by generating a synthetic set of CDC changes that, after processing, leads the target system to be in the same state as if it had read the entire new snapshot.

[0046] Such an embodiment essentially involves adding only the CDC messages that correspond to the rows that have changed, thus allowing the target system to not need to be aware that it must take any particular action on the target or switch topic, as the difference between the new snapshot and the CDC log is small and only a small amount of data needs to be read by downstream systems.

[0047] Without loss of generality, we may assume that each row in the table (and therefore each message in the topic) can be identified by a unique key. As described in the previous section, if no such key exists, then it can be created on the fly, for example, by hashing the contents of the row.

[0048] As can be realized, the comparison S50 of the CDC log with the new snapshot actually leads to one of the following cases: The row exists in the old CDC log but not in the new snapshot. The lines are present in both and are the same. The row exists in both snapshots, but a non-key field has changed in the snapshot. The row exists only in the snapshot.

[0049] In the first case, a synthetic DELETE message may be created in the CDC log for the row. In the second case, the row already exists in the CDC log, so no action is required. In the third case, a synthetic UPDATE message may be created in the CDC log. In the fourth case, a synthetic INSERT message may be created in the CDC log.

[0050] This effectively reduces the snapshot to a number of operations: all necessary operations are added to the CDC log, which produces the same final state that would have been produced on the target system if the target system had read the snapshot alone.

[0051] A CDC system may, for example, store rows in key order in a CDC log. Similarly, snapshots may be created with rows in key order. This then allows the CDC log and the first portion of the snapshot to be compared in linear time. The mirror portion of the CDC log may be processed independently.

[0052] In systems like Kafka, data is partitioned across multiple different partitions, where operations related to a particular row are always stored in the same partition. This can be achieved by mapping rows to partitions based on key values. Advantageously, such a solution can be parallelized in a partition-by-partition manner to ensure scalability.

[0053] Preferably, the method recognizes the degree of similarity between the snapshot and the existing CDC log to select the most appropriate algorithm, for example between those defined in Algorithms 1 and 2. This can be done in particular by keeping the keys of the rows added during the mirroring phase in a given structure, referred to herein as "mirrorSet", and then using the ratio between this and the length of the old CDC log to select the most appropriate algorithm. For example, when this ratio is small and a process is taken to write the initial snapshot in key order, the calculation of the compositing operation can be calculated in linear time.

[0054] A particularly preferred embodiment is reflected in Algorithm 1 below (pseudocode). Algorithm 1. Morphs an existing CDC log to match a given snapshot. function:MorphCDCLog(log, refreshSize, snapshot, mirrorKeys) / / Append the log to match the snapshot snapshotIndex←0 logIndex←0 toBeKept ← emptySet initialLogLength ← log.length while logIndex <refreshsize do if(snapshotindex="snapshot.length)then" break end if snapshotrecord←snapshot[snapshotindex] logrecord←log[logindex] if(snapshotrecord.key>logRecord.key)then / / logRecord no longer exists. log.append(DELETE(logRecord.key)) logIndex++ else if(snapShotRecord=logRecord)then / / Ignore if not in mirror set if(mirrorKeys.contains(snapShotRecord.key))then log.append(Record(UPSERT,snapShotRecord)) toBeKept.add(snapShotRecord.key) end if else / / Add a new snapshot record. log.append(Record(UPSERT,snapShotRecord)) toBeKept.add(snapShotRecord.key) end if snapshotIndex++ if(snapShotRecord.key=logRecord.key)then logIndex++ end if end if end while / / Delete the remaining logs that excluded the update. while(logIndex <initialLogLength)do logRecord←log[logIndex++] if!toBeKept.contains(logRecord.key)then log.append(DELETE(logRecord.key)) end if end while / / Add what remains in the snapshot while(snapshotIndex <snapshot.size)do snapShotRecord←snapshot[snapshotIndex++] log.append(snapShotRecord) end while end function

[0055] Additional technical details may be described: When the mirrorSet is large compared to the old CDC logs, the algorithm of the embodiment captured in Algorithm 1 above may become inefficient.

[0056] This is because while the snapshot portion of the log is sorted, the mirror portion is not. For that reason, current log systems are equipped with a method called "compactification" that compacts the snapshot portion and the mirror portion so that a new snapshot portion is created. A method that is compatible with compaction but allows for sorted snapshots (hereafter referred to as the sorted compaction method) is described in Algorithm 2 below.

[0057] During compaction, topic consumers cannot read the compacted log or mirrorSet marked for compaction. New records can still be added to and read from the topic unless the algorithm changes the offset for the new data. Figure 4A illustrates the flow of the sorted compaction method, and Figure 4B shows an example topic. The final compaction point in Figure 4A illustrates the offset in the log up to the point where the log was sorted and compacted. The mirror set may contain unsorted duplicate key entries. The goal of sorted compaction is to compact the current sorted compacted log (snapshot) and mirror set (new update) into a new sorted compacted log. To do this, the method first sorts and deduplicates the mirror set. Duplicates are handled in such a way that for multiple records with the same key, the one with the highest offset is taken. After sorting and replication, the mirror set essentially becomes another sorted compacted log that can be merged with the original snapshot, and the same merging rule applies: for two identical keys, the one with the higher offset is taken, which is always the value of the mirror set as the last one added. This is similar to the merge phase of the merge sort algorithm. After the algorithm shown in Algorithm 2 performs a snapshot and the mirror definition is changed, the new snapshot becomes the new sorted compacted log, and the mirror set becomes the new changes added after the algorithm started. The complexity for sort compaction is O(m log m) to sort the mirrorSet and O(m+n) to merge it with the compactedLog. Algorithm 2. Compute a new, compacted, sorted CDC log. function SORTEDCOMPACTION(compactedLog,mirrorSet) / / Create a new, sorted, compacted CDC log from an already compacted log and a mirrorSet newLog←emptyFile / / Sort and dedupe the mirrorSet. Deduplication takes the last value of the key. sortedMirrorSet←deduplicate(sort(mirrorSet)) logIndex←0 mirrorIndex←0 cLength←compactedLog:length while logIndex<cLength do logKey←compactedLog[logIndex].key if mirrorIndex> =sortedMirrorSet.size then newLog.append(compactedLog[logIndex]) logIndex++ continue end if mirrorKey←sortedMirrorSet[mirrorIndex].key if logKey <mirrorKey then newLog.append(compactedLog[logIndex]) else if sortedMirrorSet[mirrorIndex].value!=null then newLog.append(sortedMirrorSet[mirrorIndex])) else / / Does nothing. Compact out delete end if mirrorIndex++ if logkey>mirrorKey then continue end if end if logIndex++ end while / / Add the rest of the records from the mirrorSet to newLog while mirrorIndex++ <sortedMirrorSet.length do newLog.append(sortedMirrorSet[mirrorIndex])) end while end function 3. Technical Implementation Details 3.1 Computerized Systems and Devices

[0058] Computer-controlled systems and devices may be suitably designed to implement embodiments of the invention as described herein. In that regard, the methods described herein are largely non-interactive and automatic. In exemplary embodiments, the methods described herein may be implemented in either interactive, partially interactive, or non-interactive systems. The methods described herein may be implemented as software, hardware, or a combination thereof. In exemplary embodiments, the methods described herein may be implemented in software as an executable program, the latter executed by a suitable digital processing device. More generally, embodiments of the invention may be implemented using a virtual machine or a general-purpose digital computer, such as a personal computer, a workstation, or the like, or a combination thereof.

[0059] For example, FIG. 6 schematically represents a computerized unit 101 (e.g., a general-purpose or special-purpose computer), which may possibly interact with other similar units so as to be able to carry out the steps according to the method.

[0060] In an exemplary embodiment, with regard to the hardware architecture, as shown in Figure 6, each unit 101 includes at least one processor 105 and a memory 110 coupled to a memory controller 115. Several processors (CPUs or GPUs or a combination thereof) may possibly be included in each unit 101. To this end, each CPU / GPU may be assigned to a respective memory controller, as known per se.

[0061] One or more input or output or combination thereof (I / O) devices 145, 150, 155 (or peripherals) are communicatively coupled via a local input / output controller 135. The input / output controller 135 may be coupled to or include one or more buses or system buses 140, as known in the art. The input / output controller 135 may have additional elements, such as controllers, buffers (caches), drivers, repeaters, and receivers, that enable communication, which are omitted for brevity. Additionally, the local interface may include address, control, or data connections, or combinations thereof, that enable appropriate communication between the above components.

[0062] Processor 105 is a hardware device for executing software instructions. Processor 105 can be any custom-made or commercially available processor. Generally, they can include any type of semiconductor-based microprocessor (in the form of a microchip or chipset), or generally any device for executing software instructions.

[0063] Memory 110 typically includes volatile memory elements (e.g., random access memory) and may also include non-volatile memory elements. Memory 110 may also incorporate electronic, magnetic, optical, or other types of storage media, or combinations thereof. Additional storage may be provided via storage 120.

[0064] The software in memory 110 may include one or more individual programs, each of which contains executable instructions for implementing logical functions. In the example of Figure 6, the instructions loaded into memory 110 may include instructions resulting from the execution of a computer-controlled method described herein according to an exemplary embodiment. Memory 110 also loads a suitable operating system (OS) 111. OS 111 essentially controls the execution of other computer programs or instructions and may provide scheduling, input-output control, file and data management, memory management, and communication control and related services.

[0065] In some cases, a conventional keyboard and mouse may be coupled to the input / output controller 135. Other I / O devices 140-155 may be included. The computerized unit 101 may further include a display controller 125 coupled to a display 130. Any computerized unit 101 will typically include a network interface or transceiver 160 to couple to a network and in turn enable data communication with / from other external components, e.g., other units 101.

[0066] The network transmits and receives data between a given unit 101 and other devices 101. The network may in some cases be implemented in a wireless manner using wireless protocols and technologies such as, for example, Wifi, WiMax, etc. The network may be, among other things, a fixed wireless network, a wireless local area network (LAN), a wireless wide area network (WAN), a personal area network (PAN), a virtual private network (VPN), the Internet, or other suitable network system, and includes equipment for receiving and transmitting signals. Preferably, however, the network should allow very high speed message transmission between units.

[0067] The network may also be an IP-based network for communication between a given unit 101 and any external unit via a broadband connection. In an exemplary embodiment, the network may be a managed IP network operated by a service provider. The network may also be a packet-switched network such as a LAN, a WAN, an Internet network, an Internet of Things network, etc. 3.2 Computer Program Product

[0068] The present invention may be a method or a computer program product, or a combination thereof. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0069] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disks (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves having instructions recorded thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein should not be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over wires.

[0070] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may comprise copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage on a computer-readable storage medium within the respective computing / processing device.

[0071] The computer-readable program instructions for carrying out the operations of the present invention may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages. The one or more programming languages ​​include object-oriented programming languages ​​such as Smalltalk®, C++, etc., and procedural programming languages ​​such as the “C” programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry to perform aspects of the present invention.

[0072] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, systems and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0073] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus that causes a machine to perform the functions / acts specified in a flowchart or block diagram block or blocks, or combinations thereof. These computer-readable program instructions may also be stored on a computer-readable storage medium that can direct a computer, programmable data processing apparatus, or other device, or combinations thereof, to function in a particular manner, such that the computer-readable storage medium having stored thereon instructions comprises an article of manufacture containing instructions that implement aspects of the functions / acts specified in a flowchart or block diagram block or blocks, or combinations thereof.

[0074] The computer-readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device and cause a series of operational steps to be executed on the computer, other programmable apparatus, or other device to generate a computer-implemented process, whereby the instructions executed on the computer, other programmable apparatus, or other device implement the function / act identified in the flowchart or block diagram block or combination thereof.

[0075] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially in parallel, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of a block diagram or flowchart illustration, or combination thereof, and combinations of blocks in block diagrams or flowchart illustrations, or combinations thereof, may be implemented by a dedicated hardware-based system that performs the specified functions or acts or executes a combination of dedicated hardware and computer instructions. 3.3 Cloud

[0076] The computerized implementation performed by the present method may, in some cases, be provided as a cloud service. However, it should be understood that implementation of the teachings described herein is not limited to cloud computing environments. Rather, embodiments of the present invention may be implemented in conjunction with any other type of computing environment now known or later developed. Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a service provider.

[0077] While the present invention has been described with reference to a limited number of embodiments, variations, and accompanying drawings, it will be apparent to those skilled in the art that various modifications may be made and equivalents may be substituted without departing from the scope of the invention. In particular, features (such as devices or methods) described in a given embodiment, variation, or illustrated in a drawing may be combined with or substituted for other features in other embodiments, variations, or drawings without departing from the scope of the invention. Accordingly, various combinations of the features described with respect to any of the above embodiments or variations are contemplated and remain within the scope of the appended claims. Furthermore, many minor modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the scope of the invention. Therefore, the invention is not limited to the particular embodiments disclosed, but is intended to include all embodiments falling within the scope of the appended claims. Many other variations beyond those expressly mentioned above are contemplated. 。 [Item 1] 1. A method for tracking change data capture log history, or CDC log history, comprising: Take a first snapshot of the source system, and create a set of key-value pairs S that reflects the first snapshot. 1 and Execute a mirror operation on the source system, and accordingly generate a set of key-value pairs S 1 CDC change operations representing changes to be performed on a set of key-value pairs S M The phases are captured as The first CDC log is stored in the set S 1 and the set S M a first sequence S of key-value pairs containing said key-value pairs A and Taking a second snapshot of the source system, and creating a set of key-value pairs S that reflects the second snapshot. 2 and The first sequence S of key-value pairs A Let S be a set of key-value pairs. 2 Compare with the set of key-value pairs S 3 the modified CDC operation is A expressing the modification to be performed on The first sequence S A and the set S 3 a second sequence S of key-value pairs containing said key-value pairs B and obtaining a second CDC log as the modified CDC operation, B Let S be the set of key-value pairs as a whole. 2 and a stage, which ensures that the A method comprising: [Item 2] The method further comprises interpreting the second sequence of key-value pairs to modify the current state of the target system to reach a target state that is coherent with the state of the source system at the time the second snapshot is taken. The method according to item 1. [Item 3] the second sequence S of key-value pairs B is obtained as an ordered sequence, whereby the set S of key-value pairs 1 is the set S of key-value pairs M and the set S of key-value pairs precedes M itself is the set S of key-value pairs 3 precedes, The method according to item 1 or 2. [Item 4] The corrective CDC operations include one or more DELETE operations, each captured as a key-value pair. 4. The method according to any one of items 1 to 3. [Item 5] The corrective CDC operations include one or more INSERT operations, each captured as a key-value pair. 5. The method according to any one of items 1 to 4. [Item 6] The corrective CDC operations include one or more UPDATE operations, each captured as a key-value pair. 6. The method according to any one of items 1 to 5. [Item 7] The corrective CDC operations include at least one of each of the following operations, each captured as a key-value pair: a DELETE operation, an INSERT operation, and an UPDATE operation; 7. The method according to any one of items 1 to 6. [Item 8] all values ​​of the key-value pairs correspond to database rows in the source system; 8. The method according to any one of items 1 to 7. [Item 9] A given database row indexed in the first CDC log is not reflected in the second snapshot, thereby causing the first sequence S A and the set S 2 causes one of the corrective CDC operations to be derived as a DELETE operation for the given database row. The method according to item 8. [Item 10] A given database row indexed in the first CDC log is reflected in the second snapshot, and this causes the first sequence S A and the set S 2 causes one of the modified CDC operations to be derived as a corresponding UPDATE operation on that non-key field. Item 8. The method according to item 8 or 9. [Item 11] A given database row reflected in the second snapshot is not indexed in the first CDC log, thereby causing the first sequence S A and the set S 2 causes one of the modified CDC operations to be derived as the corresponding INSERT operation for that given row. 11. The method according to any one of items 8 to 10. [Item 12] A given database row indexed in the first CDC log is reflected identically in the second snapshot, thereby A and the set S 2 does not result in any corrective CDC operations for a given row. 12. The method according to any one of items 8 to 11. [Item 13] The method further comprises obtaining a sorted compaction of the first CDC log. 13. The method according to any one of items 1 to 12. [Item 14] the method is performed by a CDC system configured to divide data into different partitions; 14. The method according to any one of items 1 to 13. [Item 15] The method further comprises mapping the database rows according to the different partitions of the CDC system based on the key-value pairs. Item 15. The method according to item 14, which is dependent on any one of items 8 to 12. [Item 16] The first sequence S A and the set S 2 The comparison with the first sequence S A and the set S 2 and further evaluating a degree of similarity between the two, whereby the modified CDC operation is derived based on an algorithm selected according to the evaluated degree of similarity. 16. The method according to any one of items 1 to 15. [Item 17] The method further comprises: if a unique key is missing in the source system, 1 , the set S 2 , the set S M , the first sequence S A , the set S 3 , and the second sequence S B generating this unique key for each one or more of the one or more key-value pairs in 17. The method according to any one of items 1 to 16. [Item 18] 1. A computer program for tracking change data capture log history, or CDC log history, comprising: and program instructions embodied therein, the program instructions causing a processing means to: Take a first snapshot of the source system, and create a set of key-value pairs S that reflects the first snapshot. 1 The procedure to derive Execute a mirror operation on the source system, and accordingly generate a set of key-value pairs S 1 CDC change operations representing changes to be performed on a set of key-value pairs S M The steps are captured as The first CDC log is stored in the set S 1 and the set S M a first sequence S of key-value pairs containing said key-value pairs A and the procedure to obtain it as Taking a second snapshot of the source system, and creating a set of key-value pairs S that reflects the second snapshot. 2 The procedure to derive The first sequence S of key-value pairs A Let S be a set of key-value pairs. 2 Compare with the set of key-value pairs S 3 the modified CDC operation is A a procedure representing the modifications to be performed on The first sequence S A and the set S 3 a second sequence S of key-value pairs containing said key-value pairs B and obtaining a second CDC log as the second sequence S of key-value pairs, B Let S be the set of key-value pairs as a whole. 2 and a procedure to ensure that the a computer program executable by said processing means to cause said processing means to [Item 19] the program instructions are further executable by the processing means to cause the processing means to interpret the second sequence of key-value pairs and cause a target system to reach a state reflecting the second snapshot. Item 19. The computer program according to item 18. [Item 20] The program instructions cause the processing means to generate the second sequence S of key-value pairs as an ordered sequence. B whereby the set S of key-value pairs is obtained. 1 is the set S of key-value pairs M and the set S of key-value pairs precedes M itself is the set S of key-value pairs 3 further executable by the processing means to perform the steps preceding 20. The computer program according to item 18 or 19. < / refreshsize>

Claims

1. 1. A method for tracking change data capture (CDC) log history, comprising: A processor takes a first snapshot of a source system and creates a set S of key-value pairs reflecting the first snapshot. １ and the processor performing a mirror operation on the source system by replicating operations captured by the first snapshot, the mirror operation producing one or more results on the source system; The processor generates a set S of key-value pairs that reflects the one or more results of the mirror operation. １ obtaining a CDC modification operation representing a modification to be performed on a set of key-value pairs S Ｍ The phases are captured as The processor stores the first CDC log in the set S １ and the set S Ｍ a first sequence S of key-value pairs including said key-value pairs of Ａ and The processor takes a second snapshot of the source system and generates a set S of key-value pairs reflecting the second snapshot. ２ and The processor generates a first sequence S of key-value pairs. Ａ Let S be a set of key-value pairs. ２ Compare with the set of key-value pairs S ３ derive a modified CDC operation as Ａ expressing the modification to be performed on The processor generates the first sequence S Ａ and the set S ３ a second sequence S of key-value pairs containing the key-value pairs of Ｂ obtaining a second CDC log as Ｂ is the set S of key-value pairs as a whole. ２ and a stage, which ensures that the A method comprising:

2. The method further comprises the processor interpreting the second sequence of key-value pairs to modify the current state of the target system to reach a target state that is coherent with the state of the source system at the time the second snapshot is taken. The method of claim 1.

3. The second sequence S of key-value pairs Ｂ is taken as an ordered sequence, whereby the set S of key-value pairs １ is the set S of key-value pairs Ｍ and the set S of key-value pairs Ｍ itself is the set S of key-value pairs ３ precedes, The method according to claim 1 or 2.

4. A method for tracking change data capture log history or CDC log history, comprising: a processor taking a first snapshot of a source system and deriving a set of key-value pairs S 1 that reflects said first snapshot; the processor performing a mirror operation of the source system and responsively obtaining CDC change operations representing changes performed on a set of key-value pairs S 1 , the CDC change operations being captured as a set of key-value pairs S M ; The processor obtaining a first CDC log as a first sequence S A of key-value pairs that includes the key-value pairs of the set S 1 and the set S M ; the processor taking a second snapshot of the source system and deriving a set of key-value pairs S2 that reflects the second snapshot; the processor comparing a first sequence of key-value pairs S A with a set of key-value pairs S 2 and deriving a modified CDC operation as a set of key-value pairs S 3 , the modified CDC operation representing a modification to be performed on the first sequence of key-value pairs S A ; the processor obtaining a second CDC log as a second sequence S B of key-value pairs that includes the first sequence S A and the key-value pairs of the set S 3 , the modified CDC operation ensuring that the second sequence S B of key-value pairs as a whole is coherent with the set S 2 of key-value pairs; Including, the second sequence S B of key-value pairs is obtained as an ordered sequence, whereby the set S 1 of key-value pairs precedes the set S M of key-value pairs, which itself precedes the set S 3 of key-value pairs; method.

5. The modifying CDC operations include one or more DELETE operations, each captured as a key-value pair.

5. The method according to any one of claims 1 to 4.

6. The modified CDC operations include one or more INSERT operations, each captured as a key-value pair.

6. The method according to any one of claims 1 to 5.

7. The modifying CDC operation includes one or more UPDATE operations, each captured as a key-value pair.

7. The method according to any one of claims 1 to 6.

8. The modifying CDC operations include at least one of each of the following operations, each captured as a key-value pair: a DELETE operation, an INSERT operation, and an UPDATE operation.

8. The method according to any one of claims 1 to 7.

9. all values ​​of the key-value pairs correspond to database rows in the source system; 9. The method according to any one of claims 1 to 8.

10. A given database row indexed in the first CDC log is not reflected in the second snapshot, thereby causing the first sequence S Ａ and the set S ２ causes one of the modified CDC operations to be derived as a DELETE operation for the given database row.

10. The method of claim 9.

11. A given database row indexed in the first CDC log is reflected in the second snapshot, thereby retaining the first sequence S Ａ and the set S ２ causes one of the modified CDC operations to be derived as the corresponding UPDATE operation for that non-key field.

11. The method according to claim 9 or 10.

12. A given database row reflected in the second snapshot is not indexed in the first CDC log, thereby causing the first sequence S Ａ and the set S ２ causes one of the modified CDC operations to be derived as the corresponding INSERT operation for that given row.

12. The method according to any one of claims 9 to 11.

13. A given database row indexed in the first CDC log is reflected identically in the second snapshot, thereby Ａ and the set S ２ does not lead to any modified CDC operation for a given row.

13. The method according to any one of claims 9 to 12.

14. The method further comprises the processor obtaining a sorted compaction of the first CDC log.

14. The method of any one of claims 1 to 13.

15. the method is performed by the processor of a CDC system configured to divide data into different partitions; 15. The method of any one of claims 1 to 14.

16. The method further comprises the processor mapping the database rows according to the different partitions of the CDC system based on the key-value pairs.

16. The method of claim 15 when dependent on any one of claims 9 to 13.

17. The first sequence S Ａ and the set S ２ The comparison with the first sequence S Ａ and the set S ２ and further comprising evaluating a degree of similarity between the two, whereby the modified CDC operation is derived based on an algorithm selected according to the evaluated degree of similarity.

17. The method of any one of claims 1 to 16.

18. A method for tracking change data capture log history or CDC log history, comprising: a processor taking a first snapshot of a source system and deriving a set of key-value pairs S 1 that reflects said first snapshot; the processor performing a mirror operation of the source system and responsively obtaining CDC change operations representing changes performed on a set of key-value pairs S 1 , the CDC change operations being captured as a set of key-value pairs S M ; The processor obtaining a first CDC log as a first sequence S A of key-value pairs that includes the key-value pairs of the set S 1 and the set S M ; the processor taking a second snapshot of the source system and deriving a set of key-value pairs S2 that reflects the second snapshot; the processor comparing a first sequence of key-value pairs S A with a set of key-value pairs S 2 and deriving a modified CDC operation as a set of key-value pairs S 3 , the modified CDC operation representing a modification to be performed on the first sequence of key-value pairs S A ; the processor obtaining a second CDC log as a second sequence S B of key-value pairs that includes the first sequence S A and the key-value pairs of the set S 3 , the modified CDC operation ensuring that the second sequence S B of key-value pairs as a whole is coherent with the set S 2 of key-value pairs; Including, The comparison of the first sequence S A with the set S 2 further includes evaluating a degree of similarity between the first sequence S A and the set S 2 , whereby the modified CDC operation is derived based on an algorithm selected according to the evaluated degree of similarity. method.

19. The method further comprises the step of: if the unique key is missing in the source system, the processor extracts the unique key from the set S １ , the set S ２ , the set S Ｍ , the first sequence S Ａ , the set S ３ , and the second sequence S Ｂ generating this unique key for each of one or more of the one or more key-value pairs in 19. The method of any one of claims 1 to 18.

20. 1. A computer program for tracking change data capture (CDC) log history, comprising: and program instructions embodied therein, the program instructions causing a processing means to: Taking a first snapshot of the source system and creating a set of key-value pairs S that reflects the first snapshot １ The procedure to derive performing a mirror operation on the source system by replicating operations captured by the first snapshot, the mirror operation producing one or more results on the source system; a set S of key-value pairs reflecting the one or more results of the mirror operation; １ a procedure for obtaining a CDC change operation representing a change to be performed on a set of key-value pairs S Ｍ The steps are captured as The first CDC log is １ and the set S Ｍ a first sequence S of key-value pairs including said key-value pairs of Ａ and the procedure to obtain it as taking a second snapshot of the source system and creating a set S of key-value pairs reflecting the second snapshot; ２ The procedure to derive A first sequence S of key-value pairs Ａ Let S be a set of key-value pairs. ２ Compare with the set of key-value pairs S ３ derive a modified CDC operation as Ａ a procedure representing the modifications to be performed on The first sequence S Ａ and the set S ３ a second sequence S of key-value pairs containing the key-value pairs of Ｂ , wherein the modified CDC operation obtains a second CDC log as the second sequence S of key-value pairs. Ｂ is the set S of key-value pairs as a whole. ２ and a procedure to ensure that the a computer program executable by said processing means to cause said processing means to

21. the program instructions are further executable by the processing means to cause the processing means to interpret the second sequence of key-value pairs and cause a target system to reach a state reflecting the second snapshot.

21. A computer program according to claim 20.

22. The program instructions cause the processing means to generate the second sequence S of key-value pairs as an ordered sequence. Ｂ whereby the set S of key-value pairs is obtained. １ is the set S of key-value pairs Ｍ and the set S of key-value pairs Ｍ itself is the set S of key-value pairs ３ further executable by the processing means to perform the steps preceding 22. A computer program according to claim 20 or 21.

23. A computer program for tracking change data capture log history or CDC log history, comprising: and program instructions embodied therein, the program instructions causing a processing means to: taking a first snapshot of a source system and deriving a set of key-value pairs S 1 that reflects said first snapshot; performing a mirror operation of said source system and, in response, obtaining CDC change operations representing changes to be performed on a set of key-value pairs S 1 , said CDC change operations being captured as a set of key-value pairs S M ; obtaining a first CDC log as a first sequence S A of key-value pairs that includes the key-value pairs of the set S 1 and the set S M ; taking a second snapshot of the source system and deriving a set S2 of key-value pairs reflecting the second snapshot; a procedure for comparing a first sequence of key-value pairs S A with a set of key-value pairs S 2 and deriving a modified CDC operation as a set of key-value pairs S 3 , said modified CDC operation representing a modification to be performed on said first sequence of key-value pairs S A ; obtaining a second CDC log as a second sequence S B of key-value pairs that includes the first sequence S A and the key-value pairs of the set S 3 , wherein the modified CDC operation ensures that the second sequence S B of key-value pairs as a whole is coherent with the set S 2 of key-value pairs; executable by the processing means to cause the program instructions are further executable by the processing means to cause the processing means to obtain the second sequence S B of key-value pairs as an ordered sequence, whereby the set S 1 of key-value pairs precedes the set S M of key-value pairs, and the set S M of key-value pairs itself precedes the set S 3 of key-value pairs; Computer program.

24. A computer program for tracking change data capture log history or CDC log history, comprising: and program instructions embodied therein, the program instructions causing a processing means to: taking a first snapshot of a source system and deriving a set of key-value pairs S 1 that reflects said first snapshot; performing a mirror operation of said source system and, in response, obtaining CDC change operations representing changes to be performed on a set of key-value pairs S 1 , said CDC change operations being captured as a set of key-value pairs S M ; obtaining a first CDC log as a first sequence S A of key-value pairs that includes the key-value pairs of the set S 1 and the set S M ; taking a second snapshot of the source system and deriving a set S2 of key-value pairs reflecting the second snapshot; a procedure for comparing a first sequence of key-value pairs S A with a set of key-value pairs S 2 and deriving a modified CDC operation as a set of key-value pairs S 3 , said modified CDC operation representing a modification to be performed on said first sequence of key-value pairs S A ; obtaining a second CDC log as a second sequence S B of key-value pairs that includes the first sequence S A and the key-value pairs of the set S 3 , wherein the modified CDC operation ensures that the second sequence S B of key-value pairs as a whole is coherent with the set S 2 of key-value pairs; executable by the processing means to cause The comparison of the first sequence S A with the set S 2 further includes evaluating a degree of similarity between the first sequence S A and the set S 2 , whereby the modified CDC operation is derived based on an algorithm selected according to the evaluated degree of similarity. Computer program.

Images