A method, apparatus and medium for incremental data synchronization

Abstract

The application discloses a kind of methods, devices and media of incremental data synchronization, applicable to computer technology field.Base on current time, obtain the current snapshot of main volume, last synchronization period snapshot and reference initial synchronization snapshot;According to the relationship between the latest snapshot and last synchronization period snapshot, determine the periodic synchronization situation of the snapshot of last synchronization period, and according to the corresponding periodic synchronization situation and the relationship of reference initial synchronization snapshot, determine the snapshot creation state of main volume;When snapshot creation state is not created, compare the data under each snapshot to determine incremental data by fast-diff characteristics.The method can quickly obtain information about whether the data has changed through the update of object-map value by fast-diff characteristics, so that the data object is obtained in a resource-saving manner during data incremental synchronization, and the volume only needs to be read when opening, and further processing is required when taking snapshots.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus and medium for incremental data synchronization. Background Technology

[0002] In storage systems, asynchronous remote replication is used to achieve data disaster recovery between different storage systems. The usual practice is to perform a full copy of the primary and backup volumes when creating a volume replication pair, and then perform incremental data synchronization at fixed intervals or at a fixed time point.

[0003] Incremental synchronization calculates the difference between the current volume data and the data from the last synchronization time. If a difference exists, it is synchronized to the backup volume. A common approach is to divide the data into multiple data objects based on a certain unit, read and process the data, and then compare the differences between each data object. Due to the large volume of data on the primary volume, this process is time-consuming. Furthermore, when maintaining many pairs in a system, the primary volume often reads all data objects during each synchronization cycle, even if no data modifications have occurred. This results in significant system resource consumption during data reading, creating unnecessary system load and impacting overall system read / write performance.

[0004] Therefore, finding a method for incremental data synchronization is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, and medium for incremental data synchronization. This method creates snapshots of backup volumes only when data changes, reducing system load and improving system read / write performance. The rapid difference feature ensures that the absence of a snapshot does not necessarily mean the current data has not been updated, thus identifying incremental data and avoiding resource consumption.

[0006] To address the aforementioned technical problems, this invention provides a method for incremental data synchronization, comprising:

[0007] Based on the current time, obtain the current snapshot ID number of the main volume, the snapshot ID number of the previous synchronization period, and the initial synchronization snapshot ID number;

[0008] The periodic synchronization status of the snapshot identity number in the previous synchronization period is determined based on the relationship between the current snapshot identity number and the snapshot identity number in the previous synchronization period, and the snapshot creation status of the main volume is determined based on the corresponding periodic synchronization status and the relationship between the baseline initial synchronization snapshot identity number.

[0009] When the snapshot creation status is not created, the data under each snapshot identification number is obtained through the fast difference feature and compared to determine the incremental data;

[0010] When the snapshot creation status is "created", the data corresponding to the created snapshot is obtained as the incremental data.

[0011] Preferably, determining the periodic synchronization status of the snapshot identity number in the previous synchronization period based on the relationship between the current snapshot identity number and the snapshot identity number in the previous synchronization period includes:

[0012] Determine whether the current snapshot identification number is the same as the snapshot identification number of the previous synchronization period;

[0013] If they are the same, then it is confirmed that the primary volume has not been snapshotted during the current synchronization cycle;

[0014] If they are different, then determine whether the identity identifier number of the previous synchronization cycle snapshot is 0;

[0015] If the value is 0, then the identity number of the previous synchronization cycle snapshot is determined to be the first cycle synchronization;

[0016] If it is not 0, then the identity number of the previous synchronization cycle snapshot is determined to be a non-first-cycle synchronization.

[0017] Preferably, when the previous synchronization cycle snapshot identification number is the first cycle synchronization, determining the snapshot creation status of the main volume based on the corresponding cycle synchronization status and the relationship between the baseline initial synchronization snapshot identification number includes:

[0018] Determine whether the identity identifier number of the previous synchronization cycle snapshot is the same as the identity identifier number of the baseline initial synchronization snapshot;

[0019] If they are the same, then the snapshot creation status of the main volume is determined to be not created;

[0020] If they are different, then the snapshot creation status of the main volume is determined to be created.

[0021] Preferably, when the identity identifier number of the previous synchronization cycle snapshot is not the first cycle synchronization, determining the snapshot creation status of the main volume based on the relationship between the corresponding cycle synchronization status and the identity identifier number of the baseline initial synchronization snapshot includes:

[0022] If the identity identifier number of the previous synchronization cycle snapshot is not the first cycle synchronization and the identity identifier number of the current snapshot is different from the identity identifier number of the previous synchronization cycle snapshot, then the snapshot creation status of the main volume is determined to be created;

[0023] If the identity identifier number of the previous synchronization cycle snapshot is not the first synchronization cycle and the identity identifier number of the current snapshot is the same as the identity identifier number of the previous synchronization cycle snapshot, then the snapshot creation status of the main volume is determined to be not created.

[0024] Preferably, the step of comparing data obtained from each snapshot identity number using rapid difference characteristics to determine incremental data includes:

[0025] The current data of the main volume and the snapshot data under the snapshot identity number of the previous synchronization cycle are obtained through the fast difference feature under the object mapping feature;

[0026] Determine whether the current data and the snapshot data are the same;

[0027] If they are the same, then the bitmap corresponding to the main volume is determined to be 0;

[0028] If they are different, the difference between the current data of the main volume and the snapshot data is determined as the incremental data.

[0029] Preferably, the current data and the snapshot data under the identity number of the previous synchronization cycle snapshot are obtained by acquiring the data between every two snapshots between the snapshot corresponding to the current data and the snapshot data under the identity number of the previous synchronization cycle snapshot.

[0030] Preferably, when the bitmap corresponding to the main volume is 0, the method further includes:

[0031] End the current period synchronization and update the synchronization time of the current period.

[0032] To address the aforementioned technical problems, the present invention also provides an apparatus for incremental data synchronization, comprising:

[0033] The acquisition module is used to acquire the current snapshot ID number, the snapshot ID number of the previous synchronization period, and the baseline initial synchronization snapshot ID number of the main volume, based on the current time.

[0034] The first determining module is used to determine the periodic synchronization status of the snapshot identity number of the previous synchronization period based on the relationship between the current snapshot identity number and the snapshot identity number of the previous synchronization period, and to determine the snapshot creation status of the main volume based on the corresponding periodic synchronization status and the relationship between the baseline initial synchronization snapshot identity number.

[0035] The second determining module is used to determine incremental data by comparing the data under each snapshot identification number obtained through the fast difference feature when the snapshot creation status is not created.

[0036] The third determining module is used to obtain the data corresponding to the created snapshot as the incremental data when the snapshot creation status is "created".

[0037] To address the aforementioned technical problems, the present invention also provides an apparatus for incremental data synchronization, comprising:

[0038] Memory, used to store computer programs;

[0039] A processor, used to implement the incremental data synchronization method as described above when executing the computer program.

[0040] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the incremental data synchronization method described above.

[0041] This invention provides a method for incremental data synchronization, comprising: obtaining the current snapshot identifier number, the snapshot identifier number of the previous synchronization period, and the initial synchronization snapshot identifier number of the primary volume based on the current time; determining the periodic synchronization status of the snapshot identifier number of the previous synchronization period based on the relationship between the current snapshot identifier number and the snapshot identifier number of the previous synchronization period, and determining the snapshot creation status of the primary volume based on the corresponding periodic synchronization status and the relationship between the initial synchronization snapshot identifier number of the benchmark; when the snapshot creation status is not created, obtaining data under each snapshot identifier number through fast difference feature for comparison to determine incremental data; when the snapshot creation status is created, obtaining the data corresponding to the created snapshot as incremental data. This method, through the fast difference feature for updating object mapping values, can quickly obtain information on whether data has changed, enabling resource-efficient acquisition of data objects during incremental data synchronization. Opening the volume only requires reading, and further processing is only required when a snapshot needs to be taken, necessitating a write operation. Snapshots are taken only when data changes, reducing system load and improving system read / write performance. The fast difference feature ensures that the absence of a snapshot does not necessarily mean the current data has not been updated, thus determining incremental data and avoiding resource consumption.

[0042] In addition, the present invention also provides an apparatus and medium for incremental data synchronization, which have the same beneficial effects as the incremental data synchronization method described above. Attached Figure Description

[0043] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A flowchart illustrating an incremental data synchronization method provided in an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of volume snapshot changes provided in an embodiment of the present invention;

[0046] Figure 3 A structural diagram of an incremental data synchronization device provided in an embodiment of the present invention;

[0047] Figure 4 This is a structural diagram of another incremental data synchronization device provided in an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0049] The core of this invention is to provide a method, apparatus, and medium for incremental data synchronization. A snapshot is taken of the backup volume only when data changes, reducing system load and improving system read / write performance. The fast-diff feature ensures that the absence of a snapshot does not necessarily mean the current data has not been updated, thus identifying incremental data and avoiding resource consumption.

[0050] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] It should be noted that how the primary volume detects data changes and the nature of those changes typically involves calculating the difference between the current volume data and the data from the last synchronization. If a difference exists, this difference is used as incremental data and synchronized to the backup volume. The incremental data synchronization method provided in this invention relies on the fast-diff feature of the data volume. Whether the fast-diff feature supports fast comparison operations between snapshots depends on the object-map feature. Whether the object-map feature supports object bitmaps, primarily used to accelerate import, export, and used capacity statistics, depends on the exclusive-lock feature. The object-map feature of the data volume refers to the maintenance of a 2-bit bitmap within the metadata. Each bitmap represents a data object, describing the relationship between the current data of the data volume and the snapshot. Each unit in the bitmap has four states: 0, 1, 2, and 3. 0 indicates that the data object contains no data; 1 indicates that data was updated after the last snapshot or that the data was modified even without a snapshot; 2 indicates that the data was not updated after the last snapshot; and 3 indicates an object to be deleted.

[0052] The Fast-Diff feature refers to updating the object-map value whenever volume data is updated. Normally, if Fast-Diff is not enabled, creating a snapshot of a data volume will not set the corresponding bitmap to 2, because updating the bitmap also takes time. By comparing the changes between the current object-map and a specific snapshot, it's possible to determine whether the object has changed since that snapshot, thus achieving the goal of quickly identifying whether data has changed.

[0053] This invention, by making reasonable use of the fast-diff feature and by recording changes in snapshot identification numbers (IDs), can quickly calculate the data read and write behavior of the main volume resynchronization cycle, making decisions for further periodic incremental synchronization. This allows for rapid periodic synchronization in most cases, thereby avoiding unnecessary system consumption and improving performance and user experience.

[0054] Remote snapshot replication is an authorized feature for disaster recovery. This feature asynchronously (batch) replicates block-level data from a volume on the local storage system to volumes on the same system or different, independent systems. These different systems can be located at the same site as the first system or at a different site. The primary volume is the data source in the replication set and can be mapped to the host. For disaster recovery purposes, if the primary volume goes offline, the standby volume is designated as the primary volume. The primary volume resides on the primary virtual disk on the primary system. The standby volume is the target of the data in the replication set and is not accessible to the host. For disaster recovery purposes, if the primary volume goes offline, the standby volume is designated as the primary volume. The standby volume resides on the standby virtual disk on the standby system. Replication snapshots are a special type of snapshot that preserves the data state of the primary volume in the replication set at the time the snapshot is created. For the primary volume, after the replication process has completed copying the primary volume data to the standby volume, a replication snapshot is created simultaneously on both the primary and standby systems. Although replication snapshots are counted in the system's maximum number of volumes, they are not mappable and do not count towards license limits. Replication snapshots can be exported as regular licensed snapshots.

[0055] Figure 1 A flowchart of an incremental data synchronization method provided in an embodiment of the present invention is shown below. Figure 1 As shown, the method includes:

[0056] S11: Based on the current time, obtain the current snapshot ID number of the main volume, the snapshot ID number of the previous synchronization period, and the initial synchronization snapshot ID number;

[0057] S12: Determine the periodic synchronization status of the snapshot identity number in the previous synchronization period based on the relationship between the current snapshot identity number and the snapshot identity number in the previous synchronization period, and determine the snapshot creation status of the main volume based on the corresponding periodic synchronization status and the relationship between the baseline initial synchronization snapshot identity number.

[0058] S13: Determine whether the snapshot creation status is not created. If yes, proceed to step S14; otherwise, proceed to step S15.

[0059] S14: Use the fast difference feature to obtain data under each snapshot identity number and compare them to determine incremental data;

[0060] S15: Retrieve the data corresponding to the created snapshot as incremental data.

[0061] Specifically, using the current time as a reference, the current snapshot ID of the primary volume snapshot is obtained. This can be understood as working backwards from the current time, finding the snapshot ID closest to the current time. The previous synchronization cycle snapshot ID is based on the current time. For example, if the synchronization cycle is every Friday, and today is Thursday, then the previous synchronization cycle snapshot ID corresponds to the snapshot ID from the synchronization cycle last week. The baseline initial synchronization snapshot ID is the snapshot ID from the initial full synchronization before the first synchronization cycle, or the snapshot ID corresponding to the primary volume initialization.

[0062] The periodic synchronization status of the snapshot ID in the previous synchronization period is determined by the relationship between the current snapshot ID and the snapshot ID of the previous synchronization period. It's important to understand that the periodic synchronization status primarily determines whether it's the first periodic synchronization. If the two snapshot IDs are the same, it means that the user did not create a snapshot during this synchronization period. When the remote service performs periodic synchronization for the first time, the two will definitely be different; at this time, the snapshot ID of the previous synchronization period will be 0.

[0063] As one embodiment, determining the periodic synchronization status of the snapshot ID of the previous synchronization period based on the relationship between the current snapshot ID and the snapshot ID of the previous synchronization period includes:

[0064] Determine if the current snapshot ID is the same as the snapshot ID from the previous synchronization cycle;

[0065] If they are the same, then it is confirmed that no snapshot has been created on the primary volume during the current synchronization cycle;

[0066] If they are different, then check if the snapshot ID of the previous synchronization cycle is 0;

[0067] If it is 0, then the snapshot ID of the previous synchronization period is determined to be the first periodic synchronization;

[0068] If it is not 0, then the snapshot ID of the previous synchronization cycle is determined to be a non-first-cycle synchronization.

[0069] Specifically, the system checks if the current snapshot ID is the same as the snapshot ID from the previous synchronization period. If they are the same, it means that the user did not create any snapshots during the synchronization period from the previous period to the current period, and the primary volume has no snapshots created during the current synchronization period. If the two snapshot IDs are different, the system checks if the snapshot ID from the previous synchronization period is 0 to determine if the previous synchronization period was the first synchronization period. If the snapshot ID from the previous synchronization period is 0, it is the first synchronization period; otherwise, it is not the first synchronization period.

[0070] The snapshot creation status of the primary volume is determined based on different periodic synchronization conditions and the relationship with the baseline initial synchronization snapshot ID. The method for determining the snapshot creation status varies depending on the specific periodic synchronization condition. For example, if there are several categories of periodic synchronization conditions, then there will also be several methods for determining the snapshot creation status.

[0071] As one embodiment, when the snapshot ID of the previous synchronization cycle is the first cycle synchronization, the snapshot creation status of the main volume is determined based on the relationship between the corresponding cycle synchronization status and the baseline initial synchronization snapshot ID, including:

[0072] Determine whether the snapshot ID of the previous synchronization cycle is the same as the baseline initial synchronization snapshot ID;

[0073] If they are the same, then the snapshot creation status of the main volume is determined to be not created;

[0074] If they are different, then the snapshot creation status of the main volume is determined to be created.

[0075] As one method to determine the snapshot creation status, when the snapshot ID of the previous synchronization cycle is the first cycle synchronization, it is checked whether the snapshot ID of the previous synchronization cycle is the same as the baseline initial synchronization snapshot ID. If they are the same, it means that no other snapshots have been created on the primary volume since the last synchronization, and the snapshot creation status of the primary volume is determined to be uncreated. If they are different, it means that other snapshots have been taken on the primary volume since the last synchronization, and the snapshot creation status of the primary volume is determined to be created.

[0076] As another embodiment, when the snapshot ID of the previous synchronization cycle is not the first cycle synchronization, the snapshot creation status of the main volume is determined according to the relationship between the corresponding cycle synchronization status and the baseline initial synchronization snapshot ID, including:

[0077] If the snapshot ID of the previous synchronization cycle was not the first synchronization cycle and the current snapshot ID is different from the snapshot ID of the previous synchronization cycle, then the snapshot creation status of the main volume is determined to be created.

[0078] If the snapshot ID of the previous synchronization cycle was not the first synchronization cycle and the current snapshot ID is the same as the snapshot ID of the previous synchronization cycle, then the snapshot creation status of the main volume is determined to be not created.

[0079] As one way to determine the snapshot creation status, if the snapshot ID of the previous synchronization cycle was not the first synchronization cycle, and the current snapshot ID is different from the snapshot ID of the previous synchronization cycle, it means that a new snapshot has been generated since the last synchronization cycle, indicating that other snapshots have been taken on the main volume since the last synchronization. In this case, we can directly take a new snapshot to determine that the snapshot creation status of the main volume is created.

[0080] If the snapshot ID of the previous synchronization cycle was not the first synchronization cycle, and the current snapshot ID is the same as the snapshot ID of the previous synchronization cycle, it means that no new snapshot has been generated from the last synchronization cycle to the present, and the snapshot status of the main volume is determined to be uncreated.

[0081] Based on the above determination of the snapshot creation status of the main volume, when no snapshot has been created, the data under each snapshot ID is obtained through the fast-diff feature for comparison to determine incremental data; data differences are compared by obtaining the object map and using the fast-diff feature. The method is to obtain the current data of the main volume and compare it with the snapshot at the time of the last synchronization. If no data has been written, the corresponding bitmap value is 0.

[0082] If a snapshot has already been created, a new snapshot is created, and the data differences corresponding to the new snapshot are compared to achieve incremental data import. It can use object-map to identify which objects have changed, and then read the changed data objects to synchronize to the backup. It should be noted that although both object-map and fast-diff features use bitmap markers to obtain data changes, this embodiment identifies which objects have changed when a snapshot has already been created using object-map, unlike the method used by fast-diff to obtain incremental data when data updates may occur without a snapshot. This distinction is made here.

[0083] This invention provides an incremental data synchronization method, comprising: obtaining the current snapshot ID, the snapshot ID of the previous synchronization period, and the baseline initial synchronization snapshot ID of the primary volume based on the current time; determining the periodic synchronization status of the snapshot ID of the previous synchronization period based on the relationship between the current snapshot ID and the snapshot ID of the previous synchronization period, and determining the snapshot creation status of the primary volume based on the relationship between the corresponding periodic synchronization status and the baseline initial synchronization snapshot ID; when the snapshot creation status is not created, obtaining data under each snapshot ID through the fast-diff feature for comparison to determine incremental data; when the snapshot creation status is created, obtaining the data corresponding to the created snapshot as incremental data. This method, through the fast-diff feature updating the object-map value, can quickly obtain information on whether data has changed, enabling resource-efficient acquisition of data objects during incremental data synchronization. Opening the volume only requires reading, and further processing is only required when a snapshot needs to be taken, necessitating a write operation. Snapshots are taken only when data changes, reducing system load and improving system read / write performance. The fast-diff feature ensures that the absence of a snapshot does not necessarily mean the current data has not been updated, thus determining incremental data and avoiding resource consumption.

[0084] As one example, when the snapshot creation status is "not created", the data under each snapshot ID is obtained through the fast-diff feature and compared to determine the incremental data, including:

[0085] The current data of the primary volume and the snapshot data under the snapshot ID of the previous synchronization cycle can be obtained through the fast-diff feature under the object map feature;

[0086] Determine if the current data and the snapshot data are the same;

[0087] If they are the same, then the bitmap corresponding to the main volume is set to 0;

[0088] If they are different, the difference between the current data of the main volume and the snapshot data is determined as the incremental data.

[0089] Specifically, by comparing the current data of the fast-diff feature under the current object map feature with the snapshot data under the snapshot ID of the previous synchronization cycle, it is possible to determine whether the current object has changed since a certain snapshot, thereby achieving the purpose of quickly identifying whether the data has changed. It can quickly calculate the data read and write behavior of the main volume during the synchronization cycle, making a decision dependency for further periodic incremental synchronization, so that periodic synchronization can be completed quickly in most cases, thereby avoiding useless system consumption and improving performance and user experience.

[0090] The method involves comparing the current data of the main volume with the snapshot from the last synchronization. If no data has been written, the corresponding bitmap value is 0; otherwise, the data is considered to have changed. During the comparison, since multiple snapshots are compared, the object-map value used for the fast-diff comparison represents the change from the last snapshot to the current time. If multiple snapshots were taken after the last synchronization, each snapshot needs to be compared individually. As one implementation, the current data and the snapshot data under the snapshot ID of the previous synchronization period are obtained by comparing the data between every two snapshots corresponding to the current data and the snapshot data under the snapshot ID of the previous synchronization period.

[0091] Specifically, Figure 2 This is a schematic diagram of a volume snapshot change provided in an embodiment of the present invention, such as... Figure 2 As shown, the process compares the data changes between each pair of snapshots (snap 1, snap 2, etc.) from the snapshot pair's identification number (pair_snap_id) to the current time. Finally, the changes are summarized to obtain the changes from the current time to the snapshot ID of the previous synchronization cycle. Since the snapshots in the main volume grow linearly, there is one snapshot for each corresponding time period. This allows for the summation of every two snapshots, providing a snapshot summary from the previous synchronization cycle to the current time for comparison.

[0092] As one embodiment, when the bitmap corresponding to the main volume is 0, the method further includes:

[0093] End the current period synchronization and update the current period synchronization time.

[0094] Specifically, when the bitmap corresponding to the main volume is 0, it means that no data has been written or changed, and the current synchronization cycle ends directly, and the synchronization time is updated directly.

[0095] The incremental data synchronization process provided in this embodiment involves two scenarios. One scenario involves comparing data differences using the fast-diff feature of the object-map. This is done by comparing the current data on the main volume with the snapshot from the last synchronization. If no data has been written, the corresponding bitmap value is 0; otherwise, the data is considered changed. If no change is considered, the current synchronization cycle ends directly, and the synchronization time is updated. The other scenario involves creating a new snapshot. If the data has changed, a new snapshot is created, and the difference between the new snapshot and the pair_snap_id is compared to achieve incremental data import.

[0096] The foregoing has described in detail various embodiments of the incremental data synchronization method. Based on this, the present invention also discloses an apparatus for incremental data synchronization corresponding to the above-described method. Figure 3 This is a structural diagram of an incremental data synchronization device provided in an embodiment of the present invention. Figure 3 As shown, the device for incremental data synchronization includes:

[0097] Module 11 is used to obtain the current snapshot ID of the main volume, the snapshot ID of the previous synchronization cycle, and the baseline initial synchronization snapshot ID, based on the current time.

[0098] The first determining module 12 is used to determine the periodic synchronization status of the snapshot ID of the previous synchronization period based on the relationship between the current snapshot ID and the snapshot ID of the previous synchronization period, and to determine the snapshot creation status of the main volume based on the relationship between the corresponding periodic synchronization status and the baseline initial synchronization snapshot ID.

[0099] The second determining module 13 is used to determine incremental data by comparing the data under each snapshot ID through the fast-diff feature when the snapshot creation status is not created.

[0100] The third determining module 14 is used to obtain the data corresponding to the created snapshot as incremental data when the snapshot creation status is "created".

[0101] Since the embodiments of the device part correspond to the embodiments described above, please refer to the embodiments described in the method part for the embodiments of the device part, and will not be repeated here.

[0102] This invention provides an incremental data synchronization apparatus, comprising: obtaining the current snapshot ID, the snapshot ID of the previous synchronization period, and the baseline initial synchronization snapshot ID of the primary volume based on the current time; determining the periodic synchronization status of the snapshot ID of the previous synchronization period according to the relationship between the current snapshot ID and the snapshot ID of the previous synchronization period, and determining the snapshot creation status of the primary volume according to the relationship between the corresponding periodic synchronization status and the baseline initial synchronization snapshot ID; when the snapshot creation status is not created, obtaining data under each snapshot ID through the fast-diff feature for comparison to determine incremental data; when the snapshot creation status is created, obtaining the data corresponding to the created snapshot as incremental data. This apparatus, through the fast-diff feature updating the object-map value, can quickly obtain information on whether data has changed, enabling resource-efficient acquisition of data objects during incremental data synchronization. Opening the volume only requires reading, and further processing is only required when a snapshot needs to be taken, necessitating a write operation. Snapshots are taken only when data changes, reducing system load and improving system read / write performance. The fast-diff feature ensures that the absence of a snapshot does not necessarily mean the current data has not been updated, thus determining incremental data and avoiding resource consumption.

[0103] Figure 4 A structural diagram of another incremental data synchronization device provided in an embodiment of the present invention is shown below. Figure 4 As shown, the device includes:

[0104] Memory 21 is used to store computer programs;

[0105] Processor 22, steps for implementing incremental data synchronization when executing a computer program.

[0106] The device for incremental data synchronization provided in this embodiment may include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0107] The processor 22 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 22 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 22 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 22 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 22 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0108] The memory 21 may include one or more computer-readable storage media, which may be non-transitory. The memory 21 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 21 is used to store at least the following computer program 211, which, after being loaded and executed by the processor 22, is capable of implementing the relevant steps of the incremental data synchronization method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 21 may also include an operating system 212 and data 213, etc., and the storage method may be temporary storage or permanent storage. The operating system 212 may include Windows, Unix, Linux, etc. The data 213 may include, but is not limited to, the data involved in the incremental data synchronization method, etc.

[0109] In some embodiments, the device for incremental data synchronization may further include a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27.

[0110] Those skilled in the field can understand, Figure 4 The structure shown does not constitute a limitation on the device for incremental data synchronization and may include more or fewer components than illustrated.

[0111] The processor 22 implements the incremental data synchronization method provided in any of the above embodiments by calling instructions stored in the memory 21.

[0112] This invention provides an incremental data synchronization apparatus, comprising: obtaining the current snapshot ID, the snapshot ID of the previous synchronization period, and the baseline initial synchronization snapshot ID of the primary volume based on the current time; determining the periodic synchronization status of the snapshot ID of the previous synchronization period according to the relationship between the current snapshot ID and the snapshot ID of the previous synchronization period, and determining the snapshot creation status of the primary volume according to the relationship between the corresponding periodic synchronization status and the baseline initial synchronization snapshot ID; when the snapshot creation status is not created, obtaining data under each snapshot ID through the fast-diff feature for comparison to determine incremental data; when the snapshot creation status is created, obtaining the data corresponding to the created snapshot as incremental data. This apparatus, through the fast-diff feature updating the object-map value, can quickly obtain information on whether data has changed, enabling resource-efficient acquisition of data objects during incremental data synchronization. Opening the volume only requires reading, and further processing is only required when a snapshot needs to be taken, necessitating a write operation. Snapshots are taken only when data changes, reducing system load and improving system read / write performance. The fast-diff feature ensures that the absence of a snapshot does not necessarily mean the current data has not been updated, thus determining incremental data and avoiding resource consumption.

[0113] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by processor 22, implements the steps of the incremental data synchronization method described above.

[0114] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] This invention provides a computer medium comprising: acquiring the current snapshot ID, the snapshot ID of the previous synchronization period, and the baseline initial synchronization snapshot ID of the primary volume based on the current time; determining the periodic synchronization status of the snapshot ID of the previous synchronization period according to the relationship between the current snapshot ID and the snapshot ID of the previous synchronization period, and determining the snapshot creation status of the primary volume according to the relationship between the corresponding periodic synchronization status and the baseline initial synchronization snapshot ID; when the snapshot creation status is not created, acquiring data under each snapshot ID through the fast-diff feature for comparison to determine incremental data; when the snapshot creation status is created, acquiring the data corresponding to the created snapshot as incremental data. This medium, through the fast-diff feature updating the object-map value, can quickly obtain information on whether data has changed, enabling resource-efficient acquisition of data objects during incremental data synchronization. Opening the volume only requires reading, and further processing is only required when a snapshot needs to be taken, necessitating a write operation. Snapshots are taken only when data changes, reducing system load and improving system read / write performance. The fast-diff feature ensures that the absence of a snapshot does not necessarily mean the current data has not been updated, thus determining incremental data and avoiding resource consumption.

[0116] The above provides a detailed description of the incremental data synchronization method, apparatus, and medium provided by the present invention. The various embodiments are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0117] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for incremental data synchronization, characterized in that, include: Based on the current time, obtain the current snapshot ID number of the main volume, the snapshot ID number of the previous synchronization period, and the initial synchronization snapshot ID number; The periodic synchronization status of the snapshot identity number in the previous synchronization period is determined based on the relationship between the current snapshot identity number and the snapshot identity number in the previous synchronization period, and the snapshot creation status of the main volume is determined based on the corresponding periodic synchronization status and the relationship between the baseline initial synchronization snapshot identity number. When the snapshot creation status is not created, the data under each snapshot identification number is obtained through the fast difference feature and compared to determine the incremental data; When the snapshot creation status is "created", the data corresponding to the created snapshot is obtained as the incremental data; Correspondingly, determining the periodic synchronization status of the snapshot identity number in the previous synchronization period based on the relationship between the current snapshot identity number and the snapshot identity number in the previous synchronization period includes: Determine whether the current snapshot identification number is the same as the snapshot identification number of the previous synchronization period; If they are the same, then it is confirmed that the primary volume has not been snapshotted during the current synchronization cycle; If they are different, then determine whether the identity identifier number of the previous synchronization cycle snapshot is 0; If the value is 0, then the identity number of the previous synchronization cycle snapshot is determined to be the first cycle synchronization; If it is not 0, then the identity number of the previous synchronization cycle snapshot is determined to be a non-first-cycle synchronization; Correspondingly, when the previous synchronization cycle snapshot identification number is the first cycle synchronization, determining the snapshot creation status of the main volume based on the corresponding cycle synchronization status and the relationship between the baseline initial synchronization snapshot identification number includes: Determine whether the identity identifier number of the previous synchronization cycle snapshot is the same as the identity identifier number of the baseline initial synchronization snapshot; If they are the same, then the snapshot creation status of the main volume is determined to be not created; If they are different, then the snapshot creation status of the main volume is determined to be created; Correspondingly, when the identity identifier number of the previous synchronization cycle snapshot is not the first cycle synchronization, determining the snapshot creation status of the main volume based on the relationship between the corresponding cycle synchronization status and the identity identifier number of the baseline initial synchronization snapshot includes: If the identity identifier number of the previous synchronization cycle snapshot is not the first cycle synchronization and the identity identifier number of the current snapshot is different from the identity identifier number of the previous synchronization cycle snapshot, then the snapshot creation status of the main volume is determined to be created; If the identity identifier number of the previous synchronization cycle snapshot is not the first synchronization cycle and the identity identifier number of the current snapshot is the same as the identity identifier number of the previous synchronization cycle snapshot, then the snapshot creation status of the main volume is determined to be not created.

2. The incremental data synchronization method according to claim 1, characterized in that, The step of comparing data obtained from each snapshot identity number using fast difference characteristics to determine incremental data includes: The current data of the main volume and the snapshot data under the snapshot identity number of the previous synchronization cycle are obtained through the fast difference feature under the object mapping feature; Determine whether the current data and the snapshot data are the same; If they are the same, then the bitmap corresponding to the main volume is determined to be 0; If they are different, the difference between the current data of the main volume and the snapshot data is determined as the incremental data.

3. The incremental data synchronization method according to claim 2, characterized in that, The current data and the snapshot data under the identity number of the previous synchronization cycle snapshot are obtained by acquiring the data between every two snapshots between the snapshot corresponding to the current data and the snapshot data under the identity number of the previous synchronization cycle snapshot.

4. The incremental data synchronization method according to claim 2, characterized in that, When the bitmap corresponding to the main volume is 0, the following is also included: End the current period synchronization and update the synchronization time of the current period.

5. An apparatus for incremental data synchronization, characterized in that, include: The acquisition module is used to acquire the current snapshot ID number, the snapshot ID number of the previous synchronization period, and the baseline initial synchronization snapshot ID number of the main volume, based on the current time. The first determining module is used to determine the periodic synchronization status of the snapshot identity number of the previous synchronization period based on the relationship between the current snapshot identity number and the snapshot identity number of the previous synchronization period, and to determine the snapshot creation status of the main volume based on the corresponding periodic synchronization status and the relationship between the baseline initial synchronization snapshot identity number. The second determining module is used to determine incremental data by comparing the data under each snapshot identification number obtained through the fast difference feature when the snapshot creation status is not created. The third determining module is used to obtain the data corresponding to the created snapshot as the incremental data when the snapshot creation status is "created"; Correspondingly, determining the periodic synchronization status of the snapshot identity number in the previous synchronization period based on the relationship between the current snapshot identity number and the snapshot identity number in the previous synchronization period includes: Determine whether the current snapshot identification number is the same as the snapshot identification number of the previous synchronization period; If they are the same, then it is confirmed that the primary volume has not been snapshotted during the current synchronization cycle; If they are different, then determine whether the identity identifier number of the previous synchronization cycle snapshot is 0; If the value is 0, then the identity number of the previous synchronization cycle snapshot is determined to be the first cycle synchronization; If it is not 0, then the identity number of the previous synchronization cycle snapshot is determined to be a non-first-cycle synchronization; Correspondingly, when the previous synchronization cycle snapshot identification number is the first cycle synchronization, determining the snapshot creation status of the main volume based on the corresponding cycle synchronization status and the relationship between the baseline initial synchronization snapshot identification number includes: Determine whether the identity identifier number of the previous synchronization cycle snapshot is the same as the identity identifier number of the baseline initial synchronization snapshot; If they are the same, then the snapshot creation status of the main volume is determined to be not created; If they are different, then the snapshot creation status of the main volume is determined to be created; Correspondingly, when the identity identifier number of the previous synchronization cycle snapshot is not the first cycle synchronization, determining the snapshot creation status of the main volume based on the relationship between the corresponding cycle synchronization status and the identity identifier number of the baseline initial synchronization snapshot includes: If the identity identifier number of the previous synchronization cycle snapshot is not the first cycle synchronization and the identity identifier number of the current snapshot is different from the identity identifier number of the previous synchronization cycle snapshot, then the snapshot creation status of the main volume is determined to be created; If the identity identifier number of the previous synchronization cycle snapshot is not the first synchronization cycle and the identity identifier number of the current snapshot is the same as the identity identifier number of the previous synchronization cycle snapshot, then the snapshot creation status of the main volume is determined to be not created.

6. An apparatus for incremental data synchronization, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the incremental data synchronization method as described in any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the incremental data synchronization method as described in any one of claims 1 to 4.

Images