A snapshot capacity calculation method and device based on multi-version data

By establishing a snapshot chain in a multi-version data environment and managing the version number, the complexity and inaccuracy of snapshot capacity calculation are solved, and efficient and accurate snapshot capacity statistics are achieved, which is suitable for cloud storage systems.

CN117873958BActive Publication Date: 2024-11-26CHINA TELECOM CLOUD TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311729561.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-15
Publication Date
2024-11-26
Estimated Expiration
2043-12-15

AI Technical Summary

Technical Problem

Existing snapshot capacity calculation methods are highly complex, data integrity and consistency are difficult to guarantee in multi-snapshot environments, capacity estimation is inaccurate, and performance overhead is high, especially in incremental snapshots and rollback operations, which are prone to repeated calculations. .

Method used

Using the snapshot capacity calculation method based on multi-version data, the first full snapshot and subsequent incremental snapshot are written to the data block, the snapshot chain is established, the version number interval is accumulated and the redundant version number is eliminated, and the total capacity of the snapshot chain is calculated based on the data block size. And perform the difference calculation when adding or deleting snapshots to avoid repeated calculations.

Benefits of technology

High-performance snapshot capacity calculation is realized, ensuring the accuracy of capacity statistics, avoiding memory space waste, and improving the efficiency of snapshot chain capacity statistics.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117873958B_ABST
    Figure CN117873958B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of storage data management, and provides a method and device for calculating snapshot capacity based on multi-version data, including: writing multiple data blocks through the first full snapshot and subsequent incremental snapshots to form a snapshot chain corresponding to a version number interval for each snapshot; accumulating all version numbers in the version number interval of the snapshot and removing the redundant version numbers generated by rollback to obtain the total capacity of the snapshot chain; when adding a new snapshot, removing the redundant version number of the rollback-related snapshot from the version number interval of the new snapshot to obtain the capacity of the new single snapshot; when deleting a snapshot, retaining the referenced version number in the rollback-related snapshot of the deleted snapshot, and correspondingly inheriting it by the referenced subject, incrementing the version number of the referenced subject according to the inherited referenced version number and removing the version number of the deleted snapshot to obtain the capacity of the deleted single snapshot; updating the total capacity of the snapshot chain. The present invention can avoid repeated calculations and efficiently and correctly calculate the capacity of the snapshot chain.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of storage data management, and in particular relates to a snapshot capacity calculation method and device based on multi-version data. Background Art

[0002] With the rise of cloud computing, snapshot technology has been widely used. Cloud service providers use snapshots to provide services such as data backup, disaster recovery, and data replication. Users can easily create, manage, and use snapshots without investing a lot of local storage resources.

[0003] In cloud computing environments, snapshot capacity statistics is a critical task that helps users manage their cloud storage resources and ensure sufficient space for storing snapshots and backup data. Existing snapshot capacity calculations still have some potential flaws and challenges: Snapshot chain complexity: When there are multiple snapshots that depend on previous snapshots, calculating capacity and managing snapshot chains may become complicated, and data integrity and consistency are difficult to ensure; Inaccurate capacity estimation: Existing technologies (such as ceph, gfs) usually calculate snapshot capacity based on recording data differences between snapshots, but incremental snapshots and snapshot rollback operations result in a large number of referenced data blocks in the snapshot chain, resulting in repeated calculations; Performance overhead: Currently, the data differences between snapshots are mainly recorded in bitmap format. If the data block granularity is too large, the calculated value will be significantly different from the actual capacity; if the data block granularity is too small, it will consume more storage resources and memory space. Summary of the invention

[0004] In view of the above deficiencies in the prior art, an object of the present invention is to provide a method and device for calculating snapshot capacity based on multi-version data.

[0005] A first aspect of the present invention provides a method for calculating snapshot capacity based on multi-version data, comprising:

[0006] S1: writing multiple data blocks through the first full snapshot and subsequent incremental snapshots to establish a snapshot chain, wherein the snapshot chain includes multiple snapshots corresponding to version number intervals;

[0007] S2: Accumulate the total number of all version numbers in the version number range of each snapshot and remove the redundant version numbers generated by rollback. Combined with the data block size corresponding to a single snapshot, calculate the total capacity of the snapshot chain.

[0008] S3: When a new snapshot is added to the snapshot chain, the redundant version numbers of all snapshots associated with the rollback are removed from the version number range of the new snapshot, and the capacity of the new single snapshot is calculated;

[0009] S4: When a snapshot is deleted from the snapshot chain, the referenced version number in the rollback associated snapshot of the deleted snapshot is retained, and the referenced version number is correspondingly inherited by the referenced subject. The version number of the referenced subject is incremented according to the corresponding inherited referenced version number and the version number of the deleted snapshot is removed to calculate the deleted single snapshot capacity;

[0010] S5: updating the total capacity of the snapshot chain according to the newly added single snapshot capacity and the deleted single snapshot capacity to complete the snapshot capacity calculation.

[0011] Furthermore, step S1 is implemented by RoW redirection on write.

[0012] Furthermore, step S1 further includes:

[0013] S11: Associating the first data block written into the storage system with the version number of the current storage system;

[0014] S12: incrementally updating the version number of the storage system in sequence according to the writing order of the data blocks after the first data block;

[0015] S13: establishing a corresponding version number for each data block after the first data block according to the incrementally updated version number, and obtaining multiple snapshots;

[0016] S14: For a single snapshot, the version number and snapshot data of the single snapshot are merged into the next snapshot, and redundant data blocks in the merged snapshot are deleted to obtain the snapshot chain.

[0017] Furthermore, step S1 also includes:

[0018] S151: When a snapshot in the snapshot chain is rolled back, all non-redundant data blocks between the initial version number corresponding to the first data block and the maximum version number corresponding to the rolled back snapshot are traversed:

[0019] S152: Create a new version number index for each non-redundant data block;

[0020] S153: After the snapshot rollback is completed, the current version number of the storage system is incrementally updated again based on the new version number index.

[0021] Furthermore, the new version number index established in step S152 is greater than the version number of the current storage system.

[0022] Furthermore, the expression of the total capacity of the snapshot chain in step S2 is:

[0023] δ=(∑S z -∑S h )S;

[0024] Among them, δ is the calculated total capacity of the snapshot chain, S z is the total number of all version numbers in the version number range of all snapshots on the snapshot chain, S h The total amount of redundant version numbers generated for rollback, p is the size of a single data block.

[0025] Furthermore, the reference subject in step S4 is an upstream snapshot that is rolled back to the deleted snapshot.

[0026] A second aspect of the present invention provides a snapshot capacity calculation device based on multi-version data, comprising:

[0027] Snapshot chain establishment module: used to write multiple data blocks through the first full snapshot and subsequent incremental snapshots to establish a snapshot chain, wherein the snapshot chain includes multiple snapshots corresponding to version number intervals;

[0028] Total amount calculation module: used to accumulate the total amount of all version numbers in the version number range of each snapshot and eliminate the redundant version numbers generated by rollback, and calculate the total capacity of the snapshot chain based on the data block size corresponding to a single snapshot;

[0029] Incremental calculation module: when a new snapshot is added to the snapshot chain, the redundant version numbers of all snapshots associated with the rollback are removed from the version number interval of the new snapshot, and the capacity of the new single snapshot is calculated;

[0030] Reduction calculation module: when a snapshot is deleted from the snapshot chain, the referenced version number in the rollback associated snapshot of the deleted snapshot is retained, and the referenced version number is correspondingly inherited by the referenced subject. The version number of the referenced subject is incremented according to the corresponding inherited referenced version number and the version number of the deleted snapshot is removed to calculate the deleted single snapshot capacity;

[0031] Total amount update module: used to update the total capacity of the snapshot chain according to the newly added single snapshot capacity and the deleted single snapshot capacity to complete the snapshot capacity calculation.

[0032] A third aspect of the present invention provides a snapshot capacity calculation device based on multi-version data, comprising:

[0033] A memory and at least one processor, wherein instructions are stored in the memory;

[0034] At least one of the processors calls the instructions in the memory to enable the document database audit device to execute a snapshot capacity calculation method based on multi-version data as described in any one of the above items.

[0035] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, and when the instructions are executed by a processor, a snapshot capacity calculation method based on multi-version data as described in any one of the above items is implemented.

[0036] The beneficial effects of the present invention are as follows:

[0037] The present invention provides a snapshot capacity calculation method and device based on multi-version data, aiming to realize an algorithm with high calculation performance and capable of ensuring snapshot capacity accuracy. The algorithm adopts a finer-grained version number to manage data differences between different snapshots based on RoW technology and multi-version data management technology, thereby solving the problems of inaccurate snapshot capacity and wasted memory space. The algorithm can avoid repeated calculation of data blocks when there are referenced data blocks in incremental snapshots and rollbacks, thereby ensuring the accuracy of capacity statistics, and uses a differential method to calculate the changes in snapshot capacity in the snapshot chain after creating and deleting snapshots, thereby improving the performance of statistical snapshot chain capacity. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are only used to illustrate specific embodiments and are not considered to limit the present invention. In the entire drawings, the same reference symbols represent the same components. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0039] Figure 1 A flow chart of a snapshot capacity calculation method based on multi-version data provided by an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of a snapshot capacity calculation device based on multi-version data provided by an embodiment of the present invention;

[0041] Figure 3 It is a schematic diagram of the principle of calculating the total capacity of a snapshot chain and the snapshot chain established in a snapshot capacity calculation method based on multi-version data provided by an embodiment of the present invention;

[0042] Figure 4 It is a schematic diagram of calculating the capacity of a single snapshot and the capacity of a newly added single snapshot in a snapshot capacity calculation method based on multi-version data provided by an embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram of calculating the capacity of a deleted single snapshot in a snapshot capacity calculation method based on multi-version data provided by an embodiment of the present invention;

[0044] Figure 6The invention is a schematic diagram of the inheritance principle of deleting redundant data blocks in single snapshot capacity calculation in a snapshot capacity calculation method based on multi-version data provided by an embodiment of the present invention.

[0045] Reference numerals:

[0046] 100, snapshot chain establishment module; 200, total amount calculation module; 300, increment calculation module; 400, decrement calculation module; 500, total amount update module. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work should fall within the scope of protection of the present invention.

[0048] Furthermore, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts disclosed in the present invention.

[0049] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the orientation or position relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. The terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0050] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of methods and systems consistent with some aspects of the present invention as detailed in the appended claims.

[0051] In order to better understand the present invention, some special terms appearing in the present invention are explained below.

[0052] RoW: Redirect on Write, redirect on write;

[0053] LBA: Logical Block Address, logical block address;

[0054] CoW: Copy on Write, copy on write.

[0055] Combine the following Figures 1 to 6 Embodiments of the present invention are described.

[0056] Method Embodiment

[0057] A first aspect of the present invention provides a method for calculating snapshot capacity based on multi-version data, comprising:

[0058] S1: writing multiple data blocks through the first full snapshot and subsequent incremental snapshots to establish a snapshot chain, wherein the snapshot chain includes multiple snapshots corresponding to version number intervals;

[0059] Furthermore, the system uses RoW technology to form a snapshot chain through the first full snapshot and subsequent incremental snapshots to complete the storage of historical data, where each snapshot has a corresponding version number range.

[0060] Wherein, step S1 is implemented by RoW write-time redirection.

[0061] Wherein, step S1 further comprises:

[0062] S11: Associating the first data block written into the storage system with the version number of the current storage system;

[0063] S12: incrementally updating the version number of the storage system in sequence according to the writing order of the data blocks after the first data block;

[0064] S13: establishing a corresponding version number for each data block after the first data block according to the incrementally updated version number, and obtaining multiple snapshots;

[0065] S14: For a single snapshot, the version number and snapshot data of the single snapshot are merged into the next snapshot, and redundant data blocks in the merged snapshot are deleted to obtain the snapshot chain.

[0066] Furthermore, the specific process of step S1 includes: writing data: each data block written to the system needs to be associated with the latest version number of the current system, and each data block written to the storage system has a corresponding version number; creating a snapshot: adding 1 to the latest system version number, for example, the version number interval of the previous snapshot is [2,3), the latest system version number is 3, and the version number interval of the snapshot created at this time is [3,4), and the latest system version number is updated to 4; deleting the snapshot: merging the version number interval and data of the snapshot into the next snapshot, and then deleting the redundant data blocks in the snapshot, that is, obtaining the data blocks with the same LBA in the two snapshots.

[0067] Wherein, step S1 also includes:

[0068] S151: When a snapshot in the snapshot chain is rolled back, all non-redundant data blocks between the initial version number corresponding to the first data block and the maximum version number corresponding to the rolled back snapshot are traversed:

[0069] S152: Create a new version number index for each non-redundant data block;

[0070] S153: After the snapshot rollback is completed, the current version number of the storage system is incrementally updated again based on the new version number index.

[0071] The new version number index established in step S152 is greater than the version number of the current storage system.

[0072] Furthermore, the specific process also includes snapshot rollback: since all snapshots except the first one are incremental snapshots, rolling back to a snapshot requires traversing all non-redundant data blocks between [starting version number, maximum version number of the snapshot] and adding indexes of new version numbers to these data blocks. The new version number needs to be greater than the latest version number of the current system. After the rollback is completed, the latest version number of the system is set to the new version number + 1.

[0073] Combine the following Figure 3 The snapshot chain established in step S1 of the snapshot capacity calculation method based on multi-version data provided by the present invention, that is, the multi-version snapshot chain data storage system is introduced.

[0074] Take a single cloud disk data operation as an example. Figure 3The figure shows a simple demonstration of writing 6 blocks of data. The cloud disk forms a snapshot chain in chronological order, and sets the starting version number of the snapshot chain to 1. The first snapshot, which is also a full snapshot, writes 6 new data blocks with version number 1. The second snapshot contains 4 data blocks with version number 2, and the third snapshot contains 3 data blocks with version number 3. Before creating the fourth snapshot, the cloud disk rolls back to the first snapshot, so the fourth snapshot contains version numbers 4 and 5, where the data block with version number 4 is associated with the data block with version number 1. The fifth snapshot contains 2 data blocks with version number 6. Finally, the cloud disk rolls back to the fourth snapshot and creates the sixth snapshot. The sixth snapshot contains version numbers 7 and 8, where the data block with version number 7 is associated with the data blocks with version number interval [1,5].

[0075] From a business perspective, cloud service providers need to count the total capacity of the snapshot chain or the capacity of each snapshot and bill services based on the capacity. Therefore, this application introduces a method for calculating snapshot capacity in a multi-version snapshot chain data storage system from three aspects: total snapshot chain capacity, single snapshot capacity, and differential capacity.

[0076] S2: Accumulate the total number of all version numbers in the version number range of each snapshot and remove the redundant version numbers generated by rollback. Combined with the data block size corresponding to a single snapshot, calculate the total capacity of the snapshot chain.

[0077] Because snapshot rollback will add a new version number index to the data blocks in a certain version number range, directly counting the number of all version numbers will result in duplicate calculation of data blocks. Therefore, it is necessary to eliminate the redundant version numbers generated by the rollback.

[0078] The expression of the total capacity of the snapshot chain in step S2 is:

[0079] δ=(∑S z -∑S h )*p;

[0080] Among them, δ is the calculated total capacity of the snapshot chain, S z is the total number of all version numbers in the version number range of all snapshots on the snapshot chain, S h The total amount of redundant version numbers generated for rollback, p is the size of a single data block.

[0081] Combine the following Figure 3 This section describes how to calculate the total snapshot capacity in step S2.

[0082] The calculation method consists of two steps: the first step is to count the version number in each snapshot; the second step is to count the redundant version numbers that appear after the rollback. Figure 3Take the snapshot chain in as an example, first count the number of all version numbers in the first to sixth snapshots, where the number of version number 1 in the first snapshot is 6; the number of version number 2 in the second snapshot is 4; the number of version number 3 in the third snapshot is 3; the number of version number 4 in the fourth snapshot is 4, the number of version number 5 is 2; the number of version number 6 in the fifth snapshot is 2; the number of version number 7 in the sixth snapshot is 4, the number of version number 8 is 2. Then count the redundant version numbers 4 and 7 caused by two rollbacks in the snapshot chain, where the data blocks of version number 7 are associated with the version number interval [1,5] and all the corresponding data exist, so the 4 blocks of data of version number 7 are not included in the total capacity; similarly, the 4 blocks of data of version number 4 are not included in the total capacity. Finally, the total number of data blocks in the snapshot chain can be obtained as (6+4+3+4+2+2+4+2)-(4+4)=19.

[0083] Combine the following Figure 3 This section describes how to calculate the capacity of a single snapshot.

[0084] by Figure 3 Taking the fourth snapshot in as an example, the fourth snapshot contains version numbers 4 and 5, with a total of 6. The data block with version number 4 is associated with version number 1 and all the corresponding data exists, so the 4 blocks of data with version number 4 are not included in the capacity. Therefore, the number of data blocks in the fourth snapshot is (4+2)-4=2.

[0085] Combine the following Figure 4 This section describes another method for calculating the capacity of a single snapshot.

[0086] Unlike the total capacity of the snapshot chain, which requires obtaining the version number interval of all snapshots, although the total capacity can also calculate the capacity of all snapshots, the efficiency is too low. The capacity of a single snapshot only requires the snapshot version number interval associated with the snapshot and its rollback version number, and all snapshots associated with the rollback version number must be obtained through the algorithm. Therefore, the calculation method is divided into three steps: (1) counting the version number identifiers in the snapshot; (2) obtaining all snapshots associated with the rollback version number in the snapshot; (3) counting the redundant version numbers that appear after the rollback. Methods (1) and (3) are based on the same principle as the total capacity calculation method of the snapshot chain. The method of step (2) is introduced below.

[0087] like Figure 4 Take the snapshot chain in as an example. The snapshot contains 15 snapshots, where the size of the sequence number indicates the creation order. The snapshot with sequence number 1 is the starting full snapshot. The arrow indicates that a rollback operation was performed before the snapshot was created. For example, 2←5 means that the cloud disk rolled back to snapshot 2 before snapshot 5 was created. Now calculate the capacity of snapshot 7. Since snapshot 7 does not contain the rollback version number, you only need to count the number of data blocks in the version number range of snapshot 7.

[0088] The following focuses on calculating the capacity of snapshot 12. Before creating snapshot 12, the cloud disk rolled back to snapshot 10, so snapshot 12 contains the version number generated by the rollback, and the rollback version number should be associated with snapshots 1-10. However, before creating snapshot 8, the cloud disk rolled back to snapshot 3, overwriting the data of snapshots 4-7, so the rollback version number is no longer associated with snapshots 4-7. Because snapshot 5 rolled back to snapshot 2 before, the rollback version number in snapshot 5 is associated with snapshots 1 and 2. According to the previous analysis, snapshot 12 is also associated with snapshots 1 and 2. Therefore, there may be duplicated data blocks between snapshot 5 and snapshot 12. Therefore, the rollback version number of snapshot 12 must be associated with snapshot 5. Similarly, it can be concluded that it also needs to be associated with snapshots 14 and 15.

[0089] Figure 4 All snapshots associated with the rollback version number of snapshot 12 are marked in the figure. Through the above analysis, the associated snapshots can be divided into two categories: the snapshot interval corresponding to the rollback version number, which includes snapshots 1, 2, 3, 8, 9, and 10; the first snapshot created after rolling back to the first type of snapshot, which includes snapshots 5, 14, and 15. In general, it can be defined that if there are multiple version number indexes for the same data block, this data block belongs to the snapshot corresponding to the smallest version number, so the data of snapshots 14 and 15 do not need to be traversed, and only the data blocks in snapshot 12 that are the same as snapshots 1, 2, 3, 5, 8, 9, and 10 need to be removed.

[0090] S3: When a new snapshot is added to the snapshot chain, the redundant version numbers of all snapshots associated with the rollback are removed from the version number range of the new snapshot, and the capacity of the new single snapshot is calculated;

[0091] Furthermore, in a storage system with a large number of snapshots, if the total capacity of the snapshot chain is recalculated every time the snapshot capacity changes, the system performance will be degraded. Therefore, this application proposes a differential capacity calculation method based on a single snapshot. In a multi-version snapshot chain data storage system, only the creation and deletion of snapshots will change the snapshot capacity. The differential capacity calculation method in these two scenarios is described below.

[0092] Combine the following Figure 4 This section describes how to calculate differential capacity when creating a snapshot.

[0093] like Figure 4As shown in the figure, there are 14 snapshots in the current snapshot chain, and the cloud disk is currently rolled back to snapshot 13. After snapshot 15 is created, the snapshot capacity changes. Now the differential statistics method is used to recalculate the capacity. The method is divided into two steps. The first step is to find all snapshots associated with the rollback version number of snapshot 15 according to the single snapshot capacity calculation method, including snapshots 1, 2, 3, 5, 8, 9, 10, 12, 13, and 14; the second step is to calculate the capacity of snapshot 15 using the single snapshot capacity calculation method. According to the definition, when a data block has multiple version numbers, it belongs to the snapshot corresponding to the smallest version number. Therefore, creating a snapshot will not affect other snapshots, and there is no need to update the capacity of other snapshots. At the same time, the total capacity of the snapshot chain needs to be added with the capacity of snapshot 15.

[0094] S4: When a snapshot is deleted from the snapshot chain, the referenced version number in the rollback associated snapshot of the deleted snapshot is retained, and the referenced version number is correspondingly inherited by the referenced subject. The version number of the referenced subject is incremented according to the corresponding inherited referenced version number and the version number of the deleted snapshot is removed to calculate the deleted single snapshot capacity;

[0095] The reference subject in step S4 is the upstream snapshot that is rolled back to the deleted snapshot.

[0096] Combine the following Figure 5 This section describes how to calculate the differential capacity when deleting a snapshot.

[0097] Now delete snapshot 7, such as Figure 5 As shown, the steps for calculating the total capacity of the snapshot chain and the single snapshot capacity after deleting snapshot 7 are as follows: first, find all snapshots associated with the rollback version number of snapshot 7 according to the algorithm in the single snapshot capacity calculation method, including snapshots 1, 2, 3, 4, 5, 10, 12, and 13; secondly, calculate the capacity of snapshot 8, that is, the version number interval containing snapshot 7; then update the snapshot capacity associated with the rollback version number, where the capacity of snapshots 10, 12, and 13 needs to be updated.

[0098] Because some redundant data blocks in snapshot 8 need to be deleted, if these data blocks are referenced by the rollback version numbers in snapshots 10, 12, and 13, then these data blocks cannot be deleted and should be inherited by snapshots 10, 12, and 13. Finally, the total capacity of the snapshot chain plus the increments of snapshots 10, 12, and 13 minus the capacity of snapshot 7.

[0099] Figure 6The figure shows the version numbers of the four data blocks of snapshots 7, 8, 10, and 13. Version number 4 of snapshot 10 and version number 6 of snapshot 13 are rollback version numbers, which are associated with the data blocks of snapshot 7. Now snapshot 7 is deleted. Normally, data blocks 2, 3, and 4 in snapshot 7 need to be deleted, and then data block 1 is inherited by snapshot 8. However, data block 3 of snapshot 7 is associated with version 4 of snapshot 10, so data block 3 cannot be deleted and should be transferred to snapshot 10 for management. Similarly, data block 2 should be transferred to snapshot 13 for management. Finally, the capacity changes of each snapshot can be obtained: the number of data blocks of snapshots 8, 10, and 13 increases by 1 respectively; the change in the total capacity of the snapshot chain is the increment of snapshots 8, 10, and 13 minus the capacity of snapshot 7: (1+1+1)-4=-1.

[0100] S5: updating the total capacity of the snapshot chain according to the newly added single snapshot capacity and the deleted single snapshot capacity to complete the snapshot capacity calculation.

[0101] Device Embodiment

[0102] A second aspect of the present invention provides a snapshot capacity calculation device based on multi-version data, comprising:

[0103] The snapshot chain establishment module 100 is used to write multiple data blocks through the first full snapshot and subsequent incremental snapshots to establish a snapshot chain, wherein the snapshot chain includes multiple snapshots corresponding to version number intervals;

[0104] Total amount calculation module 200: used to accumulate the total amount of all version numbers in the version number interval of each snapshot and remove the redundant version numbers generated by rollback, and calculate the total capacity of the snapshot chain in combination with the data block size corresponding to a single snapshot;

[0105] Increment calculation module 300: when a new snapshot is added to the snapshot chain, redundant version numbers of all snapshots associated with rollback are removed from the version number interval of the new snapshot, and the capacity of the new single snapshot is calculated;

[0106] The reduction calculation module 400 is used to retain the referenced version number in the rollback associated snapshot of the deleted snapshot when the snapshot chain deletes the snapshot, and the referenced version number is correspondingly inherited by the referenced subject, and the version number of the referenced subject is incremented according to the corresponding inherited referenced version number and the version number of the deleted snapshot is removed, so as to calculate and obtain the deleted single snapshot capacity;

[0107] The total amount updating module 500 is used to update the total capacity of the snapshot chain according to the newly added single snapshot capacity and the deleted single snapshot capacity to complete the snapshot capacity calculation.

[0108] A third aspect of the present invention provides a snapshot capacity calculation device based on multi-version data, comprising:

[0109] A memory and at least one processor, wherein instructions are stored in the memory;

[0110] At least one of the processors calls the instructions in the memory to enable the document database audit device to execute a snapshot capacity calculation method based on multi-version data as described in any one of the above items.

[0111] In some embodiments, the device may have relatively large differences due to different configurations or performances, and may include one or more processors, for example, one or more processors and memories, one or more storage media storing applications or data, such as one or more mass storage devices. Among them, the memory and storage medium can be short-term storage or persistent storage. The program stored in the storage medium may include one or more modules, each module may include a series of instruction operations in the device. Furthermore, the processor may be configured to communicate with the storage medium and execute a series of instruction operations in the storage medium on the device.

[0112] The device may also include one or more power supplies, one or more wired or wireless network interfaces, one or more input and output interfaces, and / or one or more operating systems, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that the device structure shown above does not constitute a limitation on the device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0113] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, and when the instructions are executed by a processor, a snapshot capacity calculation method based on multi-version data as described in any one of the above items is implemented.

[0114] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0115] If the integrated unit is implemented in the form of a software functional unit and sold or passed as an independent product, it can be stored in a computer-readable storage medium. Based on the understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0116] The present invention provides a snapshot capacity calculation method and device based on multi-version data. Aiming at the current RoW and multi-version data management technology, a snapshot chain data storage system is proposed, and operation methods for writing data, creating snapshots, deleting snapshots and rolling back snapshots are designed in the system, which ensures the data integrity and consistency of incremental snapshots and rollback operations, and avoids consuming more storage resources and memory space when calculating snapshot capacity; in addition, a method for calculating the total capacity of snapshots in a multi-version snapshot chain data storage system is proposed, which solves the problem of inaccurate capacity calculation caused by data block redundancy in operations such as rollback; on this basis, a differential capacity statistics method is proposed, which calculates the change in snapshot capacity in the snapshot chain after creating and deleting snapshots, thereby improving the efficiency and performance of capacity statistics.

[0117] The present invention provides a method and device for calculating snapshot capacity based on multi-version data, which is applicable to storage system products, including block storage products such as cloud hard disks and local disks. Based on RoW and multi-version data management technology, this application can use version numbers to manage fine-grained data blocks, and use the differential method to calculate the change in snapshot capacity in the snapshot chain after creating and deleting snapshots, which can avoid repeated calculations and efficiently and correctly calculate the capacity of the snapshot chain.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the above embodiments, or replace some of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or replacements that can be easily thought of by those skilled in the art within the technical scope disclosed in the present invention should be covered within the protection scope of the present invention.

Claims

1. A snapshot capacity calculation method based on multi-version data, characterized in that: include: S1: writing multiple data blocks through the first full snapshot and subsequent incremental snapshots to establish a snapshot chain, wherein the snapshot chain includes multiple snapshots corresponding to version number intervals; Wherein, step S1 further comprises: S11: Associating the first data block written into the storage system with the version number of the current storage system; S12: incrementally updating the version number of the storage system in sequence according to the writing order of the data blocks after the first data block; S13: establishing a corresponding version number for each data block after the first data block according to the incrementally updated version number, and obtaining multiple snapshots; S14: For a single snapshot, merge the version number and snapshot data of the single snapshot into the next snapshot, and delete the redundant data blocks in the merged snapshot to obtain the snapshot chain; Wherein, step S1 also includes: S151: When a snapshot in the snapshot chain is rolled back, all non-redundant data blocks between the initial version number corresponding to the first data block and the maximum version number corresponding to the rolled back snapshot are traversed: S152: Create a new version number index for each non-redundant data block, where the created new version number index is greater than the version number of the current storage system; S153: After the snapshot rollback is completed, the current version number of the storage system is incrementally updated again based on the new version number index; S2: Accumulate the total number of all version numbers in the version number range of each snapshot and remove the redundant version numbers generated by rollback. Combined with the data block size corresponding to a single snapshot, calculate the total capacity of the snapshot chain. S3: When a new snapshot is added to the snapshot chain, the redundant version numbers of all snapshots associated with the rollback are removed from the version number range of the new snapshot, and the capacity of the new single snapshot is calculated; S4: When a snapshot is deleted from the snapshot chain, the referenced version number in the rollback associated snapshot of the deleted snapshot is retained, and the referenced version number is correspondingly inherited by the referenced subject. The version number of the referenced subject is incremented according to the corresponding inherited referenced version number and the version number of the deleted snapshot is removed to calculate the deleted single snapshot capacity; Wherein, the reference subject in step S4 is the upstream snapshot rolled back to the deleted snapshot; S5: updating the total capacity of the snapshot chain according to the newly added single snapshot capacity and the deleted single snapshot capacity to complete the snapshot capacity calculation.

2. The method for calculating snapshot capacity based on multi-version data according to claim 1, characterized in that: Step S1 is implemented by RoW redirection on write.

3. The method for calculating snapshot capacity based on multi-version data according to claim 1, characterized in that: The expression of the total capacity of the snapshot chain in step S2 is: δ=(∑S z -∑S h )*p; Among them, δ is the calculated total capacity of the snapshot chain, S z is the total number of all version numbers in the version number range of all snapshots on the snapshot chain, S h The total amount of redundant version numbers generated for rollback, p is the size of a single data block.

4. A snapshot capacity calculation device based on multi-version data, characterized in that: include: Snapshot chain establishment module: used to write multiple data blocks through the first full snapshot and subsequent incremental snapshots to establish a snapshot chain, wherein the snapshot chain includes multiple snapshots corresponding to version number intervals; When establishing a snapshot chain, the snapshot chain establishing module associates the first data block written into the storage system with the version number of the current storage system; According to the writing order of the data blocks after the first data block, the version number of the storage system is incrementally updated in sequence; according to the incrementally updated version number, a corresponding version number is established for each data block after the first data block to obtain multiple snapshots; A single snapshot, merging the version number and snapshot data of a single snapshot into the next snapshot, and deleting redundant data blocks in the merged snapshot to obtain the snapshot chain; When a snapshot in the snapshot chain is rolled back, the snapshot chain establishment module traverses all non-redundant data blocks between the initial version number corresponding to the first data block and the maximum version number corresponding to the rolled back snapshot: a new version number index is established for each non-redundant data block, wherein the established new version number index is greater than the version number of the current storage system; After the snapshot rollback is completed, the current version number of the storage system is incrementally updated again based on the new version number index; Total amount calculation module: used to accumulate the total amount of all version numbers in the version number range of each snapshot and eliminate the redundant version numbers generated by rollback, and calculate the total capacity of the snapshot chain based on the data block size corresponding to a single snapshot; Incremental calculation module: when a new snapshot is added to the snapshot chain, the redundant version numbers of all snapshots associated with the rollback are removed from the version number interval of the new snapshot, and the capacity of the new single snapshot is calculated; Reduction calculation module: when a snapshot is deleted from the snapshot chain, the referenced version number in the rollback associated snapshot of the deleted snapshot is retained, and the referenced version number is correspondingly inherited by the referenced subject. The version number of the referenced subject is incremented according to the corresponding inherited referenced version number and the version number of the deleted snapshot is removed to calculate the deleted single snapshot capacity; The reference subject is an upstream snapshot rolled back to the deleted snapshot; Total amount update module: used to update the total capacity of the snapshot chain according to the newly added single snapshot capacity and the deleted single snapshot capacity to complete the snapshot capacity calculation.

5. A snapshot capacity calculation device based on multi-version data, characterized in that: include: A memory and at least one processor, wherein instructions are stored in the memory; At least one of the processors calls the instructions in the memory so that the document database audit device executes the snapshot capacity calculation method based on multi-version data as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed by the processor, a snapshot capacity calculation method based on multi-version data as described in any one of claims 1 to 3 is implemented.