An epoch dag-based copy-free snapshot management method and device

Abstract

This invention relates to the field of snapshot management technology, providing a copyless snapshot management method and device based on Epoch DAG, comprising: in response to a creation operation, generating snapshot nodes based on the Epoch mechanism, constructing all snapshot nodes into a directed acyclic graph according to their dependencies, and associating each snapshot node with a skip interval for data filtering; in response to a rollback operation, switching the logical view to the data state represented by the target snapshot only by adjusting the currently active snapshot and the current skip interval, without performing data copying; in response to a deletion operation, executing the corresponding metadata update strategy according to whether the deleted snapshot is the currently active snapshot, updating the directed acyclic graph and adjusting the relevant skip intervals; in response to a read operation, logically filtering the read data according to the current skip interval and the skip interval of the active snapshot, and returning data that conforms to the snapshot view. This invention can achieve copyless rollback of any snapshot within seconds, with zero space overhead, and supports branch evolution.

Description

Technical Field

[0001] This invention relates to the field of snapshot management technology, and in particular to a copyless snapshot management method and device based on Epoch DAG. Background Technology

[0002] In distributed storage systems that implement snapshots based on the Epoch mechanism, Epoch is used as a globally incrementing timestamp to mark the version of each data write, thus providing a version management basis for snapshot creation and recovery. Existing technologies mainly include the following two types of snapshot management schemes:

[0003] The first type is the linear rollback scheme. This scheme only supports rolling back the system to the most recently created snapshot. If a user needs to restore to a historical snapshot, they must roll back step by step from the latest snapshot until the target snapshot. The advantage of this scheme is its simplicity and low metadata overhead, but its disadvantages are equally significant: the step-by-step rollback operation is time-consuming, and all snapshot data created after the target snapshot will be lost during the rollback process. For users, this linear rollback mechanism cannot provide true point-in-time recovery capability and is difficult to meet diverse and granular data management needs.

[0004] The second type is a rollback scheme that supports any historical snapshot. This scheme allows users to directly roll back to any historical snapshot, overcoming the limitations of linear rollback. However, to achieve independent recoverability of any snapshot, this type of scheme usually requires performing a data copy operation during snapshot creation or rollback, physically copying the data corresponding to the snapshot to an independent storage area. This approach directly leads to two major problems: First, the large amount of data copying brings huge IO overhead, causing the time spent on snapshot creation and rollback operations to increase linearly with the data size, failing to meet the performance requirements of second-level response; second, the existence of multiple data copies causes a serious space bloat problem, namely the write amplification effect, which significantly increases storage costs, especially in scenarios with multiple snapshots and multiple branches.

[0005] Chinese patent CN105095016A discloses a disk snapshot rollback method and apparatus. This solution introduces a global snapshot number and a local numbering mechanism to achieve metadata pointer switching during snapshot rollback, which reduces the reference count adjustment overhead in the rollback operation to a certain extent. However, its essential defects are: the forced deletion of subsequent snapshots after rollback leads to the break of the snapshot chain and the inability to retain branch versions; the physical deletion of data brings additional IO overhead, making it impossible to achieve true second-level copy-free rollback; and linear version management cannot support snapshot branch evolution, making it difficult to meet the needs of complex business scenarios.

[0006] Therefore, how to achieve second-level, copy-free rollback of any historical snapshot while ensuring low space overhead has become a key technical challenge that needs to be solved in distributed storage systems. Summary of the Invention

[0007] In view of this, in order to overcome the shortcomings of the prior art, the present invention aims to provide a copyless snapshot management method and device based on Epoch DAG.

[0008] According to a first aspect of the present invention, a copyless snapshot management method based on Epoch DAG is provided, the method comprising:

[0009] Step S1: In response to the creation operation, generate snapshot nodes based on the Epoch mechanism, construct a directed acyclic graph of all snapshot nodes according to their dependencies, and associate each snapshot node with a skip interval for data filtering;

[0010] Step S2: In response to the rollback operation, switch the logical view to the data state represented by the target snapshot by adjusting the current active snapshot and the current skip range, without performing a data copy;

[0011] Step S3: In response to the deletion operation, execute the corresponding metadata update strategy based on whether the deleted snapshot is the currently active snapshot, update the directed acyclic graph and adjust the relevant skip intervals;

[0012] Step S4: In response to the read operation, perform logical filtering on the read data based on the current skip range and the skip range of the active snapshot, and return the data that matches the snapshot view.

[0013] Optionally, the copyless snapshot management method based on Epoch DAG of the present invention includes step S1, which comprises:

[0014] When a snapshot creation instruction is received, the current system's Epoch value, the currently active snapshot, its associated skip interval, and the current skip interval are obtained.

[0015] A new snapshot node is generated based on the current Epoch value, the current active snapshot and its skip interval, and the current skip interval. The new snapshot node contains at least the current Epoch value, a pointer to its previous snapshot, and the newly generated skip interval.

[0016] Add the new snapshot node to the snapshot list and establish its unidirectional dependency with the previous snapshot node in the directed acyclic graph;

[0017] Update the currently active snapshot to the new snapshot node to complete snapshot creation.

[0018] Optionally, in the Epoch DAG-based copyless snapshot management method of the present invention, the skip interval of the newly generated snapshot node is represented in the form of a starting Epoch and an ending Epoch. The starting Epoch of each skip interval is the creation time corresponding to the target snapshot to be rolled back, and the ending Epoch is the time when the snapshot rollback is initiated.

[0019] Optionally, in the Epoch DAG-based copyless snapshot management method of the present invention, each snapshot node in step S1 contains at least the following information:

[0020] The Epoch value of the current snapshot is used to identify the data version at the time the snapshot was created;

[0021] A pointer to its preceding snapshot, used to maintain the dependencies and version evolution paths between snapshot nodes;

[0022] The associated skip range is used to filter data that does not belong to that snapshot when reading data.

[0023] Optionally, in the Epoch DAG-based copyless snapshot management method of the present invention, step S2 includes:

[0024] When a rollback instruction is received pointing to a certain historical snapshot, the corresponding snapshot node is located from the snapshot list based on the identifier of the target snapshot to be rolled back;

[0025] Obtain the current system's Epoch value, and generate a new skip interval based on the current system's Epoch value and the Epoch value of the rollback target snapshot;

[0026] The current active snapshot is updated to the rollback target snapshot, and the current skip range is updated to the newly generated skip range, completing the switch of the logical view and enabling the system view to seamlessly switch to the data state represented by the rollback target snapshot.

[0027] Optionally, in the Epoch DAG-based copyless snapshot management method of the present invention, after completing the snapshot rollback, garbage data is marked and reclaimed in the following manner:

[0028] The system scans all data and marks data whose Epoch value is within the current skip interval and is later than the Epoch value corresponding to the latest snapshot as garbage data.

[0029] The marked garbage data is asynchronously physically reclaimed through a background recycling thread. Before the recycling is completed, the garbage data is not visible to user read operations.

[0030] Optionally, in the Epoch DAG-based copyless snapshot management method of the present invention, step S3 includes:

[0031] When a command to delete a snapshot is received, it is determined whether the target snapshot to be deleted is the currently active snapshot;

[0032] If the target snapshot is the currently active snapshot, merge its associated skip range into the skip range of its predecessor snapshot, and update the predecessor snapshot as the new active snapshot.

[0033] If the target snapshot is not currently active, remove the snapshot node from the snapshot list and update the directed acyclic graph structure, breaking its dependencies with the preceding and following nodes.

[0034] After deleting a snapshot, perform a consistent update on the snapshot list and the directed acyclic graph.

[0035] Optionally, in the Epoch DAG-based copyless snapshot management method of the present invention, if the target snapshot is a currently active snapshot, its associated skip interval is merged into the skip interval of its predecessor snapshot in the following manner: when it is necessary to merge the skip interval of a snapshot node into the skip interval of its predecessor snapshot, the two skip intervals are combined to form a new skip interval, which covers the entire range of the original two skip intervals.

[0036] Optionally, in the Epoch DAG-based copyless snapshot management method of the present invention, step S4 includes:

[0037] When a user's data read request is received, the original data corresponding to the request is read from the underlying storage system;

[0038] Get the skip interval of the currently active snapshot and the current skip interval, and form the set of currently effective skip intervals;

[0039] The raw data is filtered one by one to determine whether the Epoch value of each data item falls within any interval of the currently effective skip interval set;

[0040] If the Epoch value of the data falls within any skip interval, the data is removed from the returned results; if the Epoch value of the data does not fall within any skip interval, the data is retained and added to the returned results.

[0041] The filtered data results are returned to the user.

[0042] According to a second aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect of the present invention.

[0043] The present invention provides a copyless snapshot management method and device based on Epoch DAG, which has the following beneficial technical effects:

[0044] 1. Enables second-level rollback of any snapshot, improving operational efficiency.

[0045] Logical view switching is achieved through the skip interval mechanism. The snapshot rollback operation only involves recalculating the skip interval and adjusting the active snapshot pointer. The entire process is a metadata-level operation, which is completely independent of the data size. This completely solves the technical bottleneck that the rollback time in traditional solutions increases linearly with the amount of data, providing users with true fast recovery capabilities at any point in time.

[0046] Second, completely eliminate data copying to achieve zero space overhead growth.

[0047] This approach fundamentally avoids the write amplification problem caused by data replication in traditional solutions, making the additional space overhead in scenarios with multiple snapshots and branches approach zero. Users can create snapshots frequently and retain multiple historical versions without worrying about the rapid expansion of storage capacity.

[0048] Third, it supports snapshot branching evolution to meet the needs of complex business scenarios.

[0049] The dependency relationships between snapshot nodes are built based on a directed acyclic graph, which naturally supports the derivation of multiple parallel branches from the same previous snapshot. Each branch has an independent skip interval for evolution, and the data views do not interfere with each other. Users can flexibly switch and roll back between different branches.

[0050] IV. Data reading performance is lossless; snapshot queries are as efficient as normal read operations.

[0051] This makes the performance of reading snapshot data almost identical to that of reading ordinary data. Users can get a consistent response speed when querying under any snapshot view, and will not sacrifice access performance due to the existence of the snapshot mechanism.

[0052] Fifth, asynchronous garbage data recycling maximizes system resource utilization.

[0053] Separating resource reclamation operations from front-end business I / O avoids interference with user read and write operations, while ensuring that storage space can be released and reused in a timely manner, thereby achieving efficient management of system resources and continuous optimization of utilization.

[0054] VI. The metadata structure is concise, and the system complexity is controllable.

[0055] A concise metadata design reduces the implementation complexity and maintenance cost of the system, while making it easier to guarantee the atomicity and consistency of snapshot operations, thereby improving the overall stability and reliability of the system. Attached Figure Description

[0056] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram illustrating a snapshot operation performed using the copyless snapshot management method based on Epoch DAG according to the present invention.

[0058] Figure 2 This is a logical schematic diagram of the copyless snapshot management method based on Epoch DAG according to the present invention;

[0059] Figure 3 This is a schematic diagram of the snapshot management process according to the copyless snapshot management method based on Epoch DAG according to Exemplary Example 2 of the present invention;

[0060] Figure 4 This is a schematic diagram illustrating a first application of the copyless snapshot management method based on Epoch DAG according to an exemplary embodiment 2 of the present invention for snapshot management.

[0061] Figure 5 This is a schematic diagram illustrating a second application of the copyless snapshot management method based on Epoch DAG according to an exemplary embodiment 2 of the present invention for snapshot management;

[0062] Figure 6 This is a schematic diagram illustrating a third application of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 2 of the present invention for snapshot management;

[0063] Figure 7 This is a schematic diagram illustrating the snapshot management process according to the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 3 of the present invention;

[0064] Figure 8 This is an application diagram illustrating the snapshot management method based on Epoch DAG according to Exemplary Embodiment 3 of the present invention.

[0065] Figure 9 This is a schematic diagram illustrating the snapshot management process according to the copyless snapshot management method based on Epoch DAG according to exemplary embodiment 4 of the present invention;

[0066] Figure 10This is a schematic diagram illustrating a first application of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 4 of the present invention for snapshot management.

[0067] Figure 11 This is a schematic diagram illustrating a second application of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 4 of the present invention for snapshot management.

[0068] Figure 12 This is a schematic diagram illustrating a third application of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 4 of the present invention for snapshot management.

[0069] Figure 13 This is a schematic diagram illustrating a fourth application of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 4 of the present invention for snapshot management.

[0070] Figure 14 This is a schematic diagram of the structure of the device provided by the present invention. Detailed Implementation

[0071] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0072] It should be noted that, in the absence of conflict, the following embodiments and features can be combined with each other; and, based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0073] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0074] Figure 1 This is a schematic diagram illustrating a snapshot operation performed using the copyless snapshot management method based on Epoch DAG according to the present invention. Figure 2 This is a logical schematic diagram of the copyless snapshot management method based on Epoch DAG according to the present invention, as shown below. Figure 1 and Figure 2As shown, snapshot S1 is created at Epoch 100, snapshot S2 is created at Epoch 200, snapshot S3 is created at Epoch 300, snapshot S4 is created at Epoch 400, and the snapshot is rolled back to snapshot S2 at Epoch 500. Snapshot S5 is created at Epoch 600, snapshot S6 is created at Epoch 700, and the snapshot is rolled back to snapshot S5 at Epoch 800. Snapshot S7 is created at Epoch 900, snapshot S8 is created at Epoch 1000, and the snapshot is rolled back to snapshot S7 at Epoch 1100.

[0075] The following specific embodiments illustrate the copyless snapshot management method based on Epoch DAG of the present invention in detail.

[0076] Example 1

[0077] Exemplary Example 1 of the present invention provides a copyless snapshot management method based on Epoch DAG. In this embodiment, the method of the present invention is implemented in the following manner:

[0078] Step S1: In response to the creation operation, generate snapshot nodes based on the Epoch mechanism, construct a directed acyclic graph of all snapshot nodes according to their dependencies, and associate each snapshot node with a skip interval for data filtering.

[0079] In storage systems, Epoch is used as a timestamp that represents a specific point in time and marks the written version of data.

[0080] As an optional example, in this embodiment, when a snapshot creation instruction is received, the Epoch value of the current system, the currently active snapshot, its associated skip interval, and the current skip interval are obtained; a new snapshot node is generated based on the current Epoch value, the currently active snapshot, its skip interval, and the current skip interval. The new snapshot node at least includes the current Epoch value, a pointer to its predecessor snapshot, and the newly generated skip interval; the new snapshot node is added to the snapshot list, and a unidirectional dependency relationship between it and the predecessor snapshot node is established in the directed acyclic graph; the currently active snapshot is updated to the new snapshot node, and the snapshot creation is completed.

[0081] In this embodiment, the skip interval of the newly generated snapshot node is represented in the form of a start Epoch and an end Epoch. The start Epoch of each skip interval is the creation time corresponding to the target snapshot to be rolled back, and the end Epoch is the time when the snapshot rollback is initiated.

[0082] In practical applications, each snapshot node in this embodiment contains at least the following information:

[0083] The Epoch value of the current snapshot is used to identify the data version at the time the snapshot was created;

[0084] A pointer to its preceding snapshot, used to maintain the dependencies and version evolution paths between snapshot nodes;

[0085] The associated skip range is used to filter data that does not belong to that snapshot when reading data.

[0086] Step S2: In response to the rollback operation, switch the logical view to the data state represented by the target snapshot by adjusting the current active snapshot and the current skip range, without performing a data copy.

[0087] As an optional example, in this embodiment, when a rollback instruction pointing to a certain historical snapshot is received, the corresponding snapshot node is located from the snapshot list according to the identifier of the rollback target snapshot; the Epoch value of the current system is obtained, and a new skip interval is generated based on the Epoch value of the current system and the Epoch value of the rollback target snapshot; the currently active snapshot is updated to the rollback target snapshot, and the current skip interval is updated to the newly generated skip interval, completing the switch of the logical view, so that the system view can be seamlessly switched to the data state represented by the rollback target snapshot, and the whole process does not require any data copying.

[0088] It should be noted that in this embodiment, after the snapshot rollback is completed, garbage data is marked and reclaimed in the following manner:

[0089] The system scans all data and marks data whose Epoch value is within the current skip interval and is later than the Epoch value corresponding to the latest snapshot as garbage data. The marked garbage data is asynchronously physically reclaimed by a background reclamation thread. Before the reclamation is completed, the garbage data is not visible to user read operations to ensure the consistency and correctness of the data view.

[0090] Step S3: In response to the deletion operation, execute the corresponding metadata update strategy based on whether the deleted snapshot is the currently active snapshot, update the directed acyclic graph and adjust the relevant skip intervals.

[0091] As an optional example, in this embodiment, when an instruction to delete a snapshot is received, it is determined whether the target snapshot to be deleted is the currently active snapshot; if the target snapshot is the currently active snapshot, its associated skip interval is merged into the skip interval of its predecessor snapshot, and the predecessor snapshot is updated to the new active snapshot; if the target snapshot is not the currently active snapshot, the snapshot node is removed from the snapshot list, and the directed acyclic graph structure is updated to break its dependency relationship with the preceding and following nodes; after deleting the snapshot, the snapshot list and the directed acyclic graph are updated consistently.

[0092] In either case, after deleting a snapshot, the snapshot list and the directed acyclic graph must be updated consistently to ensure that the system's metadata structure and data filtering logic remain correct after the snapshot is deleted.

[0093] It should be noted that if the target snapshot is the currently active snapshot, its associated skip interval is merged into the skip interval of its predecessor snapshot in the following way: When it is necessary to merge the skip interval of a snapshot node into the skip interval of its predecessor snapshot, the union of the two skip intervals is taken to form a new skip interval. This new skip interval covers the entire range of the original two skip intervals, ensuring that after the active snapshot is deleted, the data originally filtered by the active snapshot can still be filtered correctly, maintaining the logical integrity of the data view.

[0094] Step S4: In response to the read operation, perform logical filtering on the read data based on the current skip range and the skip range of the active snapshot, and return the data that matches the snapshot view.

[0095] As an optional example, in this embodiment, when a user's data read request is received, the original data corresponding to the request is read from the underlying storage system; the skip interval of the currently active snapshot and the current skip interval are obtained to form a set of currently effective skip intervals; the read original data is filtered one by one to determine whether the Epoch value of each data falls within any interval in the set of currently effective skip intervals; if the Epoch value of the data falls within any skip interval, the data is removed from the returned result; if the Epoch value of the data is not within any skip interval, the data is retained and added to the returned result; the filtered data result is returned to the user to ensure that the user reads the correct data view represented by the currently active snapshot.

[0096] In practical applications, the method in this embodiment also supports the branching evolution and management of snapshots:

[0097] When multiple subsequent snapshots are created from the same preceding snapshot, multiple branch nodes are generated in the directed acyclic graph structure, each representing a different data evolution path. The skip interval of each branch node is maintained independently and does not interfere with each other. The data view of each branch is defined independently by its own skip interval. Users can perform snapshot rollback operations between different branches, that is, users can roll back from a snapshot of one branch to a snapshot of another branch, realizing branched management and flexible switching of data versions. The branched snapshot management mechanism can maintain multiple data version evolution paths at the same time, meeting the data management needs of complex business scenarios.

[0098] Example 2

[0099] Exemplary Example 2 of the present invention provides a copyless snapshot management method based on Epoch DAG. Figure 3 This is a schematic diagram illustrating the snapshot management process of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 2 of the present invention, as shown below. Figure 3 As shown, in this embodiment, the method of the present invention is implemented in the following manner:

[0100] Step 1: Generate snapshot nodes

[0101] The system first obtains the current Epoch value and generates a new snapshot node based on this Epoch value. This node will be used to represent the data version corresponding to this snapshot.

[0102] Step 2: Determine if the current skip interval exists.

[0103] The system checks if a valid skip interval already exists:

[0104] If a current skip range exists, then bind that skip range to the newly generated snapshot node;

[0105] If the current skip interval does not exist, skip this step and continue to the next step.

[0106] Step 3: Determine if a previous snapshot exists.

[0107] The system checks if a previous snapshot (i.e., a parent snapshot) exists in the current snapshot:

[0108] If there is no previous snapshot, it means that the current snapshot is the first snapshot, and the process jumps directly to the last step;

[0109] If a previous snapshot exists, proceed to the next step of the judgment.

[0110] Step 4: Determine if the previous snapshot has a skip interval.

[0111] The system checks whether previous snapshots are associated with skip intervals:

[0112] If the previous snapshot has a skip range, then the skip range is also bound to the newly generated snapshot node;

[0113] If the previous snapshot does not have a skip range, skip this step.

[0114] Step 5: Complete snapshot creation

[0115] The system adds the newly generated snapshot node to the snapshot list and updates the currently active snapshot to the newly generated snapshot node, thus completing the entire snapshot creation process.

[0116] The method of this embodiment will be further described in detail below in a specific scenario.

[0117] Figure 4 This is a schematic diagram illustrating a first application of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 2 of the present invention, such as... Figure 4 As shown, when a snapshot is created, snapshot nodes are generated in sequence and added to the snapshot list. A snapshot node includes the Epoch at which the current snapshot was generated, its preceding snapshot, and the skip interval. The preceding snapshot is used to maintain snapshot branch information, and the skip interval is used to filter data when reading data. At the same time, every snapshot-related operation will trigger the update of the active snapshot and the current skip interval. For example, at Epoch 100, snapshot S1 is created. At this time, a snapshot node is created and added to the snapshot list. The snapshot node's Epoch is 100, the preceding snapshot is NA, the skip interval is NA, the active snapshot is updated to S1, and the current skip interval is updated to NA. At Epoch 200, snapshot S2 is created. At this time, a snapshot node is created and added to the snapshot list. The snapshot node's Epoch is 200, the preceding snapshot is S1, the skip interval is NA, the active snapshot is updated to S2, and the current skip interval remains NA. At Epoch 300, snapshot S3 is created. At this time, a snapshot node is created and added to the snapshot list. The snapshot node's Epoch is 300, the preceding snapshot is S2, the skip interval is NA, the active snapshot is updated to S3, and the current skip interval remains NA. At 400, a snapshot S4 is created. At this time, a snapshot node is created and added to the snapshot list. The snapshot node Epoch is 400, the previous snapshot is S3, the skip interval is NA, the active snapshot is updated to S4, and the current skip interval remains NA.

[0118] Figure 5 This is a second application illustration of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 2 of the present invention, such as... Figure 5As shown, at Epoch 500, the system rolls back to snapshot S2, the active snapshot is updated to S2, and the current skip range is updated to [200, 500]. At Epoch 600, snapshot S5 is created. At this time, a snapshot node is created and added to the snapshot list. The snapshot node's Epoch is 600, the preceding snapshot is S2, the skip range is [200, 500], the active snapshot is updated to S5, and the current skip range remains [200, 500]. At Epoch 700, snapshot S6 is created. At this time, a snapshot node is created and added to the snapshot list. The snapshot node's Epoch is 700, the preceding snapshot is S5, the skip range is [200, 500], the active snapshot is updated to S6, and the current skip range remains [200, 500].

[0119] Figure 6 This is a schematic diagram illustrating a third application of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 2 of the present invention, such as... Figure 6 As shown, at Epoch 800, the system rolls back to snapshot S5, updates the active snapshot to S5, and updates the current skip range to [600, 800]. At Epoch 900, snapshot S7 is created. At this time, a snapshot node is created and added to the snapshot list. The snapshot node's Epoch is 900, the preceding snapshot is S5, and the skip range is [200, 500] and [600, 800]. The active snapshot is updated to S7, and the current skip range remains [600, 800]. At Epoch 1000, snapshot S8 is created. At this time, a snapshot node is created and added to the snapshot list. The snapshot node's Epoch is 1000, the preceding snapshot is S7, and the skip range is [200, 500] and [600, 800]. The active snapshot is updated to S8, and the current skip range remains [600, 800].

[0120] Example 3

[0121] Exemplary Example 3 of the present invention provides a copyless snapshot management method based on Epoch DAG. Figure 7 This is a schematic diagram illustrating the snapshot management process of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 3 of the present invention, as follows: Figure 7 As shown, in this embodiment, the method of the present invention is implemented in the following manner:

[0122] When the system receives a snapshot rollback command, requesting that the current system view be rolled back to the specified target snapshot, the following steps are executed in sequence:

[0123] Step 1: Generate new skip intervals

[0124] The system first obtains the current Epoch value and locates the corresponding snapshot node based on the identifier of the target snapshot. Then, based on the current Epoch value and the Epoch value of the target snapshot, a new skip interval is generated. This new skip interval will be used to define which data should be filtered and which data should be visible after the rollback.

[0125] Step 2: Update the current skip interval

[0126] The system updates the currently active skip range to the newly generated skip range from the first step. This update operation is the core of the logical view switching; it changes the filtering rules during data reading but does not affect any actual data.

[0127] Step 3: Determine if the target snapshot is a currently active snapshot.

[0128] The system checks whether the target snapshot to be rolled back is already the current active snapshot:

[0129] If the target snapshot is not the currently active snapshot, proceed to step four;

[0130] If the target snapshot is the currently active snapshot, no further action is required, and the process ends directly.

[0131] Step 4: Update active snapshots

[0132] When the target snapshot is not the currently active snapshot, the system updates the currently active snapshot to the target snapshot to be rolled back. At this point, the system's active snapshot pointer points to the target snapshot, and the logical view switch is complete.

[0133] Step 5: End of process

[0134] Once the snapshot rollback operation is complete, the system's data view has seamlessly switched to the data state represented by the target snapshot.

[0135] Figure 8 This is an application illustration of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 3 of the present invention, such as... Figure 8 As shown, at Epoch 1100, the system rolls back to snapshot S7, the active snapshot is updated to S7, and the current skip range is updated to [900, 1100].

[0136] It should be noted that in this embodiment, when rolling back a snapshot, data not protected by the snapshot should be reclaimed in a timely manner. Therefore, data within the current skip interval and after the latest snapshot in the Epoch will be marked as garbage. This garbage data cannot be read before it is reclaimed by the background task. For any user read operation at any time, it is only necessary to filter out the data belonging to the current skip interval and the active snapshot skip interval after the data is read, and then return it to the user to achieve the effect of reading snapshot data.

[0137] Example 4

[0138] Exemplary Example 4 of the present invention provides a copyless snapshot management method based on Epoch DAG. Figure 9 This is a schematic diagram illustrating the snapshot management process of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 4 of the present invention, as shown below. Figure 9 As shown, in this embodiment, the method of the present invention is implemented in the following manner:

[0139] When the system receives a snapshot deletion command, requesting the removal of a specified target snapshot from the system, the following steps are executed in sequence:

[0140] Step 1: Determine if the target snapshot is a currently active snapshot.

[0141] The system first checks whether the target snapshot to be deleted is currently the active snapshot. This determination will decide which processing path to follow.

[0142] If the target snapshot is a currently active snapshot, proceed to step two;

[0143] If the target snapshot is not currently active, proceed directly to step four.

[0144] Step 2: Merge skip ranges (only performed if the target snapshot is an active snapshot)

[0145] When the target snapshot is the currently active snapshot, the system merges its bound skip range into the skip range bound to its preceding snapshot (i.e., the parent snapshot). The specific rule for this merging operation is: take the union of the two skip ranges to form a new skip range, which covers the entire range of the original two ranges. The purpose of this operation is to ensure that after deleting an active snapshot, the data originally filtered by that active snapshot can still be filtered correctly, maintaining the logical integrity of the data view.

[0146] Step 3: Update the active snapshot (only performed if the target snapshot is an active snapshot)

[0147] After merging the skip intervals, the system updates the currently active snapshot to the snapshot preceding the target snapshot. At this point, the active snapshot pointer has been transferred from the deleted snapshot to its parent snapshot, and the system's logical view baseline has been migrated.

[0148] Step 4: Update the snapshot node tree structure

[0149] Regardless of whether the target snapshot is an active snapshot, the system needs to update the directed acyclic graph (DAG) structure:

[0150] Remove the target snapshot node from the graph;

[0151] Disconnect the node from its preceding and following nodes;

[0152] Re-establish connections between its preceding and following nodes (if necessary) to ensure the integrity and acyclic properties of the graph.

[0153] Step 5: Remove the target snapshot from the snapshot list

[0154] Finally, the system officially removes the target snapshot node from the snapshot list, releasing the associated metadata resources. At this point, the snapshot deletion operation is complete.

[0155] Step 6: End of process

[0156] Once the snapshot deletion operation is complete, the system no longer contains the deleted snapshot node, the related skip interval information has been merged or cleared, and the directed acyclic graph structure remains intact and consistent.

[0157] Figure 10 This is a schematic diagram illustrating a first application of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 4 of the present invention for snapshot management. Figure 11 This is a second application illustration of the copyless snapshot management method based on Epoch DAG according to exemplary embodiment 4 of the present invention, such as... Figure 10 and Figure 11 As shown, snapshot S5 was deleted at Epoch 1300. Since snapshot S5 is not an active snapshot, there is no need to update the active snapshot. However, the predecessor snapshot of snapshot node S6 needs to be updated to S2, and the predecessor snapshot of snapshot node S7 also needs to be updated to S2. Finally, snapshot node S5 was removed from the snapshot list.

[0158] Figure 12 This is a schematic diagram illustrating a third application of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 4 of the present invention for snapshot management. Figure 13 This is a schematic diagram illustrating a fourth application of the copyless snapshot management method based on Epoch DAG according to Exemplary Embodiment 4 of the present invention, as follows: Figure 12 and Figure 13 As shown, snapshot S7 was deleted at Epoch 1400. Since snapshot S7 is an active snapshot, its predecessor snapshot S2 needs to be updated to be an active snapshot. At the same time, the skip intervals [200, 500] and [600, 800] bound to snapshot S7 need to be merged into the skip interval bound to its predecessor snapshot S2. That is, the skip interval of snapshot S2 is updated to [200, 500] and [600, 800]. The predecessor snapshot of snapshot node S8 needs to be updated to S2. Finally, snapshot node S7 is removed from the snapshot list.

[0159] The copyless snapshot management method based on Epoch DAG in this invention has the following beneficial technical effects:

[0160] 1. Enables second-level rollback of any snapshot, improving operational efficiency.

[0161] Logical view switching is achieved through a skip interval mechanism. Snapshot rollback operations only involve recalculating the skip interval and adjusting the active snapshot pointer. The entire process is a metadata-level operation, completely independent of data size. Regardless of whether the total data volume of the storage system reaches the petabyte or even exabyte level, and regardless of the depth of the snapshot chain, rollback operations can be completed within milliseconds to seconds. This completely solves the technical bottleneck of traditional solutions where rollback time increases linearly with data volume, providing users with truly rapid recovery capabilities at any point in time.

[0162] Second, completely eliminate data copying to achieve zero space overhead growth.

[0163] No physical data copying is performed in any snapshot operation: only metadata nodes are generated when a snapshot is created, only skip intervals are adjusted when a snapshot is rolled back, and only metadata information is merged when a snapshot is deleted. This fundamentally avoids the write amplification problem caused by data copying in traditional solutions, making the additional space overhead of the system in multi-snapshot and multi-branch scenarios approach zero. Users can create snapshots frequently and retain multiple historical versions without worrying about the rapid expansion of storage capacity.

[0164] Third, it supports snapshot branching evolution to meet the needs of complex business scenarios.

[0165] Based on a directed acyclic graph (DAG) to build dependencies between snapshot nodes, it naturally supports the derivation of multiple parallel branches from the same previous snapshot. Each branch has an independent skip interval for evolution, and the data views do not interfere with each other. Users can flexibly switch and roll back between different branches. It provides technical support for complex business scenarios such as data version management, parallel development and testing, and multi-tenant isolation, enabling the distributed storage system to adapt to more diverse application needs.

[0166] IV. Data reading performance is lossless; snapshot queries are as efficient as normal read operations.

[0167] When reading data, the results only need to be logically filtered based on the skip interval of the active snapshot and the current skip interval. The filtering operation is completed in memory without additional IO access, making the performance of reading snapshot data almost the same as that of reading ordinary data. Users can get a consistent response speed when querying under any snapshot view, and will not sacrifice access performance due to the existence of the snapshot mechanism.

[0168] Fifth, asynchronous garbage data recycling maximizes system resource utilization.

[0169] Garbage data is automatically marked after snapshot rollback, and physical reclamation is performed asynchronously by a background thread. Resource reclamation operations are separated from foreground business I / O, avoiding interference with user read / write operations, while ensuring that storage space can be released and reused in a timely manner, achieving efficient management and continuous optimization of system resources.

[0170] VI. The metadata structure is concise, and the system complexity is controllable.

[0171] Only a small amount of metadata information, such as Epoch value, previous pointer, and skip range, needs to be maintained in the snapshot node, without the need to introduce complex index structures or mapping tables. The concise metadata design reduces the implementation complexity and maintenance cost of the system, while making it easier to guarantee the atomicity and consistency of snapshot operations, thereby improving the overall stability and reliability of the system.

[0172] like Figure 14 As shown, the present invention also provides a device including a processor 310, a communication interface 320, a memory 330 for storing processor-executable computer programs, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 implements the aforementioned copyless snapshot management method based on Epoch DAG by running the executable computer program.

[0173] The computer program in memory 330, when implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0174] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected based on actual needs to achieve the purpose of this embodiment. Those skilled in the art can understand and implement this without any creative effort.

[0175] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0176] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A copyless snapshot management method based on Epoch DAG, characterized in that, The method includes: Step S1: In response to the creation operation, generate snapshot nodes based on the Epoch mechanism, construct a directed acyclic graph of all snapshot nodes according to their dependencies, and associate each snapshot node with a skip interval for data filtering; Step S2: In response to the rollback operation, switch the logical view to the data state represented by the target snapshot by adjusting the current active snapshot and the current skip range, without performing a data copy; Step S3: In response to the deletion operation, execute the corresponding metadata update strategy based on whether the deleted snapshot is the currently active snapshot, update the directed acyclic graph and adjust the relevant skip intervals; Step S4: In response to the read operation, perform logical filtering on the read data based on the current skip range and the skip range of the active snapshot, and return the data that matches the snapshot view.

2. The copyless snapshot management method based on Epoch DAG according to claim 1, characterized in that, Step S1 includes: When a snapshot creation instruction is received, the current system's Epoch value, the currently active snapshot, its associated skip interval, and the current skip interval are obtained. A new snapshot node is generated based on the current Epoch value, the current active snapshot and its skip interval, and the current skip interval. The new snapshot node contains at least the current Epoch value, a pointer to its previous snapshot, and the newly generated skip interval. Add the new snapshot node to the snapshot list and establish its unidirectional dependency with the previous snapshot node in the directed acyclic graph; Update the currently active snapshot to the new snapshot node to complete snapshot creation.

3. The copyless snapshot management method based on Epoch DAG according to claim 2, characterized in that, The skip intervals of newly generated snapshot nodes are represented in the form of a start epoch and an end epoch. The start epoch of each skip interval is the creation time corresponding to the target snapshot to be rolled back, and the end epoch is the time when the snapshot rollback is initiated.

4. The copyless snapshot management method based on Epoch DAG according to claim 1, characterized in that, In step S1, each snapshot node contains at least the following information: The Epoch value of the current snapshot is used to identify the data version at the time the snapshot was created; A pointer to its preceding snapshot, used to maintain the dependencies and version evolution paths between snapshot nodes; The associated skip range is used to filter data that does not belong to that snapshot when reading data.

5. The copyless snapshot management method based on Epoch DAG according to claim 1, characterized in that, Step S2 includes: When a rollback instruction is received pointing to a certain historical snapshot, the corresponding snapshot node is located from the snapshot list based on the identifier of the target snapshot to be rolled back; Obtain the current system's Epoch value, and generate a new skip interval based on the current system's Epoch value and the Epoch value of the rollback target snapshot; The current active snapshot is updated to the rollback target snapshot, and the current skip range is updated to the newly generated skip range, completing the switch of the logical view and enabling the system view to seamlessly switch to the data state represented by the rollback target snapshot.

6. The copyless snapshot management method based on Epoch DAG according to claim 5, characterized in that, In step S2, after completing the snapshot rollback, garbage data is marked and reclaimed in the following manner: The system scans all data and marks data whose Epoch value is within the current skip interval and is later than the Epoch value corresponding to the latest snapshot as garbage data. The marked garbage data is asynchronously physically reclaimed through a background recycling thread. Before the recycling is completed, the garbage data is not visible to user read operations.

7. The copyless snapshot management method based on Epoch DAG according to claim 1, characterized in that, Step S3 includes: When a command to delete a snapshot is received, it is determined whether the target snapshot to be deleted is the currently active snapshot; If the target snapshot is the currently active snapshot, merge its associated skip range into the skip range of its predecessor snapshot, and update the predecessor snapshot as the new active snapshot. If the target snapshot is not currently active, remove the snapshot node from the snapshot list and update the directed acyclic graph structure, breaking its dependencies with the preceding and following nodes. After deleting a snapshot, perform a consistent update on the snapshot list and the directed acyclic graph.

8. The copyless snapshot management method based on Epoch DAG according to claim 7, characterized in that, In step S3, if the target snapshot is the currently active snapshot, its associated skip interval is merged into the skip interval of its predecessor snapshot in the following way: when it is necessary to merge the skip interval of a snapshot node into the skip interval of its predecessor snapshot, the two skip intervals are combined to form a new skip interval, which covers the entire range of the original two skip intervals.

9. The copyless snapshot management method based on Epoch DAG according to claim 1, characterized in that, Step S4 includes: When a user's data read request is received, the original data corresponding to the request is read from the underlying storage system; Get the skip interval of the currently active snapshot and the current skip interval, and form the set of currently effective skip intervals; The raw data is filtered one by one to determine whether the Epoch value of each data item falls within any interval of the currently effective skip interval set; If the Epoch value of the data falls within any skip interval, the data is removed from the returned results; if the Epoch value of the data does not fall within any skip interval, the data is retained and added to the returned results. The filtered data results are returned to the user.

10. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method according to any one of claims 1-9.

Images