A snapshot processing method and device based on blind area snapshots

Abstract

The application provides a snapshot processing method and device based on a blind area snapshot, and relates to the technical field of snapshot processing based on a blind area snapshot, wherein the method comprises: the snapshot processing method based on the blind area snapshot of the application embodiment can effectively avoid the damage of a snapshot rollback operation to a snapshot chain, maintain the integrity of the dependency relationship between each snapshot version, and thus improve the reliability and consistency of a distributed storage system in a data recovery process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of snapshot management and rollback mechanism optimization of distributed storage systems, and particularly relates to a snapshot processing method and device based on blind area snapshots, an electronic device and a computer readable storage medium. BACKGROUND

[0002] As the core infrastructure of cloud computing and data centers, distributed storage systems are widely used in large-scale data management, virtualization platforms and enterprise-level storage services. In related technologies, a storage architecture with high availability and scalability is constructed through the cooperative work of object storage devices (OSDs), placement groups (PGs) and metadata management. Specifically, the system covers the whole process from data writing, snapshot generation to version rollback, including key links such as object cloning (clone), forward relationship maintenance, snapshot chain construction, etc. With the development of storage pool append write technology, snapshot operations gradually evolve into a fine management mode based on object information (OI) and clone hierarchy (clone_list) to improve storage efficiency and data consistency.

[0003] However, in the existing snapshot rollback scheme, the version tracing mechanism based on the snapshot chain is directly used, which usually only copies basic information when cloning objects, without complete copying of the forward relationship, which may cause the snapshot chain to be destroyed in the rollback operation. Specifically, when the system performs rollback, the dependency relationship between the original snapshot versions may be invalid, which further causes the subsequent snapshots to be unable to correctly access the historical data segments, affecting the reliability and consistency of data recovery. In addition, the traditional scheme needs to traverse the snapshot chain layer by layer to find dependent objects when processing read requests, which has high time complexity and storage overhead, and is difficult to meet the needs of large-scale distributed systems for efficient and stable snapshot management. SUMMARY

[0004] The present application aims to at least solve one of the technical problems in the related art to some extent.

[0005] To this end, a first object of the present application is to provide a snapshot processing method based on blind area snapshots.

[0006] A second object of the present application is to provide a device.

[0007] A third object of the present application is to provide an electronic device.

[0008] A fourth object of the present application is to provide a computer readable storage medium.

[0009] A fifth object of the present application is to provide a computer program product.

[0010] To achieve the above object, the embodiment of the first aspect of the present application provides a snapshot processing method based on a blind area snapshot, comprising the following steps: S1, determining a hierarchical clone object to which a target rollback clone object belongs according to a clone_list in an OI of a header object and a rollback target snapshot id, and obtaining an OI of the hierarchical clone object; S2, calculating the target rollback clone object based on clone_overlap information in the OI of the hierarchical clone object, and deleting a positive relationship of the header object; S3, generating a blind area snapshot, wherein a key of the blind area snapshot is a latest snapshot id of a current object, and a value is a target snapshot id to which the rollback is performed; S4, modifying the clone_overlap information in the OI of the header object according to the generated blind area snapshot, and assembling the positive relationship and the OI information into a kv format and sending to a distributed kv storage for storage.

[0011] In an embodiment of the present application, the step of determining a hierarchical clone object to which a target rollback clone object belongs according to a clone_list in an OI of a header object and a rollback target snapshot id, and obtaining an OI of the hierarchical clone object further comprises the following steps: S11, traversing a hierarchical clone object list recorded in the clone_list, and matching a rollback target snapshot id with snapshot id ranges of each hierarchical clone object to determine a hierarchical clone object in which the target rollback clone object is located; S12, extracting OI information of the matched hierarchical clone object, and verifying integrity and validity of the OI information to ensure data consistency of subsequent operations.

[0012] In an embodiment of the present application, the step of calculating the target rollback clone object based on clone_overlap information in the OI of the hierarchical clone object, and deleting the positive relationship of the header object further comprises the following steps: S21, determining a start position and a length of a positive relationship data segment to be rolled back according to a dependence range of a current clone object on a previous clone object recorded in the clone_overlap; S22, marking and deleting the positive relationship data segment corresponding to the rollback target snapshot id in the header object, and retaining the positive relationship data not affected by the rollback to reduce the risk of data loss.

[0013] In an embodiment of the present application, the generation of the blind area snapshot, wherein the key of the blind area snapshot is the latest snapshot id of the current object, and the value is the target snapshot id to which the rollback is performed, further comprises: S31, setting the key of the blind area snapshot as the latest snapshot id of the header object before rollback, and setting the value as the snapshot id actually pointed to by the rollback, and storing the mapping relationship in the OI of the header object; S32, assigning a unique identifier to each blind area snapshot, and establishing an index in the distributed kv storage, so as to quickly locate the dependency relationship in subsequent read operations.

[0014] In an embodiment of the present application, further comprising: S5, according to the snapshot id carried in the read request, finding the corresponding layered clone object from the clone_list in the OI of the header object, and determining the dependent object of the target data segment based on the clone_overlap information of the layered clone object, so as to support the read operation of the snapshot object.

[0015] To achieve the above purpose, the second aspect embodiment of the present application proposes a snapshot processing device based on blind area snapshot, comprising: a layered object determination module, configured to determine the layered clone object to which the target rollback clone object belongs according to the clone_list and the rollback target snapshot id in the OI of the header object, and obtain the OI of the layered clone object; a forward relationship processing module, configured to calculate the target rollback clone object based on the clone_overlap information in the OI of the layered clone object, and delete the forward relationship of the header object; a blind area snapshot generation module, configured to generate a blind area snapshot, wherein the key of the blind area snapshot is the latest snapshot id of the current object, and the value is the target snapshot id to which the rollback is performed; an information assembling and storage module, configured to modify the clone_overlap information in the OI of the header object according to the generated blind area snapshot, and assemble the forward relationship and the OI information into kv format and send to the distributed kv storage for storage.

[0016] In an embodiment of the present application, the layered object determination module is further configured to: traverse the list of layered clone objects recorded in the clone_list, match the rollback target snapshot id with the snapshot id range of each layered clone object, and determine the layered clone object in which the target rollback clone object is located; extract the OI information of the matched layered clone object, and verify the integrity and validity of the OI information, so as to ensure the data consistency of subsequent operations.

[0017] In an embodiment of the present application, the forward relationship processing module is further configured to: determine the start position and length of the forward relationship data segment to be rolled back according to the dependency range of the current clone object on the previous clone object recorded in the clone_overlap; and mark for deletion the forward relationship data segment corresponding to the rollback target snapshot id in the header object, and retain the forward relationship data not affected by the rollback, so as to reduce the risk of data loss.

[0018] In an embodiment of the present application, the blind area snapshot generation module is further configured to: set the key of the blind area snapshot as the latest snapshot id of the header object before rollback, and set the value as the snapshot id pointed to by the actual rollback, and store the mapping relationship in the OI of the header object; and assign a unique identifier to each blind area snapshot, and establish an index in the distributed kv storage, so as to quickly locate the dependency relationship in subsequent read operations.

[0019] In an embodiment of the present application, the read support module is further configured to: find the corresponding layered clone object from the clone_list in the OI of the header object according to the snapshot id carried in the read request, and determine the dependency object of the target data segment based on the clone_overlap information of the layered clone object, so as to support the read operation of the snapshot object.

[0020] To achieve the above object, a third aspect of the present application provides an electronic device, comprising: a processor, and a memory connected with the processor in communication;

[0021] The memory stores computer execution instructions;

[0022] The processor executes the computer execution instructions stored in the memory, so as to implement the method according to any one of the first aspect.

[0023] To achieve the above object, a fourth aspect of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the method according to any one of the first aspect.

[0024] To achieve the above object, a fifth aspect of the present application provides a computer program product, wherein the computer program is executed by a processor to implement the method according to any one of the first aspect.

[0025] The method, device, electronic device and computer readable storage medium according to the embodiments of the present application can effectively avoid the damage of the snapshot rollback operation to the snapshot chain, maintain the dependency relationship integrity between the snapshot versions, and thus improve the system data consistency and reliability.

[0026] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the attendant drawings or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0027] The foregoing and / or additional aspects and advantages of the application are achieved by providing what is described below and / or claimed by the application as set forth in more detail in the accompanying drawings, wherein:

[0028] Figure 1 A flowchart of a snapshot processing method based on a blind area snapshot provided by an embodiment of the application;

[0029] Figure 2 A structural diagram of a snapshot processing device based on a blind area snapshot provided by an embodiment of the application. DETAILED DESCRIPTION

[0030] Embodiments of the application are described in detail below with reference to the attached drawing figures, wherein the same or similar components have the same or similar designations throughout the several figures of the drawing and any description of the same or similar components can be understood throughout the description of the drawings. The embodiments described below are illustrative of the application and are not meant to be limiting of the application as one of ordinary skill in the art would understand the application is capable of use in various arrangements and situations.

[0031] To solve this problem, an embodiment of the application provides a snapshot processing method based on a blind area snapshot, Figure 1 is a flowchart of a snapshot processing method based on a blind area snapshot of an embodiment of the application.

[0032] As shown in Figure 1 , the snapshot processing method based on a blind area snapshot includes the following steps:

[0033] S1, according to the clone_list in the OI of the header object and the rollback target snapshot id, determine the hierarchical clone object to which the target rollback clone object belongs, and obtain the OI of the hierarchical clone object.

[0034] Specifically, in some implementations, this step determines the hierarchical clone object to which the target rollback clone object belongs by parsing the clone_list field in the OI (Object Information) structure of the header object and the specified rollback target snapshot ID, and further obtains the OI information of the hierarchical clone object. Specifically, clone_list is an ordered list that records the clone objects of the header object under different snapshot versions and their corresponding snapshot IDs. In the rollback operation, the system first traverses the list and determines the position of the target clone object in the hierarchical structure by comparing the snapshot ID of each clone object with the target rollback snapshot ID. Further, the system traces up to the corresponding hierarchical clone object through the parent-child relationship of the clone object and reads the OI information of the hierarchical clone object from the distributed metadata storage.

[0035] From the perspective of parameter indicators, clone_list is usually stored in the form of an array or a linked list, and each element contains metadata such as snapshot ID, clone object ID, and timestamp. The rollback target snapshot ID needs to comply with the system-defined snapshot naming specification (such as UUID or incremental integer identification) and needs to have a valid mapping in clone_list. In addition, the system supports accelerating the positioning of the target snapshot through binary search or hash index, with a time complexity of O(log n) or O(1), significantly improving the rollback efficiency.

[0036] In application scenarios, this step is commonly used in the snapshot rollback process in distributed storage systems, especially under the multi-version concurrency control (MVCC) mechanism, to ensure that the rollback operation does not damage the integrity of the snapshot chain. Through this step, the system can quickly identify the dependency structure of the target snapshot, providing basic data support for subsequent forward relationship reconstruction and blind area snapshot generation.

[0037] The technical effect of this step is to avoid the snapshot chain breakage problem caused by rollback in traditional solutions, ensuring that the system can still maintain the traceability and consistency of data versions after rollback. At the same time, through the OI acquisition of hierarchical clone objects, the system can achieve precise control of data dependency relationships, thereby improving the reliability and performance of snapshot processing.

[0038] Further, S1 includes:

[0039] S11, traverses the list of hierarchical clone objects recorded in clone_list, matches the rollback target snapshot id with the snapshot id range of each hierarchical clone object, and determines the hierarchical clone object in which the target rollback clone object is located.

[0040] Specifically, in some implementations, traversing the list of hierarchical clone objects recorded in clone_list and matching the rollback target snapshot ID with the snapshot ID range of each hierarchical clone object is one of the key steps in the snapshot rollback process of the present application. The core technical principle of this step is to achieve ordered access to each hierarchical clone object in the snapshot chain by maintaining the clone_list structure of the object, and to quickly locate the clone object level involved in the target rollback operation in combination with the clone_overlap field and the blind area snapshot structure.

[0041] In a specific implementation, clone_list is an ordered list that records hierarchical clone object information of an object under different snapshot versions. Each clone object contains its corresponding snapshot ID range (such as [start_id, end_id]) and a pointer to the OI (Object Info) metadata of the level. When performing snapshot rollback, the system first obtains the target snapshot ID, then traverses clone_list, and compares whether the snapshot ID falls within the snapshot ID interval of the current clone object one by one. If a match is found, the clone object is determined to be the target level of the rollback operation, and its OI information is extracted for subsequent processing.

[0042] At the parameter index level, the snapshot ID range of each clone object in clone_list is usually stored in the form of a closed interval, such as [1000, 2000], indicating that the clone object covers all versions from snapshot ID 1000 to 2000. The system uses binary search or linear scanning strategy when traversing, depending on the length of clone_list and performance requirements. In large-scale distributed storage systems, binary search algorithm is usually used to improve efficiency, with a time complexity of O(logn), which is significantly better than the O(n) of linear scanning.

[0043] The application scenario of this step mainly focuses on the execution of snapshot rollback operations in distributed storage systems, especially in key business processes such as multi-version object management, data consistency guarantee, and system fault recovery. By accurately matching the target snapshot ID with the ID range of the hierarchical clone object, the system can avoid invalid traversal of the entire snapshot chain, thereby reducing the number of metadata access times and system overhead.

[0044] From the technical effect, this step effectively solves the problem of invalid snapshot version caused by the destruction of chain dependency in traditional snapshot rollback. By introducing the blind zone snapshot structure, the system does not need to copy the forward relationship or object data when rolling back, only needs to update the clone_overlap field in the OI and generate a new blind zone snapshot record, thereby significantly improving the efficiency of the rollback operation and the data reliability of the system. In addition, this mechanism also supports quick positioning of the required snapshot object in the read operation, enhancing the concurrent processing capability and response speed of the system.

[0045] S12, extract the OI information from the matched hierarchical clone object, and verify the integrity and validity of the OI information to ensure the data consistency of subsequent operations.

[0046] Specifically, in some implementations, extracting the OI (Object Information) information from the matched hierarchical clone object and verifying the integrity and validity of the OI information is one of the key steps in the snapshot processing optimization scheme based on blind zone snapshots. This step aims to ensure that the system can accurately identify and access the correct data version in snapshot rollback or read operation, thereby guaranteeing data consistency and system reliability.

[0047] From the technical implementation point of view, this step first determines the hierarchical clone object involved in the current request by traversing the clone_list field in the OI structure of the header object and combining the target snapshot ID. Each hierarchical clone object only copies the basic metadata of the object when created, not the complete forwarding relationship, so its OI information contains the clone_overlap field, which records the dependency range of the current clone object on the previous version of the forwarding relationship. When extracting the OI information, the system needs to parse the starting offset and length of the field, and combine the mapping relationship in the blind zone snapshot structure (such as (3, 0) or (4, 1)) to determine whether the target data segment exists in the current clone object or other snapshot objects it depends on.

[0048] At the parameter index level, the integrity verification of the OI information usually includes checking whether the is_exist flag is true and whether there is a record matching the target snapshot ID in the clone_list. The validity verification involves range checking of the clone_overlap field to ensure that its offset and length are within the legal interval (e.g., 0 ≤ offset < object_size, 0 < length ≤ object_size - offset) and meet the metadata specification defined in Ceph or Ceph-compatible distributed storage systems. In addition, the key-value pair in the blind zone snapshot structure needs to meet the requirements of monotonicity of the snapshot ID and reachability of the rollback path to ensure the logical consistency of the snapshot chain.

[0049] At the application scenario level, this step is widely used in snapshot rollback and snapshot reading operations in distributed storage systems. For example, when performing snapshot rollback, the system needs to extract the OI information from the target clone object and verify whether it contains the data segment forward relationship required for rollback. When reading the snapshot object, the client request carries the snapshot ID, and the system needs to analyze the OI information and the blind zone snapshot structure to determine the ownership of the data segment, thereby realizing efficient data positioning and access.

[0050] At the technical effect level, this step accurately extracts and verifies the OI information, effectively avoiding the performance overhead caused by forward relationship copying during snapshot rollback, while ensuring the integrity of the snapshot chain is not damaged, thereby improving the stability and reliability of the system in multi-version data management. Further, this mechanism supports efficient and low-latency snapshot operations in large-scale distributed environments, providing a solid technical support for disaster recovery and version control of storage systems.

[0051] S2, based on the clone_overlap information in the hierarchical clone object OI, calculate the target rollback clone object, and delete the forward relationship of the header object.

[0052] Specifically, in some implementations, based on the clone_overlap information in the hierarchical clone object OI, the target rollback clone object is calculated, and the forward relationship of the header object is deleted, which is one of the key steps in the snapshot processing optimization scheme of the present application. The core technical principle of this step is to use the clone_overlap field to record the dependency range of the current clone object on the previous version forward relationship, so as to accurately locate the clone object that needs to be restored during rollback, avoid redundant copying of data or forward relationship of the entire object, reduce system overhead, and maintain the integrity of the snapshot chain.

[0053] The specific implementation includes: first, the system determines the hierarchical structure where the target rollback clone object is located according to the clone_list maintained in the OI (Object Information) of the header object and the user-specified rollback target snapshot ID, and obtains the OI of the hierarchical clone object. Subsequently, by parsing the clone_overlap field in the OI, the system can identify on which data intervals the current clone object depends on the forward relationship of the previous version. On this basis, the system calculates the target rollback clone object, that is, the specific version object that needs to be restored.

[0054] Further, after completing the identification of the target clone object, the system will delete the forward relationship currently saved in the header object to prevent data conflicts or inconsistencies after rollback. Subsequently, the system generates a blind zone snapshot (for example, (4, 1) represents rolling back from snapshot 4 to snapshot 1), and updates the clone_overlap field in the OI of the header object to reflect the new dependency relationship. Finally, the system assembles the forward relationship and the updated OI into a KV (Key-Value) structure, which is persisted through a distributed KV storage system.

[0055] In terms of parameters, clone_overlap is usually represented in the form of an interval (offset, length), such as (1M, 2M), to identify the data range of the previous version object that the current clone object depends on. Snapshot ID is uniquely identified using a 64-bit integer or UUID format, ensuring good scalability and uniqueness in large-scale distributed systems. The generation of blind zone snapshots needs to follow certain naming conventions and storage strategies, usually consistent with the topology of the snapshot chain to support subsequent read jumps.

[0056] This step is mainly applied to the execution phase of snapshot rollback operations in distributed storage systems, especially in scenarios where historical version data needs to be restored without affecting the current snapshot chain structure. For example, when data anomalies or version conflicts occur in the storage pool, the system can quickly locate and restore to the specified snapshot version while avoiding the destruction of dependency relationships for other snapshot versions, thereby ensuring data reliability and consistency.

[0057] In terms of technical effects, this step effectively solves the performance degradation and snapshot chain failure problems caused by forward relationship copying in the traditional snapshot rollback process. By introducing the blind zone snapshot mechanism, the system does not need to perform data replication when rolling back, only needs to update the dependency relationship and generate metadata records, significantly reducing I / O overhead and storage resource consumption. In addition, this method ensures the integrity of the snapshot chain, so that intermediate version snapshots can still be accessed correctly, thereby improving the data protection capability and version management efficiency of the system.

[0058] Further, S2 comprises:

[0059] S21, according to the dependency range of the current clone object to the previous clone object recorded in clone_overlap, determining the starting position and length of the forward relationship data segment that needs to be rolled back.

[0060] Specifically, in some implementations, according to the dependency range of the current clone object to the previous clone object recorded in clone_overlap, determining the starting position and length of the forward relationship data segment that needs to be rolled back is one of the core steps of the present application to implement snapshot rollback optimization based on the blind area snapshot mechanism. This step realizes accurate positioning and rollback operation of forward relationship data by parsing the clone_overlap field in the OI (Object Info) information of the object, combined with the rollback target snapshot ID, thereby avoiding the destruction of the snapshot chain.

[0061] In a specific technical implementation, clone_overlap is an interval mapping structure used to record the dependency relationship of the current clone object relative to its predecessor clone object in the dimensions of data offset (offset) and length (length). For example, if the current clone object has a forward relationship that depends on the previous snapshot object between offset 1M and 2M, clone_overlap will record this interval. In the snapshot rollback process, the system first obtains clone_list from the OI of the header object, and determines the level where the target clone object is located according to the rollback target snapshot ID. Then, the system calculates the starting offset and length of the forward relationship data segment that needs to be rolled back by traversing clone_overlap in the OI of this level, usually in bytes, with a precision of 1M (1024KB) or finer granularity, such as 512KB, depending on the block size parameter configured by the system.

[0062] At the parameter index level, the key parameters involved in this step include snapshot_id (snapshot identifier), offset (data offset), length (data length), the interval structure of clone_overlap (such as [start_offset, end_offset]), and the hierarchical index of clone_list. The system usually uses B+ tree or hash table structure for efficient indexing and querying of clone_overlap to meet the performance requirements in high-concurrency read-write scenarios.

[0063] In application scenarios, this step is widely used in snapshot rollback operations in distributed storage systems, especially in multi-version concurrency control (MVCC) and append-only storage architectures. Through this step, the system can restore specific version data without breaking the snapshot chain, thereby improving the data reliability and consistency guarantee capability of the system.

[0064] In terms of technical effects, this step effectively avoids the performance loss and storage overhead caused by data or forward relationship copying in traditional snapshot rollback, and at the same time, through precise dependency range calculation, ensures that the rollback operation only affects the target data segment and does not affect the integrity of other snapshot versions. Further, this mechanism supports multi-level snapshot rollback, enhancing the fault tolerance and maintainability of the system in complex data operation scenarios.

[0065] S22, mark and delete the forward relationship data segment corresponding to the rollback target snapshot id in the header object, and retain the forward relationship data not affected by the rollback to reduce the risk of data loss.

[0066] Specifically, in some implementations, this step involves marking and deleting the forward relationship data segment corresponding to the rollback target snapshot ID in the header object when performing a snapshot rollback operation in a distributed storage system, and retaining the forward relationship data not affected by the rollback. The core technical principle of this operation is to realize intelligent jumping of the dependency relationship in the snapshot chain by introducing a "blind area snapshot" mechanism, thereby avoiding breaking the integrity of the snapshot chain during rollback and reducing the risk of data loss.

[0067] The specific implementation includes: first, the system determines the hierarchical clone object to which the target rollback clone object belongs according to the clone_list recorded in the OI (Object Info) of the header object and the rollback target snapshot ID, and obtains its OI information. Then, by parsing the clone_overlap field in the OI, the coverage of the target rollback clone object is calculated. On this basis, the system marks and deletes the forward relationship data segment related to the rollback target snapshot ID in the header object, rather than physically deleting it, to retain the data segment not affected by the rollback. The deletion operation is usually based on offset and length parameters to accurately control the deletion range of the data segment and ensure that the data consistency of other snapshot versions is not affected.

[0068] Further, the system generates a blind area snapshot record with a key-value pair structure, where the key is the latest snapshot ID of the current object, and the value is the rollback target snapshot ID. The record is written into the OI of the header object and is persisted by the distributed KV storage system. In the read operation, the system will jump the snapshot chain according to the blind area snapshot and the clone overlap field, so as to quickly locate the forward relationship of the required data segment.

[0069] This step plays a key role in the snapshot rollback process, and its technical value lies in avoiding the problem of broken snapshot chain caused by rollback in the traditional scheme, while reducing the performance overhead caused by data copying. Through the precise data segment marking deletion mechanism, the system improves the storage efficiency and the reliability of data recovery while ensuring data consistency. This technology is suitable for multi-version snapshot management scenarios in distributed storage systems, especially in business systems that require frequent rollback and version switching, such as cloud storage, database backup and recovery, virtual machine snapshot, etc.

[0070] S3, generating a blind area snapshot, wherein the key of the blind area snapshot is the latest snapshot id of the current object, and the value is the target snapshot id to which the rollback is performed.

[0071] Specifically, in the distributed storage system, generating a blind area snapshot is one of the core steps to realize snapshot rollback optimization. This step introduces a special key-value structure in the OI (Object Information) of the header object, where the key is the latest snapshot ID of the current object, and the value is the target snapshot ID relied on by the rollback operation, so as to realize efficient data rollback and reading without breaking the dependency relationship of the snapshot chain.

[0072] In specific implementation, when the system performs a snapshot rollback operation, it first determines the hierarchical clone structure to which the target rollback clone object belongs according to the clone_list and target snapshot ID in the OI of the header object, and obtains its OI information. Then, the system calculates the dependency range of the target rollback clone object according to the clone_overlap field in the OI. On this basis, the system will delete the forward relationship of the current header object, and generate a new blind area snapshot record to bind the latest snapshot ID with the rollback target snapshot ID. The record is then written into the distributed KV storage system for subsequent read operation.

[0073] From the technical index level, the generation of the blind area snapshot needs to meet certain data consistency requirements, for example, the incremental rule of snapshot ID should follow the snapshot management specification of Ceph or Ceph-like system to ensure the orderliness of the snapshot chain. In addition, the clone_overlap field usually adopts the interval mapping (offset-length mapping) method to record the dependency range of the current clone object on the previous snapshot object, and its data structure can be an array or a hash table, supporting O(1) or O(log n) lookup efficiency.

[0074] In application scenarios, this step is widely used in version rollback, data recovery and consistency guarantee scenarios of distributed storage systems. For example, in Ceph or similar architecture, when the client requests to roll back to a certain historical snapshot version, the system quickly locates the dependency relationship through the blind area snapshot structure, avoiding the problem of data unrecoverability caused by the broken snapshot chain in the traditional scheme.

[0075] The technical value of this step lies in that by introducing the blind area snapshot mechanism, the problem of broken snapshot chain dependency relationship in the snapshot rollback process is effectively solved, thereby improving the data reliability and snapshot management efficiency of the system. At the same time, this mechanism avoids large-scale data or forward relationship copy operations, significantly reducing storage overhead and I / O performance loss.

[0076] Further, S3 includes:

[0077] S31, set the key of the blind area snapshot as the latest snapshot id of the header object before rollback, set the value as the snapshot id actually pointed to by the rollback, and store the mapping relationship in the OI of the header object.

[0078] Specifically, in some implementations, setting the key of the blind area snapshot as the latest snapshot id of the header object before rollback, setting the value as the snapshot id actually pointed to by the rollback, and storing the mapping relationship in the OI (Object Info) of the header object is one of the core steps of the present application to realize snapshot rollback optimization based on the blind area snapshot mechanism. This step solves the problem of invalid snapshot version caused by broken dependency relationship in the rollback process of the traditional snapshot chain by introducing the blind area snapshot structure in the OI of the header object.

[0079] In a specific technical implementation, when performing a snapshot rollback operation, the system first determines the hierarchical clone object to which the target rollback clone object belongs according to the clone_list recorded in the OI of the header object and the target snapshot id, and obtains the OI information thereof. Subsequently, the system calculates the dependency range of the target rollback clone object according to the clone_overlap field in the OI. After completing the deletion of the forward relationship of the header object, the system generates a blind area snapshot whose key is the latest snapshot id (such as Snapshot3) of the header object before rollback and whose value is the snapshot id (such as Snapshot0) pointed to by the actual rollback, and writes the blind area snapshot into the OI of the header object. The mapping relationship is used for quickly locating the dependent snapshot in subsequent read operations, avoiding direct access to invalid snapshot versions that have been rolled back.

[0080] At the parameter index level, the key and value of the blind area snapshot are both 64-bit integer snapshot identifiers (SnapshotID) for uniquely identifying snapshot versions. The clone_overlap field usually represents the dependency range of the current clone object on the previous clone object in the form of an interval (such as [0, 1M]), with a unit of bytes (Byte) and supporting a maximum of 2^64 bytes of data objects. The structure follows the standard protocol of a distributed KV storage system when storing, such as using a consistent hashing algorithm for distributed storage of key-value pairs, to ensure data consistency and read-write performance under high concurrency access.

[0081] In application scenarios, this step is widely used in snapshot rollback, data recovery, and version control scenarios in distributed storage systems. For example, when data corruption or version conflict occurs in a storage pool, the system can quickly locate the valid snapshot version through the blind area snapshot, thereby recovering data consistency and avoiding the problem of snapshot chain breakage caused by rollback operations.

[0082] The technical effect of this step is that by introducing the blind area snapshot mapping mechanism, the integrity of the snapshot chain is effectively maintained, avoiding the problem of invalid snapshot versions caused by rollback operations in traditional solutions. At the same time, this mechanism does not require large-scale data or forward relationship copying, significantly reducing system overhead and improving the efficiency and reliability of snapshot processing.

[0083] S32, a unique identifier is allocated for each blind area snapshot, and an index is established in the distributed kv storage to quickly locate the dependency relationship in subsequent read operations.

[0084] Specifically, assigning a unique identifier to each blind area snapshot and establishing an index in the distributed KV storage is the key step to achieve efficient management and rapid positioning of snapshot dependency relationship in the present application. In some implementations, this step builds a dependency graph between snapshot objects by introducing Snapshot ID as a unique identifier, combined with the clone_list and clone_overlap fields in the Object Information (OI). The structure of the blind area snapshot adopts the form of key-value pair, where the key is the latest snapshot ID of the current object, and the value is the snapshot ID that the snapshot depends on after rollback, for example, (3, 0) represents that snapshot 3 depends on snapshot 0 after rollback.

[0085] In specific operations, when performing snapshot rollback, the system first determines the hierarchical clone object to which the target rollback snapshot belongs according to the OI information of the header object, and calculates its clone_overlap range. Subsequently, the system generates a new blind area snapshot record and writes it as a KV entry into the distributed KV storage system. The KV storage system can achieve data distribution based on the CRUSH algorithm of Ceph, support high-concurrency read and write and low-latency access, and usually adopts consistent hashing or range partitioning strategy for data sharding to ensure the efficiency and scalability of the index.

[0086] At the parameter level, the snapshot ID is usually a 64-bit integer or a UUID format to ensure global uniqueness; the clone_overlap field records the dependent interval of the current clone object on the previous version object, such as (offset, length), with units of bytes or blocks. In read operations, the system quickly locates the snapshot object where the required data segment is located by parsing the clone_list and clone_overlap in the OI and combining the blind area snapshot index, thereby avoiding the dependency breakage problem caused by traditional snapshot chain rollback.

[0087] This step is widely applicable to snapshot management scenarios in distributed storage systems in practical applications, especially in environments that require frequent rollback or concurrent reading of multiple snapshot versions, such as cloud backup, virtual machine snapshot recovery, database point-in-time recovery, etc. By establishing an index in the KV storage, the system can complete the search of snapshot dependency relationship within O(1) or O(logN) time complexity, significantly improving the snapshot reading efficiency and overall system stability, while reducing storage overhead and computational resource consumption, embodying the innovation and practicality of the present application in the snapshot processing mechanism.

[0088] S4, modify the clone_overlap information in the header object OI according to the generated blind area snapshot, and assemble the forward relationship with the OI information into kv format and send to the distributed kv storage for saving.

[0089] Specifically, in some implementations, the clone_overlap information in the header object OI (Object Info) is modified according to the generated blind area snapshot, and the forward relationship is assembled with the OI information into KV (Key-Value) format and sent to the distributed KV storage for saving, which is the key step to realize snapshot chain integrity protection and efficient data reading in the present application. This step dynamically maintains the dependency relationship of the object, ensures that the structure of the original snapshot chain will not be damaged during snapshot rollback, and improves the system data consistency and reliability.

[0090] From the technical implementation point of view, when the system detects a snapshot rollback operation, it first determines the hierarchical clone object to which the target rollback object belongs according to the clone_list in the header object OI and the target snapshot ID, and obtains its OI information. Then, the system calculates the dependency range of the target rollback clone object according to the clone_overlap field in the OI. On this basis, the system deletes the forward relationship of the current header object, and generates a new blind area snapshot in KV structure, where the key is the latest snapshot ID of the current object, and the value is the rollback target snapshot ID. This blind area snapshot records the jump relationship in the snapshot rollback process, which is used for fast positioning of dependent objects in subsequent read operations.

[0091] From the parameter index level, the clone_overlap field is usually represented in interval form, such as (1M, 2M), which is used to identify the forward relationship range of the current clone object dependent on the previous version. The generation of blind area snapshot needs to follow certain naming conventions, such as using the format "snapshot_id:rollback_id", to ensure good retrievability and consistency in distributed KV storage. In the KV assembly process, the forward relationship needs to be serialized, usually in JSON or Protobuf format, to improve transmission efficiency and storage compatibility.

[0092] From the application scenario level, this step is widely used in the snapshot management module of distributed storage system, especially in the multi-version concurrency control (MVCC) and data rollback scenarios. For example, in a cloud storage platform, when a user requests to roll back to a historical snapshot version, the system quickly builds the dependency relationship through this step to avoid data inconsistency problems caused by broken snapshot chain.

[0093] From the technical effect level, this step effectively avoids the performance loss and storage overhead caused by directly copying the positive relationship in the traditional snapshot rollback, and at the same time, through the blind area snapshot mechanism, the integrity of the snapshot chain is ensured, thereby improving the stability and data recovery capability of the system in the high concurrency and large-scale data scene.

[0094] The snapshot processing method based on the blind area snapshot of the embodiment of the application effectively avoids the damage to the snapshot chain in the snapshot rollback process by introducing the blind area snapshot structure, maintains the dependency relationship between the snapshot versions, and improves the system data reliability and snapshot reading efficiency

[0095] S5, according to the snapshot id carried in the read request, find the corresponding layered clone object in the clone_list in the header object OI, and determine the dependent object of the target data segment based on the clone_overlap information of the layered clone object, to support the read operation of the snapshot object.

[0096] Specifically, in the distributed storage system, when a read request is received, the request usually carries an identifier (snapshot id) of a target snapshot. The core of this step is to find the corresponding layered clone object in the clone_list in the header object OI (Object Information) according to the snapshot id, and determine the dependent object of the target data segment based on the clone_overlap information of the clone object, so as to support the read operation of the snapshot object. This process is a key link for implementing the blind area snapshot mechanism, and ensures that the historical version data can be correctly accessed after the snapshot rollback.

[0097] In some implementations, a clone_list structure is maintained in the header object OI, which records the layered clone objects and their snapshot ids of the object under different snapshot versions. When a read request arrives, the system first parses the snapshot id in the request and performs a matching search in the clone_list to determine the layered clone object corresponding to the snapshot id. The clone_list usually adopts an ordered list or hash table structure to support O(1) or O(log n) time complexity search operation, and the specific implementation can be selected according to the system performance requirement.

[0098] Further, after finding the corresponding hierarchical clone object, the system will read the clone_overlap field in its OI. clone_overlap is an interval mapping structure used to describe the positive relationship of the current clone object depending on the previous snapshot object in which data offset interval. For example, clone_overlap can be represented as a series of (offset, length) interval pairs, each corresponding to a dependent snapshot object. When reading the target data segment, the system will determine whether the segment exists in the current clone object or needs to jump to the dependent snapshot object for reading according to the offset and length of the segment and the clone_overlap information.

[0099] Optionally, in the snapshot rollback operation, the system will generate a blind area snapshot record in the form of (current_snapshot_id, rollback_target_id) stored in the header object OI. When reading the (2M, 3M) data segment of Snapshot4, if the segment is marked as dependent on Snapshot1 in clone_overlap, the system will perform snapshot chain jumping according to the blind area snapshot record, thereby ensuring the correctness of the reading path.

[0100] The technical implementation of this step relies on an efficient management mechanism of object metadata in the storage system, involving the maintenance of snapshot chain, the hierarchical structure design of clone object, and the interval mapping algorithm of dependency relationship. The key parameters include the length of snapshot id (usually 64-bit or 128-bit UUID), the interval granularity of clone_overlap (such as 1MB or 4KB), the storage structure of clone_list (such as B+ tree or hash table), etc. Through this step, the system can achieve efficient and reliable historical version data reading without breaking the snapshot chain dependency relationship, thereby significantly improving the data consistency and access performance of the distributed storage system in the snapshot rollback scenario.

[0101] The snapshot processing method based on blind area snapshot of the embodiment of the application further improves the accuracy and efficiency of snapshot reading and enhances the data consistency guarantee capability of the system in the complex snapshot dependency scenario by finding the corresponding hierarchical clone object from the clone_list of the header object OI according to the snapshot ID in the reading request and using the clone_overlap information to accurately locate the dependent object of the target data segment.

[0102] In order to realize the above-mentioned embodiment, the application further provides a snapshot processing device based on blind area snapshot. Figure 2A structure schematic diagram of a snapshot processing device based on a blind area snapshot is provided for an embodiment of the present application. Figure 2 As shown in the figure, the device comprises:

[0103] A hierarchical object determination module 100 is configured to determine a hierarchical clone object to which a target rollback clone object belongs according to a clone_list and a rollback target snapshot id in an OI of a header object, and obtain an OI of the hierarchical clone object.

[0104] A forward relationship processing module 200 is configured to calculate a target rollback clone object based on clone_overlap information in an OI of a hierarchical clone object, and delete a forward relationship of a header object.

[0105] A blind area snapshot generation module 300 is configured to generate a blind area snapshot, wherein a key of the blind area snapshot is a latest snapshot id of a current object, and a value is a target snapshot id to which a rollback is performed.

[0106] An information assembling and storage module 400 is configured to modify clone_overlap information in an OI of a header object according to the generated blind area snapshot, and assemble a forward relationship and the OI information into a kv format and send to a distributed kv storage for storage.

[0107] As to the device in the above-mentioned embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0108] In order to realize the above-mentioned embodiments, the present application further provides an electronic device, comprising a processor and a memory connected with the processor in communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to realize the method provided by the foregoing embodiments.

[0109] In order to realize the above-mentioned embodiments, the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to realize the method provided by the foregoing embodiments.

[0110] In order to realize the above-mentioned embodiments, the present application further provides a computer program product, comprising a computer program, wherein the computer program is executed by a processor to realize the method provided by the foregoing embodiments.

[0111] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the present application comply with relevant laws and regulations and do not violate public order and good customs.

[0112] It is important to note that user's personal information should be collected for legitimate and reasonable uses of the service and not shared or sold outside of those legitimate uses. Further, such collection / sharing should occur after the users are notified of these uses and agree to the terms, such as by agreeing to the terms of service, where they are given the opportunity to opt in to such uses. Still further, any desired step can be taken by the operator to avoid collecting personal information, such as avoiding collecting personal information on-line. In addition, users should be provided with an opportunity to opt out of collection of personal information, where appropriate, and / or subsequent transfer to a third party.

[0113] Embodiments are contemplated that provide users with the ability to selectively allow or block the use or access of personal information data. That is, the disclosure contemplates providing users with control to permit or deny permission to access such personal information data. Such control can be provided in a variety of manners such as by allowing a user to opt-in or opt-out of particular tracking services at the point of collection (e.g., during signup), providing subscribers with options for what information can be shared with third parties, providing users with the ability to prevent further collection of personal information data after it has been collected (e.g., by deleting a user account or by changing settings), providing users with the ability to limit the types of services that can use their personal information data, and so on. Further, embodiments are contemplated that provide users with the ability to access and / or edit their personal information data.

[0114] In the foregoing description of various embodiments described herein, reference has been made to the use of terms such as "one embodiment", "an embodiment", "some embodiments", "exemplary", "specific example", or the like. This terminology is not intended to refer to any particular embodiment or number of embodiments, but instead is intended to highlight that the embodiments described herein can include a variety of embodiments. Although the above description has been described with reference to particular embodiments, it is to be understood that these are only examples and modifications can be made without departing from the scope of the present application. For example, the above description describes certain embodiments using terms such as "one embodiment", "an embodiment", "some embodiments", "exemplary", "specific example", or the like, which is intended to mean that a certain feature, structure, material, or characteristic described in connection with that embodiment is included in at least one embodiment or example of the present application. The appearances of the above-described terms in various places in the specification are not intended to exclude that the terms define the same feature, structure, material, or characteristic or are kept equivalent in some way. Moreover, it is appreciated that features of the described embodiments can be combined, removed, or modified without departing from the scope of the application.

[0115] In addition, the terms "first", "second", etc. are used herein only to describe various steps in a method, program, routine or the like, and do not imply a relative importance or a specific order of steps. Thus, features defined with "first", "second" etc. can include at least one of the features. In the description of the application, the meaning of "a plurality" is at least two, for example, two, three or the like, unless otherwise specifically defined.

[0116] Any process or method described in a flowchart or otherwise described herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for performing specific logic functions (or steps) or portions of the application, and that the various elements of the flowchart can be implemented with computer-readable media that is formatted to contain the code. It should also be understood that each step within a flowchart can represent one or more modules, segments, or portions of code, which include one or more executable instructions for performing specific logic functions (or steps). As such, the various system components and process steps discussed above can be implemented with software, hardware, or a combination of these. It should be appreciated that many types of software, hardware, and / or firmware instructions can be used to implement the methods and processes described herein. The software, hardware, and / or firmware instructions will be referred to generally as "code" herein.

[0117] The logic and / or steps represented in flow diagrams or otherwise described herein, for example, can be considered as a sequence of instructions to implement logic functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this specification, a "computer-readable medium" can be any means that can contain, store, communicate, propagate or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a machine-readable storage device (e.g., magnetic, optical or other) a machine-readable storage diskette (e.g., floppy disk, optical disk, etc.), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), optical fibers, and a portable compact disc read-only memory (CDROM), among others. Additionally, the computer-readable medium can be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0118] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. As such, in some embodiments, the steps or methods can be implemented in a combination of hardware and software. If implemented in hardware, as in another embodiment, any of the above techniques can be implemented with or without the use of the following technologies, which are well known in the art: discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0119] Those of skill in the art would understand that information and signals can be represented using any of a variety of technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that can be referenced throughout the above description can be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0120] In addition, each function unit in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0121] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

[0122] It should be understood that various forms of flow shown above can be reordered, added or deleted. For example, each step described in the present application can be executed in parallel, sequentially or in different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0123] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and replacements can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method of snapshot processing based on blind area snapshots, characterized in that, The method comprises the following steps: S1, according to the clone_list and the rollback target snapshot id in the OI of the header object, determine the hierarchical clone object to which the target rollback clone object belongs, and obtain the OI of the hierarchical clone object; S2, based on the clone_overlap information in the OI of the hierarchical clone object, calculate the target rollback clone object, and delete the forward relationship of the header object; S3, generate a blind area snapshot, wherein the key of the blind area snapshot is the latest snapshot id of the current object, and the value is the target snapshot id to which the rollback is performed; S4, modify the clone_overlap information in the OI of the header object according to the generated blind area snapshot, and assemble the forward relationship and the OI information into kv format and send to the distributed kv storage for saving.

2. The method of claim 1, wherein, According to the clone_list and the rollback target snapshot id in the OI of the header object, the method further comprises the steps of: S11, traversing the hierarchical clone object list recorded in the clone_list, matching the rollback target snapshot id with the snapshot id range of each hierarchical clone object to determine the hierarchical clone object in which the target rollback clone object is located; S12, extracting the OI information of the matched hierarchical clone object, and verifying the integrity and validity of the OI information to ensure the data consistency of subsequent operations.

3. The method of claim 1, wherein, According to the clone_overlap information in the OI of the hierarchical clone object, the method further comprises the steps of: S21, according to the dependency range of the current clone object on the last clone object recorded in the clone_overlap, determining the starting position and length of the forward relationship data segment to be rolled back; S22, marking and deleting the forward relationship data segment corresponding to the rollback target snapshot id in the header object, and retaining the forward relationship data not affected by the rollback to reduce the risk of data loss.

4. The method of claim 1, wherein, The method of generating a blind area snapshot, wherein the key of the blind area snapshot is the latest snapshot id of the current object, and the value is the target snapshot id to which the rollback is performed, further comprises the steps of: S31, setting the key of the blind area snapshot as the latest snapshot id of the header object before rollback, and setting the value as the snapshot id actually pointed to by the rollback, and storing the mapping relationship in the OI of the header object; S32, assigning a unique identifier to each blind area snapshot, and establishing an index in the distributed kv storage to facilitate subsequent read operation to quickly locate the dependency relationship.

5. The method of claim 1, wherein, The method further comprises the steps of: S5, according to the snapshot id carried in the read request, find the corresponding layered clone object in the clone_list in the header object OI, and determine the dependent object of the target data segment based on the clone_overlap information of the layered clone object, to support the read operation of the snapshot object.

6. A blind area snapshot-based snapshot processing apparatus characterized by comprising: Comprise: The layered object determination module is used for determining the layered clone object to which the target rollback clone object belongs according to the clone_list and the rollback target snapshot id in the OI of the header object, and obtaining the OI of the layered clone object; The forward relationship processing module is used for calculating the target rollback clone object based on the clone_overlap information in the OI of the layered clone object, and deleting the forward relationship of the header object; The blind area snapshot generation module is used for generating a blind area snapshot, wherein the key of the blind area snapshot is the latest snapshot id of the current object, and the value is the target snapshot id to which the rollback is performed; The information assembly and storage module is used for modifying the clone_overlap information in the OI of the header object according to the generated blind area snapshot, and assembling the forward relationship and the OI information into kv format and sending to the distributed kv storage for saving.

7. The apparatus of claim 6, wherein, The layered object determination module is further used for: Traversing the layered clone object list recorded in the clone_list, matching the rollback target snapshot id with the snapshot id range of each layered clone object, and determining the layered clone object in which the target rollback clone object is located; Extracting the OI information of the matched layered clone object, and verifying the integrity and validity of the OI information, to ensure the data consistency of subsequent operations.

8. An electronic device, comprising: Comprise: A processor, and a memory connected with the processor in communication; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, to realize the method in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to realize the method in any one of claims 1-6.

10. A computer program product, characterised in that, Comprise a computer program, which is executed by the processor to realize the method in any one of claims 1-6.

Images