Snapshot data validity verification method and device, electronic equipment and program product

Abstract

The invention discloses a snapshot data validity verification method and device, electronic equipment and a program product, and relates to the technical field of snapshots.The verification method comprises the steps that a deleted snapshot identifier set in a deleted state is determined on the basis of a snapshot list, and a deleted snapshot identifier set in a deleted state is determined on the basis of the deleted snapshot identifier set; determining each target storage unit of which the snapshot identifier is deleted, and for each target storage unit, based on a snapshot reference table and a metadata index carried by the target storage unit, verifying the snapshot data on the target storage unit to obtain a verification result, and under the condition that the verification result indicates that the snapshot data in the invalid state exists on the target storage unit, recovering the snapshot data in the invalid state. According to the method and the device, the technical problem of relatively low efficiency of checking the snapshot data in related technologies is solved.

Description

Technical Field

[0001] The present invention relates to the field of snapshot technology, and in particular to a method and device for verifying the validity of snapshot data, an electronic device, and a program product. Background Art

[0002] In all-flash block storage systems, snapshots are a mechanism for preserving the data state of a volume at a specific point in time. They can be used for data backup and recovery. Snapshots use a copy-on-write mechanism, overwriting write operations to generate snapshot data. Deleting a snapshot may invalidate a large amount of snapshot data, requiring snapshot garbage collection (GC) to identify the invalid data.

[0003] Data is stored in chunks (e.g., 64MB), consisting of header metadata, data areas, and tail metadata (including extents (metadata describing data chunks) and a garbage bitmap). The head space quickly determines data validity using the garbage bitmap. However, snapshot garbage collection (GC) requires a full metadata scan to determine reference relationships because data may be referenced by multiple snapshots. This involves a large amount of random I / O (input / output), resulting in low efficiency. Especially for high-performance all-flash storage, the query overhead in snapshot GC becomes a performance bottleneck.

[0004] In related technologies, in order to perform snapshot GC, a full metadata scan method is required to verify the validity of snapshot data. That is, when performing GC, the snapshot metadata of the entire chunk is scanned and combined with the list of deleted snapshots to determine whether the data is referenced by other snapshots. If the data is not introduced by any snapshot, it means that the data is invalid and can be recycled. However, this full metadata scan method has the following problems:

[0005] (1) Low verification efficiency: Full metadata scanning requires scanning the entire chunk (involving a 64MB logical range), which involves a large amount of random I / O, a long verification time, and limited GC performance, affecting the high IOPS (Input / Output Operations Per Second) characteristics of all-flash storage.

[0006] (2) High resource consumption: Frequent index queries (metadata indexes) lead to high memory cache and disk I / O overhead, high CPU (Central Processing Unit) usage, and difficulty meeting the low latency requirements of all-flash storage.

[0007] (3) Complex reference relationship processing: Snapshot data may be referenced by multiple snapshots. After deleting a snapshot, the reference relationships need to be checked one by one, which is inefficient.

[0008] Moreover, if a garbage bitmap is used for snapshot GC, although invalid data can be marked through the garbage bitmap, the index still needs to be queried to confirm the snapshot reference status, which involves a large amount of random I / O and results in low efficiency.

[0009] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0010] Embodiments of the present invention provide a method and device for verifying the validity of snapshot data, an electronic device, and a program product, so as to at least solve the technical problem of low efficiency in verifying snapshot data in related technologies.

[0011] According to one aspect of an embodiment of the present invention, a method for verifying the validity of snapshot data is provided, comprising: determining a set of deleted snapshot identifiers in a deleted state based on a snapshot list; determining a target storage unit for each deleted snapshot identifier based on the deleted snapshot identifier set, wherein the target storage unit stores snapshot data protected by the snapshot; for each target storage unit, verifying the snapshot data on the target storage unit based on a snapshot reference table and a metadata index carried by the target storage unit to obtain a verification result, wherein the snapshot reference table is a reference table generated when the snapshot is created, and the metadata index records the snapshot identifier referenced by each snapshot data; if the verification result indicates that there is snapshot data in an invalid state on the target storage unit, the invalid snapshot data is recycled.

[0012] Furthermore, before determining the deleted snapshot identification set in the deleted state based on the snapshot list, it also includes: for each storage volume, when the first data block of the initial data of the storage volume is overwritten by the first new data, creating a first snapshot, wherein the first snapshot is used to protect the initial data of the storage volume, and the first data block is any data block on the initial data; writing the first data block protected by the first snapshot into the storage unit, and generating a snapshot reference table, wherein there is a correspondence between the snapshot reference table and the storage unit.

[0013] Furthermore, the step of generating the snapshot reference table includes: determining a snapshot identifier of the first snapshot; determining a first reference sequence based on the snapshot identifier and the length of the first new data; and writing the first reference sequence into the snapshot reference table.

[0014] Furthermore, the step of determining the first reference sequence based on the snapshot identifier and the length of the first new data includes: determining the reference length of the reference sequence based on the length of the first new data and a preset length unit; determining the reference value at each length in the reference sequence based on the snapshot identifier; and determining the first reference sequence based on the reference length and the reference value at each length.

[0015] Furthermore, after generating the snapshot reference table, it also includes: creating a second snapshot when the second data block of the current data of the storage volume is overwritten by the second new data, wherein the second snapshot is used to protect the current data of the storage volume, and the current data includes: the first new data and the remaining initial data after being overwritten by the first new data; determining whether the second data block is a data block in the remaining initial data; when the second data block is a data block in the remaining initial data, determining the second data block protected by both the first snapshot and the second snapshot, and writing the second data block to the storage unit; determining a second reference sequence based on the snapshot identifier of the first snapshot, the snapshot identifier of the second snapshot, and the length of the second new data, wherein the minimum snapshot identifier between the snapshot identifier of the first snapshot and the snapshot identifier of the second snapshot is determined as the reference value on the second reference sequence; and writing the second reference sequence to the snapshot reference table.

[0016] Furthermore, after writing the second data block to the storage unit, the method further includes: when the storage unit is full, determining that the writing of the snapshot reference table is completed; compressing the snapshot reference table after writing, and adding the compressed snapshot reference table to the end of the storage unit; for each data block on the storage unit, writing all snapshot identifiers of the protected data block to the metadata index, and adding the metadata index to the end of the storage unit.

[0017] Furthermore, the step of verifying the snapshot data on the target storage unit includes: querying a reference sequence corresponding to the snapshot data from a snapshot reference table; determining, based on the reference sequence, a target snapshot identifier indicated by a minimum snapshot identifier that protects the snapshot data; and determining that the snapshot data is in an invalid state when the target snapshot identifier belongs to a deleted snapshot identifier set and there are no remaining snapshot identifiers that do not belong to the deleted snapshot identifier set, wherein the remaining snapshot identifiers are snapshot identifiers other than the target snapshot identifier that protect the snapshot data, determined by querying a metadata index.

[0018] According to another aspect of an embodiment of the present invention, a device for verifying the validity of snapshot data is also provided, including: a first determination unit, for determining a set of deleted snapshot identifiers in a deleted state based on a snapshot list; a second determination unit, for determining a target storage unit for each deleted snapshot identifier based on the set of deleted snapshot identifiers, wherein the target storage unit stores snapshot data protected by the snapshot; a verification unit, for verifying the snapshot data on each target storage unit based on a snapshot reference table and a metadata index carried by the target storage unit to obtain a verification result, wherein the snapshot reference table is a reference table generated when the snapshot is created, and the metadata index records the snapshot identifier referenced by each snapshot data; a recycling unit, for recycling the invalid snapshot data when the verification result indicates that there is invalid snapshot data on the target storage unit.

[0019] Furthermore, the verification device also includes: a first creation unit, used to create a first snapshot for each storage volume before determining a set of deleted snapshot identifiers in a deleted state based on the snapshot list, when the first data block of the initial data of the storage volume is overwritten by the first new data, wherein the first snapshot is used to protect the initial data of the storage volume, and the first data block is any data block on the initial data; a first generation unit, used to write the first data block protected by the first snapshot into the storage unit, and generate a snapshot reference table, wherein there is a corresponding relationship between the snapshot reference table and the storage unit.

[0020] Furthermore, the first generating unit includes: a first determining module for determining a snapshot identifier of the first snapshot; a second determining module for determining a first reference sequence based on the snapshot identifier and the length of the first new data; and a first writing module for writing the first reference sequence into a snapshot reference table.

[0021] Furthermore, the second determination module includes: a first determination submodule, used to determine the reference length of the reference sequence based on the length of the first new data and a preset length unit; a second determination submodule, used to determine the reference value at each length on the reference sequence based on the snapshot identifier; and a third determination submodule, used to determine the first reference sequence based on the reference length and the reference value at each length.

[0022] Furthermore, the verification device also includes: a second creation unit, configured to, after generating the snapshot reference table, create a second snapshot when the second data block of the current data of the storage volume is overwritten by the second new data, wherein the second snapshot is used to protect the current data of the storage volume, and the current data includes: the first new data and the remaining initial data after being overwritten by the first new data; a third determination unit, configured to determine whether the second data block is a data block in the remaining initial data; a fourth determination unit, configured to, when the second data block is a data block in the remaining initial data, determine the second data block protected by both the first snapshot and the second snapshot, and write the second data block into the storage unit; a fifth determination unit, configured to determine a second reference sequence based on the snapshot identifier of the first snapshot, the snapshot identifier of the second snapshot, and the length of the second new data, wherein the minimum snapshot identifier between the snapshot identifier of the first snapshot and the snapshot identifier of the second snapshot is determined as the reference value on the second reference sequence; and a first writing unit, configured to write the second reference sequence into the snapshot reference table.

[0023] Furthermore, the verification device also includes: a sixth determination unit, used to determine that the writing of the snapshot reference table is completed after the second data block is written into the storage unit when the storage unit is full; a first compression unit, used for the second writing unit, used to compress the snapshot reference table that has been written, and add the compressed snapshot reference table to the end of the storage unit; a third writing unit, used to write all snapshot identifiers of the protection data block to the metadata index for each data block on the storage unit, and add the metadata index to the end of the storage unit.

[0024] Furthermore, the verification unit includes: a first query module, used to query the reference sequence corresponding to the snapshot data from the snapshot reference table; a third determination module, used to determine the target snapshot identifier indicated by the minimum snapshot identifier that protects the snapshot data based on the reference sequence; a fourth determination module, used to determine that the snapshot data is in an invalid state when the target snapshot identifier belongs to a deleted snapshot identifier set and there are no remaining snapshot identifiers that do not belong to the deleted snapshot identifier set, wherein the remaining snapshot identifiers are snapshot identifiers other than the target snapshot identifier that protect the snapshot data, determined by querying the metadata index.

[0025] According to another aspect of an embodiment of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and the computer program, when executed by a processor, implements any of the above-mentioned methods for verifying the validity of snapshot data.

[0026] According to another aspect of an embodiment of the present invention, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any of the above-mentioned methods for verifying the validity of snapshot data.

[0027] In the present invention, based on the snapshot list, a set of deleted snapshot identifiers in a deleted state is determined, and based on the deleted snapshot identifier set, a target storage unit for each deleted snapshot identifier is determined. For each target storage unit, based on the snapshot reference table and metadata index carried by the target storage unit, the snapshot data on the target storage unit is verified to obtain a verification result. When the verification result indicates that there is snapshot data in an invalid state on the target storage unit, the snapshot data in the invalid state is recovered, thereby solving the technical problem of low efficiency in verifying snapshot data in the related art.

[0028] In the present invention, a corresponding snapshot reference table is pre-created for each storage unit. Based on the snapshot list, the identification set of deleted snapshots can be determined, and then the target storage unit (Chunk) storing the snapshot data can be located. By parsing the snapshot reference table at the end of the Chunk and combining it with the metadata index, it can be efficiently determined whether the snapshot data is in an invalid state. If the data is invalid, garbage collection can be performed, reducing random I / O operations, improving verification efficiency, and achieving the purpose of quickly verifying the validity of snapshot data, thereby realizing the technical effect of improving snapshot GC efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0030] Figure 1 is a flowchart of an optional method for verifying the validity of snapshot data according to an embodiment of the present invention;

[0031] Figure 2 1 is a schematic diagram of an optional volume overwriting method for writing data A once according to an embodiment of the present invention;

[0032] Figure 3 is a schematic diagram of an optional method of overwriting new data on a volume according to an embodiment of the present invention;

[0033] Figure 4 This is a schematic diagram of an optional method of overwriting new data at volume offset = 32K according to an embodiment of the present invention;

[0034] Figure 5 1 is a schematic diagram of optionally overwriting new data at volume offset = 64k and = 0 according to an embodiment of the present invention;

[0035] Figure 6 is a schematic diagram of an optional Chunk1 view according to an embodiment of the present invention;

[0036] Figure 7 is a schematic diagram of an optional Chunk1 structure according to an embodiment of the present invention;

[0037] Figure 8 is a schematic diagram of an optional Chunk2 structure according to an embodiment of the present invention;

[0038] Figure 9 is a schematic diagram of an optional device for verifying the validity of snapshot data according to an embodiment of the present invention;

[0039] Figure 10The figure is a hardware structure block diagram of an electronic device (or mobile device) for verifying the validity of snapshot data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0041] It should be noted that the terms "first," "second," and the like in the present invention are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatuses.

[0042] To facilitate those skilled in the art to understand the present invention, some of the terms or nouns involved in the embodiments of the present invention are explained below:

[0043] HEAD: Used to store the latest data version currently in use by the volume.

[0044] Snapshot: used to save snapshot data at a specific point in time, that is, a copy of the entire volume's data at the moment the snapshot is created.

[0045] Extent: Metadata that describes the data block and is used to record the mapping from logical addresses to physical addresses.

[0046] Snapshot Extent: This is the metadata of the snapshot data block, used to store the logical range of the snapshot and its corresponding physical address.

[0047] Chunk: The basic unit of data storage in the storage system, with a size of 64MB, including header metadata (header), data area, and tail metadata (tail metadata).

[0048] Index: This is the metadata index used to query the mapping between logical addresses and physical addresses in the storage system. It is stored on disk and the query involves random I / O.

[0049] Snapshot GC: Snapshot garbage collection identifies and deletes data blocks that become invalid due to snapshot deletion, freeing up storage space.

[0050] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) collected and involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation portals for users to choose to authorize or refuse. For example, an interface is set up between this system and the relevant users or institutions. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or institution through the interface, and obtain the relevant information after receiving the consent information fed back by the aforementioned user or institution.

[0051] This invention introduces a snapshot reference table as the core mechanism for verifying snapshot data validity in all-flash block storage systems. Using a key-value data structure of approximately 8KB, it records the snapshot dependencies of each 4KB data block. This effectively addresses the inefficiency and high resource consumption caused by the current snapshot garbage collection (GC) requiring the scanning of large amounts of metadata. While minimizing performance impact, this effectively improves the efficiency of snapshot GC in all-flash block storage systems, ensuring high performance and resource utilization in storage systems under large-scale snapshot scenarios. It addresses the following issues:

[0052] (1) Low efficiency in snapshot data validity verification: The current snapshot GC needs to scan metadata in a 64MB logical range, which involves a large amount of random I / O and is time-consuming. However, the present invention can quickly verify data validity in memory by loading a snapshot reference table of approximately 8KB, effectively improving efficiency.

[0053] (2) High resource consumption of metadata query: Frequent index query currently results in high memory cache and disk I / O overhead, which makes it difficult to meet the high IOPS requirements of all-flash storage. However, the present invention reduces I / O overhead by eliminating index query, and CPU usage is also greatly reduced;

[0054] (3) The processing of snapshot dynamic reference relationships is complex: Currently, multiple snapshot reference relationships need to be checked one by one, which is inefficient. However, the present invention can quickly determine the dynamic reference status through the snapshot reference table and the memory snapshot list, thereby improving processing efficiency.

[0055] The present invention will be described in detail below with reference to various embodiments.

[0056] Example 1

[0057] According to an embodiment of the present invention, an embodiment of a method for verifying the validity of snapshot data is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0058] Figure 1 FIG. 1 is a flow chart of an optional method for verifying the validity of snapshot data according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0059] Step S101: Determine a deleted snapshot identifier set in a deleted state based on a snapshot list.

[0060] In this embodiment of the present invention, the storage system's memory maintains a snapshot list containing information about all created snapshots. When a snapshot is deleted by a user, its status is marked as "deleted" in the snapshot list. To conserve storage space, all deleted snapshots can be periodically filtered from the snapshot list to form a collection, known as a "deleted snapshot identifier collection." This facilitates locating the snapshot data corresponding to these deleted snapshot identifiers and deleting the expired snapshot data to reclaim storage space.

[0061] Step S102: determining a target storage unit for each snapshot deletion identifier based on the snapshot deletion identifier set, wherein the target storage unit stores snapshot data protected by the snapshot.

[0062] In an embodiment of the present invention, after determining the set of snapshot identifiers to be deleted, the target storage units (i.e., Chunks) associated with these snapshot identifiers can be located. Because snapshot data is protected when the snapshot is created, each Chunk may contain one or more snapshot data.

[0063] Step S103: For each target storage unit, the snapshot data on the target storage unit is verified based on the snapshot reference table and metadata index carried by the target storage unit to obtain a verification result, wherein the snapshot reference table is a reference table generated when the snapshot is created, and the metadata index records the snapshot identifier referenced by each snapshot data.

[0064] In this embodiment of the present invention, the validity of the stored snapshot data can be verified for each chunk (target storage unit) based on the snapshot reference table and metadata index stored at the end of the chunk. The snapshot reference table records which snapshots reference each 4KB data block within the chunk. The metadata index is a tool for finding data in the storage system and records the snapshot identifier referenced by each snapshot data.

[0065] In this embodiment of the present invention, the presence of a snapshot reference table reduces reliance on metadata indexes. By comparing the snapshot IDs in the snapshot reference table with those in the "deleted snapshot identifier set," the validity of the data within a chunk can be determined. If a data chunk only has references to deleted snapshots, that data is marked as invalid. This allows for rapid determination of data validity using the snapshot reference table, avoiding the resource waste and performance bottlenecks associated with full metadata scans.

[0066] Step S104: When the verification result indicates that invalid snapshot data exists on the target storage unit, the invalid snapshot data is recycled.

[0067] In this embodiment of the present invention, if invalid snapshot data (i.e., data not referenced by any undeleted snapshots) is detected on the target storage unit, a data recycling mechanism can be triggered to remove this invalid data from the chunk, freeing up storage space. This not only helps optimize storage space but also improves overall storage system performance, especially when handling large-scale snapshots.

[0068] In this embodiment, a snapshot reference table is introduced into the chunk's tail metadata to record the snapshot IDs that each 4KB data block depends on. This table, combined with the in-memory snapshot list, allows for rapid data validity determination without querying index metadata. The snapshot reference table is generated when a chunk is written and, when full, compressed and stored in the chunk's tail metadata. This table, approximately 8KB in size, reduces I / O, CPU, and memory overhead, optimizing all-flash storage performance.

[0069] In this embodiment, the snapshot reference table is a piece of data recorded when writing chunk data, detailing which snapshots the chunk data depends on. These records are combined at the chunk granularity to form a snapshot reference table with a maximum length of 64MB / 4KB = 16384.

[0070] In summary, by pre-creating a corresponding snapshot reference table for each storage unit, the identification set of deleted snapshots can be determined based on the snapshot list, and then the target storage unit (Chunk) storing the snapshot data can be located. By parsing the snapshot reference table at the end of the Chunk and combining it with the metadata index, it is possible to efficiently determine whether the snapshot data is in an invalid state. If the data is invalid, garbage collection can be performed, reducing random I / O operations, improving verification efficiency, and achieving the purpose of quickly verifying the validity of snapshot data, thereby achieving the technical effect of improving the snapshot GC efficiency, and further solving the technical problem of low efficiency in verifying snapshot data in related technologies.

[0071] In order to generate a corresponding snapshot reference table for each storage unit, in the snapshot data validity verification method provided in the first embodiment of the present application, before determining the deleted snapshot identification set in the deleted state based on the snapshot list, for each storage volume, when the first data block of the initial data of the storage volume is overwritten by the first new data, a first snapshot is created, wherein the first snapshot is used to protect the initial data of the storage volume, and the first data block is any data block on the initial data; the first data block protected by the first snapshot is written to the storage unit, and a snapshot reference table is generated, wherein there is a corresponding relationship between the snapshot reference table and the storage unit.

[0072] In an embodiment of the present invention, in the initial state, when data is first written to each storage volume (i.e., data is written to the HEAD space of the volume), the initial data of the volume is obtained. If the client mounts data (i.e., writes data) on the storage volume (i.e., the first data block of the initial data of the storage volume is overwritten by the first new data, and the first data block is any data block on the initial data, such as 0-32k), the first snapshot is automatically created. The purpose of the first snapshot is to protect the initial data state of the storage volume. At this point, all data in the volume (i.e., all data before the new data is overwritten) belongs to the initial data, and the snapshot will completely save a copy of this data for subsequent data recovery or historical status backtracking. For example, the client mounts the storage volume, writes 32k of data at volume offset = 0, creates snapshot 1, and then overwrites it, the protected snapshot 1 data (i.e., the first data block) will be generated.

[0073] In this embodiment of the present invention, when the first data block in a storage volume is overwritten by new data for the first time—for example, when the first data block within a logical range of the volume is overwritten by the first new data—the first data block before the overwriting (i.e., the snapshot data) can be written to the storage unit (Chunk), and a corresponding snapshot reference table is generated. The snapshot reference table records which snapshots reference each 4KB data block within the Chunk, specifically, which snapshots are associated with the historical versions of the data block.

[0074] In this embodiment of the present invention, a one-to-one correspondence exists between the snapshot reference table and the storage unit (chunk). That is, whenever a new snapshot data block is written to a chunk, the chunk's snapshot reference table is updated accordingly to reflect the latest snapshot dependencies. This correspondence ensures that when snapshot data validity verification is required, the specific location of the data storage can be quickly located and the validity of the data block can be determined through the snapshot reference table, without requiring extensive metadata queries.

[0075] Figure 2 FIG. 1 is a schematic diagram of an optional volume overwriting method for writing data A once according to an embodiment of the present invention. Figure 2 As shown, it shows the volume storage view when only the initial data is overwritten once, that is, the initial state, where data A is written to the HEAD space of the volume, and the HEAD space starts from 0K.

[0076] Figure 3 FIG. 1 is a schematic diagram of an optional method of overwriting new data on a volume according to an embodiment of the present invention. Figure 3 As shown, when the client is mounted on the storage volume and writes 32k data B at the volume's HEAD space offset = 0, it creates snapshot 1 and overwrites B from 0-32K. After overwriting B, data protected by snapshot 1 is generated, namely snapshot1 (A referenced by snapshot 1). The storage processes the data and writes the protected snapshot 1 data (A referenced by snapshot 1) into Chunk1 to protect the snapshot data. Because the size of the generated snapshot data is 32k, 32k data is written to Chunk1. At the same time, the corresponding Chunk1 snapshot reference table {1, 1, 1, 1, 1, 1, 1} is generated, with record 1 and a length of 8. Because each unit represents 4k, it means that this 32k data is only referenced by snapshot 1. As shown Figure 3 As shown in the figure, the volume storage view and Chunk1 view show the situation when data B overwrites the 0-32K data block of data A.

[0077] This embodiment effectively manages snapshot data in storage volumes. When initial data is written, the first snapshot fully protects the initial state of the storage volume. During data updates, the protected snapshot data is precisely located and written to the storage unit, while a snapshot reference table is generated for rapid data validity verification. This precise snapshot data management and verification method not only improves snapshot GC efficiency but also optimizes the I / O performance of the all-flash storage system, reducing CPU and memory resource consumption, thereby improving the overall performance of the storage system.

[0078] In order to improve the accuracy of generating a snapshot reference table, in the snapshot data validity verification method provided in Example 1 of the present application, the snapshot identifier of the first snapshot is determined; based on the snapshot identifier and the length of the first new data, the first reference sequence is determined; and the first reference sequence is written to the snapshot reference table.

[0079] In an embodiment of the present invention, when a snapshot is created, the storage system assigns a unique snapshot ID to represent the snapshot. This ID is used to track and identify the snapshot in snapshot lists and other metadata. After determining the snapshot ID of the first snapshot, a first reference sequence can be generated based on the length of the new data. Assuming that the first new data overwrites 32KB of continuous data starting from the starting offset on the storage volume, a reference sequence with a length of 8 will be generated in the snapshot reference table (because each unit represents 4KB, and 32KB of data corresponds to 8 units). This reference sequence includes the snapshot ID of the first snapshot, indicating that the state of these data blocks before being overwritten was protected by the first snapshot. If the data overwritten by the same write operation is referenced by multiple snapshots, the reference sequence will include the minimum of these snapshot IDs to reflect the snapshot dependency of the data blocks. The generated first reference sequence is then written to the snapshot reference table.

[0080] In this embodiment, when the first new data is written to the storage volume and overwrites the first data block, the snapshot dependency can be recorded. The snapshot reference table is a data structure that records which snapshots reference each data block (4KB) within a chunk. By adding the first reference sequence to the snapshot reference table, the storage system can quickly identify which data blocks are invalid and which are still valid during snapshot garbage collection. This effectively avoids a large number of random I / O operations, speeds up snapshot data validity verification, and reduces CPU and memory resource consumption.

[0081] In this embodiment, by generating and updating the snapshot reference table in real time during data block overwrite operations, the snapshot data lifecycle is effectively managed. During snapshot garbage collection, data block validity can be determined directly from the snapshot reference table and the in-memory snapshot list, without querying index metadata. This reduces verification time. Furthermore, because the reference table is relatively fixed and small (approximately 8KB), it can be easily loaded into memory, avoiding frequent disk I / O overhead and further optimizing storage system performance. This not only improves snapshot management efficiency but also enhances the stability and responsiveness of all-flash storage systems in large-scale snapshot scenarios.

[0082] In order to improve the accuracy of determining the first reference sequence, in the snapshot data validity verification method provided in Example 1 of the present application, the reference length of the reference sequence is determined based on the length of the first new data and the preset length unit; the reference value at each length in the reference sequence is determined based on the snapshot identifier; and the first reference sequence is determined based on the reference length and the reference value at each length.

[0083] In an embodiment of the present invention, when a client performs an overwrite operation on a storage volume, writing the first new data, the reference length of the reference sequence can be determined based on the actual length of the new data and a preset length unit (e.g., 4KB). For example, if the first new data is 32KB long, then based on the 4KB unit, the reference sequence length should be 8 units (32KB / 4KB = 8). This reference length refers to the number of elements that need to be updated or written in the snapshot reference table, ensuring that the snapshot dependencies of each data block within the coverage area of the new data are accurately recorded. The reference value of each position in the reference sequence is then determined based on the snapshot identifier. When the overwrite operation occurs, it is possible to check which snapshots reference the pre-overwrite data, i.e., the first data block. Assuming that the pre-overwrite data is referenced only by the first snapshot, the reference value at each position in the reference sequence will be set to the identifier of the first snapshot. If the same data block is referenced by multiple snapshots, the reference value will be set to the minimum of all relevant snapshot identifiers. This ensures that the snapshot reference table accurately reflects the snapshot dependency status of each data block within the storage unit (chunk), providing critical information for data validity verification during snapshot garbage collection.

[0084] Afterwards, the first reference sequence is constructed based on the reference length and the determined reference value. The reference sequence consists of the reference length and the reference value at each position. It is a data structure that represents the snapshot dependency of the overwritten data block. For example, if the reference length is 8 and the reference value at all positions is the identifier 1 of the first snapshot, then the first reference sequence will be {1, 1, 1, 1, 1, 1, 1}. This reference sequence will be written to the snapshot reference table on the chunk where the protected snapshot data is located, forming a direct association with the written data block for subsequent snapshot data validity verification.

[0085] In this embodiment, by dynamically generating a reference sequence and writing it into a snapshot reference table, the current problems of low efficiency and high resource consumption in snapshot garbage collection are resolved. After a data block is overwritten, its snapshot dependency status can be immediately updated without the need for large-scale metadata queries during snapshot garbage collection. This enables fast and accurate data block validity verification, eliminates a large number of random I / O operations, reduces CPU and memory overhead, and optimizes the performance of all-flash storage systems. In actual deployments, this reference sequence generation and update method ensures efficient snapshot data management, especially when the storage system faces frequent write operations and large-scale snapshot scenarios, improving system processing capabilities and maintaining low latency and high throughput.

[0086] In order to accurately record the reference relationship in the snapshot reference table when new data continues to be written to the storage volume, in the snapshot data validity verification method provided in the first embodiment of the present application, after generating the snapshot reference table, when the second data block of the current data of the storage volume is overwritten by the second new data, a second snapshot is created, wherein the second snapshot is used to protect the current data of the storage volume, and the current data includes: the first new data and the remaining initial data after being overwritten by the first new data; determining whether the second data block is a data block in the remaining initial data; when the second data block is a data block in the remaining initial data, determining the second data block protected by both the first snapshot and the second snapshot, and writing the second data block to the storage unit; determining a second reference sequence based on the snapshot identifier of the first snapshot, the snapshot identifier of the second snapshot, and the length of the second new data, wherein the minimum snapshot identifier between the snapshot identifier of the first snapshot and the snapshot identifier of the second snapshot is determined as the reference value on the second reference sequence; and writing the second reference sequence to the snapshot reference table.

[0087] In this embodiment of the present invention, after a storage volume is overwritten by first new data, if the client continues to overwrite the volume, such as writing second new data to a second data block, a second snapshot is automatically created. This second snapshot not only protects the current state of the volume, including the portion overwritten by the first new data, but also the remaining original data, i.e., the data areas that have not been updated. By creating this second snapshot, the storage system can maintain the historical state of the volume, facilitating data recovery or analysis when needed.

[0088] In an embodiment of the present invention, when a second new data overwrite operation occurs, it can be determined whether the second data block belongs to the remaining initial data. If the second data block belongs to the portion of initial data remaining after the first overwrite operation, this means that the data block before the overwrite operation has not been overwritten by any snapshot. Therefore, the snapshot dependency status of this data block requires special attention to ensure that subsequent snapshot data validity checks can accurately reflect its historical status. If the second data block is determined to be a data block in the remaining initial data and has been overwritten, it will be determined that this data block is protected by both the first snapshot and the second snapshot. This means that before the second new data is written, the snapshot dependency of the second data block not only includes the first snapshot, but now also extends to the second snapshot. In order to protect the original data of these two snapshots, the second data block before the overwrite can be written to the storage unit (Chunk), and the snapshot reference table can be updated to reflect the latest snapshot dependency.

[0089] In an embodiment of the present invention, after determining that the second data block is protected by both the first and second snapshots, a second reference sequence can be generated based on the snapshot identifiers and the actual length of the second new data. Because the data block is protected by both snapshots, the reference value in the second reference sequence will be determined to be the minimum of the two snapshot identifiers (i.e., the snapshot identifier created first). This simplifies the data structure in the snapshot reference table, reduces redundant information, and ensures that the snapshot dependencies of the data block are correctly recorded. For example, if the second new data is 32KB long, the second reference sequence will contain 8 units (each representing 4KB), and the reference value of each unit will be the minimum of the two snapshot identifiers. The second reference sequence is then written to the snapshot reference table of the storage unit, directly associating it with the location of the overwrite operation of the second new data. By adding the latest snapshot dependency information to the snapshot reference table, subsequent snapshot data validity checks can directly use the snapshot reference table to determine which data blocks are still valid and which are invalid, without the need for additional metadata queries. This reduces random I / O operations during snapshot garbage collection, lowers CPU and memory overhead, and optimizes the performance of the all-flash storage system.

[0090] For example, in Figure 3 Based on the above, if the client continues to write 32KB of data at volume offset = 32KB, this generates the protected data for snapshot 2. The storage system processes the data and writes the protected snapshot 2 data to Chunk 1, protecting the snapshot data. The 32KB of data is appended to Chunk 1, and the corresponding snapshot reference table is generated. Record 1 is still required to indicate that this 32KB of data is referenced by both snapshots 1 and 2.

[0091] Figure 4FIG. 1 is a schematic diagram of an optional method of overwriting new data at volume offset = 32K according to an embodiment of the present invention. Figure 4 As shown, in Figure 3 On the basis of , when the client writes 32K data C at the HEAD space offset = 32K of the volume, it creates snapshot 2 and overwrites C at 32K-64K. After overwriting C, data protected by snapshot 1 and snapshot 2 will be generated, namely snapshot2 (A referenced by snapshots 1 and 2). The storage processes the data and writes the data protected by snapshot 1 and snapshot 2 (A referenced by snapshots 1 and 2) into Chunk1 to protect the snapshot data. Because the size of the generated snapshot data is 32K, 32K data is written to Chunk1. At the same time, the corresponding Chunk1 snapshot reference table {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1} is generated, and record 1 is recorded. This is because the minimum identifier of snapshot 1 and snapshot 2 is used. For example Figure 4 As shown in the figure, the volume storage view and Chunk1 view show the situation when data C overwrites the 32K-64K data blocks of data A.

[0092] For example, in Figure 4 Based on this, if the client continues to write 32KB of data at volume offset = 64KB, protection data for snapshot 3 is generated. The storage system processes the data and appends the protected snapshot 3 data to Chunk 1, protecting the snapshot data. The 32KB of data is appended to Chunk 1, and the corresponding snapshot reference table is generated. Record 1 is still required, indicating that this 32KB of data is referenced by snapshots 1, 2, and 3. Snapshot data is now generated for the entire volume range 0-96KB. Data 0-32KB is referenced by snapshot 1, data 32K-64KB is referenced by snapshots 1 and 2, and data 64K-96KB is referenced by snapshots 1, 2, and 3. This 96KB of data is stored in the same chunk, and the reference relationship between snapshot data is described by the same snapshot reference table. If the volume range 0-32KB is overwritten, that is, the client writes 32KB of data at volume offset = 0, protection data for snapshot 3 is generated. The storage processes the data and continues to append Chunk1 to the protected snapshot 3 data. At the same time, the corresponding snapshot reference table is generated and record 2 is required, indicating that this 32KB data is referenced by snapshots 2 and 3. Because snapshot 1 has already generated snapshot data in the 0-32KB range of the volume, this data is only referenced by snapshots 2 and 3.

[0093] Figure 5 FIG. 1 is a schematic diagram of optionally overwriting new data at volume offset = 64k and = 0 according to an embodiment of the present invention. Figure 5 As shown, in Figure 4Based on this, the client simultaneously writes 32KB of data D and data E at offsets = 32KB and 0, respectively, to the volume's HEAD space. Snapshot 3 is created, overwriting D between 64KB and 96KB, and overwriting data5 (E) between 0KB and 32KB. Overwriting D creates data protected by snapshots 1, 2, and 3, namely snapshot 3 (A referenced by snapshots 1, 2, and 3). Overwriting E creates data protected by snapshots 2 and 3, namely snapshot 3 (B referenced by snapshots 2 and 3). The storage processes the data and writes the data protected by snapshots 1, 2, and 3 (A referenced by snapshots 1, 2, and 3) and the data protected by snapshots 2 and 3 (B referenced by snapshots 2 and 3) into Chunk 1. At the same time, it generates the corresponding Chunk 1 snapshot reference table {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, …} and adds the snapshot reference table to the tail of Chunk 1 (tail meta). Figure 5 As shown, the volume storage view and Chunk1 view show the situation when data D covers the 32K-64K data blocks of data A and data E covers the 0K-32K data blocks of data B.

[0094] In this embodiment, by dynamically updating the snapshot reference table during data block overwrite operations, the efficiency and accuracy of snapshot data management are effectively improved. During the snapshot garbage collection process, data blocks no longer referenced by any active snapshots can be quickly located and reclaimed, while ensuring that data blocks referenced by active snapshots are retained. This is suitable for large-scale snapshots and frequent data overwrite scenarios. It not only reduces the time to detect invalid data, but also avoids the resource consumption of metadata queries, thereby improving the throughput and responsiveness of the all-flash storage system and meeting the low latency and high IOPS requirements of high-performance storage systems.

[0095] In order to facilitate the verification of snapshot data validity, in the method for verifying the validity of snapshot data provided in Example 1 of the present application, after the second data block is written to the storage unit, if the storage unit is full, it is determined that the writing of the snapshot reference table is completed; the snapshot reference table that has been written is compressed, and the compressed snapshot reference table is added to the end of the storage unit; for each data block on the storage unit, all snapshot identifiers of the protection data block are written to the metadata index, and the metadata index is added to the end of the storage unit.

[0096] In this embodiment of the present invention, as data is repeatedly written to a storage volume, the data in a chunk continues to grow. When a chunk is filled with snapshot data, a complete snapshot reference table is generated. This table is compressed and appended to the metadata at the end of the chunk. This reference table is generated for all subsequent snapshot data writes. Therefore, the snapshot reference table has a one-to-one correspondence with each chunk and is stored in the metadata at the end of the chunk. It is approximately 8KB in length.

[0097] Figure 6 is a schematic diagram of an optional Chunk1 view according to an embodiment of the present invention, such as Figure 6 The figure shows various snapshot data written in Chunk 1. For example, A referenced by snapshot 1 is stored in 0-32K, A referenced by snapshots 1 and 2 is stored in 32K-64K, A referenced by snapshots 1, 2, and 3 is stored in 64K-96K, and B referenced by snapshots 2 and 3 is stored in 96K-128K. The Chunk 1 snapshot reference table {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, …} is generated accordingly, and the Chunk 1 snapshot reference table is compressed and written to the tail of the chunk (tail meta).

[0098] Specifically, when a storage cell is fully written, this means that every data block (4KB) within the storage cell has been written to the snapshot data. At this point, the storage cell's snapshot reference table can be checked to confirm whether the snapshot dependencies for each data block have been fully recorded. The snapshot reference table is a data structure that records which snapshots protect each data block within the storage cell. When the storage cell reaches its storage capacity limit (64MB), the snapshot reference table is marked as "written complete," indicating that all relevant snapshot dependency information has been collected. If the snapshot reference table is confirmed to be fully written, it can be compressed. Compressing the snapshot reference table saves storage space and reduces the size of the storage cell's tail metadata. The compression mechanism merges or encodes duplicate snapshot identification information based on the snapshot dependencies of the data blocks within the storage cell, reducing redundancy. The compressed snapshot reference table is stored in the storage cell's tail metadata, facilitating subsequent data management and fast access during snapshot garbage collection. This ensures efficient utilization of the storage cell while reducing the storage system's metadata management overhead.

[0099] In addition, the metadata index can be updated based on the reference relationship of the snapshot data. The metadata index is a data structure used to query the mapping from the logical address to the physical address of the data block. For each data block on the storage unit, all snapshot identification information that protects the data block can be recorded and the metadata index can be updated. The updated metadata index will be appended to the end of the storage unit and stored together with the snapshot reference table. Although the snapshot reference table is mainly used to determine the validity of the data block during the snapshot data validity verification process, if the identifier of the deleted snapshot is greater than the identifier recorded in the snapshot reference table, the metadata index can be used to query other identifiers referenced by the data block to accurately determine the validity of the data block.

[0100] In this embodiment, by marking the snapshot reference table upon completion of a storage unit write, and then compressing and storing this reference table, the size of the tail metadata can be reduced, optimizing storage space utilization. Updating and appending the metadata index to the tail of the storage unit ensures that the snapshot dependency information of the data block is fully recorded, providing support for possible data queries.

[0101] In order to improve the accuracy of verifying the snapshot data on the target storage unit, in the method for verifying the validity of snapshot data provided in Example 1 of the present application, a reference sequence corresponding to the snapshot data is queried from a snapshot reference table; based on the reference sequence, a target snapshot identifier indicated by the minimum snapshot identifier protecting the snapshot data is determined; when the target snapshot identifier belongs to a deleted snapshot identifier set and there are no remaining snapshot identifiers that do not belong to the deleted snapshot identifier set, it is determined that the snapshot data is in an invalid state, wherein the remaining snapshot identifiers are snapshot identifiers other than the target snapshot identifier that are determined by querying the metadata index to protect the snapshot data.

[0102] In an embodiment of the present invention, when a snapshot GC process begins, a snapshot reference table can be loaded from the metadata at the end of a storage unit (chunk). The snapshot reference table contains snapshot dependency information for each data block (4KB) in the storage unit, specifically which snapshots reference these data blocks. Based on the logical location of the snapshot data, the corresponding reference sequence in the snapshot reference table can be queried. The reference sequence is a data structure that records all snapshot dependencies for a specific data block, allowing for quick identification of which snapshots reference the specific data block. After retrieving the reference sequence, the minimum snapshot identifier protecting the snapshot data, namely the "target snapshot identifier," can be determined. This identifier indicates the snapshot that initially referenced the snapshot data, i.e., the earliest snapshot created to protect the data block. By determining the target snapshot identifier, the protection status of the snapshot data can be effectively determined based on the chronological order of the snapshot creation. The target snapshot identifier is then checked to see if it belongs to a predefined "deleted snapshot identifier set." The deleted snapshot identifier set contains the identifiers of deleted snapshots. If the target snapshot identifier belongs to this set, and a metadata index query reveals no remaining snapshot identifiers that are not in the deleted snapshot identifier set (i.e., snapshot identifiers greater than the target snapshot identifier that also protect the snapshot data), the snapshot data can be determined to be invalid. Here, remaining snapshot identifiers refer to those snapshot identifiers that protect snapshot data but have not yet been deleted. This determination process, through the combined use of snapshot reference tables and metadata indexes, can quickly and accurately identify which data can be safely recycled without requiring expensive full metadata queries.

[0103] For example, when a snapshot is deleted, the data referenced by the snapshot becomes invalid data. The snapshot GC verifies the invalid data in the chunk, moves the valid data (i.e., copies the valid data to the new chunk), and deletes the old chunk.

[0104] Figure 7 is a schematic diagram of an optional Chunk1 structure according to an embodiment of the present invention, such as Figure 7As shown in the figure, the Chunk1 view is displayed. For example, snapshot data (A protected by snap1 (snapshot 1)) is stored on 0-32K, snapshot data (A protected by snap2 (snapshot 1, 2)) is stored on 32K-64K, snapshot data (A protected by snap3 (snapshot 1, 2, 3)) is stored on 64K-96K, and snapshot data (B protected by snap3 (snapshot 2, 3)) is stored on 96K-128K. The Chunk1 snapshot reference table {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, …} is generated accordingly. The Chunk1 snapshot reference table is compressed and written to the tail of the chunk (tail meta). Among them, A protected by snap1 (snapshot 1) corresponds to 1, 1, 1, 1, 1, 1, 1 on the snapshot reference table; A protected by snap2 (snapshot 1, 2) corresponds to 1, 1, 1, 1, 1, 1, 1 on the snapshot reference table; A protected by snap3 (snapshot 1, 2, 3) corresponds to 1, 1, 1, 1, 1, 1, 1 on the snapshot reference table; B protected by snap3 (snapshot 2, 3) corresponds to 2, 2, 2, 2, 2, 2, 2 on the snapshot reference table.

[0105] exist Figure 7 Based on this, if snapshot 1 and snapshot 2 are deleted, the snapshot GC selects Chunk 1 for garbage collection and loads the corresponding extents (metadata index) and snapshot reference table through the metadata at the end of Chunk 1.

[0106] like Figure 7 As shown, the first data in Chunk1 corresponds to the data of snapshot 1 in the volume logical range 0-32k. It can be confirmed through the snapshot reference table that this data is only referenced by snapshot 1. Because snapshot 1 has been deleted, the data is invalid.

[0107] The second data entry in Chunk1 corresponds to the data of snapshot 2 in the volume logical range 32k-64k. The snapshot reference table confirms that this data entry is referenced by both snapshot 1 and snapshot 2. However, since both snapshot 1 and snapshot 2 have been deleted, the data entry is invalid.

[0108] The third data entry in Chunk1 corresponds to the data of snapshot 3 in the volume logical range 64k-96k. The snapshot reference table confirms that this data entry is referenced by snapshots 1, 2, and 3. Although snapshots 1 and 2 have been deleted, snapshot 3 still references this data entry, so the data is valid.

[0109] The fourth data entry in Chunk1 corresponds to the data of snapshot 3 in the volume logical range 0k-32k. The snapshot reference table confirms that this data entry is referenced by snapshots 2 and 3. Even though snapshot 2 has been deleted, snapshot 3 still exists and references the data, so the data is valid.

[0110] Therefore, the snapshot GC can move the third and fourth data items and then delete Chunk 1, freeing up storage space. This way, through the aforementioned fast verification method, each data item only needs to be compared with the snapshot reference table to determine its validity, avoiding the need to query index metadata, effectively improving performance and verification efficiency.

[0111] Figure 8 is a schematic diagram of an optional Chunk2 structure according to an embodiment of the present invention, such as Figure 8 As shown, the Chunk2 view is displayed by Figure 7 The two data of snap3 of Chunk1 are moved to the new chunk (Chunk2), and a new reference table is generated at the same time (Chunk2 snapshot reference table {1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, …}). Then Chunk1 is deleted to obtain Chunk2, in which Chunk2 contains two data: snap3A on 0-32K and snap3B on 32K-64K.

[0112] In this embodiment, by querying the reference sequence from the snapshot reference table and determining the validity of the snapshot data based on the reference sequence and metadata index, the efficiency of the snapshot GC is improved, the full metadata scan is avoided, random I / O operations are reduced, and the burden on the CPU and memory is reduced.

[0113] In an embodiment of the present invention, through the design of a snapshot reference table, the snapshot dependency of each 4KB block can be recorded at the chunk granularity and stored in the metadata at the end of the chunk. This eliminates the need to query metadata on a large scale during snapshot GC, shortening the verification range from scanning a 64MB logical range to loading approximately 8KB of data. Furthermore, through the generation and compression of the reference table, the storage space table is compressed based on the snapshot's continuity ID, thereby optimizing storage efficiency. In addition, by judging data validity through the snapshot reference table of the metadata at the end of the chunk, there is no need to perform a large-scale metadata query, thus achieving rapid verification of snapshot GC. This is applicable to all scenarios requiring verification of snapshot data validity, such as GC, data migration, replication, etc., and is not limited to all-flash storage.

[0114] The following describes it in detail with reference to another embodiment.

[0115] Example 2

[0116] The snapshot data validity verification device provided in this embodiment includes multiple implementation units, each implementation unit corresponding to each implementation step in the above-mentioned embodiment 1.

[0117] Figure 9 FIG. 1 is a schematic diagram of an optional device for verifying the validity of snapshot data according to an embodiment of the present invention. Figure 9 As shown, the verification device may include: a first determination unit 90 , a second determination unit 91 , a verification unit 92 , and a recovery unit 93 .

[0118] The first determining unit 90 is configured to determine a deleted snapshot identifier set in a deleted state based on the snapshot list;

[0119] A second determining unit 91 is configured to determine a target storage unit for each snapshot deletion identifier based on the snapshot deletion identifier set, wherein the target storage unit stores snapshot data protected by the snapshot;

[0120] Verification unit 92 is configured to verify the snapshot data on each target storage unit based on a snapshot reference table and a metadata index carried by the target storage unit, and obtain a verification result, wherein the snapshot reference table is a reference table generated when the snapshot is created, and the metadata index records the snapshot identifier referenced by each snapshot data;

[0121] The recycling unit 93 is configured to recycle the invalid snapshot data when the verification result indicates that the invalid snapshot data exists on the target storage unit.

[0122] The above-mentioned verification device can pre-create a corresponding snapshot reference table for each storage unit, determine the identification set of deleted snapshots based on the snapshot list, and then locate the target storage unit (Chunk) storing the snapshot data. By parsing the snapshot reference table at the end of the Chunk and combining it with the metadata index, it can efficiently determine whether the snapshot data is in an invalid state. If the data is invalid, garbage collection can be performed, reducing random I / O operations, improving verification efficiency, and achieving the purpose of quickly verifying the validity of snapshot data, thereby realizing the technical effect of improving snapshot GC efficiency.

[0123] Optionally, the verification device also includes: a first creation unit, used to create a first snapshot for each storage volume before determining a set of deleted snapshot identifiers in a deleted state based on the snapshot list, when the first data block of the initial data of the storage volume is overwritten by the first new data, wherein the first snapshot is used to protect the initial data of the storage volume, and the first data block is any data block on the initial data; a first generation unit, used to write the first data block protected by the first snapshot into the storage unit, and generate a snapshot reference table, wherein there is a correspondence between the snapshot reference table and the storage unit.

[0124] Optionally, the first generating unit includes: a first determining module for determining a snapshot identifier of the first snapshot; a second determining module for determining a first reference sequence based on the snapshot identifier and the length of the first new data; and a first writing module for writing the first reference sequence into a snapshot reference table.

[0125] Optionally, the second determination module includes: a first determination submodule, used to determine the reference length of the reference sequence based on the length of the first new data and a preset length unit; a second determination submodule, used to determine the reference value at each length on the reference sequence based on the snapshot identifier; and a third determination submodule, used to determine the first reference sequence based on the reference length and the reference value at each length.

[0126] Optionally, the verification device further includes: a second creation unit for creating a second snapshot after generating the snapshot reference table, when the second data block of the current data of the storage volume is overwritten by the second new data, wherein the second snapshot is used to protect the current data of the storage volume, and the current data includes: the first new data and the remaining initial data after being overwritten by the first new data; a third determination unit for determining whether the second data block is a data block in the remaining initial data; a fourth determination unit for determining, when the second data block is a data block in the remaining initial data, the second data block protected by both the first snapshot and the second snapshot, and writing the second data block into the storage unit; a fifth determination unit for determining a second reference sequence based on the snapshot identifier of the first snapshot, the snapshot identifier of the second snapshot, and the length of the second new data, wherein the smallest snapshot identifier between the snapshot identifier of the first snapshot and the snapshot identifier of the second snapshot is determined as the reference value on the second reference sequence; and a first writing unit for writing the second reference sequence into the snapshot reference table.

[0127] Optionally, the verification device also includes: a sixth determination unit, used to determine that the writing of the snapshot reference table is completed after the second data block is written into the storage unit when the storage unit is full; a first compression unit, used for the second writing unit, used to compress the snapshot reference table that has been written, and add the compressed snapshot reference table to the end of the storage unit; a third writing unit, used to write all snapshot identifiers of the protection data block to the metadata index for each data block on the storage unit, and add the metadata index to the end of the storage unit.

[0128] Optionally, the verification unit includes: a first query module, used to query a reference sequence corresponding to the snapshot data from a snapshot reference table; a third determination module, used to determine, based on the reference sequence, a target snapshot identifier indicated by a minimum snapshot identifier that protects the snapshot data; and a fourth determination module, used to determine that the snapshot data is in an invalid state when the target snapshot identifier belongs to a deleted snapshot identifier set and there are no remaining snapshot identifiers that do not belong to the deleted snapshot identifier set, wherein the remaining snapshot identifiers are snapshot identifiers other than the target snapshot identifier that protect the snapshot data, determined by querying the metadata index.

[0129] The above-mentioned verification device can also include a processor and a memory. The above-mentioned first determination unit 90, second determination unit 91, verification unit 92, recovery unit 93, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0130] The processor includes a kernel that retrieves the corresponding program unit from the memory. One or more kernels may be configured to reclaim invalid snapshot data by adjusting kernel parameters when a check result indicates that invalid snapshot data exists on the target storage unit.

[0131] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0132] The present invention also provides a computer program product which, when executed on a data processing device, is suitable for executing a program initialized with the following method steps: based on a snapshot list, determining a set of deleted snapshot identifiers in a deleted state; based on the set of deleted snapshot identifiers, determining a target storage unit for each deleted snapshot identifier; for each target storage unit, verifying the snapshot data on the target storage unit based on a snapshot reference table and a metadata index carried by the target storage unit to obtain a verification result; and if the verification result indicates that there is snapshot data in an invalid state on the target storage unit, reclaiming the invalid snapshot data.

[0133] According to another aspect of an embodiment of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and the computer program, when executed by a processor, implements any of the above-mentioned methods for verifying the validity of snapshot data.

[0134] According to another aspect of an embodiment of the present invention, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the above-mentioned method for verifying the validity of snapshot data.

[0135] Figure 10 FIG. 1 is a hardware structure block diagram of an electronic device (or mobile device) for verifying the validity of snapshot data according to an embodiment of the present invention. Figure 10 As shown, the electronic device may include one or more processors (e.g., Figure 10 The processors 1002a, 1002b, ..., 1002n, etc., which may include but are not limited to processing devices such as microprocessors (MCUs) or programmable logic devices (FPGAs), and a memory 1004 for storing data. In addition, the system may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. It will be understood by those skilled in the art that Figure 10 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 10 More or fewer components than shown, or with Figure 10 Different configurations shown.

[0136] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0137] The embodiments or examples of the present disclosure are not exhaustive, but are merely illustrations of some embodiments or examples, and are not intended to be specific limitations on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment or example can be implemented as an independent example, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain embodiment or example can be arbitrarily combined; in addition, the various embodiments or examples can be arbitrarily combined. For example, some or all steps of different embodiments or examples can be arbitrarily combined, and a certain embodiment or example can be arbitrarily combined with the optional methods or optional examples of other embodiments or examples.

[0138] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0139] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, and can be electrical or other forms.

[0140] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0141] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0142] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk.

[0143] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for verifying the validity of snapshot data, characterized in that: include: Based on the snapshot list, determining a set of deleted snapshot identifiers in a deleted state; Determining a target storage unit for each snapshot deletion identifier based on the snapshot deletion identifier set, wherein the target storage unit stores snapshot data protected by the snapshot; For each target storage unit, verifying the snapshot data on the target storage unit based on a snapshot reference table and a metadata index carried by the target storage unit to obtain a verification result, wherein the snapshot reference table is a reference table generated when a snapshot is created, and the metadata index records a snapshot identifier referenced by each snapshot data; When the verification result indicates that the snapshot data in an invalid state exists on the target storage unit, the snapshot data in the invalid state is recycled.

2. The verification method according to claim 1, wherein: Before determining the deleted snapshot identifier set in the deleted state based on the snapshot list, the following steps are also included: For each storage volume, when a first data block of initial data of the storage volume is overwritten by first new data, creating a first snapshot, wherein the first snapshot is used to protect the initial data of the storage volume, and the first data block is any data block on the initial data; The first data block protected by the first snapshot is written into a storage unit, and the snapshot reference table is generated, wherein the snapshot reference table corresponds to the storage unit.

3. The verification method according to claim 2, wherein: The step of generating the snapshot reference table includes: determining a snapshot identifier of the first snapshot; Determining a first reference sequence based on the snapshot identifier and the length of the first new data; The first reference sequence is written to the snapshot reference table.

4. The verification method according to claim 3, wherein: The step of determining a first reference sequence based on the snapshot identifier and the length of the first new data includes: Determining a reference length of the reference sequence based on the length of the first new data and a preset length unit; Determining a reference value for each length of the reference sequence based on the snapshot identifier; The first reference sequence is determined based on the reference length and the reference value at each of the lengths.

5. The verification method according to claim 2, wherein: After generating the snapshot reference table, the following steps are further included: creating a second snapshot when a second data block of the current data of the storage volume is overwritten by second new data, wherein the second snapshot is used to protect the current data of the storage volume, the current data including: the first new data and remaining initial data after being overwritten by the first new data; determining whether the second data block is a data block in the remaining initial data; In a case where the second data block is a data block in the remaining initial data, determining the second data block protected by both the first snapshot and the second snapshot, and writing the second data block into the storage unit; determining a second reference sequence based on the snapshot identifier of the first snapshot, the snapshot identifier of the second snapshot, and the length of the second new data, wherein the smallest snapshot identifier between the snapshot identifier of the first snapshot and the snapshot identifier of the second snapshot is determined as a reference value on the second reference sequence; The second reference sequence is written to the snapshot reference table.

6. The verification method according to claim 5, characterized in that: After writing the second data block into the storage unit, the method further includes: When the storage unit is full, determining that writing of the snapshot reference table is completed; compressing the snapshot reference table after writing, and adding the compressed snapshot reference table to the end of the storage unit; For each data block on the storage unit, all the snapshot identifiers protecting the data block are written into the metadata index, and the metadata index is added to the end of the storage unit.

7. The verification method according to claim 1, wherein: The step of verifying the snapshot data on the target storage unit includes: Querying the reference sequence corresponding to the snapshot data from the snapshot reference table; Determining, based on the reference sequence, a target snapshot identifier indicated by a minimum snapshot identifier that protects the snapshot data; When the target snapshot identifier belongs to the deleted snapshot identifier set and there are no remaining snapshot identifiers that do not belong to the deleted snapshot identifier set, it is determined that the snapshot data is in the invalid state, wherein the remaining snapshot identifiers are snapshot identifiers other than the target snapshot identifier that protect the snapshot data, determined by querying the metadata index.

8. A device for verifying the validity of snapshot data, characterized in that: include: A first determining unit is configured to determine a deleted snapshot identifier set in a deleted state based on the snapshot list; a second determining unit, configured to determine a target storage unit for each snapshot deletion identifier based on the snapshot deletion identifier set, wherein the target storage unit stores snapshot data protected by the snapshot; a verification unit configured to verify the snapshot data on each target storage unit based on a snapshot reference table and a metadata index carried by the target storage unit, to obtain a verification result, wherein the snapshot reference table is a reference table generated when a snapshot is created, and the metadata index records a snapshot identifier referenced by each snapshot data; A recycling unit is configured to recycle the snapshot data in an invalid state when the verification result indicates that the snapshot data exists in an invalid state on the target storage unit.

9. A computer program product, characterized in that The method comprises a non-volatile computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method for verifying the validity of snapshot data according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The device comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for verifying the validity of snapshot data as described in any one of claims 1 to 7.

Images