Method and apparatus for checking validity of snapshot data, electronic device, and program product

Abstract

The application discloses a snapshot data validity checking method and device, electronic equipment and program product, and relates to the technical field of snapshots, and the checking method comprises the following steps: determining a set of deleted snapshot identifiers in a deleted state based on a snapshot list; determining a target storage unit of each deleted snapshot identifier based on the set of deleted snapshot identifiers; for each target storage unit, checking snapshot data on the target storage unit based on a snapshot reference table and metadata index carried by the target storage unit to obtain a checking result; and in the case where the checking result indicates that there is snapshot data in an invalid state on the target storage unit, recycling the snapshot data in the invalid state. The application solves the technical problem of low efficiency in checking snapshot data in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of snapshots, in particular to a snapshot data validity checking method and device, an electronic device and a program product. BACKGROUND

[0002] In a full flash block storage system, snapshots are a mechanism for saving the data state of a volume at a certain time point, which can be applied to data backup, recovery and other scenarios. Snapshots use a write-once copy mechanism, and snapshot data is generated by an overwrite operation. After a snapshot is deleted, a large amount of snapshot data may become invalid, and the invalid data needs to be identified through snapshot GC (Garbage Collection, i.e., snapshot garbage collection).

[0003] Data is stored in units of chunks (data blocks, such as 64 MB), including header metadata, a data area, and tail metadata (including extent (metadata describing data blocks) and a garbage bitmap). The HEAD space quickly determines data validity through the garbage bitmap, but snapshot GC needs to scan the metadata in full to determine the reference relationship because data may be referenced by multiple snapshots, which involves a large amount of random I / O (input / output), resulting in low efficiency. In particular, in a full flash storage system with high performance requirements, the query overhead in snapshot GC becomes a performance bottleneck.

[0004] In related technologies, in order to perform snapshot GC, a full metadata scanning method is needed to check the validity of snapshot data, i.e., when performing GC, the snapshot metadata of the entire chunk is scanned, and in combination with a list of deleted snapshots, it is determined whether the data is referenced by other snapshots. If the data is not referenced by any snapshot, it means that the data is invalid and can be recycled. However, this full metadata scanning method has the following problems:

[0005] (1) Low verification efficiency: full metadata scanning needs to scan the entire chunk (involving a 64M logical range), which involves a large amount of random I / O, long verification time, limited GC performance, and affects the high IOPS (Input / Output Operations Per Second) characteristics of the full flash storage system.

[0006] (2) Large resource consumption: frequent queries to the Index (metadata index) result in high memory cache and disk I / O overhead, high CPU (Central Processing Unit) occupancy, and difficulty in meeting the low latency requirements of the full flash storage system.

[0007] (3) Complex reference relationship processing: snapshot data may be referenced by multiple snapshots, and after a snapshot is deleted, the reference relationship needs to be checked one by one, which is inefficient.

[0008] And if the garbage bitmap is used for snapshot GC, although the invalid data can be marked by the garbage bitmap, the Index needs to be queried to confirm the snapshot reference state, which involves a large amount of random I / O, resulting in low efficiency.

[0009] At present, no effective solution has been proposed for the above problems. SUMMARY

[0010] Embodiments of the present application provide a snapshot data validity verification method and device, electronic equipment and program product, to at least solve the technical problem of low efficiency in verifying snapshot data in related technologies.

[0011] According to an aspect of an embodiment of the present application, a snapshot data validity verification method is provided, comprising: determining a set of deleted snapshot identifiers in a deleted state based on a snapshot list; determining a target storage unit of each deleted snapshot identifier based on the set of deleted snapshot identifiers, wherein the target storage unit stores snapshot data protected by the snapshot; for each target storage unit, verifying the snapshot data on the target storage unit based on a snapshot reference table carried by the target storage unit and a metadata index, to obtain a verification result, wherein the snapshot reference table is a reference table generated when the snapshot is created, and the metadata index records snapshot identifiers referenced by each snapshot data; and in the case where the verification result indicates that there is snapshot data in an invalid state on the target storage unit, recycling the snapshot data in the invalid state.

[0012] Further, before determining the set of deleted snapshot identifiers in the deleted state based on the snapshot list, it further comprises: for each storage volume, creating a first snapshot in the case where a first data block of initial data of the storage volume is covered by a first new data, wherein the first snapshot is used to protect the initial data of the storage volume, and the first data block is any data block on the initial data; writing the first data block protected by the first snapshot to a storage unit, and generating a snapshot reference table, wherein the snapshot reference table has a corresponding relationship with the storage unit.

[0013] Further, the step of generating the snapshot reference table comprises: determining a snapshot identifier of the first snapshot; determining a first reference sequence based on the snapshot identifier and a length of the first new data; and writing the first reference sequence to the snapshot reference table.

[0014] Further, the step of determining the first reference sequence based on the snapshot identifier and the length of the first new data comprises: determining a reference length of the reference sequence based on the length of the first new data and a preset length unit; determining a reference value on each length on the reference sequence based on the snapshot identifier; and determining the first reference sequence based on the reference length and the reference value on each length.

[0015] Further, after generating the snapshot reference table, further comprising: creating a second snapshot in a case that the second data block of the current data of the storage volume is covered by second new data, wherein the second snapshot is used to protect the current data of the storage volume, and the current data comprises the first new data and the remaining initial data covered by the first new data; determining whether the second data block is a data block in the remaining initial data; in a case that the second data block is a data block in the remaining initial data, determining the second data block protected by both the first snapshot and the second snapshot, and writing the second data block into the storage unit; determining a second reference sequence based on a snapshot identifier of the first snapshot, a snapshot identifier of the second snapshot, and a length of the second new data, wherein the minimum snapshot identifier in the snapshot identifier of the first snapshot and the snapshot identifier of the second snapshot is determined as a reference value on the second reference sequence; and writing the second reference sequence into the snapshot reference table.

[0016] Further, after writing the second data block into the storage unit, further comprising: determining that the snapshot reference table is written completely in a case that the storage unit is written full; compressing the snapshot reference table written completely, and adding the compressed snapshot reference table to the tail of the storage unit; and writing all snapshot identifiers of the data block protected by the data block on the storage unit into a metadata index, and adding the metadata index to the tail of the storage unit.

[0017] Further, the step of checking the snapshot data on the target storage unit comprises: querying the reference sequence corresponding to the snapshot data from the snapshot reference table; determining the target snapshot identifier indicated by the minimum snapshot identifier protecting the snapshot data based on the reference sequence; and determining that the snapshot data is in an invalid state in a case that the target snapshot identifier belongs to the deleted snapshot identifier set and there is no remaining snapshot identifier not belonging to the deleted snapshot identifier set, wherein the remaining snapshot identifier is determined by querying the metadata index and is a snapshot identifier protecting the snapshot data except the target snapshot identifier.

[0018] According to another aspect of the embodiments of the present application, a snapshot data validity checking device is further provided, comprising: a first determining unit configured to determine a deleted snapshot identifier set in a deleted state based on a snapshot list; a second determining unit configured to determine a target storage unit of each deleted snapshot identifier based on the deleted snapshot identifier set, wherein the target storage unit stores snapshot data protected by the snapshot; a checking unit configured to check, for each target storage unit, snapshot data on the target storage unit based on a snapshot reference table carried by the target storage unit and a metadata index, and obtain a checking result, wherein the snapshot reference table is a reference table generated when the snapshot is created, and the metadata index records snapshot identifiers referenced by each snapshot data; and a recycling unit configured to recycle snapshot data in an invalid state in a case that the checking result indicates that there is snapshot data in the invalid state on the target storage unit.

[0019] Further, the checking device further comprises: a first creating unit, configured to, before determining the set of deleted snapshot identifiers in the deleted state based on the snapshot list, create a first snapshot for each storage volume in a case that a first data block of initial data of the storage volume is covered by a first new data, wherein the first snapshot is used to protect the initial data of the storage volume, and the first data block is any data block on the initial data; and a first generating unit, configured to write the first data block protected by the first snapshot to a storage unit and generate a snapshot reference table, wherein the snapshot reference table has a corresponding relationship with the storage unit.

[0020] Further, the first generating unit comprises: a first determining module, configured to determine a snapshot identifier of the first snapshot; a second determining module, configured to determine a first reference sequence based on the snapshot identifier and a length of the first new data; and a first writing module, configured to write the first reference sequence to the snapshot reference table.

[0021] Further, the second determining module comprises: a first determining submodule, configured to determine a reference length of the reference sequence based on the length of the first new data and a preset length unit; a second determining submodule, configured to determine a reference value on each length of the reference sequence based on the snapshot identifier; and a third determining submodule, configured to determine the first reference sequence based on the reference length and the reference value on each length.

[0022] Further, the checking device further comprises: a second creating unit, configured to, after generating the snapshot reference table, create a second snapshot in a case that a second data block of current data of the storage volume is covered by a second new data, wherein the second snapshot is used to protect the current data of the storage volume, and the current data comprises the first new data and remaining initial data covered by the first new data; a third determining unit, configured to determine whether the second data block is a data block in the remaining initial data; a fourth determining unit, configured to, in a case that the second data block is a data block in the remaining initial data, determine the second data block protected by both the first snapshot and the second snapshot and write the second data block to the storage unit; a fifth determining unit, configured to determine a second reference sequence based on a snapshot identifier of the first snapshot, a snapshot identifier of the second snapshot and a length of the second new data, wherein a minimum snapshot identifier in the snapshot identifier of the first snapshot and the snapshot identifier of the second snapshot is determined as a reference value on the second reference sequence; and a first writing unit, configured to write the second reference sequence to the snapshot reference table.

[0023] Further, the checking device further comprises: a sixth determining unit, configured to determine that the snapshot reference table is written completely in a case that the storage unit is full after the second data block is written into the storage unit; a first compressing unit, configured to compress the snapshot reference table written completely by the second writing unit, and add the compressed snapshot reference table to the tail of the storage unit; and a third writing unit, configured to write all snapshot identifiers protecting the data block into the metadata index for each data block on the storage unit, and add the metadata index to the tail of the storage unit.

[0024] Further, the checking unit comprises: a first querying module, configured to query the reference sequence corresponding to the snapshot data from the snapshot reference table; a third determining module, configured to determine the target snapshot identifier indicated by the minimum snapshot identifier protecting the snapshot data based on the reference sequence; and a fourth determining module, configured to determine that the snapshot data is in the invalid state in a case that the target snapshot identifier belongs to the set of deleted snapshot identifiers, and there is no remaining snapshot identifier that does not belong to the set of deleted snapshot identifiers, wherein the remaining snapshot identifier is determined by querying the metadata index and is the snapshot identifier protecting the snapshot data except the target snapshot identifier.

[0025] According to another aspect of the embodiments of the present application, a computer program product is also provided, comprising a non-volatile computer readable storage medium, the non-volatile computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the snapshot data validity checking method of any of the above.

[0026] According to another aspect of the embodiments of the present application, an electronic device is also provided, comprising one or more processors and a memory, the memory being configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the snapshot data validity checking method of any of the above.

[0027] In the present application, based on the snapshot list, the set of deleted snapshot identifiers in the deleted state is determined, based on the set of deleted snapshot identifiers, the target storage unit of each deleted snapshot identifier is determined, for each target storage unit, the snapshot data on the target storage unit is checked based on the snapshot reference table and the metadata index carried by the target storage unit, and the checking result is obtained, in a case that the checking result indicates that there is snapshot data in the invalid state on the target storage unit, the snapshot data in the invalid state is recycled, thereby solving the technical problem of low efficiency of checking snapshot data in the related art.

[0028] In the present application, for each storage unit, a corresponding snapshot reference table is created in advance, the identification set of the deleted snapshot can be determined according to the snapshot list, and then the target storage unit (Chunk) storing the snapshot data is located. By analyzing the snapshot reference table at the tail of the Chunk and combining the metadata index, it can be efficiently judged whether the snapshot data is in an invalid state. If the data is invalid, garbage collection can be performed, random I / O operation is reduced, checking efficiency is improved, the purpose of quickly checking the validity of the snapshot data is achieved, and the technical effect of improving the snapshot GC efficiency is realized. BRIEF DESCRIPTION OF DRAWINGS

[0029] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0030] Figure 1 is a flow chart of an optional snapshot data validity checking method according to an embodiment of the present application;

[0031] Figure 2 is a schematic view of an optional volume overwriting data A according to an embodiment of the present application;

[0032] Figure 3 is a schematic view of an optional volume overwriting new data according to an embodiment of the present application;

[0033] Figure 4 is a schematic view of an optional volume overwriting new data at volume offset = 32K according to an embodiment of the present application;

[0034] Figure 5 is a schematic view of an optional volume overwriting new data at volume offset = 64K and = 0 according to an embodiment of the present application;

[0035] Figure 6 is a schematic view of an optional Chunk1 view according to an embodiment of the present application;

[0036] Figure 7 is a schematic view of an optional Chunk1 structure according to an embodiment of the present application;

[0037] Figure 8 is a schematic view of an optional Chunk2 structure according to an embodiment of the present application;

[0038] Figure 9 is a schematic view of an optional snapshot data validity checking device according to an embodiment of the present application;

[0039] Figure 10is a hardware structure block diagram of an electronic device (or mobile device) for a snapshot data validity checking method according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the personnel in the technical field better understand the present application scheme, the technical scheme in the embodiment of the present application will be clearly and completely described below in combination with the drawings in the embodiment of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the scope of protection of the present application.

[0041] It should be noted that the terms "first", "second" and the like in the present application are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0042] In order to facilitate the understanding of the present application by those skilled in the art, the following explains some terms or nouns involved in the embodiments of the present application:

[0043] HEAD: the latest data version currently used by the storage volume.

[0044] Snapshot: used to save snapshot data at a specific time point, that is, a copy of the entire volume data at the moment when the snapshot is created.

[0045] Extent: metadata describing data blocks, used to record the mapping of logical addresses to physical addresses.

[0046] Snapshot Extent: metadata of snapshot data blocks, used to store the logical range of the snapshot and its corresponding physical address.

[0047] Chunk: the basic unit of data storage in the storage system, with a size of 64MB, including header metadata (header), data area and tail metadata (tail meta).

[0048] Index: metadata index in the storage system used to query the mapping of logical addresses to physical addresses, stored in the disk, and the query involves random I / O.

[0049] Snapshot GC: also known as snapshot garbage collection, identifies and deletes data blocks that have become invalid due to snapshot deletion, thereby freeing up storage space.

[0050] It should be noted that all related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) collected and involved in this invention are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and it does not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0051] This invention introduces a snapshot reference table as the core mechanism for validating snapshot data in an all-flash block storage system. It records the snapshot dependencies of each 4KB data block using an approximately 8KB key-value data structure, effectively solving the inefficiency and high resource consumption problems caused by the current snapshot GC requiring the scanning of large amounts of metadata. With limited performance impact, it effectively improves the efficiency of snapshot GC in all-flash block storage systems, ensuring high performance and resource utilization of the storage system in large-scale snapshot scenarios. It solves the following problems:

[0052] (1) Low efficiency of snapshot data validity verification: The current snapshot GC needs to scan 64MB of metadata, which involves a large amount of random I / O and takes a long time. However, the present invention can quickly verify the validity of data in memory by loading an approximately 8KB snapshot reference table, which effectively improves efficiency.

[0053] (2) High resource consumption for metadata query: Currently, frequent index queries lead to high memory caching and disk I / O overhead, making it difficult to meet the high IOPS requirements of all-flash storage. However, this invention reduces I / O overhead by eliminating index queries, and CPU usage is also greatly reduced.

[0054] (3) The dynamic reference relationship of snapshots is complicated: Currently, multiple snapshot reference relationships need to be checked one by one, which is inefficient. However, the present invention can quickly determine the dynamic reference status through the snapshot reference table and the memory snapshot list, thus improving the processing efficiency.

[0055] The present invention will now be described in detail with reference to various embodiments.

[0056] Example 1

[0057] According to an embodiment of the present application, an embodiment of a method for checking validity of snapshot data is provided. It should be noted that the steps shown in the flowcharts of the drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0058] Figure 1 is a flowchart of an optional method for checking validity of snapshot data according to an embodiment of the present application, as shown in Figure 1 The method comprises the following steps:

[0059] In step S101, based on a snapshot list, a set of deleted snapshot identifiers is determined.

[0060] In an embodiment of the present application, in the memory of a storage system, a snapshot list is maintained, which includes information of all created snapshots. When a snapshot is deleted by a user, its state will be marked as "deleted" in the snapshot list. In order to save storage space, all deleted snapshots can be screened from the snapshot list periodically to form a set, i.e., a set of deleted snapshot identifiers, so as to find out snapshot data corresponding to these deleted snapshot identifiers, thereby deleting snapshot data in an invalid state to recover storage space.

[0061] In step S102, based on the set of deleted snapshot identifiers, a target storage unit of each deleted snapshot identifier is determined, wherein the target storage unit stores snapshot data protected by the snapshot.

[0062] In an embodiment of the present application, after the set of deleted snapshot identifiers is determined, the target storage unit (i.e., Chunk) associated with these snapshot identifiers can be located. Because snapshot data is protected when a snapshot is created, one or more snapshot data can be contained in each Chunk.

[0063] In step S103, for each target storage unit, based on a snapshot reference table carried by the target storage unit and a metadata index, snapshot data on the target storage unit is checked to obtain a checking result, wherein the snapshot reference table is a reference table generated when a snapshot is created, and the metadata index records a snapshot identifier referenced by each snapshot data.

[0064] In an embodiment of the present application, for each Chunk (target storage unit), the validity of the snapshot data stored therein can be checked according to the snapshot reference table stored at the tail of the Chunk and the metadata index. Here, the snapshot reference table records information that each 4KB data block in the Chunk is referenced by which snapshot. The metadata index is a tool for finding data in a storage system, which records a snapshot identifier referenced by each snapshot data.

[0065] In the embodiment of the present application, the dependence on the metadata index can be reduced due to the existence of the snapshot reference table. Whether the data in the Chunk is still valid can be determined by comparing the snapshot ID (identifier) in the snapshot reference table with the snapshot ID in the "deleted snapshot identifier set". If a data block only has references to deleted snapshots, the part of the data is marked as invalid. In this way, the validity of the data can be quickly determined through the snapshot reference table, and the resource waste and performance bottleneck caused by full metadata scanning can be avoided.

[0066] In step S104, if the check result indicates that there is snapshot data in the invalid state on the target storage unit, the snapshot data in the invalid state is recycled.

[0067] In the embodiment of the present application, if it is checked that there is snapshot data in the invalid state (i.e., invalid data, which is not referenced by an undeleted snapshot) on the target storage unit, a data recycling mechanism can be triggered to remove the invalid data from the Chunk and release the storage space. In this way, not only the optimization of the storage space is facilitated, but also the overall performance of the storage system is improved, especially when a large-scale snapshot scenario is processed.

[0068] In the present embodiment, by introducing a snapshot reference table in the tail meta of the Chunk, the snapshot ID on which each 4KB data block depends is recorded, and the data validity is quickly determined in combination with the snapshot list in the memory, without the need to query the index metadata. The snapshot reference table is generated when the Chunk is written, and is compressed and stored in the tail meta of the Chunk when it is full. The size of the snapshot reference table is about 8KB. In this way, the I / O, CPU and memory overheads are reduced, and the all-flash storage performance is optimized.

[0069] In the present embodiment, the snapshot reference table is a piece of data recorded when the Chunk data is written, which records in detail which snapshots the Chunk data depends on. These records are combined in the granularity of the Chunk to form a snapshot reference table with a maximum length of 64M / 4k=16384.

[0070] In summary, by pre-creating the corresponding snapshot reference table for each storage unit, the identification set of the deleted snapshots can be determined according to the snapshot list, and then the target storage unit (Chunk) storing the snapshot data is located. By analyzing the snapshot reference table at the tail of the Chunk and combining the metadata index, it can be efficiently judged whether the snapshot data is in an invalid state. If the data is invalid, garbage collection can be performed, reducing random I / O operations, improving the checking efficiency, achieving the purpose of quickly checking the validity of the snapshot data, thereby realizing the technical effect of improving the snapshot GC efficiency, and further solving the technical problem of low efficiency of checking the snapshot data in the related art.

[0071] In order to generate the corresponding snapshot reference table for each storage unit, in the snapshot data validity checking method provided by the embodiment one of the present application, before determining the identification set of the deleted snapshots in the deleted state based on the snapshot list, for each storage volume, in the case that the first data block of the initial data of the storage volume is covered by the first new data, a first snapshot is created, wherein the first snapshot is used to protect the initial data of the storage volume, and the first data block is any data block on the initial data; the first data block protected by the first snapshot is written into the storage unit, and a snapshot reference table is generated, wherein the snapshot reference table has a corresponding relationship with the storage unit.

[0072] In the embodiment of the present application, when each storage volume is first written with data (i.e. data is written to the HEAD space of the volume) in the initial state, the initial data of the volume is obtained. If the client mounts data (i.e. writes data) on the storage volume (i.e. the first data block of the initial data of the storage volume is covered by the first new data, and the first data block is any data block on the initial data, such as 0-32k), a first snapshot is automatically created. The purpose of the first snapshot is to protect the initial data state of the storage volume. At this time, all data in the volume (i.e. all data before being covered by new data) belongs to the initial data, and the snapshot will completely save a copy of these data for subsequent data recovery or historical state backtracking. For example, the client mounts the storage volume and writes 32k data at the volume offset = 0, creates snapshot 1, and then covers the write, which will generate the data of the protected snapshot 1 (i.e. the first data block).

[0073] In the embodiment of the present application, when the first data block in the storage volume is first covered by new data, for example, the first data block in the logical range of the volume is covered by the first new data, the first data block before being covered (i.e. snapshot data) can be written into the storage unit (Chunk), and a corresponding snapshot reference table is generated. The snapshot reference table records the information that each 4KB data block in the Chunk is referenced by which snapshot, i.e. the historical version of the data block is associated with which snapshot.

[0074] In the embodiment of the present application, the snapshot reference table has a one-to-one correspondence with the storage unit (Chunk), that is, whenever a new snapshot data block is written into the Chunk, the snapshot reference table of the Chunk is updated accordingly to reflect the latest snapshot dependency. This correspondence can ensure that when snapshot data validity checking is required, the specific location of data storage can be quickly located, and the validity of the data block can be determined through the snapshot reference table, without the need for extensive metadata queries.

[0075] Figure 2 is a schematic diagram of an optional volume overwriting data A according to an embodiment of the present application, as shown in Figure 2 , a volume storage view when only overwriting data once is shown, that is, the initial state, data A is written into the HEAD space of the volume, and the HEAD space starts from 0K.

[0076] Figure 3 is a schematic diagram of an optional volume overwriting new data according to an embodiment of the present application, as shown in Figure 3 , the client hangs on the storage volume, and when 32k data B is written at the HEAD space offset = 0 of the volume, snapshot 1 is created and B is overwritten at 0-32K, and after B is overwritten, the data protected by snapshot 1, that is, snapshot1 (A referenced by snapshot 1), is generated. The storage processes the data and writes the protected snapshot 1 data (A referenced by snapshot 1) into Chunk1, and protects the snapshot data. Because the size of the generated snapshot data is 32k, 32k data is written into Chunk1, at the same time, the corresponding Chunk1 snapshot reference table {1, 1, 1, 1, 1, 1, 1, 1} is generated, record 1, length 8, because each unit represents 4k, so it indicates that this 32k data is only referenced by snapshot 1. Figure 3 , the volume storage view and Chunk1 view show the case when data B overwrites the 0-32K data block of data A.

[0077] In this embodiment, effective management of snapshot data in the storage volume is achieved. When the initial data is written, the first snapshot can completely protect the initial state of the storage volume; when the data is updated, the protected snapshot data is accurately located and written into the storage unit, and a snapshot reference table is generated at the same time, which is used to quickly check the validity of the data. This accurate snapshot data management and checking method not only improves the efficiency of snapshot GC, but also optimizes the I / O performance of the all-flash storage system, reduces the resource consumption of CPU and memory, and thus improves the overall performance of the storage system.

[0078] To improve the accuracy of generating the snapshot reference table, in the snapshot data validity checking method provided in Embodiment One of the present application, the snapshot identifier of the first snapshot is determined; the first reference sequence is determined based on the snapshot identifier and the length of the first new data; and the first reference sequence is written into the snapshot reference table.

[0079] In the embodiment of the present application, when a snapshot is created, the storage system allocates a unique snapshot identifier (Snapshot ID) to represent the snapshot. This identifier is used to track and identify the snapshot in the snapshot list and other metadata. After the snapshot identifier of the first snapshot is determined, the first reference sequence can be generated according to the length of the new data. Assuming that the first new data covers 32k continuous data on the storage volume starting from the starting offset, a reference sequence with a length of 8 will be generated in the snapshot reference table (because each unit represents 4k, 32k data corresponds to 8 units). This reference sequence contains the snapshot identifier of the first snapshot, indicating that the state of these data blocks before being overwritten is protected by the first snapshot. If the data covered by the same write operation is referenced by multiple snapshots, the reference sequence will contain the smallest identifier of these snapshot identifiers to reflect the snapshot dependency of the data block. Then, the generated first reference sequence is written into the snapshot reference table.

[0080] In the embodiment, when the first new data is written into the storage volume and covers the first data block, the snapshot dependency can be recorded. The snapshot reference table is a data structure that records which snapshots reference each data block (4KB) in the Chunk. By adding the first reference sequence to the snapshot reference table, the storage system can quickly identify which data blocks are invalid and which data blocks are still valid during snapshot GC, effectively avoiding a large number of random I / O operations, improving the speed of snapshot data validity checking, and reducing the resource consumption of CPU and memory.

[0081] In the embodiment, by generating and updating the snapshot reference table in real time during the data block overwrite operation, the life cycle of snapshot data is effectively managed. During the snapshot GC process, there is no need to query the index metadata, and the validity of the data block can be determined directly according to the snapshot reference table and the snapshot list in the memory, shortening the checking time. In addition, since the size of the reference table is relatively fixed and small (about 8KB), it can be easily loaded into the memory, avoiding frequent disk I / O overhead and further optimizing the performance of the storage system. In this way, not only the efficiency of snapshot management is improved, but also the stability and response speed of the all-flash storage system in large-scale snapshot scenarios are enhanced.

[0082] To improve the accuracy of determining the first reference sequence, in the snapshot data validity checking method provided in Embodiment One of the present application, the reference length of the reference sequence is determined based on the length of the first new data and a preset length unit; the reference value at each length on the reference sequence is determined based on the snapshot identifier; and the first reference sequence is determined based on the reference length and the reference value at each length.

[0083] In the embodiments of the present application, when the client performs an overwrite operation on the storage volume to write the first new data, the reference length of the reference sequence can be determined according to the actual length of the new data and a preset length unit (such as 4 KB). For example, if the length of the first new data is 32 KB, then according to the unit of 4 KB, the length of the reference sequence should be 8 units (32 KB / 4 KB=8). This reference length refers to the number of elements in the snapshot reference table that need to be updated or written, ensuring that the snapshot dependency relationship of each data block within the coverage range of the new data is accurately recorded. Then, the reference value at each position on the reference sequence is determined based on the snapshot identifier. When the overwrite operation occurs, it can be checked which snapshots reference the data before the overwrite, i.e., the first data block. Assuming that the data before the overwrite is only referenced by the first snapshot, then in the reference sequence, the reference value at each position will be set to the identifier of the first snapshot. If the same data block is referenced by multiple snapshots, the reference value will be set to the smallest identifier of all related snapshot identifiers, thus ensuring that the snapshot reference table can accurately reflect the snapshot dependency state of each data block within the storage unit (Chunk), and providing key information for data validity checking during snapshot GC.

[0084] Subsequently, the first reference sequence is constructed according to the reference length and the determined reference value. The reference sequence is composed of the reference length and the reference value at each position, and is a data structure representing the snapshot dependency relationship of the data block that is overwritten. For example, if the reference length is 8 and the reference value at all positions is the identifier 1 of the first snapshot, then the first reference sequence will be {1, 1, 1, 1, 1, 1, 1, 1}. This reference sequence will be written into the snapshot reference table on the Chunk where the protected snapshot data is located, and is directly associated with the written data block, for subsequent snapshot data validity checking.

[0085] In the embodiment, by dynamically generating the reference sequence and writing the snapshot reference table, the problems of low efficiency and large resource consumption in the current snapshot GC are solved. After the data block is overwritten by the overwrite operation, the snapshot dependency state thereof can be updated immediately, without large-scale metadata query during the snapshot GC, so that fast and accurate data block validity verification is realized, a large number of random I / O operations are eliminated, the CPU and memory overheads are reduced, and the performance of the all-flash storage system is optimized. In actual deployment, the generation and update method of the reference sequence ensures the efficiency of snapshot data management, especially when the storage system faces frequent write operations and large-scale snapshot scenarios, the system processing capacity can be improved, and low latency and high throughput can be maintained.

[0086] In order to accurately record the reference relationship to the snapshot reference table when new data continues to be written to the storage volume, in the snapshot data validity verification method provided in Embodiment One of the application, after the snapshot reference table is generated, in the case that the second data block of the current data of the storage volume is overwritten by the second new data, a second snapshot is created, wherein the second snapshot is used to protect the current data of the storage volume, and the current data includes the first new data and the remaining initial data overwritten by the first new data; it is determined whether the second data block is a data block in the remaining initial data; in the case that the second data block is a data block in the remaining initial data, the second data block protected by both the first snapshot and the second snapshot is determined, and the second data block is written to the storage unit; based on the snapshot identifier of the first snapshot, the snapshot identifier of the second snapshot and the length of the second new data, a second reference sequence is determined, wherein the minimum snapshot identifier in the snapshot identifier of the first snapshot and the snapshot identifier of the second snapshot is determined as the reference value on the second reference sequence; and the second reference sequence is written to the snapshot reference table.

[0087] In the embodiment of the application, after the storage volume is overwritten by the first new data, if the client continues to perform an overwrite operation on the volume, such as writing the second new data on the second data block, a second snapshot will be automatically created. The second snapshot not only protects the current state of the volume, including the part overwritten by the first new data, but also protects the remaining initial data, i.e. those data regions that have not been updated. By creating the second snapshot, the storage system can maintain the historical state of the volume for data recovery or analysis when needed.

[0088] In the embodiment of the present application, when the second new data overwriting operation occurs, it can be determined whether the second data block belongs to the remaining initial data. If the second data block belongs to the initial data part remaining after the first overwriting, it means that the data block before the overwriting operation has not been covered by any snapshot, and therefore, the snapshot dependency state of this data block needs special attention to ensure that the subsequent snapshot data validity check can accurately reflect its historical state. If the second data block is determined to be a data block in the remaining initial data and has been overwritten, it will be determined that this data block is protected by both the first snapshot and the second snapshot. This means that before the second new data is written, the snapshot dependency of the second data block not only includes the first snapshot, but also extends to the second snapshot. In order to protect the original data of the two snapshots, the second data block before overwriting can be written into a storage unit (Chunk), and the snapshot reference table is updated to reflect the latest snapshot dependency relationship.

[0089] In the embodiment of the present application, after it is determined that the second data block is protected by both the first snapshot and the second snapshot, the second reference sequence can be generated based on the identities of the snapshots and the actual length of the second new data. Since it is a data block protected by two snapshots, the reference value on the second reference sequence will be determined as the minimum value of the two snapshot identities (i.e. the snapshot identity created first). In this way, the data structure in the snapshot reference table can be simplified, and redundant information can be reduced, while ensuring that the snapshot dependency relationship of the data block can be correctly recorded. For example, if the length of the second new data is 32KB, the second reference sequence will contain 8 units (each unit represents 4KB), and the reference value of each unit will be the minimum value of the two snapshot identities. Then, the second reference sequence is written into the snapshot reference table of the storage unit, which is directly associated with the position of the second new data overwriting operation. By adding the latest snapshot dependency information to the snapshot reference table, it can be directly determined in the subsequent snapshot data validity check process which data blocks are still valid and which data blocks have been invalidated by using the snapshot reference table without additional metadata query. In this way, the random I / O operation in the snapshot GC process is reduced, the CPU and memory overhead is reduced, and the performance of the all-flash storage system is optimized.

[0090] Exemplarily, based on the above, Figure 3 If the client continues to write 32K data at the volume offset = 32K, the snapshot 2 protected data is generated. According to the storage processing data, the snapshot 2 data to be protected is continuously written into Chunk1, and the snapshot data is protected. The 32K data is appended to Chunk1, and at the same time, the corresponding snapshot reference table is generated and still needs to record 1, indicating that this 32K data is referenced by snapshot 1 and snapshot 2.

[0091] Figure 4is a schematic diagram of an optional embodiment of the present application in which new data is overwritten at volume offset = 32K, as shown in Figure 4 On the basis of Figure 3 , when the client writes 32K data C at the HEAD space offset = 32K of the volume, snapshot 2 is created and C is overwritten at 32K-64K, and after C is overwritten, data protected by snapshot 1 and snapshot 2, i.e. snapshot2 (snapshot 1, 2 referenced A) is generated. The storage processes the data and writes the data protected by snapshot 1 and snapshot 2 (snapshot 1, 2 referenced A) into Chunk 1, and protects the snapshot data. Because the size of the generated snapshot data is 32K, 32K data is written into Chunk 1, and the corresponding Chunk 1 snapshot reference table {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1} is generated and recorded 1, because the minimum identifier of snapshot 1 and snapshot 2 is used. As shown in Figure 4 , the case in which 32K-64K data blocks of data C overwrite data A is shown in the volume storage view and Chunk 1 view.

[0092] Exemplarily, on the basis of Figure 4 , if the client continues to write 32K data at the position of volume offset = 64K, snapshot 3 protected data is generated. The storage processes the data and continues to append Chunk 1 with the protected snapshot 3 data, and protects the snapshot data. 32K data is appended to Chunk 1, and the corresponding snapshot reference table is generated and recorded 1, indicating that this piece of 32K data is referenced by snapshot 1, snapshot 2 and snapshot 3. Then, the range of 0-96K of the volume has generated snapshot data, the data of 0-32K is referenced by snapshot 1, the snapshot data of 32K-64K is referenced by snapshot 1 and snapshot 2, and the snapshot data of 64K-96K is referenced by snapshot 1, snapshot 2 and snapshot 3, and the 96K data is stored in the same Chunk, and the snapshot data is described by the same snapshot reference table. If the data in the range of 0-32K of the volume is continued to be overwritten, i.e. the client writes 32K data at the position of volume offset = 0, snapshot 3 protected data is continued to be generated. The storage processes the data and continues to append Chunk 1 with the protected snapshot 3 data, and the corresponding snapshot reference table is generated and recorded 2, indicating that this piece of 32K data is referenced by snapshot 2 and snapshot 3. Because snapshot 1 has generated snapshot data in the range of 0-32K of the volume, this piece of data is only referenced by snapshot 2 and snapshot 3.

[0093] Figure 5 is a schematic diagram of an optional embodiment of the present application in which new data is overwritten at volume offset = 64K and = 0, as shown in Figure 5 On the basis of Figure 4On the basis, when the client writes 32k data D and data E at the HEAD space offset = 32K and = 0 of the volume respectively, snapshot 3 is created, and 64k-96K is overwritten with D, and 0k-32K is overwritten with data 5 (E). After overwriting D, data protected by snapshot 1, snapshot 2 and snapshot 3 is generated, that is, snapshot 3 (A referenced by snapshot 1, 2 and 3). After overwriting E, data protected by snapshot 2 and snapshot 3 is generated, that is, snapshot 3 (B referenced by snapshot 2 and 3). The storage processes the data, and writes data (A referenced by snapshot 1, 2 and 3) protected by snapshot 1, snapshot 2 and snapshot 3 and data (B referenced by snapshot 2 and 3) protected by snapshot 2 and snapshot 3 into Chunk 1, at the same time, a corresponding Chunk 1 snapshot reference table {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, …} is generated, and the snapshot reference table is added to the tail (tail meta) of Chunk 1. As shown in FIG. 8, the case when data D overwrites 32K-64K data block of data A and data E overwrites 0K-32K data block of data B is shown in the volume storage view and the Chunk 1 view. Figure 5

[0094] In the embodiment, by dynamically updating the snapshot reference table during the data block overwrite operation, the efficiency and accuracy of snapshot data management are effectively improved. In the snapshot GC process, the data block no longer referenced by any active snapshot can be quickly located and recycled, and the data block referenced by the active snapshot is ensured to be retained, which is suitable for large-scale snapshots and frequent data overwrite scenarios. The detection time of invalid data is reduced, and the resource consumption of metadata query is avoided, thereby improving the throughput and response speed of the all-flash storage system, and meeting the requirements of low latency and high IOPS of the high-performance storage system.

[0095] In order to facilitate the snapshot data validity verification, in the snapshot data validity verification method provided in Embodiment One of the present application, after the second data block is written into the storage unit, in the case that the storage unit is full, it is determined that the snapshot reference table is written; the written snapshot reference table is compressed, and the compressed snapshot reference table is added to the tail of the storage unit; for each data block on the storage unit, all snapshot identifiers protecting the data block are written into the metadata index, and the metadata index is added to the tail of the storage unit.

[0096] ​In the embodiment of the present application, when repeatedly writing data to the storage volume, the data on the Chunk is constantly increasing, and when the generated snapshot data fills up a Chunk, a completed snapshot reference table is formed, and this table is compressed and appended to the tail metadata of the Chunk. All subsequent snapshot data writing generates such a reference table, so the snapshot reference table and the Chunk are one-to-one corresponding, and are saved in the tail metadata of the Chunk, with a length of about 8k.

[0097] Figure 6 is a schematic diagram of an optional Chunk 1 view according to the embodiment of the present application, as Figure 6 shown, showing various snapshot data written in Chunk 1, for example, A stored on 0-32K is referenced by snapshot 1, A stored on 32K-64K is referenced by snapshot 1, 2, A stored on 64K-96K is referenced by snapshot 1, 2, 3, B stored on 96K-128K is referenced by snapshot 2, 3, and the Chunk 1 snapshot reference table {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, …} is generated accordingly and written to the tail of the Chunk after compression.

[0098] Specifically, when the storage unit is filled with data, it means that each data block (4KB) inside the storage unit has been written with snapshot data. At this time, the snapshot reference table of the storage unit can be checked to confirm whether the snapshot dependency relationship of each data block has been recorded completely. The snapshot reference table is a data structure that records which snapshots protect each data block in the storage unit. When the storage unit reaches its storage capacity limit (64MB), the snapshot reference table is marked as "written complete", indicating that all relevant snapshot dependency information has been collected. If it is confirmed that the snapshot reference table is written complete, the reference table can be compressed. The purpose of compressing the snapshot reference table is to save storage space and reduce the size of the tail metadata of the storage unit. The compression mechanism can merge or encode repeated snapshot identification information according to the snapshot dependency of the data blocks in the storage unit, reducing redundancy. The compressed snapshot reference table is saved in the tail metadata of the storage unit, facilitating fast access during subsequent data management and snapshot GC processes. In this way, efficient use of the storage unit is ensured, while reducing the overhead of managing metadata of the storage system.

[0099] And, according to the reference relationship of the snapshot data, the metadata index, which is a data structure for querying the logical address to physical address mapping of the data block, can be updated. For each data block on the storage unit, all snapshot identification information protecting the data block can be recorded, and the metadata index can be updated. The updated metadata index will be attached to the tail of the storage unit and stored together with the snapshot reference table. Although in the snapshot data validity checking process, the snapshot reference table is mainly relied on to determine the validity of the data block, if the identification of the deleted snapshot is greater than the identification recorded in the snapshot reference table, the metadata index can be used to query other identifications referenced by the data block to accurately determine the validity of the data block.

[0100] In the embodiment, by marking the snapshot reference table when the storage unit write is completed, and then compressing and storing the reference table, the size of the tail metadata can be reduced, and the utilization rate of the storage space can be optimized. The metadata index is updated and attached to the tail of the storage unit, which ensures the complete recording of the snapshot dependency information of the data block and provides support for possible data query.

[0101] In order to improve the accuracy of checking the snapshot data on the target storage unit, in the snapshot data validity checking method provided in Embodiment One of the present application, the reference sequence corresponding to the snapshot data is queried from the snapshot reference table; based on the reference sequence, the target snapshot identification of the minimum snapshot identification indication protecting the snapshot data is determined; in the case that the target snapshot identification belongs to the deleted snapshot identification set and there is no remaining snapshot identification not belonging to the deleted snapshot identification set, it is determined that the snapshot data is in an invalid state, wherein the remaining snapshot identification is the snapshot identification protecting the snapshot data except the target snapshot identification, which is determined by querying the metadata index.

[0102] In the embodiment of the present application, when starting the snapshot GC procedure, the snapshot reference table can be loaded from the tail metadata of the storage chunk first. The snapshot reference table contains the snapshot dependency information of each data block (4KB) in the storage chunk, i.e. which snapshots reference these data blocks. According to the logical position of the snapshot data, the corresponding reference sequence in the snapshot reference table can be queried. The reference sequence is a data structure that records all snapshot dependencies of a specific data block, allowing quick identification of which snapshots reference the specific data block. After the reference sequence is queried, the minimum snapshot identification that protects the snapshot data, i.e. the "target snapshot identification", can be determined. This identification is used to indicate the snapshot that originally references the snapshot data, i.e. the earliest created snapshot that protects the data block. By determining the target snapshot identification, the protection status of the snapshot data can be effectively judged based on the creation time sequence of the snapshots. Then, it is checked whether the target snapshot identification belongs to a predefined "deletion snapshot identification set". The deletion snapshot identification set contains the identifications of the snapshots that have been deleted. If the target snapshot identification belongs to this set, and by querying the metadata index, there is no remaining snapshot identification (i.e. greater than the target snapshot identification, also protecting the snapshot data) that does not belong to the deletion snapshot identification set, it can be concluded that the snapshot data is in an invalid state. Here, the remaining snapshot identification refers to those snapshot identifications that protect the snapshot data but have not been deleted. This judgment process uses the combination of the snapshot reference table and the metadata index, which can quickly and accurately identify which data can be safely recycled without the need for expensive full metadata queries.

[0103] Exemplarily, when the snapshot is deleted, the data referenced by the snapshot is invalid data. Through snapshot GC, the invalid data inside the chunk is checked, the valid data is moved (i.e. the valid data is copied to a new chunk), and the old chunk is deleted.

[0104] Figure 7 is a schematic diagram of an optional Chunk1 structure according to the embodiment of the present application, as Figure 7As shown, Chunk1 view is displayed, for example, snapshot data (snap1 (snapshot 1 protects A) is stored on 0-32K, snapshot data (snap2 (snapshot 1, 2 protects A) is stored on 32K-64K, snapshot data (snap3 (snapshot 1, 2, 3 protects A) is stored on 64K-96K, snapshot data (snap3 (snapshot 2, 3 protects B) is stored on 96K-128K, and the Chunk1 snapshot reference table {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, …} is generated accordingly, which is compressed and written to the chunk tail (tail meta). Among them, snap1 (snapshot 1) protects A corresponding to 1, 1, 1, 1, 1, 1, 1, 1 on the snapshot reference table; snap2 (snapshot 1, 2) protects A corresponding to 1, 1, 1, 1, 1, 1, 1, 1 on the snapshot reference table; snap3 (snapshot 1, 2, 3) protects A corresponding to 1, 1, 1, 1, 1, 1, 1, 1 on the snapshot reference table; snap3 (snapshot 2, 3) protects B corresponding to 2, 2, 2, 2, 2, 2, 2, 2 on the snapshot reference table.

[0105] On the basis of Figure 7 , if snapshot 1, snapshot 2, and snapshot GC are deleted, Chunk1 is selected for garbage collection, and the corresponding extents (metadata index) and snapshot reference table are loaded through the Chunk1 tail metadata.

[0106] As shown, Figure 7 , the first piece of data in Chunk1 corresponds to the snapshot 1 data of the volume logical range 0-32k, which can be confirmed by the snapshot reference table. This piece of data is only referenced by snapshot 1, and since snapshot 1 has been deleted, this data is invalid.

[0107] The second piece of data in Chunk1 corresponds to the snapshot 2 data of the volume logical range 32k-64k, which can be confirmed by the snapshot reference table. This piece of data is referenced by snapshot 1 and snapshot 2, and since snapshot 1 and snapshot 2 have been deleted, this data is invalid.

[0108] The third piece of data in Chunk1 corresponds to the snapshot 3 data of the volume logical range 64k-96k, which can be confirmed by the snapshot reference table. This piece of data is referenced by snapshot 1, snapshot 2, and snapshot 3, and although snapshot 1 and snapshot 2 are deleted, snapshot 3 still references this piece of data, so this data is valid.

[0109] The fourth piece of data within Chunk1 corresponds to the data in snapshot 3 within the volume logical range of 0k-32k. This data can be confirmed through the snapshot reference table to be referenced by snapshots 2 and 3. Snapshot 2 has been deleted, but snapshot 3 still exists and references the data, so this data is valid.

[0110] Therefore, the snapshot GC can move the 3rd and 4th data entries, and then delete Chunk1, freeing up storage space. In this way, through the above-described fast verification method, each data entry only needs to be compared with the snapshot reference table to determine its validity, avoiding the need to query index metadata for judgment, effectively improving performance and verification efficiency.

[0111] Figure 8 This is a schematic diagram of an optional Chunk2 structure according to an embodiment of the present invention, as shown below. Figure 8 As shown, the Chunk2 view is displayed, by... Figure 7 The two data entries of snap3 in Chunk1 are moved to a new chunk (Chunk2), and a new reference table is generated (Chunk2 snapshot reference table {1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,…}). Then Chunk1 is deleted, resulting in Chunk2, which contains two data entries: snap3A on the 0-32k range and snap3B on the 32K-64K range.

[0112] In this embodiment, by querying the reference sequence from the snapshot reference table and determining the validity of the snapshot data based on the reference sequence and metadata index, the efficiency of snapshot GC is improved, full metadata scanning is avoided, random I / O operations are reduced, and the burden on CPU and memory is reduced.

[0113] In this embodiment of the invention, the snapshot reference table design allows for recording snapshot dependencies of each 4KB block at the chunk level, stored in the metadata at the end of the chunk. This eliminates the need for extensive metadata queries during snapshot GC, reducing the verification scope from scanning a 64MB logical range to loading approximately 8KB of data. Furthermore, by generating and compressing the reference table based on the snapshot's continuity ID, the storage space table is compressed, optimizing storage efficiency. Moreover, by using the snapshot reference table in the chunk's end metadata to determine data validity, extensive metadata queries are unnecessary, enabling rapid snapshot GC verification. This approach is applicable to all scenarios requiring snapshot data validity verification, such as GC, data migration, and replication, and is not limited to all-flash storage.

[0114] The following is a detailed description with reference to another embodiment.

[0115] Example 2

[0116] The snapshot data validity checking device provided in the embodiment comprises a plurality of implementation units, each of which corresponds to the implementation steps in Embodiment 1.

[0117] Figure 9 is a schematic diagram of an optional snapshot data validity checking device according to an embodiment of the present application, as shown in the figure, the checking device can comprise a first determining unit 90, a second determining unit 91, a checking unit 92, and a recycling unit 93. Figure 9

[0118] The first determining unit 90 is configured to determine, based on the snapshot list, a set of deleted snapshot identifiers in a deleted state.

[0119] The second determining unit 91 is configured to determine, based on the set of deleted snapshot identifiers, a target storage unit for each deleted snapshot identifier, wherein the target storage unit stores snapshot data protected by the snapshot.

[0120] The checking unit 92 is configured to, for each target storage unit, check the snapshot data on the target storage unit based on a snapshot reference table carried by the target storage unit and a metadata index, to obtain a checking result, wherein the snapshot reference table is a reference table generated when the snapshot is created, and the metadata index records a snapshot identifier referenced by each snapshot data.

[0121] The recycling unit 93 is configured to recycle, in a case where the checking result indicates that there is snapshot data in an invalid state on the target storage unit, the snapshot data in the invalid state.

[0122] The above checking device can pre-create a corresponding snapshot reference table for each storage unit, determine a set of identifiers of deleted snapshots according to the snapshot list, then locate a target storage unit (Chunk) storing snapshot data, efficiently determine whether the snapshot data is in an invalid state by analyzing the snapshot reference table at the end of the Chunk and in combination with the metadata index, and perform garbage collection if the data is invalid, thereby reducing random I / O operations, improving checking efficiency, achieving the purpose of quickly checking the validity of snapshot data, and thus achieving the technical effect of improving snapshot GC efficiency.

[0123] Optionally, the checking device further comprises a first creating unit configured to, before determining, based on the snapshot list, the set of deleted snapshot identifiers in the deleted state, create, for each storage volume, a first snapshot in a case where a first data block of initial data of the storage volume is covered by first new data, wherein the first snapshot is used to protect the initial data of the storage volume, and the first data block is any data block on the initial data; and a first generating unit configured to write the first data block protected by the first snapshot to a storage unit and generate a snapshot reference table, wherein the snapshot reference table has a corresponding relationship with the storage unit.​

[0124] Optionally, the first generating unit comprises: a first determining module, configured to determine a snapshot identifier of the first snapshot; a second determining module, configured to determine the first reference sequence based on the snapshot identifier and a length of the first new data; and a first writing module, configured to write the first reference sequence into the snapshot reference table.

[0125] Optionally, the second determining module comprises: a first determining submodule, configured to determine a reference length of the reference sequence based on the length of the first new data and a preset length unit; a second determining submodule, configured to determine a reference value on each length of the reference sequence based on the snapshot identifier; and a third determining submodule, configured to determine the first reference sequence based on the reference length and the reference value on each length.

[0126] Optionally, the checking device further comprises: a second creating unit, configured to create a second snapshot after generating the snapshot reference table, in a case that a second data block of current data of the storage volume is covered by second new data, wherein the second snapshot is used to protect the current data of the storage volume, and the current data comprises the first new data and remaining initial data covered by the first new data; a third determining unit, configured to determine whether the second data block is a data block in the remaining initial data; a fourth determining unit, configured to determine a second data block protected by both the first snapshot and the second snapshot in a case that the second data block is a data block in the remaining initial data, and write the second data block into the storage unit; a fifth determining unit, configured to determine a second reference sequence based on a snapshot identifier of the first snapshot, a snapshot identifier of the second snapshot and a length of the second new data, wherein a minimum snapshot identifier in the snapshot identifier of the first snapshot and the snapshot identifier of the second snapshot is determined as a reference value on the second reference sequence; and a first writing unit, configured to write the second reference sequence into the snapshot reference table.

[0127] Optionally, the checking device further comprises: a sixth determining unit, configured to determine that the snapshot reference table is written completely in a case that the storage unit is written full after writing the second data block into the storage unit; a first compressing unit, configured to compress the snapshot reference table written completely, and add the compressed snapshot reference table to a tail of the storage unit; a second writing unit, configured to write all snapshot identifiers of a protected data block into a metadata index for each data block on the storage unit, and add the metadata index to the tail of the storage unit.

[0128] Optionally, the checking unit comprises: a first query module, configured to query the reference sequence corresponding to the snapshot data from the snapshot reference table; a third determination module, configured to determine the target snapshot identifier indicated by the minimum snapshot identifier protecting the snapshot data based on the reference sequence; and a fourth determination module, configured to determine that the snapshot data is in the invalid state in a case where the target snapshot identifier belongs to the deletion snapshot identifier set and there is no remaining snapshot identifier that does not belong to the deletion snapshot identifier set, wherein the remaining snapshot identifier is determined by querying the metadata index and is a snapshot identifier protecting the snapshot data except the target snapshot identifier.

[0129] The checking device described above can further comprise a processor and a memory, and the first determination unit 90, the second determination unit 91, the checking unit 92, the recycling unit 93, etc. are stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory.

[0130] The processor described above comprises a kernel, and the kernel calls the corresponding program units in the memory. The kernel can be one or more, and the recycling of the snapshot data in the invalid state is realized by adjusting the kernel parameters in the case where the checking result indicates that there is snapshot data in the invalid state on the target storage unit.

[0131] The memory described above can comprise a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory comprises at least one memory chip.

[0132] The present application also provides a computer program product adapted to execute the program of the following method steps when executed on a data processing device: determining a deletion snapshot identifier set in the deleted state based on a snapshot list, determining the target storage unit of each deletion snapshot identifier based on the deletion snapshot identifier set, checking the snapshot data on the target storage unit based on the snapshot reference table and the metadata index carried by the target storage unit for each target storage unit to obtain a checking result, and recycling the snapshot data in the invalid state in the case where the checking result indicates that there is snapshot data in the invalid state on the target storage unit.

[0133] According to another aspect of the embodiment of the present application, a computer program product is also provided, comprising a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the checking method of the snapshot data validity of any one of the above.

[0134] According to another aspect of the embodiments of the present application, an electronic device is also provided, which includes one or more processors and a memory storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the above-mentioned method for checking the validity of snapshot data.

[0135] Figure 10 is a hardware structural block diagram of an electronic device (or a mobile device) for a method for checking the validity of snapshot data according to an embodiment of the present application. As shown in Figure 10 , the electronic device can include one or more processors (for example, processors 1002a, 1002b, …, 1002n, etc. in the processor 1002 in Figure 10 , which can include but are not limited to processing devices such as microprocessors MCU or programmable logic devices FPGA, etc.), a memory 1004 for storing data. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. Those skilled in the art can understand that Figure 10 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the electronic device can also include more or fewer components than those shown in Figure 10 , or have a different configuration from Figure 10 .

[0136] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0137] The embodiments or examples of the present disclosure are not exhaustive, but only a part of the embodiments or examples, and are not specific limitations on the protection scope of the present disclosure. In the case of no contradiction, each step in an embodiment or example can be implemented as an independent embodiment, and the steps can be combined arbitrarily, for example, the scheme after removing part of the steps in an embodiment or example can be implemented as an independent embodiment, and the order of the steps in an embodiment or example can be exchanged arbitrarily, in addition, the optional ways or optional examples in an embodiment or example can be combined arbitrarily; in addition, the embodiments or examples can be combined arbitrarily, for example, the steps of different embodiments or examples can be combined arbitrarily, an embodiment or example can be combined with the optional ways or optional examples of other embodiments or examples.

[0138] In the above-mentioned embodiments of the present application, the description of each embodiment has its own emphasis, and the parts not described in detail in an embodiment can be referred to the related description of other embodiments.

[0139] In several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other manners. For example, the described unit embodiments can be divided into other ways, for example, the units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be implemented by using some interfaces, and the indirect couplings or communication connections can be implemented in electronic, mechanical, or other forms.

[0140] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed on multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0141] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0142] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

[0143] The above description is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method for checking the validity of snapshot data, characterized in that, The method comprises the following steps: determining a set of deleted snapshot identifiers in a deleted state based on a snapshot list; determining a target storage unit of each deleted snapshot identifier based on the set of deleted snapshot identifiers, wherein the target storage unit stores snapshot data protected by the snapshot; for each target storage unit, checking the snapshot data on the target storage unit based on a snapshot reference table carried by the target storage unit and a metadata index, to obtain a checking result, wherein the snapshot reference table is a reference table generated when a snapshot is created, and the metadata index records a snapshot identifier referenced by each snapshot data; in a case where the checking result indicates that the snapshot data in an invalid state exists on the target storage unit, recycling the snapshot data in the invalid state; wherein, before determining the set of deleted snapshot identifiers in the deleted state based on the snapshot list, the method further comprises: in a case where a first data block of initial data of a storage volume is covered by first new data, creating a first snapshot, wherein the first snapshot is used to protect the initial data of the storage volume, and the first data block is any data block on the initial data; writing the first data block protected by the first snapshot to a storage unit, and generating the snapshot reference table, wherein the snapshot reference table has a corresponding relationship with the storage unit; after generating the snapshot reference table, the method further comprises: in a case where a second data block of current data of the storage volume is covered by second new data, creating a second snapshot, wherein the second snapshot is used to protect the current data of the storage volume, and the current data comprises the first new data and remaining initial data covered by the first new data; determining whether the second data block is a data block in the remaining initial data; in a case where the second data block is a data block in the remaining initial data, determining the second data block protected by both the first snapshot and the second snapshot, and writing the second data block to the storage unit; determining a second reference sequence based on a snapshot identifier of the first snapshot, a snapshot identifier of the second snapshot, and a length of the second new data, wherein the minimum snapshot identifier in the snapshot identifier of the first snapshot and the snapshot identifier of the second snapshot is determined as a reference value on the second reference sequence; and writing the second reference sequence to the snapshot reference table.

2. The method of claim 1, wherein, The step of generating the snapshot reference table comprises: determining a snapshot identifier of the first snapshot; determining a first reference sequence based on the snapshot identifier and a length of the first new data; writing the first reference sequence to the snapshot reference table.

3. The method of claim 2, wherein, The step of determining a first reference sequence based on the snapshot identifier and the length of the first new data comprises: determining a reference length of a reference sequence based on the length of the first new data and a preset length unit; determining a reference value on each length on the reference sequence based on the snapshot identifier; determining the first reference sequence based on the reference length and the reference value on each length.

4. The method of claim 1, wherein, after the second data block is written into the storage unit, further comprising: in the case that the storage unit is written full, determining that the snapshot reference table is written complete; compressing the snapshot reference table written complete, and adding the compressed snapshot reference table to the tail of the storage unit; for each data block on the storage unit, writing all the snapshot identifiers protecting the data block to the metadata index, and adding the metadata index to the tail of the storage unit.

5. The method of claim 1, wherein, The step of checking the snapshot data on the target storage unit comprises: inquiring the reference sequence corresponding to the snapshot data from the snapshot reference table; based on the reference sequence, determining the target snapshot identifier indicated by the minimum snapshot identifier protecting the snapshot data; in the case that the target snapshot identifier belongs to the deleted snapshot identifier set, and there is no remaining snapshot identifier not belonging to the deleted snapshot identifier set, determining that the snapshot data is in the invalid state, wherein the remaining snapshot identifier is the snapshot identifier protecting the snapshot data except the target snapshot identifier, which is determined by inquiring the metadata index.

6. An apparatus for verifying the validity of snapshot data, characterized by comprise: a first determining unit configured to determine a deleted snapshot identifier set in a deleted state based on a snapshot list; a second determining unit configured to determine a target storage unit of each deleted snapshot identifier based on the deleted snapshot identifier set, wherein the target storage unit stores snapshot data protected by the snapshot; a checking unit configured to check, for each target storage unit, snapshot data on the target storage unit based on a snapshot reference table carried by the target storage unit and a metadata index, to obtain a checking result, wherein the snapshot reference table is a reference table generated when a snapshot is created, and the metadata index records snapshot identifiers referenced by each snapshot data; a recycling unit configured to recycle, in the case that the checking result indicates that there is snapshot data in an invalid state on the target storage unit, the snapshot data in the invalid state; The checking device further comprises: a first creating unit configured to, before determining the deleted snapshot identifier set in the deleted state based on the snapshot list, create a first snapshot for each storage volume in the case that a first data block of initial data of the storage volume is covered by first new data, wherein the first snapshot is used to protect the initial data of the storage volume, and the first data block is any data block on the initial data; and a first generating unit configured to write the first data block protected by the first snapshot into a storage unit, and generate a snapshot reference table, wherein the snapshot reference table has a corresponding relationship with the storage unit. The checking device further comprises: a second creating unit, configured to create a second snapshot after the generation of the snapshot reference table, in a case where a second data block of current data of the storage volume is covered by second new data, wherein the second snapshot is used to protect the current data of the storage volume, and the current data comprises the first new data and remaining initial data covered by the first new data; a third determining unit, configured to determine whether the second data block is a data block in the remaining initial data; a fourth determining unit, configured to, in a case where the second data block is a data block in the remaining initial data, determine the second data block protected by both the first snapshot and the second snapshot, and write the second data block into the storage unit; a fifth determining unit, configured to determine a second reference sequence based on a snapshot identifier of the first snapshot, a snapshot identifier of the second snapshot, and a length of the second new data, wherein a minimum snapshot identifier in the snapshot identifier of the first snapshot and the snapshot identifier of the second snapshot is determined as a reference value on the second reference sequence; and a first writing unit, configured to write the second reference sequence into the snapshot reference table.

7. A computer program product, characterised in that, A non-transitory computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the method for checking validity of snapshot data according to any one of claims 1 to 5.

8. An electronic device, comprising: One or more processors and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method for checking validity of snapshot data according to any one of claims 1 to 5.

Images