Snapshot processing method and electronic device

Abstract

The application discloses a snapshot processing method and an electronic device, and relates to the technical field of computers. The snapshot processing method comprises the following steps: dividing at least one snapshot into one or more snapshot groups by determining, from to-be-processed snapshots, that a time difference between a snapshot creation time and a to-be-processed snapshot acquisition time of the at least one snapshot is greater than a first threshold value, determining a target snapshot from the snapshot groups according to the time difference, and updating metadata of the target snapshot based on metadata of other snapshots in the snapshot groups except the target snapshot. The snapshot can be processed in a hierarchical manner according to the time difference, the recoverability of historical data can be effectively prolonged without sacrificing the recovery accuracy of recent data, the storage space occupied by the snapshot and the management complexity are reduced, and the technical problem that data recovery cannot be traced back to an earlier historical moment in related methods can be solved, thereby achieving the technical effect of reducing the storage space occupied by the snapshot and the management complexity.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to a snapshot processing method and an electronic device. Background Technology

[0002] Scheduled snapshot technology is a technique where the system automatically generates snapshots periodically at preset time intervals. The density of scheduled snapshots (CDP) can reach the second level, such as creating a CDP snapshot every 5 seconds. However, this also leads to snapshot data consuming a large amount of disk space and increasing management complexity. Related methods typically involve deleting the oldest snapshot when the maximum number of snapshots is reached. However, these methods have at least the following problems: only deleting the oldest snapshot permanently erases all data states prior to that point in time, making it impossible to revert to earlier historical moments during data recovery. Summary of the Invention

[0003] This application provides a snapshot processing method and an electronic device to at least solve the problem in related technologies where only the earliest created snapshot is permanently deleted, resulting in the permanent loss of all data states prior to the time point corresponding to the earliest created snapshot, and making it impossible to revert to earlier historical moments during data recovery.

[0004] This application provides a snapshot processing method, comprising: acquiring snapshots to be processed for a target storage space; determining at least one snapshot from the snapshots to be processed whose time difference between the snapshot creation time and the snapshot acquisition time is greater than a first threshold, and dividing the at least one snapshot into one or more snapshot groups according to a predetermined time window; for each snapshot group, determining a target snapshot from the snapshot group according to the time difference, and updating the metadata of the target snapshot based on the metadata of other snapshots in the snapshot group excluding the target snapshot, to obtain an updated snapshot for each snapshot group; and obtaining a target snapshot set using the updated snapshot and snapshots from the snapshots to be processed whose time difference is less than or equal to the first threshold, wherein the target snapshot set is used to perform predetermined data processing operations on the data stored in the target storage space.

[0005] This application also provides a snapshot processing apparatus, comprising: an acquisition module for acquiring snapshots to be processed for a target storage space; a first determination module for determining at least one snapshot from the snapshots to be processed where the time difference between the snapshot creation time and the snapshot acquisition time is greater than a first threshold, and dividing the at least one snapshot into a snapshot group according to a predetermined time window; a second determination module for determining a target snapshot from the snapshot group according to the time difference for each snapshot group, and updating the target snapshot based on the data of other snapshots in the snapshot group excluding the target snapshot, to obtain an updated snapshot for each snapshot group; and an acquisition module for obtaining a target snapshot set using the updated snapshots and snapshots from the snapshots to be processed where the time difference is less than or equal to the first threshold.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the snapshot processing method described above when executing the computer program.

[0007] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described snapshot processing methods.

[0008] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described snapshot processing methods.

[0009] This application identifies at least one snapshot from the pool of snapshots whose creation time and acquisition time are greater than a first threshold. This snapshot is then further divided into snapshot groups. A target snapshot is determined from these groups based on the time difference, and the target snapshot is updated based on data from other snapshots in the group. This allows for the omission of recent snapshots with small time differences, preserving more important recent snapshots. For earlier snapshots with larger time differences, updates are performed, reducing storage space occupied by snapshots while ensuring historical data recoverability. This hierarchical processing method based on time differences effectively extends the recoverability of historical data without sacrificing the accuracy of recent data recovery, reduces storage space occupied by snapshots, and lowers management complexity. Therefore, it solves the technical problem in related methods where data recovery cannot trace back to earlier historical moments, achieving the technical effect of reducing storage space occupied by snapshots and management complexity. Attached Figure Description

[0010] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart of a snapshot processing method provided in an embodiment of this application.

[0012] Figure 2 This is a schematic diagram of the target snapshot update provided in an embodiment of this application.

[0013] Figure 3 This is a schematic diagram of the metadata of a snapshot provided in an embodiment of this application.

[0014] Figure 4This is a schematic diagram of the modified metadata provided in an embodiment of this application.

[0015] Figure 5 This is a schematic diagram of the metadata for the updated snapshot provided in an embodiment of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0017] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0018] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] Scheduled snapshot technology is a technique in which the system automatically generates snapshots periodically according to preset time intervals. Examples include Continuous Data Protection (CDP) snapshot technology. Compared to ordinary snapshots, CDP snapshot technology supports a higher density of snapshots per source volume, and when combined with second-level scheduled snapshots, it can achieve second-level data protection.

[0020] Because CDP snapshots offer advantages such as fast creation speed, no additional volume space usage, and minimal impact on host performance, scheduled snapshot functions typically choose to create CDP snapshots on the volume itself. The density of scheduled CDP snapshots can reach the second level; for example, a CDP snapshot can be created every 5 seconds. With this extremely high density and short interval of CDP snapshots, if data is corrupted, it can be directly rolled back to the point before the problem occurred, with at most a few seconds of data lost.

[0021] However, if the CDP snapshot interval is set to a very short time, a large amount of snapshot data will be generated in a short period of time. For example, if the interval is set to 5 seconds, 17,280 snapshots can be generated in one day for one volume. This results in the snapshot data occupying a large amount of disk space, and the metadata information for managing snapshots will also occupy memory space. Both of these will compete for production resources to some extent, and the complexity of snapshot management will also increase, resulting in decreased management efficiency.

[0022] The relevant method involves consistently deleting the oldest snapshot when the number of snapshots reaches a predetermined limit, retaining only the most recent snapshot. For example, if a user sets a maximum of 500 snapshots when creating a scheduled CDP snapshot policy, the system will delete the oldest snapshot before creating the 501st snapshot, ensuring that the number of snapshots on the volume never exceeds the preset value. While this method limits the number of snapshots, consistently deleting the oldest snapshot means that once deleted, data recovery will only restore data from the most recent period; earlier data cannot be recovered.

[0023] In view of this, embodiments of this application provide a snapshot processing method, and the snapshot processing method is described in detail in conjunction with the execution flow of the snapshot processing method.

[0024] Figure 1 A flowchart illustrating the snapshot processing method provided in this application embodiment. Figure 1 As shown, the snapshot processing method of this application embodiment includes operations S110 to S140.

[0025] Operation S110: Obtain the snapshot to be processed for the target storage space.

[0026] Operation S120: Determine at least one snapshot from the snapshots to be processed whose time difference between the snapshot creation time and the snapshot acquisition time is greater than a first threshold, and divide the at least one snapshot into one or more snapshot groups according to a predetermined time window.

[0027] In operation S130, for each snapshot group, a target snapshot is determined from the snapshot group according to the time difference, and the metadata of the target snapshot is updated based on the metadata of other snapshots in the snapshot group besides the target snapshot, so as to obtain the updated snapshot of each snapshot group.

[0028] Operation S140 involves using the updated snapshots and snapshots in the pending snapshots whose time difference is less than or equal to the first threshold to obtain a target snapshot set. This target snapshot set is used to perform predetermined data processing operations on the data stored in the target storage space.

[0029] For example, the target storage space may include a logical area for storing data. Snapshot technology is a point-in-time data protection technology that saves data in the storage space at a specific moment.

[0030] The target storage space can be the source volume. For a snapshot, a new volume can be created from the source volume to obtain the target volume, and a relationship can be established between the two volumes to obtain the snapshot relationship. The target volume can store the data at the time the snapshot was created in the source volume. When the snapshot is first created, there is no data in the target volume. When data at a certain location in the source volume is modified for the first time, the data at that location in the source volume can be copied to the target volume first, and then the new data can be written. Bitmap technology can be used to record the data status of the source and target volumes, that is, which locations of data have been copied to the target volume (the corresponding bit in the bitmap is 1) and which locations of data have not yet been copied (the corresponding bit in the bitmap is 0). When the host modifies the data in the source volume, it can use the bitmap to determine whether the data written at that location is the first time it has been written, and determine whether it needs to be copied to the target volume.

[0031] For example, CDP can add a timestamp-like record (generation ID, gen_id) to each input / output (IO) of the volume to record which generation of snapshot each IO belongs to. In this way, the data corresponding to CDP snapshots taken of the volume at different times all belong to the volume itself. In the underlying storage space of the volume, the gen_id distinguishes data at the same location at different times, ensuring that earlier data is not considered invalid and reclaimed. Compared to traditional snapshots with a target volume, CDP snapshots do not require an additional target volume; the gen_id quickly locates the data of a specific generation of snapshots, omitting the traditional snapshot write-on-write data copying process.

[0032] For example, for a volume, before a CDP snapshot is created, all IO operations have a gen_id value of 0. After a CDP snapshot is created, this snapshot becomes the generation 0 snapshot, and its corresponding data is all the data in the source volume with a gen_id of 0. After the snapshot is created, the gen_id of the source volume is incremented by 1, meaning that the gen_id of subsequent IO operations will become 1. When another CDP snapshot is taken, the new CDP snapshot becomes the generation 1 snapshot, and the gen_id is incremented by 1 again, with the gen_id of subsequent IO operations becoming 2, and so on.

[0033] Because host I / O is not uniform, some locations on the source volume may receive frequent I / O while others may receive less. Therefore, the gen_id for each location is not equal. After multiple rounds of CDP snapshots, the gen_id of hot areas will often be very large or even equal to the number of snapshots, while the gen_id of cold areas may hardly change. When reading I / O from the source volume, simply read the data with the largest gen_id for each location to read the latest data. When reading data from a specific CDP snapshot, the gen_id needs to be specified. If the ID exists in that area, the corresponding data is read directly. If it does not exist, the system checks if gen_id-1 exists, and so on until data is found. The found data is the data corresponding to the specified gen_id.

[0034] For example, a snapshot that has been created for the target storage space can be obtained, such as a snapshot created periodically, to obtain a snapshot to be processed, which may include one or more snapshots.

[0035] The snapshot creation time indicates the point in time the snapshot records the data state, while the snapshot acquisition time indicates the time at which the target snapshot set is determined during the current execution. The time difference between the snapshot creation time and the snapshot acquisition time for each snapshot to be processed can be determined.

[0036] The snapshots to be processed can be divided based on the time difference. For example, they can be divided into: snapshots with a time difference less than a first threshold, which are newly created snapshots in the most recent period; and snapshots with a time difference less than or equal to the first threshold, which are snapshots created in an earlier period.

[0037] For example, snapshots to be processed can be sorted according to their creation time to obtain a time-series snapshot sequence; snapshots with a time difference less than a first threshold and snapshots with a time difference less than or equal to the first threshold can be quickly identified from the time-series snapshot sequence using a binary search method.

[0038] Newly created snapshots can be retained in their entirety, while older snapshots can be sparsified. Snapshots created recently are typically more important, such as those that can handle recent data anomalies or errors, and thus have higher data value; therefore, they should be retained as much as possible. Older snapshots have lower data value compared to newer ones. For example, for long-running and frequently used business data, if a data anomaly or error occurred several hours ago, users would usually have already noticed it and wouldn't need to wait until now to restore data from several hours ago. In other words, older snapshots don't need to maintain second-level recovery accuracy, so these types of snapshots can be sparsified to reduce the number of snapshots and free up space.

[0039] For example, for at least one snapshot, it can be divided into at least one group according to a predetermined time window. For example, if at least one snapshot includes snapshots created between 00:00:01 and 02:00:00, then snapshots created between 00:00:01 and 01:00:00 can be divided into one group, and snapshots created between 01:00:01 and 02:00:00 can be divided into another group.

[0040] Based on the time difference, the target snapshot is determined from the snapshot group. For example, the snapshot with the smallest time difference from the snapshot group can be selected as the target snapshot. For each snapshot group, the target snapshot within that group, as well as other snapshots within that group besides the target snapshot, can be determined. Data corresponding to the other snapshots can be merged into the target snapshot to update it. After the update is complete, the other snapshots can be deleted to release the space occupied by them while retaining their corresponding data. The target snapshot set can include updated data from each snapshot group, as well as snapshots among the snapshots to be processed whose time difference is less than or equal to a first threshold.

[0041] For example, after dividing the snapshots created between 00:00:01 and 01:00:00 into a snapshot group, the snapshot created at 01:00:00 can be used as the target snapshot, and the data corresponding to the snapshots created at times such as 00:59:55, 0:59:50, ..., 0:00:05 can be merged into the target snapshot.

[0042] For example, a target snapshot set can be used to perform predetermined data processing operations such as data migration and data recovery.

[0043] Figure 2 This is a schematic diagram of the target snapshot update provided in an embodiment of this application.

[0044] like Figure 2 As shown, the snapshots to be processed corresponding to the target storage space include, for example, snapshot 1, snapshot 2, and snapshot 3. Snapshot 1 is the earliest created snapshot, and snapshot 3 is the latest created snapshot. Snapshot 1 records data such as A1, A2, and A3, snapshot 2 records data such as B1 and B2, and snapshot 3 records data such as C1 and C2.

[0045] Although snapshot 1 contains data A1, snapshot 2 contains the latest data B1. Therefore, data B1 can replace data A1 and be merged into snapshot 3.

[0046] For example, for each location, if data has already been written to that location in the latest snapshot, then the data at that location can directly use the data from the latest snapshot. For instance, for locations 3 and 4 in snapshot 3, where data C1 and data C2 already exist, data C1 and data C2 can be directly retained. However, if no data has been written to that location in the latest snapshot, then the data corresponding to the latest snapshot needs to be found from other snapshots with existing data and merged into snapshot 3. For instance, for the first location, where data exists in both snapshot 1 and snapshot 2, and snapshot 2 is newer than snapshot 1, then data B1 from snapshot 2 can replace data A1 and be merged into snapshot 3. After the data at all locations has been merged, all snapshots except snapshot 3 can be deleted to free up space.

[0047] According to embodiments of this application, by identifying at least one snapshot from the snapshots to be processed whose time difference between the snapshot creation time and the snapshot acquisition time is greater than a first threshold, further dividing the at least one snapshot into a snapshot group, and determining the target snapshot from the snapshot group based on the time difference, and updating the target snapshot based on the data of other snapshots in the snapshot group excluding the target snapshot, recent snapshots with small time differences can be left unprocessed to fully preserve more important recent snapshots; earlier snapshots with large time differences are updated, reducing the storage space occupied by snapshots while ensuring the recoverability of historical data. Through the above-described method of hierarchical processing based on time differences, the recoverability of historical data can be effectively extended without sacrificing the accuracy of recent data recovery, and the storage space occupied by snapshots can be reduced, thus reducing management complexity. This solves the technical problem in related methods where data recovery cannot trace back to earlier historical moments, achieving the technical effect of reducing the storage space occupied by snapshots and management complexity.

[0048] According to embodiments of this application, the predetermined time window includes a first time window and a second time window, the snapshot group includes a first snapshot group and a second snapshot group, and at least one snapshot includes: a first snapshot with a time difference greater than a first threshold and less than a second threshold, and a second snapshot with a time difference greater than or equal to the second threshold, wherein the second threshold is greater than the first threshold; dividing at least one snapshot into a snapshot group according to the snapshot creation time includes: dividing the first snapshot into at least one first snapshot group according to the first time window based on the snapshot creation time; dividing the second snapshot into at least one second snapshot group according to the second time window, wherein the width of the first time window is less than the width of the second time window.

[0049] For example, snapshots containing recent data (e.g., time differences less than a first threshold) can be entirely retained without processing; first snapshots containing slightly earlier data (e.g., time differences greater than the first threshold and less than a second threshold) can be grouped into smaller time windows (e.g., a first time window); and second snapshots containing even earlier data (time differences greater than or equal to the second threshold) can be grouped into larger time windows (e.g., a second time window). The values ​​of the first threshold, second threshold, first time window, and second time window can be set according to actual needs.

[0050] For example, snapshots created within one hour, snapshots created more than one hour but less than one day, and snapshots created more than one day can be identified from the pending snapshots. Snapshots created within one hour can remain as they are and not processed; snapshots created more than one hour but less than one day can be grouped by hour, such as by hourly time, for example, snapshots from 00:00:01 to 01:00:00 can be grouped together, and snapshots from 01:00:01 to 02:00:00 can be grouped together; snapshots created more than one day can be grouped by day.

[0051] According to the embodiments of this application, by dividing the first snapshot into at least one first snapshot group according to a first time window, and dividing the second snapshot into at least one second snapshot group according to a second time window, the larger the time span of the snapshot, the coarser its retention granularity and the less storage resources it occupies. This achieves reasonable sparsification processing of high-density timed snapshots based on the time span, thereby retaining both high-value recent snapshot data and some relatively long-term historical snapshot data, thus improving the data recovery capability.

[0052] According to an embodiment of this application, the width of the first time window and the width of the second time window can be determined based on the service type of the data stored in the target storage space.

[0053] For example, different business types may have different requirements for data recovery accuracy. For instance, financial data may require second-level accuracy, while log archive data only requires day-level accuracy. Therefore, a smaller width can be set when the data stored in the target storage space is financial data, while a larger width can be set when the data stored in the target storage space is log archive data.

[0054] By determining the width of the first time window and the width of the second time window based on the business type of the data stored in the target storage space, the width of the time window can be flexibly set according to the business type, thereby accurately matching business needs.

[0055] According to an embodiment of this application, the target storage space is divided into at least one logical storage block; each snapshot in the snapshot group is configured with metadata, which is used to record: the logical address of at least one logical storage block in which a write operation or modification operation occurs during the time interval between the snapshot creation time of the snapshot and the previous snapshot creation time, the physical block address corresponding to the logical address, and the snapshot creation time of the snapshot.

[0056] For example, the target storage space can be divided into at least one logical storage block, each with its own logical block address (LBA). The logical address numbers can start from 0, and each storage space manages its own LBA. The actual data can be stored on a hard disk, which can also be divided into at least one physical block, each with its own physical block address (PBA). When data is written to an LBA in the target storage space, that data is ultimately written to the PBA on the hard disk.

[0057] The metadata configured in the snapshot records the correspondence between LBA and PBA. When it is necessary to read the data of a certain LBA in the target storage space, the PBA corresponding to the LBA can be determined through the metadata, and the data can be read from the physical block corresponding to the PBA.

[0058] For example, if a snapshot is created at time TIME1 (which can be abbreviated as T1) and data is written to LBA1 (which can be abbreviated as L1), and the data is stored in PBA1 (which can be abbreviated as P1), then the metadata of this snapshot can record (L1, T1) and record that L1 corresponds to P1.

[0059] According to an embodiment of this application, updating the metadata of a target snapshot based on the metadata of other snapshots in a snapshot group besides the target snapshot includes: determining, from the logical addresses of at least one logical storage block, a logical address not recorded in the metadata of the target snapshot, as the target logical address; sequentially traversing the metadata of other snapshots in order of snapshot creation time from latest to earliest, until finding the metadata to be processed that includes the target logical address; determining the target physical block address corresponding to the target logical address from the metadata to be processed; and updating the metadata of the target snapshot based on the target logical address and the target physical block address.

[0060] For example, when updating the metadata of a target snapshot, if data is directly read from one snapshot and then written to another snapshot, for example... Figure 2As shown, reading data B1 from snapshot 2 and writing it to snapshot 3 would take a lot of time, resulting in low update efficiency. Moreover, the additional I / O generated by the read and write operations would also preempt the host I / O resources on the target storage space, affecting the host I / O performance. Therefore, the target snapshot can be updated by updating the target snapshot metadata.

[0061] It can traverse the logical addresses of all logical storage blocks to determine the logical addresses that are not recorded in the metadata of the target snapshot.

[0062] Figure 3 This is a schematic diagram of the metadata of a snapshot provided in an embodiment of this application.

[0063] like Figure 3 As shown, L1~L4 are logical addresses, T1~T3 are snapshot creation times, P1~P7 are physical block addresses, and A1, A2, A3, B1, B2, C1, and C2 are the actual data content stored on the physical blocks.

[0064] For snapshot 1, at time T1, L1 points to data A1, L2 points to data A2, and L3 points to data A3, where data A1 is stored at position P1, data A2 is stored at position P2, and data A3 is stored at position P3. Correspondingly, for snapshot 2, at time T2, L1 points to data B1, and L3 points to data B2, where data B1 is stored at position P4, and data B2 is stored at position P5. For snapshot 3, at time T3, L3 points to data C1, and L4 points to data C2, where data C1 is stored at position P6, and data C2 is stored at position P7.

[0065] Snapshot 3 is the latest snapshot and therefore the target snapshot. We can sequentially query L1 to L4 from the metadata of the target snapshot to determine logical addresses without records, including L1 and L2. For example, if querying metadata (L1, T3) from snapshot 3 fails (the metadata does not exist), we can further query (L1, T2). If the query succeeds, we find that it points to P4, and then directly modify the metadata information "(L1, T2), points to P4" to "(L1, T3), points to P4". Next, we can query metadata (L2, T3). If the query fails, we can further query (L2, T2), and if that also fails, we can query (L2, T1). If the query succeeds, we can modify the metadata information "(L2, T1), points to PBA2" to "(L2, T3) points to P2".

[0066] Figure 4 This is a schematic diagram of the modified metadata provided in an embodiment of this application.

[0067] like Figure 4As shown, the information about L2 in the metadata of snapshot 1 and the information about L1 in the metadata of snapshot 2 are merged into snapshot 3.

[0068] For example, you can further query the metadata (L3,T3) and (L4,T3). If the query is successful, it means that the data of these two logical addresses are recorded in the metadata of snapshot 3, so no processing is required.

[0069] Furthermore, queries can be made for (L5,T3), (L2,T2), (L2,T1), etc. If the query fails, it indicates that the data at the above logical addresses has never been modified during the above time period, so the above logical addresses are not processed.

[0070] Figure 5 This is a schematic diagram of the metadata for the updated snapshot provided in an embodiment of this application.

[0071] like Figure 5 As shown, the metadata of the updated snapshot already includes information about L1 and L2. Accordingly, snapshots 1 and 2 can be deleted to free up the space occupied by snapshots 1 and 2.

[0072] According to an embodiment of this application, by determining the unrecorded logical address in the metadata of the target snapshot from the logical addresses of at least one logical storage block, and using it as the target logical address; sequentially traversing the metadata of other snapshots until finding the metadata to be processed that includes the target logical address; determining the target physical block address corresponding to the target logical address, and updating the metadata of the target snapshot based on the target logical address and the target physical block address, reasonable sparsification processing can be performed on time-density created snapshots, thereby preserving both the freed space and some historical snapshot data from relatively long periods of time, thus improving the data recovery capability.

[0073] According to an embodiment of this application, updating the metadata of a target snapshot based on the target logical address and the target physical block address includes adding the target logical address and the target physical block address to the metadata of the target snapshot.

[0074] contrast Figure 3 and Figure 4 It can be seen that before the update, the metadata of snapshot 3 only records L3 and L4 related information. After the update, the metadata of snapshot 3 adds L1 and L2 related information. Specifically, for example, the new records include the two target logical addresses L1 and L2, as well as the target physical block address P4 corresponding to L1 and the target physical block address P2 corresponding to L2.

[0075] According to the embodiments of this application, by adding the target logical address and the target physical block address to the metadata of the target snapshot, the space occupied by other snapshots can be released, and the valid content recorded in the metadata of other snapshots can be migrated to the metadata of the target snapshot, so that subsequent data processing operations such as data recovery can be performed based on the metadata recorded in other snapshots.

[0076] According to an embodiment of this application, determining a target snapshot from a snapshot group based on a time difference includes: determining the snapshot creation time of each snapshot in the snapshot group; and determining the snapshot in the snapshot group whose snapshot creation time is later than the creation time of other snapshots as the target snapshot.

[0077] For example, snapshots whose creation time is later than other snapshots can be the latest snapshot created in the snapshot group.

[0078] For example, snapshots created between 00:00:01 and 01:00:00 can be grouped together. The latest snapshot within that group can be retained, meaning the snapshot and data at 01:00:00 need to be kept. For other snapshots at times such as 00:59:55, 0:59:50, ..., 0:00:05, the snapshot at 01:00:00 needs to be updated based on the data from those other snapshots, and other snapshots can be deleted.

[0079] According to the embodiments of this application, by determining the snapshot whose creation time is later than that of other snapshots as the target snapshot, users can select the corresponding snapshot based solely on the time of the business transaction when performing subsequent data processing, thereby reducing the operational difficulty of subsequent data processing.

[0080] According to an embodiment of this application, the snapshot to be processed includes snapshots created periodically by the storage system corresponding to the target storage space.

[0081] For example, a snapshot set corresponding to the target storage space can be obtained. The snapshot set can include various types of snapshots, such as snapshots created manually by the user, snapshots triggered by business software, and snapshots created periodically by the storage system corresponding to the target storage space. The snapshot created periodically by the storage system corresponding to the target storage space can be determined from the snapshot set to obtain the snapshot to be processed.

[0082] Snapshots created for a target storage space can come in various types, including user-created snapshots, snapshots triggered by business software, and snapshots created periodically by the storage system corresponding to the target storage space. User-created and business software-triggered snapshots typically have specific purposes. For example, snapshots created manually at critical business nodes such as financial settlement, quarterly report generation, and year-end data processing are used for subsequent auditing or retrospective analysis. Whether or not they need to be processed is determined by the user or business software based on actual needs. Therefore, the storage system can decide independently to process only snapshots created periodically by the storage system corresponding to the target storage space.

[0083] By processing only snapshots created periodically by the storage system corresponding to the target storage space, accidental deletion of snapshots with special purposes can be avoided, ensuring effective snapshot management.

[0084] According to an embodiment of this application, the snapshots created periodically by the storage system are created periodically according to a second time interval, wherein the second time interval is shorter than the first time interval.

[0085] For example, CDP snapshots can execute a second-level time interval strategy, such as generating a snapshot every 5 seconds, with a corresponding second time interval of 5 seconds. The values ​​of the first and second time intervals can be set according to actual needs.

[0086] The first time interval can be longer than the second time interval. For example, the first time interval can be set to 1 hour. The system can start a timer with a duration of 1 hour, which will be triggered after 1 hour to perform relevant snapshot processing operations.

[0087] By repeatedly executing the target snapshot set for the target storage space according to the first time interval, and using the target snapshot set to process the data stored in the target storage space, the storage system periodically creates snapshots according to the second time interval, which is shorter than the first time interval. This decouples the creation and processing of snapshots. High-frequency snapshot creation ensures data granularity, while processing snapshots at longer time intervals avoids the impact of frequent processing on the system, thereby saving unnecessary computational overhead.

[0088] According to embodiments of this application, the predetermined data processing operation includes at least one of the following: data recovery, data backup, data migration, data cloning, and data comparison.

[0089] For example, data recovery can be performed on a target storage space using a target snapshot set. For instance, data in the target storage space can be rolled back to the state at a certain historical snapshot point. Alternatively, snapshot data in the target snapshot set can be copied to other storage spaces to achieve data backup. Or, snapshot data in the target snapshot set can be moved from the target storage space to another storage space to achieve data migration. Alternatively, a readable and writable independent data copy can be created based on the target snapshot set to achieve data cloning. Furthermore, data differences between snapshots at two different points in time can be compared to identify the changed data and achieve data comparison.

[0090] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0091] Embodiments of this application also provide a snapshot processing apparatus, comprising: an acquisition module for acquiring snapshots to be processed for a target storage space; a first determination module for determining at least one snapshot from the snapshots to be processed where the time difference between the snapshot creation time and the snapshot acquisition time is greater than a first threshold, and dividing the at least one snapshot into a snapshot group according to a predetermined time window; a second determination module for determining a target snapshot from the snapshot group according to the time difference for each snapshot group, and updating the target snapshot based on the data of other snapshots in the snapshot group excluding the target snapshot, to obtain an updated snapshot for each snapshot group; and an acquisition module for obtaining a target snapshot set using the updated snapshots and snapshots from the snapshots to be processed where the time difference is less than or equal to the first threshold.

[0092] The first determining module includes a partitioning submodule, which is used to divide the first snapshot into at least one first snapshot group according to the snapshot creation time and the second snapshot into at least one second snapshot group according to the second time window, wherein the width of the first time window is smaller than the width of the second time window.

[0093] The second determining module includes an updating submodule, which is used to determine, from the logical addresses of at least one logical storage block, the logical address not recorded in the metadata of the target snapshot, as the target logical address; sequentially traverse the metadata of other snapshots in order of snapshot creation time from latest to earliest, until the metadata to be processed containing the target logical address is found; determine the target physical block address corresponding to the target logical address from the metadata to be processed; and update the metadata of the target snapshot based on the target logical address and the target physical block address.

[0094] The update submodule includes an update unit, which is used to add the target logical address and the target physical block address to the metadata of the target snapshot.

[0095] The second determination module includes a target snapshot determination submodule, which is used to determine the snapshot creation time of each snapshot in the snapshot group; and to determine the snapshot in the snapshot group whose snapshot creation time is later than the creation time of other snapshots as the target snapshot.

[0096] For a description of the features in the embodiment corresponding to the snapshot processing device, please refer to the relevant description in the embodiment corresponding to the snapshot processing method, which will not be repeated here.

[0097] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the snapshot processing method embodiments described above.

[0098] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the snapshot processing method embodiments described above when running.

[0099] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0100] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the snapshot processing method embodiments described above.

[0101] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the snapshot processing method embodiments described above.

[0102] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0103] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0104] The snapshot processing method provided in this application has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A snapshot processing method, characterized in that, include: Obtain a snapshot to be processed for the target storage space; From the snapshots to be processed, determine at least one snapshot whose time difference between snapshot creation time and snapshot acquisition time is greater than a first threshold, and divide the at least one snapshot into one or more snapshot groups according to a predetermined time window; For each of the snapshot groups, a target snapshot is determined from the snapshot group according to the time difference, and the metadata of the target snapshot is updated based on the metadata of the other snapshots in the snapshot group excluding the target snapshot, to obtain the updated snapshot of each of the snapshot groups; By using the updated snapshots and the snapshots in the pending snapshots whose time difference is less than or equal to the first threshold, a target snapshot set is obtained, wherein the target snapshot set is used to perform predetermined data processing operations on the data stored in the target storage space.

2. The snapshot processing method according to claim 1, characterized in that, The predetermined time window includes a first time window and a second time window, the snapshot group includes a first snapshot group and a second snapshot group, and the at least one snapshot includes: a first snapshot whose time difference is greater than the first threshold and less than the second threshold, and a second snapshot whose time difference is greater than or equal to the second threshold, wherein the second threshold is greater than the first threshold. The step of dividing the at least one snapshot into one or more snapshot groups according to a predetermined time window includes: The first snapshot is divided into at least one first snapshot group according to the first time window; The second snapshot is divided into at least one second snapshot group according to the second time window, wherein the width of the first time window is smaller than the width of the second time window.

3. The snapshot processing method according to claim 1, characterized in that, The target storage space is divided into at least one logical storage block; Each snapshot in the snapshot group is configured with metadata, which is used to record: the logical address of at least one logical storage block in which a write operation or modification operation occurred during the time interval between the snapshot creation time of the snapshot and the previous snapshot creation time, the physical block address corresponding to the logical address, and the snapshot creation time of the snapshot.

4. The snapshot processing method according to claim 3, characterized in that, Updating the metadata of the target snapshot based on the metadata of other snapshots in the snapshot group besides the target snapshot includes: From the logical addresses of the at least one logical storage block, determine the logical address that is not recorded in the metadata of the target snapshot, and use it as the target logical address; Following the order of the snapshot creation time from latest to earliest, the metadata of the other snapshots are traversed sequentially until the metadata to be processed, including the target logical address, is found. Determine the target physical block address corresponding to the target logical address from the metadata to be processed; Update the metadata of the target snapshot based on the target logical address and the target physical block address.

5. The snapshot processing method according to claim 4, characterized in that, The metadata update of the target snapshot based on the target logical address and the target physical block address includes: Add the target logical address and the target physical block address to the metadata of the target snapshot.

6. The snapshot processing method according to claim 1, characterized in that, The step of determining the target snapshot from the snapshot group based on the time difference includes: Determine the snapshot creation time of each snapshot in the snapshot group; The snapshot in the snapshot group whose creation time is later than that of other snapshots is identified as the target snapshot.

7. The snapshot processing method according to claim 1, characterized in that, The snapshots to be processed include snapshots created periodically by the storage system corresponding to the target storage space.

8. The snapshot processing method according to claim 2, characterized in that, The snapshots created by the storage system at regular intervals are created periodically according to a second time interval, wherein the second time interval is shorter than the first time interval.

9. The snapshot processing method according to claim 1, characterized in that, The predetermined data processing operations include at least one of the following: data recovery, data backup, data migration, data cloning, and data comparison.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the snapshot processing method as described in any one of claims 1 to 9.

Images