Snapshot rollback method and device, electronic equipment and computer readable storage medium

Abstract

The invention discloses a snapshot rollback method and device, electronic equipment and a computer readable storage medium. The method comprises the steps that a target index is determined according to a snapshot rollback task; wherein the target index comprises a target snapshot index and a target data block index; querying a global index file by using the target snapshot index, determining target snapshot position information, and obtaining a target snapshot file according to the target snapshot position information; querying a local index file corresponding to the target snapshot file by using the target data block index, determining position information of a target data block, and obtaining the target data block according to the position information of the target data block; and carrying out snapshot rollback on a target file corresponding to the snapshot rollback task by utilizing the target data block to obtain a rollback file. By applying the technical scheme provided by the invention, the snapshot rollback efficiency can be effectively improved, and the availability of the storage system is further improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a snapshot rollback method, and also to a snapshot rollback device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Snapshot technology refers to recording the system status at a specific point in time for later recovery or viewing, providing important support for data recovery and version management. This technology is particularly important in storage systems because it allows users to create data copies without affecting the normal operation of the system to support data recovery, backup, and version control. During the snapshot rollback process, the effective management of data block index data has a direct impact on the snapshot rollback efficiency. However, traditional snapshot rollback technology requires searching for rollback data blocks in massive data to complete the snapshot rollback. Obviously, this is time-consuming and inefficient.

[0003] Therefore, how to effectively improve the efficiency of snapshot rollback and improve the availability of the storage system is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0004] The purpose of the present application is to provide a snapshot rollback method, which can effectively improve the efficiency of snapshot rollback and further improve the availability of the storage system; another purpose of the present application is to provide a snapshot rollback device, an electronic device, a computer-readable storage medium and a computer program product, all of which have the above-mentioned beneficial effects.

[0005] In a first aspect, the present application provides a snapshot rollback method, comprising:

[0006] Determine a target index according to the snapshot rollback task; wherein the target index includes a target snapshot index and a target data block index;

[0007] Using the target snapshot index to query the global index file, determine the target snapshot location information, and obtain the target snapshot file according to the target snapshot location information;

[0008] Using the target data block index to query the local index file corresponding to the target snapshot file, determine the target data block location information, and obtain the target data block according to the target data block location information;

[0009] The target data block is used to perform a snapshot rollback on the target file corresponding to the snapshot rollback task to obtain a rollback file.

[0010] Optionally, performing snapshot rollback on a target file corresponding to the snapshot rollback task using the target data block to obtain a rollback file includes:

[0011] Using the target data block to divide the snapshot rollback task into tasks, to obtain snapshot rollback subtasks;

[0012] Distribute each snapshot rollback subtask to each cluster node in the distributed system, so that each cluster node executes a corresponding snapshot rollback subtask, implements the snapshot rollback of the target file, and obtains the rollback file.

[0013] Optionally, distributing each snapshot rollback subtask to each cluster node in the distributed system so that each cluster node executes a corresponding snapshot rollback subtask to implement snapshot rollback of the target file and obtain the rollback file includes:

[0014] Performing load information statistics on each of the cluster nodes to obtain first load distribution information of the distributed system;

[0015] Distribute each snapshot rollback subtask to each cluster node according to the first load distribution information, so that each cluster node executes a corresponding snapshot rollback subtask, implements the snapshot rollback of the target file, and obtains the rollback file.

[0016] Optionally, the snapshot rollback method further includes:

[0017] Performing load information statistics on each of the cluster nodes at a preset time interval to obtain second load distribution information of the distributed system;

[0018] Determine a high-load node and a low-load node according to the second load distribution information;

[0019] The snapshot rollback subtask on the high-load node is scheduled to the low-load node to achieve load balancing.

[0020] Optionally, performing snapshot rollback on a target file corresponding to the snapshot rollback task using the target data block to obtain a rollback file includes:

[0021] Performing hash calculation on the target data block using a preset hash algorithm to obtain a hash check value;

[0022] Querying in the snapshot verification file to obtain the verification standard value corresponding to the target data block;

[0023] Determine whether the verification standard value is consistent with the hash verification value;

[0024] If the verification standard value is consistent with the hash verification value, performing snapshot rollback on the target file corresponding to the snapshot rollback task using the target data block to obtain the rollback file;

[0025] If the verification standard value is inconsistent with the hash verification value, a snapshot rollback failure prompt is output;

[0026] Accordingly, the method further comprises:

[0027] When the target file is obtained by performing a file snapshot operation on the original file, a hash calculation is performed on each data block in the target file using the preset hash algorithm to obtain a verification standard value of each data block;

[0028] A snapshot verification file corresponding to the target file is generated according to the verification standard value of each of the data blocks.

[0029] Optionally, when performing a file snapshot operation on an original file to obtain the target file, performing a hash calculation on each data block in the target file using the preset hash algorithm to obtain a checksum standard value of each data block includes:

[0030] Determine the changed data blocks in the original file;

[0031] Executing the file snapshot operation on each of the changed data blocks to obtain the target file;

[0032] The preset hash algorithm is used to perform hash calculation on each of the changed data blocks in the target file to obtain a verification standard value of each of the changed data blocks.

[0033] Optionally, the snapshot rollback method further includes:

[0034] Determine the predicted rollback data block based on the historical snapshot rollback log;

[0035] Migrating the predicted rollback data block to a preset storage medium;

[0036] Accordingly, obtaining the target data block according to the target data block position information includes:

[0037] Determining whether the target data block hits the preset storage medium;

[0038] If the target data block hits the preset storage medium, the target data block is read from the preset storage medium;

[0039] If the target data block does not hit the preset storage medium, the target data block is obtained according to the target data block position information.

[0040] In a second aspect, the present application further discloses a snapshot rollback device, comprising:

[0041] A determination module, used to determine a target index according to the snapshot rollback task; wherein the target index includes a target snapshot index and a target data block index;

[0042] A first acquisition module, configured to query a global index file using the target snapshot index, determine target snapshot location information, and obtain a target snapshot file according to the target snapshot location information;

[0043] A second acquisition module is used to query the local index file corresponding to the target snapshot file by using the target data block index to determine the target data block location information, and obtain the target data block according to the target data block location information;

[0044] The rollback module is used to perform a snapshot rollback on the target file corresponding to the snapshot rollback task using the target data block to obtain a rollback file.

[0045] In a third aspect, the present application further discloses an electronic device, comprising:

[0046] Memory for storing computer programs;

[0047] A processor is used to implement the steps of any one of the snapshot rollback methods described above when executing the computer program.

[0048] In a fourth aspect, the present application further discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any one of the snapshot rollback methods described above are implemented.

[0049] In a fifth aspect, the present invention further discloses a computer program product, including a computer program / instruction, which implements the steps of any one of the snapshot rollback methods described above when executed by a processor.

[0050] The present application provides a snapshot rollback method, comprising: determining a target index according to a snapshot rollback task; wherein the target index includes a target snapshot index and a target data block index; querying a global index file using the target snapshot index to determine target snapshot location information, and obtaining a target snapshot file according to the target snapshot location information; querying a local index file corresponding to the target snapshot file using the target data block index to determine target data block location information, and obtaining a target data block according to the target data block location information; and performing a snapshot rollback on a target file corresponding to the snapshot rollback task using the target data block to obtain a rollback file.

[0051] By applying the technical solution provided by the present application, during the snapshot generation process, the index information of each data block of each snapshot version is divided into a snapshot index and a data block index, wherein the snapshot index is stored in a global index file for implementing the query of the snapshot version, and the data block index is stored in local index files corresponding to different snapshot versions for implementing the query of the data blocks to be rolled back in a specific snapshot version. That is to say, the global index file records the status information within the system, and the local index file records the changes to specific files. Thus, effective hierarchical management of the snapshot index is achieved, which helps to quickly and efficiently locate the target data block of the target snapshot file, thereby realizing fast and efficient snapshot rollback, and effectively improving the availability of the storage system.

[0052] In one embodiment of the present application, for a snapshot rollback task, the target data block that needs to be snapshot rolled back can be used to divide the task into multiple snapshot rollback subtasks, which are then distributed to each cluster node in the distributed system to achieve parallel processing of multiple snapshot rollback subtasks. This can effectively improve the processing speed of the overall snapshot rollback task and further improve the snapshot rollback efficiency.

[0053] The snapshot rollback device, electronic device, computer-readable storage medium, and computer program product provided in this application also have the above-mentioned technical effects, and this application will not elaborate on them here. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the drawings required for use in the description of the prior art and the embodiments of the present application are briefly introduced below. Of course, the drawings described below in relation to the embodiments of the present application are only part of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work, and the obtained other drawings also belong to the protection scope of the present application.

[0055] Figure 1 A flowchart of a snapshot rollback method provided in this application;

[0056] Figure 2 A schematic diagram of an incremental rollback strategy provided by this application;

[0057] Figure 3 A schematic diagram of an optimization and acceleration strategy for index data provided in this application;

[0058] Figure 4 A schematic diagram of a data verification mechanism provided by this application;

[0059] Figure 5 A schematic diagram of a snapshot rollback performance improvement strategy provided in this application;

[0060] Figure 6 A schematic diagram of the structure of a snapshot rollback device provided by the present application;

[0061] Figure 7 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0062] The core of the present application is to provide a snapshot rollback method, which can effectively improve the efficiency of snapshot rollback and further improve the availability of the storage system; another core of the present application is to provide a snapshot rollback device, electronic device, computer-readable storage medium and computer program product, all of which have the above-mentioned beneficial effects.

[0063] In order to describe the technical solutions in the embodiments of the present application more clearly and completely, the technical solutions in the embodiments of the present application will be introduced below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0064] An embodiment of the present application provides a snapshot rollback method.

[0065] Please refer to Figure 1 , Figure 1 This is a flowchart of a snapshot rollback method provided by the present application. The snapshot rollback method may include the following S101 to S104.

[0066] S101: Determine a target index according to a snapshot rollback task; wherein the target index includes a target snapshot index and a target data block index.

[0067] This step aims to determine the target index, which can be achieved based on the currently received snapshot rollback task, that is, by parsing the snapshot rollback task to obtain the target index. The target index may include a target snapshot index and a target data block index, wherein the target snapshot index is used to index and query the snapshot version to be rolled back, that is, the target snapshot file described below; the target data block index is used to index and query the data block to be rolled back in the target snapshot file, that is, the target data block described below.

[0068] S102: querying the global index file using the target snapshot index to determine the target snapshot location information, and obtaining the target snapshot file according to the target snapshot location information.

[0069] This step is intended to determine the target snapshot file. As mentioned above, the target snapshot index is used to index and query the snapshot version that needs to be rolled back. Therefore, the target snapshot index can be used to query the global index file to obtain the location information of the target snapshot file, that is, the above-mentioned target snapshot location information, so as to obtain the target snapshot file according to the target snapshot location information. It can be understood that the global index file is used to store the correspondence between the index information (snapshot index) and the location information of each version of the snapshot file.

[0070] In one embodiment of the present application, performing snapshot rollback on a target file corresponding to a snapshot rollback task using a target data block to obtain a rollback file may include:

[0071] Perform hash calculation on the target data block using a preset hash algorithm to obtain a hash check value;

[0072] Query the snapshot verification file to obtain the verification standard value corresponding to the target data block;

[0073] Determine whether the verification standard value is consistent with the hash verification value;

[0074] If the checksum value is consistent with the hash checksum value, the target data block is used to perform snapshot rollback on the target file corresponding to the snapshot rollback task to obtain the rollback file;

[0075] If the verification standard value is inconsistent with the hash verification value, a prompt indicating that the snapshot rollback failed will be output;

[0076] Accordingly, the method further comprises:

[0077] When the file snapshot operation is performed on the original file to obtain the target file, a hash calculation is performed on each data block in the target file using a preset hash algorithm to obtain a checksum standard value for each data block;

[0078] Generate a snapshot verification file corresponding to the target file according to the verification standard value of each data block.

[0079] In order to effectively ensure the accuracy of snapshot rollback, the embodiment of the present application provides a data block verification method. Specifically, in the process of performing a file snapshot operation on the original file to obtain a target file, a hash calculation can be performed on each data block in the target file to obtain a corresponding verification standard value, and the corresponding data block information can be saved in a preset file to generate a snapshot verification file, that is, the snapshot verification file includes the corresponding relationship between the data block information (such as address information and other identification information) of each data block in the target file and the verification standard value. Therefore, in the snapshot rollback process, for the target data block that needs to be rolled back, a hash calculation can be performed on it to obtain a corresponding hash verification value, and the verification standard value corresponding to the target data block can be determined in the snapshot verification file corresponding to the target snapshot file to which the target data block belongs, and the hash verification value and the verification standard value are compared for consistency. If the two are the same, the snapshot rollback operation can be continued; otherwise, if the two are not the same, it means that the target data block is abnormal, and the snapshot rollback operation cannot be continued at this time, and a snapshot rollback failure prompt can be output.

[0080] It can be seen that in the embodiment of the present application, performing hash verification on the data block to be rolled back during the snapshot rollback process can effectively ensure the integrity and consistency of the snapshot data, that is, effectively ensure the accuracy of the snapshot rollback.

[0081] When the file snapshot operation is performed on the original file to obtain the target file, a hash calculation is performed on each data block in the target file using a preset hash algorithm to obtain a checksum standard value of each data block, which may include:

[0082] Determine the changed data blocks in the original file;

[0083] Perform a file snapshot operation on each changed data block to obtain a target file;

[0084] A preset hash algorithm is used to perform hash calculation on each changed data block in the target file to obtain a verification standard value for each changed data block.

[0085] The embodiment of the present application provides an incremental snapshot solution to effectively ensure the working efficiency of snapshots and snapshot rollbacks. Specifically, in the process of performing a file snapshot operation on an original file to obtain a target file, each changed data block in the original file, that is, the changed data block, can be determined, and the target file of the file snapshot operation is performed only on this part of the data blocks. In other words, the snapshot operation is performed only on the data blocks in the original file where the data has changed, and the snapshot operation is not performed on the data blocks where the data has not changed. Then, in the snapshot rollback process, it is only necessary to perform a snapshot rollback on the data blocks that have changed, and there is no need to perform a snapshot rollback on the data blocks that have not changed. In this way, the efficiency of file snapshots and snapshot rollbacks can be effectively guaranteed.

[0086] S103: querying the local index file corresponding to the target snapshot file using the target data block index to determine the target data block location information, and obtaining the target data block according to the target data block location information.

[0087] This step is intended to achieve the determination of the target data block. As mentioned above, the target data block index is used to index and query the data block that needs to be rolled back in the target snapshot file. Therefore, the target data block index can be used to query the local index file corresponding to the target snapshot file to obtain the location information of the target data block, that is, the above-mentioned target data block location information, so as to obtain the target data block according to the target data block location information. It can be understood that the local index file is used to store the correspondence between the index information (data block index) and the location information of each data block in the corresponding version snapshot file. It should be noted that there are multiple local index files, and one snapshot version corresponds to one local index file, that is, the number of snapshot versions is the number of local index files.

[0088] In one embodiment of the present application, the snapshot rollback method may further include:

[0089] Determine the predicted rollback data block based on the historical snapshot rollback log;

[0090] Migrating the predicted rollback data blocks to a preset storage medium;

[0091] Accordingly, the above-mentioned obtaining the target data block according to the target data block position information may include:

[0092] Determine whether the target data block hits the preset storage medium;

[0093] If the target data block hits the preset storage medium, the target data block is read from the preset storage medium;

[0094] If the target data block does not hit the preset storage medium, the target data block is obtained according to the target data block position information.

[0095] The embodiment of the present application provides a method for obtaining a target data block, which can achieve fast and efficient acquisition of the target data block, and further improve the efficiency of snapshot rollback. Specifically, the historical snapshot rollback log can be used to estimate the data block, and the data block that may need to be subsequently rolled back by the snapshot, that is, the above-mentioned predicted rollback data block, is obtained, and it is migrated to a preset storage medium in advance for temporary storage. On this basis, when obtaining the target data block based on the target data block location information, it can be determined first whether the target data block hits the preset storage medium. If it hits the preset storage medium, the target data block can be directly read from the preset storage medium. On the contrary, if it fails to hit the preset storage medium, the target data block can be directly read from the underlying storage according to the target data block location information.

[0096] Of course, the specific type of the preset storage medium does not affect the implementation of this technical solution, and can be set by the technician according to actual needs, and the embodiments of this application do not limit this. In order to effectively ensure the efficiency of snapshot rollback, in one possible implementation, the preset storage medium can be a high-performance storage medium, such as NVMe SSD (Non-Volatile Memory Express, a new type of solid-state drive).

[0097] S104: Performing snapshot rollback on the target file corresponding to the snapshot rollback task using the target data block to obtain a rollback file.

[0098] This step aims to implement snapshot rollback of the target file corresponding to the snapshot rollback task. The target file is the file that needs to be snapshot rolled back. It corresponds to the target snapshot file (the version file that needs to be snapshot rolled back). The former is the file before the snapshot rollback, and the latter is the file version that needs to be snapshot rolled back. After the snapshot rollback operation is completed on the target file, the rollback file can be obtained. Among them, the target data block is used to perform a snapshot rollback on the target file corresponding to the snapshot rollback task, which is essentially to use the target data block to replace the corresponding data block in the target file.

[0099] It can be seen that the snapshot rollback method provided in the embodiment of the present application divides the index information of each data block of each snapshot version into a snapshot index and a data block index during the snapshot generation process, wherein the snapshot index is stored in a global index file for implementing the query of the snapshot version, and the data block index is stored in local index files corresponding to different snapshot versions for implementing the query of the data blocks to be rolled back in a specific snapshot version. That is to say, the global index file records the status information within the system, and the local index file records the changes to the specific files. Thus, effective hierarchical management of the snapshot index is implemented, which helps to quickly and efficiently locate the target data blocks of the target snapshot file, thereby implementing fast and efficient snapshot rollback, and effectively improving the availability of the storage system.

[0100] Based on the above embodiments:

[0101] In one embodiment of the present application, performing snapshot rollback on a target file corresponding to a snapshot rollback task using a target data block to obtain a rollback file may include:

[0102] Use the target data block to divide the snapshot rollback task into tasks to obtain each snapshot rollback subtask;

[0103] Each snapshot rollback subtask is distributed to each cluster node in the distributed system, so that each cluster node executes the corresponding snapshot rollback subtask, realizes the snapshot rollback of the target file, and obtains the rollback file.

[0104] Specifically, in the process of using the target data block for snapshot rollback, the snapshot rollback task can be divided according to the target data block to obtain multiple snapshot rollback subtasks. For example, each target data block can be used as a snapshot rollback subtask, or a small number of target data blocks can be used as a snapshot rollback subtask and distributed to each cluster node in the distributed system. Multiple cluster nodes execute multiple snapshot rollback subtasks in parallel, that is, snapshot rollback of multiple target data blocks is achieved at the same time.

[0105] It can be seen that in the embodiment of the present application, for the snapshot rollback task, the target data block that needs to be snapshot rolled back can be used to divide the task, obtain multiple snapshot rollback subtasks, and then distribute them to each cluster node in the distributed system to realize parallel processing of multiple snapshot rollback subtasks. As a result, the processing speed of the overall snapshot rollback task can be effectively improved, and the snapshot rollback efficiency can be further improved.

[0106] Distributing each snapshot rollback subtask to each cluster node in the distributed system so that each cluster node executes the corresponding snapshot rollback subtask to implement snapshot rollback of the target file and obtain the rollback file may include:

[0107] Performing load information statistics on each cluster node to obtain first load distribution information of the distributed system;

[0108] Distribute each snapshot rollback subtask to each cluster node according to the first load distribution information, so that each cluster node executes the corresponding snapshot rollback subtask, implements the snapshot rollback of the target file, and obtains the rollback file.

[0109] On the basis of executing snapshot rollback operations in parallel on multiple cluster nodes, in order to effectively ensure the stability of the distributed system, in the process of distributing snapshot rollback subtasks, the load information of each cluster node in the distributed system can be first counted (for example, including but not limited to computing resources, storage resources, network resources, etc.), the load situation of each cluster node can be obtained, and the first load distribution information of the distributed system can be generated, thereby, the distribution of each snapshot rollback subtask can be realized with reference to the first load distribution information. For example, a larger number of snapshot rollback subtasks can be allocated to high-load cluster nodes, and a smaller number of snapshot rollback subtasks can be allocated to low-load cluster nodes, so as to effectively ensure the load balance of each cluster node in the distributed system, and further ensure the stable operation of the distributed system.

[0110] Furthermore, the snapshot rollback method may further include:

[0111] Performing load information statistics on each cluster node at a preset time interval to obtain second load distribution information of the distributed system;

[0112] determining a high-load node and a low-load node according to the second load distribution information;

[0113] Schedule the snapshot rollback subtask on the high-load node to the low-load node to achieve load balancing.

[0114] In order to further ensure the load balancing of each cluster node in the distributed system, a timed load balancing operation can be performed in the distributed system. Specifically, the load information of each cluster node in the distributed system can be counted according to a preset time interval to obtain the second load distribution information of the distributed system, so that the second load distribution information can be used to perform a load balancing operation, that is, to partially or completely schedule the snapshot rollback subtask on the high-load cluster node to the low-load cluster node, so as to achieve load balancing of each cluster node in the distributed system, and further ensure the stable operation of the distributed system. It can be understood that the specific value of the above-mentioned preset time interval does not affect the implementation of the present technical solution, and can be set by technical personnel according to actual needs, and this application does not limit this.

[0115] In one embodiment of the present application, the snapshot rollback method may further include: according to the historical snapshot rollback log, sorting the snapshot indexes in the global index file in order of access frequency from high to low, and sorting the data block indexes in the local index files in order of access frequency from high to low.

[0116] The snapshot rollback method provided by the embodiment of the present application can further improve the snapshot rollback efficiency. Specifically, by referring to the historical snapshot rollback log, the access frequency of each data block in each snapshot file can be obtained. Obviously, the higher the access frequency, the higher the frequency of the snapshot rollback operation being executed; the lower the access frequency, the lower the frequency of the snapshot rollback operation being executed. Thus, each index information in the global index file and each local index file can be sorted in descending order according to the access frequency of the corresponding data block, ensuring that the index information of the data block that is more likely to be subjected to the snapshot rollback operation is sorted higher, which helps to achieve faster and more efficient snapshot rollback.

[0117] Furthermore, each snapshot index in the global index file can be stored in the index structure of B-tree (a data storage structure) or LSM tree (another data storage structure), and each data block index in each local index file can also be stored in the index structure of B-tree or LSM tree. It can be understood that these structures support fast insertion, deletion and search operations, which helps to further improve the efficiency of snapshot rollback.

[0118] In addition, each snapshot index in the global index file and each data block index in each local index file can be stored in the form of compressed data, such as differential coding or bitmap indexing, so as to effectively reduce redundancy and save storage space.

[0119] Based on the above embodiments, an embodiment of the present application provides another snapshot rollback method.

[0120] The implementation process of a snapshot rollback method provided in this application is as follows:

[0121] 1. Incremental rollback strategy.

[0122] The problem with the traditional full rollback strategy is that it requires restoring the entire snapshot file, which results in a large number of data blocks needing to be read and written during the data recovery process, causing unnecessary waste of resources. Therefore, this application proposes an incremental rollback strategy that records the data differences between adjacent version snapshots and only restores the changed data blocks, thereby significantly reducing the amount of data blocks read and written. Figure 2 , Figure 2 The following is a schematic diagram of an incremental rollback strategy provided by this application, and its specific implementation method includes:

[0123] (1) Differential calculation and storage: Each time a snapshot file is generated, the new snapshot file is compared with the previous snapshot file, the difference data blocks are calculated and stored in the form of incremental files. This difference only contains the data blocks that are actually changed during the read and write operations.

[0124] (2) Incremental rollback algorithm: When rolling back a snapshot, the changed data blocks are restored based on the recorded incremental files, rather than the entire snapshot file. This process uses the difference information stored in the incremental files to effectively reduce I / O requests and improve the snapshot rollback speed.

[0125] 2. Optimization and acceleration strategies for index data.

[0126] In a distributed storage system, the management of index data (metadata) has a direct impact on the efficiency of snapshot rollback. Figure 3 , Figure 3 This is a schematic diagram of an index data optimization and acceleration strategy provided in this application, and its specific implementation method includes:

[0127] (1) Hierarchical management of index data: Index data is divided into two layers: global and local. Global index data records system-wide status information (such as the overall structure and version information of snapshot files. This information is usually static and updated infrequently). Local index records changes to specific files, which can effectively improve access speed (such as the modification time, size, data block location, etc. of snapshot files. This information is updated frequently and requires fast access).

[0128] Therefore, when accessing data during snapshot rollback, the global index data is used to determine the snapshot file version to be accessed, and then the local index data is used to quickly locate the specific file and data block. Obviously, this hierarchical management method reduces unnecessary global scanning and improves access speed.

[0129] (2) Efficient index structure: Establish an index structure based on B-tree or LSM tree, allowing the system to quickly query and locate data blocks that need to be rolled back. In response to rollback requirements, the sorting and compression methods of the index can be further optimized. For example, the sorting can be adjusted according to historical access conditions to make commonly used data blocks easier to access; compression technology can be used to reduce the storage space occupied by index data, such as differential encoding or bitmap indexing to effectively reduce redundancy.

[0130] 3. Data verification mechanism.

[0131] Please refer to Figure 4 , Figure 4 This is a schematic diagram of a data verification mechanism provided by this application, and its specific implementation method includes:

[0132] (1) Real-time data verification based on hash:

[0133] Data verification is the core technology to ensure the integrity and consistency of snapshot data. This application uses a hash algorithm for real-time verification. The specific methods include:

[0134] (a) Dynamic hash generation: When a data block changes, a hash value (verification standard value) of the data block is dynamically generated and stored in the corresponding snapshot data block. This process can be performed asynchronously after the data block change transaction is completed to reduce the impact on the front-end business.

[0135] (b) Fast verification mechanism: During the snapshot rollback process, fast verification is performed based on the pre-stored hash value (verification standard value), the integrity of the data block is detected, and the verification is completed through comparison. This method not only improves the verification efficiency, but also reduces the computational cost of the verification process.

[0136] (2) Decentralized verification node design:

[0137] Since the centralized verification structure is prone to single point failure, this application proposes a decentralized verification node solution:

[0138] (a) Verification task distribution: Verification tasks are distributed to multiple cluster nodes. In the decentralized verification node design, a snapshot rollback operation may involve the rollback of multiple data blocks. In the distributed underlying layer, file rollback is performed piece by piece in the form of data blocks, and each data block corresponds to a verification task. By distributing these verification tasks to multiple cluster nodes for parallel execution, the verification efficiency can be effectively improved, and single point failures and performance bottlenecks can be avoided.

[0139] (b) Dynamic node adjustment: Dynamically adjust the allocation of verification tasks according to the current load of each cluster node. Monitor the CPU, memory, network bandwidth and other resource usage of each cluster node in real time. This data can be collected through monitoring tools or custom scripts to form a real-time view of the node load. If the load of a cluster node is too high, part of its verification tasks can be transferred to cluster nodes with lower loads. In addition, distributed consistency algorithms (such as Paxos or Raft) can be used to maintain the correctness and state consistency of verification tasks.

[0140] 4. Snapshot rollback performance improvement strategy.

[0141] Please refer to Figure 5 , Figure 5 The schematic diagram of a snapshot rollback performance improvement strategy provided in this application includes:

[0142] (1) Parallel processing and load balancing:

[0143] To meet the large-scale data processing requirements during snapshot rollback, parallel processing and load balancing strategies can be used.

[0144] (a) Parallel task division: Divide the snapshot rollback task into fine-grained subtasks and distribute them to different cluster nodes for parallel execution to effectively improve the overall task processing speed. For details, please refer to the distribution plan of the verification task.

[0145] (b) Intelligent load balancing: Dynamically allocate subtasks based on the real-time load status of each cluster node to avoid affecting the overall performance due to overload of some cluster nodes. Specifically, the load prediction model can be used to achieve uniform distribution of subtasks and make full use of system resources.

[0146] (2) Intelligent data placement strategy:

[0147] (a) Access pattern analysis: By analyzing historical access logs, we predict the data blocks with the highest access volume in the future and identify the “hotspot” data blocks (especially the critical data blocks during rollback).

[0148] (b) Prioritize storage on high-performance media: Dynamically migrate hot data blocks to high-performance storage devices based on prediction results to speed up reading.

[0149] It can be seen that the snapshot rollback method provided in the embodiment of the present application effectively improves the work efficiency, reliability and performance of snapshot rollback through innovative technologies such as incremental rollback strategy, index data optimization and acceleration strategy, data verification mechanism, and snapshot rollback performance improvement strategy, and provides a strong guarantee for the stable operation and data recovery of the distributed file storage system.

[0150] An embodiment of the present application provides a snapshot rollback device.

[0151] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of a snapshot rollback device provided by the present application. The snapshot rollback device may include:

[0152] Determining module 1, used to determine the target index according to the snapshot rollback task; wherein the target index includes a target snapshot index and a target data block index;

[0153] The first acquisition module 2 is used to query the global index file using the target snapshot index, determine the target snapshot location information, and obtain the target snapshot file according to the target snapshot location information;

[0154] The second acquisition module 3 is used to query the local index file corresponding to the target snapshot file using the target data block index, determine the target data block location information, and obtain the target data block according to the target data block location information;

[0155] The rollback module 4 is used to perform a snapshot rollback on the target file corresponding to the snapshot rollback task using the target data block to obtain a rollback file.

[0156] It can be seen that the snapshot rollback device provided in the embodiment of the present application divides the index information of each data block of each snapshot version into a snapshot index and a data block index during the snapshot generation process, wherein the snapshot index is stored in a global index file for realizing the query of the snapshot version, and the data block index is stored in local index files corresponding to different snapshot versions for realizing the query of the data blocks to be rolled back in a specific snapshot version. That is to say, the global index file records the status information within the system, and the local index file records the changes of specific files. Thus, effective hierarchical management of the snapshot index is realized, which helps to quickly and efficiently locate the target data block of the target snapshot file, thereby realizing fast and efficient snapshot rollback, and effectively improving the availability of the storage system.

[0157] In one embodiment of the present application, the rollback module 4 may include:

[0158] A division unit, used for dividing the snapshot rollback task by using the target data block to obtain each snapshot rollback subtask;

[0159] The distribution unit is used to distribute each snapshot rollback subtask to each cluster node in the distributed system, so that each cluster node executes the corresponding snapshot rollback subtask, realizes the snapshot rollback of the target file, and obtains the rollback file.

[0160] In one embodiment of the present application, the above-mentioned distribution unit can be specifically used to perform load information statistics on each cluster node to obtain the first load distribution information of the distributed system; distribute each snapshot rollback subtask to each cluster node according to the first load distribution information, so that each cluster node executes the corresponding snapshot rollback subtask, realizes the snapshot rollback of the target file, and obtains the rollback file.

[0161] In one embodiment of the present application, the above-mentioned rollback module 4 may also include a balancing unit, which is used to perform load information statistics on each cluster node at a preset time interval to obtain second load distribution information of the distributed system; determine high-load nodes and low-load nodes according to the second load distribution information; and schedule the snapshot rollback subtask on the high-load node to the low-load node to achieve load balancing.

[0162] In one embodiment of the present application, the snapshot rollback device may further include a prediction module, which is used to determine a predicted rollback data block according to a historical snapshot rollback log; migrate the predicted rollback data block to a preset storage medium;

[0163] Correspondingly, the above-mentioned second acquisition module 3 can be specifically used to determine whether the target data block hits the preset storage medium; if the target data block hits the preset storage medium, the target data block is read from the preset storage medium; if the target data block does not hit the preset storage medium, the target data block is obtained according to the target data block position information.

[0164] In one embodiment of the present application, the rollback module 4 can be specifically used to perform hash calculation on the target data block using a preset hash algorithm to obtain a hash check value; query the snapshot check file to obtain a check standard value corresponding to the target data block; determine whether the check standard value is consistent with the hash check value; if the check standard value is consistent with the hash check value, use the target data block to perform a snapshot rollback on the target file corresponding to the snapshot rollback task to obtain a rollback file; if the check standard value is inconsistent with the hash check value, output a snapshot rollback failure prompt;

[0165] Correspondingly, the snapshot rollback device may also include a generation module, which is used to perform hash calculations on each data block in the target file using a preset hash algorithm when executing a file snapshot operation on the original file to obtain a target file, so as to obtain a verification standard value of each data block; and generate a snapshot verification file corresponding to the target file according to the verification standard value of each data block.

[0166] In one embodiment of the present application, the above-mentioned generation module can be specifically used to determine the changed data blocks in the original file; perform a file snapshot operation on each changed data block to obtain a target file; use a preset hash algorithm to perform hash calculation on each changed data block in the target file to obtain a verification standard value for each changed data block.

[0167] For an introduction to the device provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0168] An embodiment of the present application provides an electronic device.

[0169] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in this application, and the electronic device may include:

[0170] A memory 11, used for storing computer programs;

[0171] The processor 10 can implement the steps of any one of the above snapshot rollback methods when executing a computer program.

[0172] like Figure 7 , which is a schematic diagram of the composition structure of an electronic device, the electronic device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through the communication bus 13.

[0173] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0174] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in an embodiment of the snapshot rollback method.

[0175] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0176] Determine a target index according to the snapshot rollback task; wherein the target index includes a target snapshot index and a target data block index;

[0177] Using the target snapshot index to query the global index file, determine the target snapshot location information, and obtain the target snapshot file according to the target snapshot location information;

[0178] Using the target data block index to query the local index file corresponding to the target snapshot file, determine the target data block location information, and obtain the target data block according to the target data block location information;

[0179] The target data block is used to perform snapshot rollback on the target file corresponding to the snapshot rollback task to obtain a rollback file.

[0180] In a possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0181] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0182] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0183] Of course, it should be noted that Figure 7 The structure shown does not constitute a limitation on the electronic device in the embodiment of the present application. In actual applications, the electronic device may include Figure 7 More or fewer components than shown, or combinations of certain components.

[0184] An embodiment of the present application provides a computer-readable storage medium.

[0185] The computer-readable storage medium provided in the embodiment of the present application stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned snapshot rollback methods can be implemented.

[0186] Among them, the computer-readable storage medium can be any available medium that can be stored in a computer or a data storage device such as a server or data center that includes one or more available media integrated. For example, it can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape, etc.), an optical medium (such as a DVD) or a semiconductor medium (such as a solid-state hard disk) and other media that can store computer program codes.

[0187] For an introduction to the computer-readable storage medium provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0188] An embodiment of the present application provides a computer program product.

[0189] The computer program product provided in the embodiments of the present application includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of any one of the above-mentioned snapshot rollback methods can be implemented.

[0190] Specifically, in the above embodiments, all or part of them can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of them can be implemented in the form of a computer program product.

[0191] Among them, the computer program product may include one or more computer programs / instructions, and when the computer program / instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application may be generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line, etc.) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0192] For an introduction to the computer program product provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0193] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0194] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0195] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0196] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples herein, and the description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A snapshot rollback method, characterized in that: include: Determine a target index according to the snapshot rollback task; wherein the target index includes a target snapshot index and a target data block index; Using the target snapshot index to query the global index file, determine the target snapshot location information, and obtain the target snapshot file according to the target snapshot location information; Using the target data block index to query the local index file corresponding to the target snapshot file, determine the target data block location information, and obtain the target data block according to the target data block location information; The target data block is used to perform a snapshot rollback on the target file corresponding to the snapshot rollback task to obtain a rollback file.

2. The snapshot rollback method according to claim 1, characterized in that: Performing snapshot rollback on the target file corresponding to the snapshot rollback task by using the target data block to obtain the rollback file includes: Using the target data block to divide the snapshot rollback task into tasks, to obtain snapshot rollback subtasks; Distribute each snapshot rollback subtask to each cluster node in the distributed system, so that each cluster node executes a corresponding snapshot rollback subtask, implements the snapshot rollback of the target file, and obtains the rollback file.

3. The snapshot rollback method according to claim 2, characterized in that: Distributing each snapshot rollback subtask to each cluster node in the distributed system so that each cluster node executes the corresponding snapshot rollback subtask to implement the snapshot rollback of the target file and obtain the rollback file, including: Performing load information statistics on each of the cluster nodes to obtain first load distribution information of the distributed system; Distribute each snapshot rollback subtask to each cluster node according to the first load distribution information, so that each cluster node executes a corresponding snapshot rollback subtask, implements the snapshot rollback of the target file, and obtains the rollback file.

4. The snapshot rollback method according to claim 2, characterized in that: Also includes: Performing load information statistics on each of the cluster nodes at a preset time interval to obtain second load distribution information of the distributed system; Determine a high-load node and a low-load node according to the second load distribution information; The snapshot rollback subtask on the high-load node is scheduled to the low-load node to achieve load balancing.

5. The snapshot rollback method according to claim 1, characterized in that: Performing snapshot rollback on the target file corresponding to the snapshot rollback task by using the target data block to obtain the rollback file includes: Performing hash calculation on the target data block using a preset hash algorithm to obtain a hash check value; Querying in the snapshot verification file to obtain the verification standard value corresponding to the target data block; Determine whether the verification standard value is consistent with the hash verification value; If the verification standard value is consistent with the hash verification value, performing snapshot rollback on the target file corresponding to the snapshot rollback task using the target data block to obtain the rollback file; If the verification standard value is inconsistent with the hash verification value, a snapshot rollback failure prompt is output; Accordingly, the method further comprises: When the target file is obtained by performing a file snapshot operation on the original file, a hash calculation is performed on each data block in the target file using the preset hash algorithm to obtain a verification standard value of each data block; A snapshot verification file corresponding to the target file is generated according to the verification standard value of each of the data blocks.

6. The snapshot rollback method according to claim 5, characterized in that: When the file snapshot operation is performed on the original file to obtain the target file, the preset hash algorithm is used to perform hash calculation on each data block in the target file to obtain the verification standard value of each data block, including: Determine the changed data blocks in the original file; Executing the file snapshot operation on each of the changed data blocks to obtain the target file; The preset hash algorithm is used to perform hash calculation on each of the changed data blocks in the target file to obtain a verification standard value of each of the changed data blocks.

7. The snapshot rollback method according to claim 1, characterized in that: Also includes: Determine the predicted rollback data block based on the historical snapshot rollback log; Migrating the predicted rollback data block to a preset storage medium; Accordingly, obtaining the target data block according to the target data block position information includes: Determining whether the target data block hits the preset storage medium; If the target data block hits the preset storage medium, the target data block is read from the preset storage medium; If the target data block does not hit the preset storage medium, the target data block is obtained according to the target data block position information.

8. A snapshot rollback device, characterized in that: include: A determination module, used to determine a target index according to the snapshot rollback task; wherein the target index includes a target snapshot index and a target data block index; A first acquisition module, configured to query a global index file using the target snapshot index, determine target snapshot location information, and obtain a target snapshot file according to the target snapshot location information; A second acquisition module is used to query the local index file corresponding to the target snapshot file by using the target data block index to determine the target data block location information, and obtain the target data block according to the target data block location information; The rollback module is used to perform a snapshot rollback on the target file corresponding to the snapshot rollback task using the target data block to obtain a rollback file.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the snapshot rollback method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the snapshot rollback method according to any one of claims 1 to 7 are implemented.

Images