Statistical method for snapshot capacity of distributed storage system

Abstract

The invention particularly relates to a snapshot capacity statistical method for a distributed storage system. According to the snapshot capacity statistical method for the distributed storage system, an execution node is initialized in a slow loading mode, and the snapshot capacity is obtained and recorded to a database node; the management node judges the block storage state and obtains the allocated capacity, and the execution node creates a snapshot according to an instruction and obtains a capacity record; the management node checks the snapshot capacity information of the database node, if the snapshot capacity information does not exist, the execution node is called to obtain the snapshot capacity information, and the snapshot capacity information is recorded; for directly deleting the snapshots, the management node issues a deletion instruction, the execution node deletes the snapshots, and database node information is updated; and aiming at cloning block storage deletion of the snapshots, the management node issues a deletion block storage instruction, and the execution node deletes the snapshots and updates the capacity. According to the statistical method for the snapshot capacity of the distributed storage system, the snapshot management efficiency is improved, automation and process of snapshot operation are achieved, the operation process is simplified, and the management efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and in particular to a method for calculating the snapshot capacity of a distributed storage system. Background Technology

[0002] In today's digital age, data volumes are exploding, and distributed storage systems, with their advantages of high scalability, high availability, and fault tolerance, have become the preferred solution for storing massive amounts of data. However, as the importance of data becomes increasingly apparent, data security and traceability have become key challenges facing distributed storage systems.

[0003] Snapshot technology, as an important means of ensuring data security and traceability, can create static copies of data at a specific point in time, providing strong support for data recovery, backup, and auditing. However, in the complex architecture of distributed storage systems, the reasonable capacity calculation of snapshots is crucial to the stability of the storage system.

[0004] On the one hand, accurate snapshot capacity statistics help to rationally plan storage resources and avoid waste or insufficiency of storage space.

[0005] On the other hand, real-time monitoring of snapshot capacity changes can help identify potential performance issues promptly, such as slow system response caused by snapshots occupying too much storage space.

[0006] Therefore, building a system that can calculate snapshot usage capacity in real time based on user operations is of great practical significance.

[0007] Based on the above, this invention proposes a method for calculating the snapshot capacity of a distributed storage system. Summary of the Invention

[0008] To overcome the shortcomings of existing technologies, this invention provides a simple and efficient method for calculating the snapshot capacity of a distributed storage system.

[0009] This invention is achieved through the following technical solution:

[0010] A method for calculating snapshot capacity in a distributed storage system includes the following steps:

[0011] Step S1: Perform node initialization

[0012] The execution nodes are initialized using a slow loading method. Each execution node executes a read command based on its own storage node type to obtain the snapshot capacity and record it to the database node.

[0013] In step S1, the execution node will match the real-time capacity information of all snapshots in the block storage with the snapshot ID, record the successfully matched capacity information in detail in the information of each snapshot of the database node, and return the data processing result in JSON format.

[0014] Step S2, Snapshot Creation

[0015] The management node determines the block storage status and obtains the allocated capacity, while the execution node creates a snapshot and obtains the capacity record according to the instructions.

[0016] In step S2, when creating a snapshot, the management node first determines whether the block storage status is normal, and at the same time obtains the allocated capacity of the block storage.

[0017] The execution node creates snapshots based on the instructions of the management node and the corresponding creation commands on different storage nodes. After successful creation, the execution node uses the read command again to directly obtain the capacity of the snapshot and records it in the snapshot information.

[0018] Step S3, Snapshot Details and List Display

[0019] The management node checks the snapshot capacity information of the database node. If it does not exist, it calls the execution node to obtain the snapshot capacity information and records it.

[0020] In step S3, when the management node displays snapshot details and a list, it first checks whether the snapshot information of the database node includes snapshot capacity information.

[0021] If included, the snapshot details will be displayed directly, providing users with clear and comprehensive snapshot information;

[0022] If not included, the execution node is invoked. The execution node obtains the corresponding capacity of the snapshot by interacting with the storage node and records it in the database node to ensure the integrity and accuracy of the snapshot information and provide reliable data support for users.

[0023] Step S4: Snapshot Deletion

[0024] Snapshot deletion includes two scenarios: directly deleting the snapshot and deleting from the clone block storage of the snapshot.

[0025] For direct snapshot deletion, the management node issues a deletion command, the execution node deletes the snapshot, and updates the database node information;

[0026] For deleting cloned block storage of snapshots, the management node issues a delete block storage command, the execution node deletes the snapshot and updates the capacity, and if the deletion fails, it performs soft deletion marking and capacity synchronization.

[0027] In step S4, the specific processing flow for directly deleting snapshots is as follows:

[0028] Step S4.1.1: The management node sends a command to the execution node to delete the snapshot, specifying the deletion target;

[0029] Step S4.1.2: The execution node executes the corresponding deletion command according to the storage node type to delete the snapshot;

[0030] After successful deletion, the execution node retrieves all snapshot information of the corresponding block storage from the database node, sorts them by time, records the information of the next snapshot A after the deleted snapshot, and immediately sets the deleted snapshot record to the deleted state to ensure the consistency of data state.

[0031] Step S4.1.3: Use the read command again to directly obtain the capacity of snapshot A and update it in the snapshot information to ensure the real-time and accuracy of the snapshot capacity information.

[0032] In step S4, the specific process for deleting the clone block storage of the snapshot is as follows:

[0033] Step S4.2.1: The management node sends a command to the execution node to delete the block storage, thus initiating the deletion process;

[0034] Step S4.2.2: The execution node queries the snapshot information corresponding to the corresponding clone block storage through the database node and deletes the corresponding snapshot;

[0035] Similar to the scenario of directly deleting a snapshot, after a snapshot is successfully deleted, the capacity of the next snapshot after the deleted snapshot is updated to ensure the accuracy of the data;

[0036] When the deletion of a cloned block storage fails, the corresponding block storage and the corresponding snapshot are marked as soft-deleted, and the current full capacity of the snapshot is synchronized to the next snapshot to avoid data corruption and ensure system stability and data integrity.

[0037] A distributed storage system snapshot capacity statistics system for implementing the above method includes a management node, an execution node, a database node, and a storage node;

[0038] The management node is used to issue operation commands to the execution node according to the user's operation instructions, and to interact with the database node to obtain snapshot records and capacity information, and to provide feedback on the snapshot status to the user.

[0039] The execution node is used to perform operations on the storage node according to the operation commands issued by the management node, collect snapshot capacity information, and record it to the database node;

[0040] The database node is used to store snapshot capacity information and provide data support for the management node and execution node.

[0041] The storage nodes are used to store user data, providing the physical basis for the creation, management and use of snapshots.

[0042] A distributed storage system snapshot capacity statistics device includes a memory and a processor; the memory is used to store a computer program, and the processor is used to execute the computer program to implement the above-described method steps.

[0043] A readable storage medium storing a computer program that, when executed by a processor, implements the above-described method steps.

[0044] The beneficial effects of this invention are: the snapshot capacity statistics method of the distributed storage system can efficiently obtain snapshot records and detailed capacity information of each snapshot, provide users with comprehensive and accurate snapshot status feedback, improve snapshot management efficiency, realize the automation and process of snapshot operation, simplify the operation process, and improve management efficiency. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram illustrating the process of obtaining snapshot capacity when querying details according to the present invention.

[0047] Figure 2 This is a schematic diagram of the process of updating snapshot capacity when deleting a snapshot according to the present invention. Detailed Implementation

[0048] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions in the embodiments of this invention will be clearly and completely described below in conjunction with the embodiments of this invention. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0049] The method for calculating snapshot capacity in this distributed storage system includes the following steps:

[0050] Step S1: Perform node initialization

[0051] The execution nodes are initialized using a slow loading method. Each execution node executes a read command based on its own storage node type to obtain the snapshot capacity and record it to the database node.

[0052] In step S1, the initialization of the execution nodes adopts a slow loading method during system startup. This design cleverly utilizes a delayed loading strategy. At the moment of system startup, excessive queries to the storage nodes are avoided, effectively reducing the load pressure on the storage nodes and ensuring a smooth and fast system startup.

[0053] Secondly, depending on the type of storage node, the execution node needs to execute different commands to obtain the snapshot capacity. Taking the RBD backend of Ceph storage as an example, it uses the "rbd du --image" command to accurately obtain the real-time capacity information of all snapshots under a specific block storage.

[0054] Subsequently, the execution node will precisely match the real-time capacity information of all snapshots in the block storage with the snapshot ID, and record the successfully matched capacity information in detail in the information of each snapshot on the database node. To facilitate subsequent data processing and analysis, the "--format=json" parameter can be specified, and the data processing results will be returned in JSON format. This standardized data format greatly improves data readability and processing efficiency.

[0055] Step S2, Snapshot Creation

[0056] The management node determines the block storage status and obtains the allocated capacity, while the execution node creates a snapshot and obtains the capacity record according to the instructions.

[0057] In step S2, the management node plays a crucial role in judgment and retrieval during snapshot creation. It first carefully assesses the block storage status to ensure it is normal, and then retrieves the allocated capacity of the block storage. This step is critical; by taking preventative measures, it effectively avoids excessive impact on the storage node due to high I / O or excessive capacity, ensuring the stability and reliability of the snapshot creation process.

[0058] Secondly, the execution nodes execute corresponding creation commands based on the instructions of the management nodes and different storage nodes. Taking the RBD backend of Ceph storage as an example, a snapshot can be easily created using the "rbd snap create" command. After successful creation, the execution nodes use read commands such as "rbd du-image" to directly obtain the capacity of the snapshot and accurately record it in the snapshot information, providing detailed basic information for subsequent snapshot management and use.

[0059] Step S3, Snapshot Details and List Display

[0060] The management node checks the snapshot capacity information of the database node. If it does not exist, it calls the execution node to obtain the snapshot capacity information and records it.

[0061] When displaying snapshot details and lists, the management node needs to perform rigorous judgment. In step S3, it first checks whether the snapshot information of the database node contains snapshot capacity information:

[0062] If included, the snapshot details will be displayed directly, providing users with clear and comprehensive snapshot information;

[0063] If not included, the execution node is quickly invoked. The execution node obtains the corresponding capacity of the snapshot by interacting with the storage node and records it in the database node to ensure the integrity and accuracy of the snapshot information and provide reliable data support for users.

[0064] Step S4: Snapshot Deletion

[0065] Snapshot deletion includes two scenarios: directly deleting the snapshot and deleting from the clone block storage of the snapshot.

[0066] For direct snapshot deletion, the management node issues a deletion command, the execution node deletes the snapshot, and updates the database node information;

[0067] For deleting cloned block storage of snapshots, the management node issues a delete block storage command, the execution node deletes the snapshot and updates the capacity, and if the deletion fails, it performs soft deletion marking and capacity synchronization.

[0068] In step S4, the specific processing flow for directly deleting snapshots is as follows:

[0069] Step S4.1.1: The management node sends a command to the execution node to delete the snapshot, specifying the deletion target;

[0070] Step S4.1.2: The execution node executes the corresponding deletion command according to the storage node type. For example, the snapshot of the RBD backend of Ceph storage is deleted using the "rbd snap delete" command.

[0071] After successful deletion, the execution node retrieves all snapshot information of the corresponding block storage from the database node, sorts them by time, records the information of the next snapshot A after the deleted snapshot, and immediately sets the deleted snapshot record to the deleted state to ensure the consistency of data state.

[0072] Step S4.1.3: Use the "rbd du--image" command again to directly obtain the capacity of snapshot A and update it in the snapshot information to ensure the real-time and accuracy of the snapshot capacity information.

[0073] In step S4, the specific process for deleting the clone block storage of the snapshot is as follows:

[0074] Step S4.2.1: The management node sends a command to the execution node to delete the block storage, thus initiating the deletion process;

[0075] Step S4.2.2: The execution node queries the snapshot information corresponding to the corresponding clone block storage through the database node and deletes the corresponding snapshot;

[0076] Similar to the scenario of directly deleting a snapshot, after a snapshot is successfully deleted, the capacity of the next snapshot after the deleted snapshot is updated to ensure the accuracy of the data;

[0077] When a cloned block storage fails to be deleted for some reason, the corresponding block storage and the corresponding snapshot are marked as soft-deleted, and the current full capacity of the snapshot is synchronized to the next snapshot to avoid data corruption and ensure system stability and data integrity.

[0078] In summary, the snapshot capacity statistics system in the distributed storage system, through its carefully designed architecture and rigorous snapshot lifecycle management process, achieves refined management of snapshot capacity, providing a solid guarantee for the stable operation of the distributed storage system and the security of user data.

[0079] The distributed storage system snapshot capacity statistics system is used to implement the above method, including management nodes, execution nodes, database nodes, and storage nodes;

[0080] The management node is used to issue operation commands to the execution node according to the user's operation instructions, and to interact with the database node to obtain snapshot records and capacity information, and to provide feedback on the snapshot status to the user.

[0081] The execution node is used to perform operations on the storage node according to the operation commands issued by the management node, collect snapshot capacity information, and record it to the database node;

[0082] The database node is used to store snapshot capacity information and provide data support for the management node and execution node.

[0083] The storage nodes are used to store user data, providing the physical basis for the creation, management and use of snapshots.

[0084] The distributed storage system snapshot capacity statistics device includes a memory and a processor; the memory is used to store a computer program, and the processor is used to execute the computer program to implement the above-described method steps.

[0085] The readable storage medium stores a computer program that, when executed by a processor, implements the above-described method steps.

[0086] Compared with existing technologies, this method for calculating snapshot capacity in a distributed storage system has the following characteristics:

[0087] 1) Improved snapshot management efficiency

[0088] The execution node can collect snapshot usage capacity information in a flexible and diverse manner according to different storage node types, and accurately record this data in the database node. The management node, through deep interaction with the database module, can efficiently obtain snapshot records and detailed capacity information for each snapshot, providing users with comprehensive and accurate snapshot status feedback.

[0089] For example, administrators can monitor the size and usage of each snapshot in real time, and adjust storage resource allocation strategies accordingly to improve storage resource utilization.

[0090] 2) Simplified operating procedures

[0091] Through close collaboration among management nodes, execution nodes, database nodes, and storage nodes, snapshot operations are automated and streamlined. Users simply issue operation commands, and the management node precisely sends specific operation commands to the execution nodes. The execution nodes then perform the corresponding operations on the storage nodes according to the commands and record the results in the database nodes. The entire process requires no manual intervention from the user at multiple stages, simplifying the operation process and improving management efficiency.

[0092] For example, when creating a snapshot, users only need to send a creation command to the management node, and the system will automatically complete operations such as status judgment, capacity acquisition, and snapshot creation, which greatly saves users' time and effort.

[0093] The embodiments described above are merely one specific implementation of the present invention. Ordinary changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for calculating snapshot capacity in a distributed storage system, characterized in that: Includes the following steps: Step S1: Perform node initialization The execution nodes are initialized using a slow loading method. Each execution node executes a read command based on its own storage node type to obtain the snapshot capacity and record it to the database node. Step S2, Snapshot Creation The management node determines the block storage status and obtains the allocated capacity, while the execution node creates a snapshot and obtains the capacity record according to the instructions. Step S3, Snapshot Details and List Display The management node checks the snapshot capacity information of the database node. If it does not exist, it calls the execution node to obtain the snapshot capacity information and records it. Step S4: Snapshot Deletion Snapshot deletion includes two scenarios: directly deleting the snapshot and deleting from the clone block storage of the snapshot. For direct snapshot deletion, the management node issues a deletion command, the execution node deletes the snapshot, and updates the database node information; For deleting cloned block storage of snapshots, the management node issues a delete block storage command, the execution node deletes the snapshot and updates the capacity, and if the deletion fails, it performs soft deletion marking and capacity synchronization.

2. The method for calculating snapshot capacity in a distributed storage system according to claim 1, characterized in that: In step S1, the execution node will match the real-time capacity information of all snapshots in the block storage with the snapshot ID, record the successfully matched capacity information in detail in the information of each snapshot of the database node, and return the data processing result in JSON format.

3. The method for calculating snapshot capacity in a distributed storage system according to claim 1, characterized in that: In step S2, when creating a snapshot, the management node first determines whether the block storage status is normal, and at the same time obtains the allocated capacity of the block storage. The execution node creates snapshots by executing corresponding creation commands based on different storage nodes according to the instructions of the management node. After successful creation, the execution node uses the read command again to directly obtain the snapshot capacity and records it in the snapshot information.

4. The method for calculating snapshot capacity in a distributed storage system according to claim 1, characterized in that: In step S3, when the management node displays snapshot details and a list, it first checks whether the snapshot information of the database node includes snapshot capacity information. If included, the snapshot details will be displayed directly, providing users with clear and comprehensive snapshot information; If not included, the execution node is invoked. The execution node obtains the corresponding capacity of the snapshot by interacting with the storage node and records it in the database node to ensure the integrity and accuracy of the snapshot information and provide reliable data support for users.

5. The method for calculating snapshot capacity in a distributed storage system according to claim 1, characterized in that: In step S4, the specific processing flow for directly deleting snapshots is as follows: Step S4.1.1: The management node sends a command to the execution node to delete the snapshot, specifying the deletion target; Step S4.1.2: The execution node executes the corresponding deletion command according to the storage node type to delete the snapshot; After successful deletion, the execution node retrieves all snapshot information of the corresponding block storage from the database node, sorts them by time, records the information of the next snapshot A after the deleted snapshot, and immediately sets the deleted snapshot record to the deleted state to ensure the consistency of data state. Step S4.1.3: Use the read command again to directly obtain the capacity of snapshot A and update it in the snapshot information to ensure the real-time and accuracy of the snapshot capacity information.

6. The method for calculating snapshot capacity in a distributed storage system according to claim 5, characterized in that: In step S4, the specific process for deleting the clone block storage of the snapshot is as follows: Step S4.2.1: The management node sends a command to the execution node to delete the block storage, thus initiating the deletion process; Step S4.2.2: The execution node queries the snapshot information corresponding to the corresponding clone block storage through the database node and deletes the corresponding snapshot; Similar to the scenario of directly deleting a snapshot, after a snapshot is successfully deleted, the capacity of the next snapshot after the deleted snapshot is updated to ensure the accuracy of the data; When the deletion of a cloned block storage fails, the corresponding block storage and the corresponding snapshot are marked as soft-deleted, and the current full capacity of the snapshot is synchronized to the next snapshot to avoid data corruption and ensure system stability and data integrity.

7. A snapshot capacity statistics system for a distributed storage system, characterized in that: The method for implementing any one of claims 1 to 6 includes a management node, an execution node, a database node, and a storage node; The management node is used to issue operation commands to the execution node according to the user's operation instructions, and to interact with the database node to obtain snapshot records and capacity information, and to provide feedback on the snapshot status to the user. The execution node is used to perform operations on the storage node according to the operation commands issued by the management node, collect snapshot capacity information, and record it to the database node; The database node is used to store snapshot capacity information and provide data support for the management node and execution node. The storage nodes are used to store user data, providing the physical basis for the creation, management and use of snapshots.

8. A snapshot capacity statistics device for a distributed storage system, characterized in that: It includes a memory and a processor; the memory is used to store a computer program, and the processor is used to execute the computer program to implement the method according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 6.

Images