A snapshot method, device and equipment of a distributed storage system and a storage medium

Abstract

The application relates to the technical field of distributed storage, and discloses a snapshot method, device and equipment of a distributed storage system and a storage medium, the method comprising the following steps: creating an initial namespace of a distributed storage system; starting a shared service on the initial namespace; performing target processing on the initial namespace based on the shared service to obtain a current namespace, wherein the target processing comprises writing data and / or function configuration; and creating a snapshot of the current namespace to determine a target snapshot. The application can solve the problem of complex snapshot creation process of the distributed storage system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed storage, and particularly relates to a snapshot method and device of a distributed storage system, computer equipment and a computer readable storage medium. BACKGROUND

[0002] The current distributed storage system takes a distributed file system as a base to realize S3 (Simple Storage Service) simple storage service and NAS (Network Attached Storage) network attached storage protocol intercommunication. However, with more and more storage protocols supported by the distributed storage system, more and more functions supported by the storage protocols, and more and more complex compatibility of various functions and the snapshot function of the original storage system, the distributed storage system needs to be configured to use the functions of multiple storage protocols, and the snapshot function of the original storage system needs to be compatible with the functions supported by various storage protocols, so that the snapshot function of the original storage system can be used to create a snapshot of a shared directory of the distributed storage system, thereby causing the snapshot creation process of the distributed storage system to be more and more complex. SUMMARY

[0003] Therefore, the present application provides a snapshot method, device, equipment and storage medium of a distributed storage system to solve the problem of complex snapshot creation process of the distributed storage system.

[0004] In a first aspect, the present application provides a snapshot method of a distributed storage system, which comprises:

[0005] creating an initial namespace of the distributed storage system;

[0006] starting a shared service on the initial namespace;

[0007] performing target processing on the initial namespace based on the shared service to obtain a current namespace, wherein the target processing comprises writing data and / or function configuration;

[0008] creating a snapshot of the current namespace to determine a target snapshot.

[0009] In this way, the shared service is started on the initial namespace by creating the initial namespace, so that the client can perform target processing on the initial namespace through the shared service to create a snapshot on the current namespace, thereby avoiding the problem of complex snapshot creation process caused by the complex compatibility of the snapshot function of the original storage system and the functions supported by the storage protocols thereof, and simplifying the snapshot creation process.

[0010] In an optional implementation, the creating of the initial namespace of the distributed storage system comprises:

[0011] creating an authentication user group of the distributed storage system and an authentication user belonging to the authentication user group;

[0012] creating a tenant of the distributed storage system and a simple storage user belonging to the tenant;

[0013] building a mapping relationship between the authentication user and the simple storage user;

[0014] creating an initial namespace by the tenant and the authentication user under the authentication user group.

[0015] In this way, by mounting the simple storage user under the tenant, the mapping relationship between the simple storage user and the authentication user is built, and then the initial namespace is created by the tenant and the authentication user, so that the NFS client and the FTP client can access the namespace according to the authentication user, and the S3 protocol client can also access the namespace through the simple storage user, so that the client can access the namespace through multiple storage protocols to create a snapshot.

[0016] In an optional implementation, the target processing of the initial namespace based on the shared service to obtain a current namespace comprises:

[0017] determining a storage protocol corresponding to a current shared service;

[0018] target processing of the initial namespace based on the storage protocol to obtain the current namespace.

[0019] In this way, when the client target processes the initial namespace through the shared service, the storage protocol corresponding to the current shared service is determined first, and then the initial namespace is target processed based on the corresponding storage protocol, so that the operation failure of the initial namespace due to the unclear storage protocol or the calling error can be avoided.

[0020] In an optional implementation, when the target processing is writing data, the target processing of the initial namespace based on the storage protocol to obtain the current namespace comprises:

[0021] if the storage protocol is the NFS protocol, connecting to the NFS client based on the NFS protocol, mounting the initial namespace to a local path of the NFS client, writing data into the local path of the NFS client, and obtaining the current namespace;

[0022] If the storage protocol is FTP, then an FTP connection object is created based on the FTP protocol, and the local files of the FTP client are written to the initial namespace through the FTP connection object to obtain the current namespace;

[0023] If the storage protocol is S3, then connect to the S3 protocol client based on the S3 protocol, write the local files of the S3 protocol client into the initial namespace, and obtain the current namespace.

[0024] In this approach, the client can choose to write data to the namespace using NFS, FTP, or S3 protocols, making the way the client writes data to the namespace more flexible and suitable for different types of clients.

[0025] In an optional implementation, the method further includes:

[0026] In response to a snapshot rollback command, the current namespace is rolled back to a target time, where the target time is the time node when the target snapshot corresponding to the snapshot rollback command was created;

[0027] Obtain the namespace data at the target time, the namespace data including the data and function configuration within the namespace at the target time;

[0028] Based on the shared service, the snapshot rollback result is obtained from the target client. The snapshot rollback result includes the data and function configuration in the namespace of the target snapshot corresponding to the snapshot rollback instruction.

[0029] Compare the namespace data with the snapshot rollback results;

[0030] If the namespace data is successfully compared with the snapshot rollback result, the snapshot rollback is determined to be successful.

[0031] If the namespace data fails to match the snapshot rollback result, the snapshot rollback is determined to have failed.

[0032] In this approach, firstly, compared to snapshot rollback via the original storage system's snapshot functionality, snapshot rollback via namespaces does not require compatibility configuration between the original storage system's snapshot functionality and the various functions supported by the storage protocol, thus making the process relatively simple and convenient. Secondly, after snapshot rollback, the rollback result is further compared with the namespace data recorded when the target snapshot was created, thereby further confirming the correctness and completeness of the rolled-back data and the namespace function configuration.

[0033] In an optional implementation, after determining that the snapshot rollback was successful, the method further includes:

[0034] Delete the data and function configurations in the current namespace;

[0035] Delete the target snapshot that was created;

[0036] Disable the shared services of the current namespace;

[0037] Delete the current namespace and the data used when the initial namespace was created.

[0038] In this approach, after the task is completed, the discarded files of the distributed storage system are further cleaned up and storage space is released, thus further improving the usability of the distributed storage system.

[0039] In one optional implementation, the configuration items in the function configuration include at least one of file number quota, capacity quota, access control list configuration, WORM configuration, bucket policy configuration, anti-hotlinking configuration, static website hosting configuration, bucket tag configuration, and object tag configuration.

[0040] In this approach, namespaces can be flexibly configured to enable various functions under shared services.

[0041] Secondly, embodiments of the present invention provide a snapshot device for a distributed storage system, characterized in that the device comprises:

[0042] The namespace creation module is used to create the initial namespace of the distributed storage system;

[0043] A namespace sharing module is used to enable sharing services on the initial namespace;

[0044] The data processing module is used to perform target processing on the initial namespace based on the shared service to obtain the current namespace. The target processing includes writing data and / or configuring functions.

[0045] The snapshot creation module is used to create a snapshot of the current namespace and determine the target snapshot.

[0046] Thirdly, embodiments of the present invention provide a computer device, including: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the snapshot method of the distributed storage system described in the first aspect or any corresponding embodiment.

[0047] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the snapshot method of the distributed storage system of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0048] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating a snapshot method for a distributed storage system according to some embodiments of the present invention;

[0050] Figure 2 This is a flowchart illustrating a snapshot method for another distributed storage system according to some embodiments of the present invention;

[0051] Figure 3 This is a schematic diagram of the process of creating an initial namespace according to some embodiments of the present invention;

[0052] Figure 4 This is a schematic diagram illustrating the enabling of shared services on an initial namespace according to some embodiments of the present invention;

[0053] Figure 5 This is a flowchart illustrating a snapshot method for another distributed storage system according to some embodiments of the present invention;

[0054] Figure 6 This is a flowchart illustrating a snapshot method for another distributed storage system according to some embodiments of the present invention;

[0055] Figure 7 This is a schematic diagram of the data writing process according to some embodiments of the present invention;

[0056] Figure 8 This is a flowchart illustrating the functional configuration according to some embodiments of the present invention;

[0057] Figure 9 This is a flowchart illustrating a snapshot method for another distributed storage system according to some embodiments of the present invention;

[0058] Figure 10 This is a schematic diagram of the process of snapshot control in a namespace according to some embodiments of the present invention;

[0059] Figure 11This is a schematic diagram of the snapshot verification process according to some embodiments of the present invention;

[0060] Figure 12 This is a flowchart illustrating a snapshot method for another distributed storage system according to some embodiments of the present invention;

[0061] Figure 13 This is a schematic diagram of a data cleaning process according to some embodiments of the present invention;

[0062] Figure 14 This is another schematic diagram of snapshot control on a namespace according to some embodiments of the present invention;

[0063] Figure 15 This is a structural block diagram of a snapshot device for a distributed storage system according to an embodiment of the present invention;

[0064] Figure 16 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] Distributed storage systems, based on distributed file systems, enable interoperability between S3 (Simple Storage Service) and NAS (Network Attached Storage) protocols. In related technologies, configuring a distributed storage system to use multiple storage protocols requires complex compatibility configurations between the original storage system's snapshot functionality and the functions supported by various storage protocols. This is necessary to create snapshots of the distributed storage system's shared directories using the original storage system's snapshot functionality, thus complicating the snapshot creation process.

[0067] Based on this, according to an embodiment of the present invention, a snapshot method embodiment for a distributed storage system is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0068] This embodiment provides a snapshot method for a distributed storage system, which can be used in the aforementioned distributed storage system. Figure 1 This is a flowchart of a snapshot method for a distributed storage system according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0069] Step S101: Create the initial namespace for the distributed storage system.

[0070] In this embodiment of the invention, an authentication user group and authentication users are created by calling Python methods; tenants are created, simple storage users (i.e., S3 users) are created, and a mapping relationship is established between them and the authentication users created above. An initial namespace is created through the tenants and authentication users, so that the simple storage users and authentication users can access this namespace.

[0071] Step S102: Enable shared services on the initial namespace.

[0072] Specifically, an NFS share is created on the initial namespace to enable the NFS sharing service, allowing NFS clients to access the share using an authenticated user; an FTP share is created on the initial namespace to enable the FTP sharing service, allowing FTP clients to access the share using an authenticated user; and a storage system service (i.e., an S3 protocol share) is enabled on the initial namespace, allowing S3 protocol clients to access the share using an S3 user.

[0073] Step S103: Perform target processing on the initial namespace based on the shared service to obtain the current namespace. The target processing includes writing data and / or configuring functions.

[0074] Specifically, the client can perform target processing on the initial namespace through NFS sharing, FTP sharing, and S3 protocol sharing to obtain the current namespace.

[0075] Step S104: Create a snapshot of the current namespace and determine the target snapshot.

[0076] Specifically, the create_snapshot( / $TENANT_NAME / $NAMESPACE_NAME, $SNAP_NAME) method is called to create multiple snapshots of the current namespace.

[0077] Understandably, step S103 described above is a normal storage business process. At one or more points in time during the execution of the storage business, a snapshot can be created for the current namespace to record information about the data and functional configuration within the current namespace.

[0078] The snapshot method for the distributed storage system provided in this embodiment starts a shared service on the initial namespace by creating an initial namespace. Therefore, the client can perform target processing on the initial namespace through the shared service to take a snapshot on the current namespace. This avoids the problem of complex snapshot creation process caused by the incompatibility between the snapshot function of the original storage system and the functions supported by its storage protocol, thus simplifying the snapshot creation process.

[0079] Figure 2 This is a flowchart of a snapshot method for another distributed storage system according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0080] Step S201: Create the initial namespace for the distributed storage system.

[0081] Step S202: Enable shared services on the initial namespace. For details, please refer to step S102 above, which will not be repeated here.

[0082] Step S203: Based on the shared service, perform target processing on the initial namespace to obtain the current namespace. The target processing includes writing data and / or configuring functions. For details, please refer to step S103 above, which will not be repeated here.

[0083] Step S204: Create a snapshot of the current namespace and determine the target snapshot. For details, please refer to step S104 above, which will not be repeated here.

[0084] Specifically, see Figure 3 The above step S201 includes:

[0085] Step S2011: Create the authentication user group of the distributed storage system and the authentication users belonging to the authentication user group.

[0086] Specifically, since a Python language environment is configured in the distributed storage system, an authenticated user group can be created using `create_authenticated_user_group($GROUP_NAME)`, and an authenticated user can be created using `create_user($USER_NAME,$GROUP_NAME)`.

[0087] Step S2012: Create tenants of the distributed storage system and simple storage users belonging to the tenants.

[0088] Step S2013: Construct a mapping relationship between the authenticated user and the simple storage user;

[0089] Specifically, in the Python language environment, a tenant is created using create_tenant ($TENANT_NAME), and a simple storage user, namely S3 user $S3_USER_NAME, is created using create_s3_user ($S3_USER_NAME, $TENANT_NAME, $USER_NAME), and the S3 user is mapped to the authenticated user $USER_NAME.

[0090] Step S2014: Create an initial namespace using the tenant and the authenticated users under the authenticated user group.

[0091] Specifically, in the Python environment, a namespace is created using `create_namespace($TENANT_NAME,$NAMESPACE_NAME, $USER_NAME, $GROUP_NAME)`, which is the initial namespace. At this point, the S3 user `$S3_USER_NAME` can access the namespace `$NAMESPACE_NAME`.

[0092] The snapshot method for a distributed storage system provided in this embodiment establishes a mapping relationship between simple storage users and authenticated users by attaching simple storage users under tenants. Then, the tenants and authenticated users create an initial namespace. This allows NFS clients and FTP clients to access the namespace through authenticated users, while S3 protocol clients can also access the namespace through simple storage users. This enables clients to access the namespace through multiple storage protocols for snapshot creation.

[0093] Specifically, enabling the shared service on the initial namespace includes:

[0094] Create an NFS share on the initial namespace to enable the NFS share service;

[0095] Create an FTP share on the initial namespace to enable the FTP sharing service;

[0096] Enable S3 protocol sharing on the initial namespace to enable the S3 protocol sharing service.

[0097] Specifically, see Figure 4In the initial namespace $NAMESPACE_NAME, calling set_nfs_share( / $TENANT_NAME / $NAMESPACE_NAME) to create an NFS share allows storage clients / NFS clients to access the share using an authenticated user; calling create_ftp_share( / $TENANT_NAME / $NAMESPACE_NAME, $USER_NAME) to create an FTP share allows FTP clients to access the share using an authenticated user $USER_NAME; enabling S3 protocol sharing allows S3 protocol clients to access the share using the S3 user $S3_USER_NAME.

[0098] Figure 5 This is a flowchart of a snapshot method for a distributed storage system according to an embodiment of the present invention, such as... Figure 5 As shown, the process includes the following steps:

[0099] Step S301: Create the initial namespace for the distributed storage system. For details, please refer to step S101 above; it will not be repeated here.

[0100] Step S302: Enable shared services on the initial namespace. For details, please refer to step S102 above, which will not be repeated here.

[0101] Step S303: Based on the shared service, target processing is performed on the initial namespace to obtain the current namespace. The target processing includes writing data and / or configuring functions. For details, please refer to step S103 above, which will not be repeated here.

[0102] Step S304: Create a snapshot of the current namespace and determine the target snapshot. For details, please refer to step S104 above, which will not be repeated here.

[0103] In some optional implementations, step S303 above includes:

[0104] Step S3031: Determine the storage protocol corresponding to the current shared service;

[0105] Step S3032: Perform target processing on the initial namespace based on the storage protocol to obtain the current namespace.

[0106] The snapshot method for the distributed storage system provided in this embodiment first determines the storage protocol corresponding to the current shared service when the client performs target processing on the initial namespace through the shared service, and then performs target processing on the initial namespace based on the corresponding storage protocol. This can avoid the failure of the operation on the initial namespace due to unclear storage protocol or incorrect calling.

[0107] Figure 6 This is a flowchart of a snapshot method for a distributed storage system according to an embodiment of the present invention, such as... Figure 6 As shown, the process includes the following steps:

[0108] Step S401: Create the initial namespace for the distributed storage system. For details, please refer to step S101 above; it will not be repeated here.

[0109] Step S402: Enable shared services on the initial namespace. For details, please refer to step S102 above, which will not be repeated here.

[0110] Step S403: Determine the storage protocol corresponding to the current shared service. For details, please refer to step S3031 above, which will not be repeated here.

[0111] Step S404: Based on the storage protocol, target processing is performed on the initial namespace to obtain the current namespace. Target processing includes writing data and / or configuring functions. For details, please refer to step S3032 above, which will not be repeated here.

[0112] Step S405: Create a snapshot of the current namespace and determine the target snapshot. For details, please refer to step S104 above, which will not be repeated here.

[0113] In some optional implementations, when the target processing is writing data, step S404 includes:

[0114] Step S4041: If the storage protocol is NFS, then connect to the NFS client based on the NFS protocol, mount the initial namespace to the local path of the NFS client, and write data into the local path of the NFS client to obtain the current namespace.

[0115] Specifically, the NFS share uses the python paramiko command to connect to the client server, then uses the mount command to mount the NFS share, and uses commands such as dd, echo, and touch to create files and write them to the initial namespace.

[0116] Step S4042: If the storage protocol is FTP, then an FTP connection object is created based on the FTP protocol, and the local files of the FTP client are written to the initial namespace through the FTP connection object to obtain the current namespace.

[0117] Specifically, FTP sharing uses the python ftplib library to connect to the FTP service and upload and download files in the initial namespace.

[0118] Step S4043: If the storage protocol is S3 protocol, then connect to the S3 protocol client based on the S3 protocol, and write the local file of the S3 protocol client into the initial namespace to obtain the current namespace.

[0119] Specifically, the S3 protocol shares the Python paramiko to connect to the client server and use the doscmd command to perform operations such as uploading, querying, and deleting files to read and write files in the initial namespace.

[0120] The snapshot method for the distributed storage system provided in this embodiment allows clients to choose between NFS, FTP, and S3 protocols to write data to the namespace, making the way clients write data to the namespace more flexible and suitable for different types of clients.

[0121] See Figure 7 In one specific implementation, the specific steps for writing data to the initial namespace based on the storage protocol include: 1. NFS shared data writing: Using the python paramiko module to connect to the NFS client server, issuing and executing shell commands, and obtaining the return value results. By issuing `mount -t nfs $MOUNT_IP: / $TENANT_NAME / $NAMESPACE_NAME $LOCAL_DIR` to the NFS client, the initial namespace is mounted to the local path $LOCAL_DIR of the NFS client, and data is written to the local path $LOCAL_DIR using commands such as `echo`, `touch`, and `dd`. 2. S3 protocol shared data writing: Using the python paramiko module, issuing `doscmd put $LOCAL_FILE S3: / / $TENANT_NAME / $NAMESPACE_NAME`, the local file $LOCAL_FILE of the S3 protocol client is uploaded to $NAMESPACE_NAME using the S3 protocol, and operations such as querying and copying are performed using commands such as `doscmd get` and `cp`. 3. FTP sharing and writing data: Import the python ftplib module, use the ftplib.FTP method to create an FTP connection object $ftp_connect, and call the $ftp_connect.storbinary method to upload the local file $LOCAL_FILE_FTP from the FTP client to the initial namespace $NAMESPACE_NAME.

[0122] See Figure 8Specifically, the configuration items in the function configuration include at least one of the following: file number quota, capacity quota, access control list configuration, WORM configuration, bucket policy configuration, anti-hotlinking configuration, static website hosting configuration, bucket tag configuration, and object tag configuration.

[0123] The snapshot method for the distributed storage system provided in this embodiment allows for flexible configuration of namespaces to achieve various functions under shared services.

[0124] In one specific implementation, the initial namespace can be configured functionally based on the following steps: Under the initial namespace $NAMESPACE_NAME, set the hard quota $HARD_LIMIT and soft quota $SOFT_LIMIT for the initial namespace based on the shared service call set_directory_quota( / $TENANT_NAME / $NAMESPACE_NAME,$HARD_LIMIT,$SOFT_LIMIT); set the hard quota $HARD_NUM_LIMIT and soft quota $SOFT_NUM_LIMIT for the number of files in the initial namespace based on the shared service call set_file_num_quota( / $TENANT_NAME / $NAMESPACE_NAME,$HARD_NUM_LIMIT,$SOFT_NUM_LIMIT); and set the hard quota $HARD_NUM_LIMIT and soft quota $SOFT_NUM_LIMIT for the number of files based on the shared service call create_acl_setting( / $TENANT_NAME / $NAMESPACE_NAME, The `$USER_NAME` and `$ACL_SETTING` methods are used to set the Access Control List (ACL); `set_namespace_worm( / $TENANT_NAME / $NAMESPACE_NAME, $GRANCE_SET, $PROTECT_SET)` is called based on the shared service to set the WROM attribute of the initial namespace; `set_namespace_strategy( / $TENANT_NAME / $NAMESPACE_NAME, $STRATEGY_SET)` is called based on the shared service to set the bucket policy; `set_namespace_stealing_link( / $TENANT_NAME / $NAMESPACE_NAME, $BLACK_IP_LIST, $WHILE_IP_LIST)` is called based on the shared service to set the blacklist and whitelist addresses for hotlinking prevention; `set_namespace_static_website_trusteeship( / $TENANT_NAME / $NAMESPACE_NAME, $TRUSTEESHIP)` is called based on the shared service to set static website hosting; `set_namespace_bucket_label( / $TENANT_NAME / $NAMESPACE_NAME, $STRUSTEESHIP)` is called based on the shared service to set the bucket policy; `set_namespace_bucket_label( / $TENANT_NAME / $NAMESPACE_NAME, $STRATEGY_SET)` is called based on the shared service to set the bucket policy; `set_namespace_bucket_label( / $TENANT_NAME / $NAMESPACE_NAME, $STRUSTEESHIP ... $LABEL) sets bucket labels; the doscmdsettagging command is used to create object labels based on the shared service.

[0125] As one optional implementation, the method further includes:

[0126] Get the remaining capacity of the current namespace in real time;

[0127] When the remaining capacity is less than a preset first capacity threshold, if multiple clients are detected writing data to the current namespace simultaneously, the priority configuration of the authenticated user or simple storage user corresponding to each client is obtained.

[0128] Based on the priority configuration of each client, clients are allowed to write data to the current namespace in descending order of priority;

[0129] When the remaining capacity is less than a preset second capacity threshold, the remaining capacity of the current namespace is fed back to the client; wherein the second capacity threshold is less than the first capacity threshold.

[0130] In this embodiment of the invention, when the remaining capacity of the current namespace is less than the first capacity threshold, data is written to the current namespace in descending order of priority according to the client's corresponding priority. When the remaining capacity is less than the second capacity threshold, the remaining capacity of the current namespace is fed back to the client. This can prevent the amount of data written to the namespace from exceeding its capacity, thus avoiding data writing errors or program errors.

[0131] Figure 9 This is a flowchart of a snapshot method for a distributed storage system according to an embodiment of the present invention, such as... Figure 9 As shown, the process includes the following steps:

[0132] Step S501: Create the initial namespace for the distributed storage system. For details, please refer to step S101 above; it will not be repeated here.

[0133] Step S502: Enable shared services on the initial namespace. For details, please refer to step S102 above, which will not be repeated here.

[0134] Step S503: Based on the shared service, target processing is performed on the initial namespace to obtain the current namespace. The target processing includes writing data and / or configuring functions. For details, please refer to step S103 above, which will not be repeated here.

[0135] Step S504: Create a snapshot of the current namespace and determine the target snapshot. For details, please refer to step S104 above, which will not be repeated here.

[0136] Step S505: In response to the snapshot rollback command, the current namespace is rolled back to the target time, where the target time is the time node when the target snapshot corresponding to the snapshot rollback command was created.

[0137] Specifically, when it is necessary to restore to a certain point in time of the original business, the recovery_snapshot( / $TENANT_NAME / $NAMESPACE_NAME,$SNAP_NAME) method is called to roll back the current namespace $NAMESPACE_NAME to the time point when $SNAP_NAME was created, which is the target time mentioned above.

[0138] Step S506: Obtain the namespace data at the target time, the namespace data including the data and function configuration within the namespace at the target time.

[0139] Step S507: Obtain the snapshot rollback result from the target client based on the shared service. The snapshot rollback result includes the data and function configuration in the namespace of the target snapshot corresponding to the snapshot rollback instruction.

[0140] Step S508: Compare the namespace data with the snapshot rollback result.

[0141] Step S509: If the namespace data is successfully compared with the snapshot rollback result, the snapshot rollback is determined to be successful.

[0142] Step S510: If the namespace data fails to match the snapshot rollback result, the snapshot rollback is determined to have failed.

[0143] The snapshot method for distributed storage systems provided in this embodiment has several advantages. First, compared to snapshot rollback via the original storage system's snapshot functionality, snapshot rollback via namespaces does not require compatibility configuration between the original storage system's snapshot functionality and the various functions supported by the storage protocol, making the process relatively simple and convenient. Second, after snapshot rollback, the rollback result is further compared with the namespace data recorded when the target snapshot was created, thereby further confirming the correctness and completeness of the rolled-back data and the namespace function configuration.

[0144] See Figure 10 In one specific implementation, during the process of writing data to the initial namespace or configuring functions, multiple snapshots can be created for the current namespace within the Python environment. Furthermore, when it is necessary to restore to a specific point in time from the original business process, the current namespace can be rolled back to the time point when the target snapshot to be restored was created, i.e., the target moment mentioned above. For example... Figure 10 As shown, the current namespace can be rolled back to the time point of namespace snapshot 1, namespace snapshot 2, or namespace snapshot 3.

[0145] See Figure 11In one specific implementation, before creating a snapshot of the current namespace at a certain point in time, it is necessary to record the data and functional configuration at that point in time for verification of the namespace's functional configuration and data after snapshot rollback. Specifically, the step of obtaining the snapshot rollback result from the target client based on the shared service can use the pythonparamiko module to pass commands such as cat, grep, and dosmcd info to the target client to perform operations on the attached... Figure 7 and attached Figure 8 The process involves acquiring the data written during the snapshot rollback and the created functional configurations, i.e., the snapshot rollback result, and comparing it with the data and functional configurations prior to the snapshot creation time. If both the functional configuration and data match successfully, the snapshot rollback is successful, and data cleanup or successful exit can proceed as needed. If either the functional configuration or the data comparison fails, the snapshot rollback is considered to have failed, an exception is thrown, the program exits, and a failure message is returned. Furthermore, it should be noted that this embodiment obtains the snapshot rollback result from the target client based on the shared service because after the snapshot rollback, the namespace data becomes the data from the time the client created the snapshot. Therefore, if it is necessary to verify the accuracy of the snapshot rollback, the snapshot rollback result needs to be obtained from the target client and compared with the record at the time the snapshot was created. It is understood that the target client is the client corresponding to the target snapshot.

[0146] Figure 12 This is a flowchart of a snapshot method for a distributed storage system according to an embodiment of the present invention, such as... Figure 12 As shown, the process includes the following steps:

[0147] Step S601: Create the initial namespace for the distributed storage system. For details, please refer to step S101 above; it will not be repeated here.

[0148] Step S602: Enable shared services on the initial namespace. For details, please refer to step S102 above, which will not be repeated here.

[0149] Step S603: Based on the shared service, target processing is performed on the initial namespace to obtain the current namespace. The target processing includes writing data and / or configuring functions. For details, please refer to step S103 above, which will not be repeated here.

[0150] Step S604: Create a snapshot of the current namespace and determine the target snapshot. For details, please refer to step S104 above, which will not be repeated here.

[0151] Step S605: In response to the snapshot rollback command, the current namespace is rolled back to the target time, where the target time is the time node when the target snapshot corresponding to the snapshot rollback command was created. For details, please refer to step S505 above, which will not be repeated here.

[0152] Step S606: Obtain the namespace data at the target time. The namespace data includes the data and function configuration within the namespace at the target time. For details, please refer to step S506 above, which will not be repeated here.

[0153] Step S607: Obtain the snapshot rollback result from the target client based on the shared service. The snapshot rollback result includes the data and function configuration within the namespace of the target snapshot corresponding to the snapshot rollback instruction. For details, please refer to step S507 above, which will not be repeated here.

[0154] Step S608: Compare the namespace data with the snapshot rollback result. For details, please refer to step S508 above, which will not be repeated here.

[0155] Step S609: If the namespace data and the snapshot rollback result are successfully compared, the snapshot rollback is determined to be successful. For details, please refer to step S509 above, which will not be repeated here.

[0156] Step S610: After determining that the snapshot rollback is successful, delete the data and function configuration in the current namespace, delete the created target snapshot, shut down the shared service of the current namespace, and delete the data in the current namespace and the data when the initial namespace was created.

[0157] It is understandable that the data cleaning process in step S610 above is the reverse process of steps S601 to S609 above. Specifically, as follows... Figure 13 The process involves sequentially deleting the data written to NFS, FTP, and S3 shares, the created snapshot $SNAP_NAME, NFS share, FTP share, namespace $NAMESPACE_NAME, S3 user $S3_USER_NAME, tenant $TENANT_NAME, authenticated user $USER_NAME, and authenticated user group $GROUP_NAME, thereby freeing up storage system space and resources.

[0158] The snapshot method for the distributed storage system provided in this embodiment further cleans up the discarded files of the distributed storage system and releases storage space after the task is completed, thus further improving the usability of the distributed storage system.

[0159] See Figure 14The overall snapshot control process of the present invention will be described below with a specific embodiment:

[0160] Namespace creation: Python methods are used to create an authentication user group and the authentication users belonging to that group; a tenant is created, along with its S3 users, and a mapping between S3 users and authentication users is established. Then, an initial namespace is created using the tenant and authentication users, allowing clients to access the initial namespace through the authentication users and S3 users.

[0161] Namespace sharing: Create an NFS share on the initial namespace so that storage clients (NFS clients) can access the share using an authenticated user; create an FTP share on the initial namespace so that FTP clients can access the share using an authenticated user; enable an S3 protocol share on the initial namespace so that S3 protocol clients can access the share using an S3 user.

[0162] Feature Configuration: Configures the initial namespace with various features. Examples include one or more of the following: file count quotas, capacity quotas, access control list settings, WORM settings, bucket policy settings, anti-hotlinking settings, static website hosting settings, bucket tags, and object tag settings.

[0163] Data writing: Files are written to the initial namespace via NFS shares, FTP shares, and S3 protocol shares. Specifically, for NFS shares, the python paramiko command is first used to connect to the client server, and the mount command is used to mount the NFS share. Then, commands such as dd, echo, and touch are used to create files and write them to the namespace. For S3 protocol shares, the python paramiko command is used to connect to the client server, and the doscmd command is used to perform operations such as uploading, querying, and deleting files in the initial namespace for reading and writing. For FTP shares, the python ftplib library is used to connect to the FTP service and upload and download files.

[0164] Snapshot creation: Create a snapshot of the current namespace at one or more points in time during feature configuration and data writing to record the data and feature configuration information in the current namespace.

[0165] Snapshot rollback: At a point in time when the original business needs to be restored, a snapshot rollback is performed on the current namespace. The data and functional configurations in the current namespace will be restored to the point in time when the snapshot was created.

[0166] Snapshot Verification: After a snapshot rollback, the rolled-back data is compared with the data records at the time the snapshot was created, and the rolled-back feature configuration is compared with the feature configuration records at the time the snapshot was created, to confirm the correctness of the rolled-back data and namespace feature configuration. If any verification of the rolled-back data or namespace feature configuration fails, the program exits abnormally. If the verification succeeds, the program exits normally or proceeds to data cleanup.

[0167] Data cleanup: After the functional configuration of data and namespaces passes verification, delete the data and snapshots of the current namespace, cancel shared services, delete the namespace, and delete S3 users, authenticated user groups, authenticated users, and tenants.

[0168] This embodiment also provides a snapshot device for a distributed storage system, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0169] This embodiment provides a snapshot device for a distributed storage system, such as... Figure 15 As shown, it includes:

[0170] Namespace creation module 701 is used to create the initial namespace of the distributed storage system;

[0171] Namespace sharing module 702 is used to enable sharing services on the initial namespace;

[0172] Data processing module 703 is used to perform target processing on the initial namespace based on the shared service to obtain the current namespace. The target processing includes writing data and / or configuring functions.

[0173] The snapshot creation module 704 is used to create a snapshot of the current namespace and determine the target snapshot.

[0174] In some alternative implementations, the namespace creation module 701 includes:

[0175] The first user creation unit is used to create the authentication user group of the distributed storage system and the authentication users belonging to the authentication user group;

[0176] The second user creation unit is used to create tenants of the distributed storage system and simple storage users belonging to the tenants;

[0177] The user relationship mapping unit is used to construct the mapping relationship between the authenticated user and the simple storage user;

[0178] The namespace creation unit is used to create an initial namespace using the tenant and the authenticated users under the authenticated user group.

[0179] In some alternative implementations, the data processing module 703 includes:

[0180] A storage protocol determination unit is used to determine the storage protocol corresponding to the current shared service;

[0181] The storage service processing unit is used to perform target processing on the initial namespace based on the storage protocol to obtain the current namespace.

[0182] In some optional implementations, the storage service processing unit includes:

[0183] The NFS protocol subunit is used to connect to the NFS client based on the NFS protocol if the storage protocol is NFS, mount the initial namespace to the local path of the NFS client, and write data to the local path of the NFS client to obtain the current namespace.

[0184] The FTP protocol subunit is used to create an FTP connection object based on the FTP protocol if the storage protocol is FTP, and write the local files of the FTP client to the initial namespace through the FTP connection object to obtain the current namespace;

[0185] The S3 protocol subunit is used to connect to the S3 protocol client based on the S3 protocol if the storage protocol is the S3 protocol, and write the local file of the S3 protocol client into the initial namespace to obtain the current namespace.

[0186] In some alternative embodiments, the apparatus further includes:

[0187] The snapshot rollback module is used to roll back the current namespace to a target time in response to a snapshot rollback command. The target time is the time node when the target snapshot corresponding to the snapshot rollback command was created.

[0188] The snapshot verification module is used for:

[0189] Obtain the namespace data at the target time, the namespace data including the data and function configuration within the namespace at the target time;

[0190] Based on the shared service, the snapshot rollback result is obtained from the target client. The snapshot rollback result includes the data and function configuration in the namespace of the target snapshot corresponding to the snapshot rollback instruction.

[0191] Compare the namespace data with the snapshot rollback results;

[0192] If the namespace data is successfully compared with the snapshot rollback result, the snapshot rollback is determined to be successful.

[0193] If the namespace data fails to match the snapshot rollback result, the snapshot rollback is determined to have failed.

[0194] In some alternative embodiments, the apparatus further includes:

[0195] The data cleaning module is used for:

[0196] Delete the data and function configurations in the current namespace;

[0197] Delete the target snapshot that was created;

[0198] Disable the shared services of the current namespace;

[0199] Delete the current namespace and the data used when the initial namespace was created.

[0200] In some optional implementations, the configuration items in the function configuration include at least one of file number quota, capacity quota, access control list configuration, WORM configuration, bucket policy configuration, anti-hotlinking configuration, static website hosting configuration, bucket tag configuration, and object tag configuration.

[0201] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0202] In this embodiment, the snapshot device of the distributed storage system is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0203] This invention also provides a computer device having the above-described features. Figure 15 The snapshot device of the distributed storage system shown.

[0204] Please see Figure 16 , Figure 16 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 16As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 16 Take a processor 10 as an example.

[0205] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0206] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0207] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0208] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0209] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0210] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0211] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A snapshot method of a distributed storage system, characterized by, The method comprises: creating an initial namespace of a distributed storage system; starting a shared service on the initial namespace; determining a storage protocol corresponding to the current shared service; target processing the initial namespace based on the storage protocol to obtain a current namespace, the target processing including writing data and / or function configuration; creating a snapshot of the current namespace to determine a target snapshot; in response to a snapshot rollback instruction, rolling back the current namespace to a target time, the target time being a time node when the target snapshot corresponding to the snapshot rollback instruction is created; obtaining namespace data at the target time, the namespace data including data and function configuration in the namespace at the target time; obtaining a snapshot rollback result based on the shared service to a target client, the snapshot rollback result including data and function configuration in the namespace under the target snapshot corresponding to the snapshot rollback instruction; comparing the namespace data with the snapshot rollback result; if the comparison of the namespace data with the snapshot rollback result is successful, determining that the snapshot rollback is successful; if the comparison of the namespace data with the snapshot rollback result fails, determining that the snapshot rollback fails; wherein, when the target processing is writing data, the target processing of the initial namespace based on the storage protocol to obtain the current namespace comprises: if the storage protocol is an NFS protocol, connecting to an NFS client based on the NFS protocol, mounting the initial namespace to a local path of the NFS client, and writing data to the local path of the NFS client to obtain the current namespace; if the storage protocol is an FTP protocol, creating an FTP connection object based on the FTP protocol, and writing a local file of an FTP client to the initial namespace through the FTP connection object to obtain the current namespace; if the storage protocol is an S3 protocol, connecting to an S3 protocol client based on the S3 protocol, and writing a local file of the S3 protocol client to the initial namespace to obtain the current namespace.

2. The method of claim 1, wherein, The method further comprises: creating an initial namespace of a distributed storage system, comprising: creating an authentication user group and an authentication user belonging to the authentication user group of the distributed storage system; creating a tenant and a simple storage user belonging to the tenant of the distributed storage system; building a mapping relationship between the authentication user and the simple storage user; 3. The method of claim 1, wherein, creating an initial namespace through the tenant and the authentication user in the authentication user group. After determining that the snapshot rollback is successful, the method further comprises: deleting data and function configuration in the current namespace; deleting the created target snapshot; closing the shared service of the current namespace; deleting the current namespace and data when creating the initial namespace.

4. The method of claim 1, wherein, The configuration items in the function configuration include at least one of a file quantity quota, a capacity quota, an access control list configuration, a WORM configuration, a bucket policy configuration, an anti-hotlinking configuration, a static website hosting configuration, a bucket tag configuration, and an object tag configuration.

5. A snapshot apparatus of a distributed storage system, characterized by comprising: The apparatus comprises: A namespace creation module configured to create an initial namespace of a distributed storage system; A namespace sharing module configured to start a sharing service on the initial namespace; A data processing module configured to perform target processing on the initial namespace based on the sharing service, to obtain a current namespace, the target processing including writing data and / or function configuration; A snapshot creation module configured to create a snapshot of the current namespace, to determine a target snapshot; A snapshot rollback module configured to, in response to a snapshot rollback instruction, roll back the current namespace to a target time point, the target time point being a time node at which the target snapshot corresponding to the snapshot rollback instruction is created; A snapshot verification module configured to: Obtain namespace data at the target time point, the namespace data including data and function configuration in the namespace at the target time point; Obtain a snapshot rollback result from a target client based on the sharing service, the snapshot rollback result including data and function configuration in the namespace under the target snapshot corresponding to the snapshot rollback instruction; Compare the namespace data with the snapshot rollback result; If the comparison of the namespace data and the snapshot rollback result is successful, determine that the snapshot rollback is successful; If the comparison of the namespace data and the snapshot rollback result fails, determine that the snapshot rollback fails; The data processing module comprises: A storage protocol determination unit configured to determine a storage protocol corresponding to a current sharing service; A storage service processing unit configured to perform target processing on the initial namespace based on the storage protocol, to obtain a current namespace; The storage service processing unit comprises: An NFS protocol subunit configured to, if the storage protocol is an NFS protocol, connect to an NFS client based on the NFS protocol, mount the initial namespace to a local path of the NFS client, and write data to the local path of the NFS client, to obtain the current namespace; An FTP protocol subunit configured to, if the storage protocol is an FTP protocol, create an FTP connection object based on the FTP protocol, and write a local file of an FTP client to the initial namespace through the FTP connection object, to obtain the current namespace; An S3 protocol subunit configured to, if the storage protocol is an S3 protocol, connect to an S3 protocol client based on the S3 protocol, and write a local file of the S3 protocol client to the initial namespace, to obtain the current namespace.

6. A computer device, comprising: comprise: a memory and a processor, which are in communication connection with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the snapshot method of the distributed storage system according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium has stored thereon computer instructions for causing a computer to execute the snapshot method of the distributed storage system of any one of claims 1 to 4.

Images