A file conversion method, device, equipment and storage medium

Abstract

The application discloses a file mutual conversion method and device, equipment and storage medium, and relates to the technical field of distribution, which comprises the following steps: creating a first authentication user group and a first authentication user; creating a first simple storage service user associated with the first authentication user; creating a target bucket by using the created second simple storage service user and uploading a target object to the target bucket; creating a namespace by using the first authentication user group and the first authentication user and creating an NFS share by using the namespace; respectively creating a second authentication user group and a second authentication user capable of being bound with the first authentication user group and the first authentication user on a client; mounting a shared directory and writing a file to be shared on the client; copying the target object to the namespace by using the second authentication user, and copying the file to be shared to the target bucket by using the first authentication user, so that the file scene and the object scene can be integrated, and the file in the shared directory and the bucket can be mutually converted and operated.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, device and storage medium for mutual file conversion. Background Art

[0002] Currently, automated operations for massive storage have been widely adopted in the IT (Information Technology) industry, cloud computing, big data, virtualization, and other fields. At the same time, the demand for integrating file and object scenarios is growing. Supporting only file operations within buckets or shared directories in file scenarios is no longer sufficient. There is a need to integrate file and object scenarios, enabling conversion between shared directories and buckets. For example, Unix users in the file scenario can operate on objects within buckets, while S3 (Simple Storage Service) users in the object scenario can operate on objects in shared directories.

[0003] However, the current automation platform for mass storage distributed systems cannot support operations such as file conversion between directories and buckets, which seriously affects the robustness, ease of use, and rich functionality of the system, as well as the competitiveness of the mass storage automation platform. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a file conversion method, device, equipment, and storage medium that can integrate the file scenario with the object scenario, enable the conversion and operation of files in shared directories and buckets, and thus improve the robustness, usability, and richness of functions of the distributed system. The specific solution is as follows:

[0005] In a first aspect, the present application discloses a method for mutual file conversion, which is applied to a distributed storage cluster, comprising:

[0006] Creating a first authentication user group, and creating a first authentication user belonging to the first authentication user group;

[0007] Creating a first simple storage service user associated with the first authenticated user;

[0008] Create an independent second Simple Storage Service user, use the second Simple Storage Service user to create a target bucket, and then upload the target object to the target bucket;

[0009] Creating a namespace using the first authenticated user group and the first authenticated user, and creating an NFS share using the namespace to share the directory of the distributed storage cluster externally;

[0010] creating, on the client side, a second authentication user group and a second authentication user that can be bound to the first authentication user group and the first authentication user, respectively, and assigning the second authentication user to the second authentication user group;

[0011] Mount a shared directory on the client and write the file to be shared into the shared directory;

[0012] The target object in the target bucket is copied and transferred to the namespace through the second authenticated user, and the file to be shared is copied and transferred from the namespace to the target bucket through the first authenticated user.

[0013] Optionally, mounting a shared directory on the client and writing a file to be shared in the shared directory includes:

[0014] Log in to the client through the second authenticated user, mount a shared directory on the client, and write the file to be shared into the shared directory;

[0015] Set the access control list permissions of the target bucket and the namespace to public read and write.

[0016] Optionally, before creating the first authentication user group, the method further includes:

[0017] Determine whether the gateway service of all nodes in the distributed storage cluster is enabled;

[0018] If the gateway services of all nodes in the distributed storage cluster are not enabled, a prompt message for enabling all gateway object services is generated to enable all gateway object services.

[0019] Optionally, the file conversion method further includes:

[0020] Verify that the NFS file service of all nodes in the distributed storage cluster is enabled;

[0021] If the NFS file services of all nodes in the distributed storage cluster are not enabled, prompt information for enabling all the NFS file services is generated to enable all the NFS file services.

[0022] Optionally, after creating the first authentication user belonging to the first authentication user group, the method further includes:

[0023] Obtaining the identity identification codes of the first authenticated user group and the first authenticated user respectively to obtain a first identity identification code and a second identity identification code;

[0024] The first identity identification code and the second identity identification code are recorded and stored persistently.

[0025] Optionally, the creating, on the client side, a second authentication user group and a second authentication user that can be bound to the first authentication user group and the first authentication user, respectively, includes:

[0026] creating a second authentication user group having the same name as the first authentication user group using the first authentication user group and the first identity identification code;

[0027] A second authentication user having the same name as the first authentication user is created using the second authentication user group and the second identity identification code.

[0028] Optionally, after using the first authenticated user to copy and transfer the file to be shared from the namespace to the target bucket, the method further includes:

[0029] Reading the content of the target object in the shared directory of the client to obtain first content, and verifying the first content. If the verification passes, generating a prompt message indicating that the bucket has been successfully converted to the shared directory;

[0030] The second authenticated user downloads and reads the content of the file to be shared from the target bucket to obtain the second content, and verifies the second content. If the verification passes, a prompt message is generated that the shared directory is successfully converted to the bucket.

[0031] In a second aspect, the present application discloses a file conversion device, which is applied to a distributed storage cluster, comprising:

[0032] A first creation module, configured to create a first authentication user group and a first authentication user belonging to the first authentication user group;

[0033] A second creation module is configured to create a first simple storage service user associated with the first authentication user;

[0034] A third creation module is configured to create an independent second simple storage service user and create a target bucket using the second simple storage service user;

[0035] An upload module, configured to upload a target object to the target bucket;

[0036] a fourth creation module, configured to create a namespace using the first authenticated user group and the first authenticated user, and to create an NFS share using the namespace to share the directory of the distributed storage cluster externally;

[0037] A fifth creation module, configured to create, on the client side, a second authentication user group and a second authentication user that can be bound to the first authentication user group and the first authentication user;

[0038] an allocating module, configured to allocate the second authenticated user to the second authenticated user group;

[0039] A shared directory mounting module, used to mount a shared directory on the client and write files to be shared in the shared directory;

[0040] The file conversion module is configured to copy and transfer the target object in the target bucket to the namespace through the second authenticated user, and simultaneously copy and transfer the file to be shared from the namespace to the target bucket using the first authenticated user.

[0041] In a third aspect, the present application discloses an electronic device comprising a processor and a memory; wherein the processor implements the aforementioned file conversion method when executing a computer program stored in the memory.

[0042] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned file conversion method is implemented.

[0043] It can be seen that this application first creates a first authentication user group and creates a first authentication user belonging to the first authentication user group, then creates a first simple storage service user associated with the first authentication user, and then creates an independent second simple storage service user, and uses the second simple storage service user to create a target bucket, and then uploads the target object to the target bucket, and then uses the first authentication user group and the first authentication user to create a namespace, and uses the namespace to create an NFS share to share the directory of the distributed storage cluster externally, and then creates a second authentication user group and a second authentication user on the client that can be bound to the first authentication user group and the first authentication user, and uploads all the objects to the target bucket. The second authenticated user is assigned to the second authenticated user group, then the shared directory is mounted on the client, and the file to be shared is written in the shared directory. Finally, the target object in the target bucket is copied and transferred to the namespace through the second authenticated user, and the file to be shared is copied and transferred from the namespace to the target bucket using the first authenticated user. In this way, the file scenario and the object scenario can be integrated, and the files in the shared directory and the bucket can be converted and operated on each other, so that users in the file scenario can operate objects from the bucket, and users in the object scenario can operate objects from the shared directory, thereby improving the robustness, usability and richness of functions of the distributed system. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0045] Figure 1 This is a flow chart of a method for mutual conversion of files disclosed in this application;

[0046] Figure 2 A flowchart of a specific method for mutual file conversion disclosed in this application;

[0047] Figure 3 This is a flowchart of a specific shared directory and bucket file intercommunication operation control method disclosed in this application;

[0048] Figure 4 This is a functional diagram of a specific GateWay service processing module disclosed in this application;

[0049] Figure 5 This is a functional diagram of a specific NFS service processing module disclosed in this application;

[0050] Figure 6 A specific virtual IP processing module disclosed in this application;

[0051] Figure 7 This is a functional diagram of a specific file user module disclosed in this application;

[0052] Figure 8 This is a functional diagram of a specific S3 and file user association module disclosed in this application;

[0053] Figure 9 This is a functional diagram of a specific S3 bucket and object module disclosed in this application;

[0054] Figure 10 This is a functional diagram of a specific namespace and NFS sharing module disclosed in this application;

[0055] Figure 11 This is a functional diagram of a specific client user and authentication user binding module disclosed in this application;

[0056] Figure 12 This is a functional diagram of a specific NFS mount write module disclosed in this application;

[0057] Figure 13This is a functional diagram of a specific S3 bucket and file directory permission setting module disclosed in this application;

[0058] Figure 14 This is a functional diagram of a specific S3 bucket and file directory permission setting module disclosed in this application;

[0059] Figure 15 This is a functional diagram of a file conversion self-check module disclosed in this application;

[0060] Figure 16 This is a schematic structural diagram of a file conversion device disclosed in this application;

[0061] Figure 17 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0063] The present application discloses a method for converting files to each other. Figure 1 As shown, applied to a distributed storage cluster, the method includes:

[0064] Step S11: creating a first authentication user group, and creating a first authentication user belonging to the first authentication user group.

[0065] In this embodiment, a first authentication user group is first created for the file scenario, and then a first authentication user belonging to the first authentication user group is created. For example, an authentication user group $AUTH_GROUP is first created, and then a Unix authentication user $AUTH_USER belonging to the group is created.

[0066] Step S12: Create a first simple storage service user associated with the first authentication user.

[0067] In this embodiment, after creating a first authentication user group and a first authentication user belonging to the first authentication user group, a simple storage service (i.e., S3) user associated with the above-mentioned first authentication user is further created, such as creating S3 user 1 $S3_USER_1 associated with the above-mentioned $AUTH_USER.

[0068] Step S13: Create an independent second Simple Storage Service user, use the second Simple Storage Service user to create a target bucket, and then upload the target object to the target bucket.

[0069] In this embodiment, after creating a first Simple Storage Service user associated with the first authenticated user, a separate Simple Storage Service user is created to obtain a second Simple Storage Service user. This second Simple Storage Service user is then used to create a bucket, and the target object is uploaded to this bucket. For example, a separate S3 user $S3_USER_2 is first created. Then, using S3 user $S3_USER_2, bucket $BUCKET_1 is created, and object $OBJECT_1 is uploaded to $BUCKET_1. The content of object $OBJECT_1 is "12345".

[0070] Step S14: creating a namespace using the first authenticated user group and the first authenticated user, and creating an NFS share using the namespace to share the directory of the distributed storage cluster externally.

[0071] In this embodiment, after uploading the target object to the target bucket, in order to share the directory in the distributed storage cluster externally, a namespace can be created using the first authenticated user group and the first authenticated user, and then an NFS (Network File System) share can be created using the created namespace. For example, the authenticated user $AUTH_USER can be used to create the namespace $NAME_SAPCE, and then the namespace $NAME_SPACE can be used to create an NFS share.

[0072] Step S15: creating a second authentication user group and a second authentication user on the client that can be bound to the first authentication user group and the first authentication user, and assigning the second authentication user to the second authentication user group.

[0073] In this embodiment, after creating an NFS share using the namespace, a second authentication user group that can be bound to the first authentication user group and the first authentication user and a second authentication user belonging to the authentication user group are created on the client, and then the second authentication user is assigned to the second authentication user group. In a specific implementation, the gid (Group ID) of the authentication user group $AUTH_GROUP is first obtained to obtain $AUTH_GROUP_ID, and then a client authentication user group $CLIENT_GROUP is created on the client with a gid of $AUTH_GROUP_ID and a user group name of $AUTH_GROUP, and a unix user $CLIENT_USER with the same name as the authentication user $AUTH_USER is created, and then the $CLIENT_USER is assigned to the $CLIENT_GROUP group.

[0074] Step S16: Mount the shared directory on the client, and write the file to be shared into the shared directory.

[0075] Furthermore, a shared directory is mounted on the client, and the file to be shared is written into the shared directory.

[0076] Specifically, mounting the shared directory on the client and writing the file to be shared into the shared directory may include: logging in to the client using the second authenticated user, mounting the shared directory on the client, and writing the file to be shared into the shared directory; and setting the access control list permissions of the target bucket and the namespace to public read and write. Specifically, the client may be logged in using the second authenticated user, then mounting the shared directory on the client, then writing the file to be shared into the shared directory, and setting the access control list (ACL) permissions of the target bucket and the namespace to public read and write. For example, use the above user $CLIENT_USER to log in to the client, and use NFS to mount the above namespace $NAME_SPACE on the client. Then, mount the shared directory $NFS_MOUNT_DIR in the above namespace $NAME_SPACE. Then, use the client to create a file $FILE_2 in the above shared directory $NFS_MOUNT_DIR with the content "67890". Then, set the permission $BUCEKT_PRIORITY of the above bucket $BUCKET_1 to public read and write (PRW), and set the permission $DIR_PRIORITY of the above namespace $SPACE_NAME to public read and write (PRW).

[0077] Step S17: The target object in the target bucket is copied and transferred to the namespace through the second authenticated user, and the file to be shared is copied and transferred from the namespace to the target bucket using the first authenticated user.

[0078] In this embodiment, after writing the file to be shared to the shared directory, the second authenticated user can copy and transfer the target object in the target bucket to the namespace, while the first authenticated user can also copy and transfer the file to be shared from the namespace to the target bucket. For example, $S3_USER_2 can copy and transfer the object $OBJECT_1 in bucket $BUCKET_1 to the shared directory $NFS_MOUNT_DIR; and $S3_USER_1 can also copy and transfer the file $FILE_2 to bucket $BUCKET_1.

[0079] Furthermore, after using the first authenticated user to copy and transfer the file to be shared from the namespace to the target bucket, the process may also include: reading the content of the target object in the shared directory of the client to obtain the first content, and verifying the first content. If the verification passes, a prompt message indicating that the bucket has been successfully converted to a shared directory is generated; downloading and reading the content of the file to be shared from the target bucket by the second authenticated user to obtain the second content, and verifying the second content. If the verification passes, a prompt message indicating that the shared directory has been successfully converted to a bucket is generated. For example, after the object $OBJECT_1 in the bucket $BUCKET_1 is copied and transferred to the above-mentioned shared directory $NFS_MOUNT_DIR by the above-mentioned $S3_USER_2, if the copy and transfer is successful, 1 is returned, otherwise 0 is returned and an error is reported; while the client is logged in using $CLIENT_USER, if $OBJECT_1 can be found in the above-mentioned shared directory $NFS_MOUNT_DIR, and the content of $NFS_MOUNT_DIR / $OBEJCT_1 is read as "123 45", appending "54321" successfully, the program returns 1, indicating that the file is successfully transferred from the bucket to the shared directory; otherwise, it returns 0 and reports an error. Use the above $S3_USER_1 to copy and transfer the file $FILE_2 to the bucket $BUCKET_1. If the copy and transfer is successful, it returns 1; otherwise, it returns 0 and reports an error. Then, on the distributed storage cluster side, download $FILE_2 from the above $BUCKET_1 and check whether the content of $FILE_2 is "67890". If so, the program returns 1; otherwise, it returns 0 and reports an error.

[0080] It can be seen that the embodiment of the present application first creates a first authentication user group and creates a first authentication user belonging to the first authentication user group, then creates a first simple storage service user associated with the first authentication user, and then creates an independent second simple storage service user, and uses the second simple storage service user to create a target bucket, and then uploads the target object to the target bucket, and then uses the first authentication user group and the first authentication user to create a namespace, and uses the namespace to create an NFS share to share the directory of the distributed storage cluster externally, and then creates a second authentication user group and a second authentication user on the client that can be bound to the first authentication user group and the first authentication user, and Assign the second authenticated user to the second authenticated user group, then mount the shared directory on the client, and write the file to be shared in the shared directory. Finally, use the second authenticated user to copy and transfer the target object in the target bucket to the namespace, and use the first authenticated user to copy and transfer the file to be shared from the namespace to the target bucket. In this way, the file scenario and the object scenario can be integrated, and the files in the shared directory and the bucket can be converted and operated on each other. Users in the file scenario can operate objects from the bucket, and users in the object scenario can operate objects from the shared directory, thereby improving the robustness, usability and functional richness of the distributed system.

[0081] The present application embodiment discloses a specific method for mutual conversion of files, which is applied to a distributed storage cluster. Figure 2 As shown, the method includes:

[0082] Step S21: Determine whether the gateway service of all nodes in the distributed storage cluster is enabled.

[0083] In this embodiment, it is first determined whether the gateway service of all nodes in the distributed storage cluster is enabled. For example, verify_all_node_s3_gateway_service() is used to determine whether the gateway service of all nodes in the distributed storage cluster is enabled.

[0084] Step S22: If the gateway services of all nodes in the distributed storage cluster are not enabled, a prompt message for enabling all gateway object services is generated to enable all gateway object services.

[0085] In this embodiment, if the gateway services of all nodes in the distributed storage cluster are not enabled, a prompt message for enabling all gateway object services may be generated, and the user may perform corresponding operations to enable all gateway object services after receiving the prompt message.

[0086] Step S23: Check whether the NFS file service of all nodes in the distributed storage cluster is enabled.

[0087] Furthermore, it is verified whether the NFS file service of all nodes in the distributed storage cluster is enabled. For example, verify_nfs_service() is used to verify whether the NFS file service of all nodes in the distributed storage cluster is enabled.

[0088] Step S24: If the NFS file services of all nodes in the distributed storage cluster are not enabled, a prompt message for enabling all the NFS file services is generated to enable all the NFS file services.

[0089] In this embodiment, if the NFS file service is not enabled on all nodes in the distributed storage cluster, a prompt message is generated to enable all NFS file services, thereby reminding the user to enable all NFS file services. Furthermore, get_ctdb_vir_ip_list() can be used to obtain the 10G CTDB (Cluster Trivial Database) virtual IP (Internet Protocol) of the distributed storage cluster for subsequent NFS mounting.

[0090] Step S25: Create a first authentication user group, and create a first authentication user belonging to the first authentication user group.

[0091] In this embodiment, after all the NFS file services are enabled, a first authenticated user group and a first authenticated user belonging to the first authenticated user group are created. For example, the command create_authenticated_user_group($AUTH_USER,$AUTH_GROUP) is used to create a local authenticated user group and authenticated user in Linux.

[0092] Step S26: respectively obtaining the identity identification codes of the first authenticated user group and the first authenticated user to obtain a first identity identification code and a second identity identification code.

[0093] In this embodiment, after creating a first authentication user group and creating a first authentication user belonging to the first authentication user group, you can further obtain the identity identification code (ID, Identity document) of the above-mentioned first authentication user group and the above-mentioned first authentication user to obtain the first identity identification code and the second identity identification code, such as recording the authentication user id: $AUTH_USER_ID and the authentication user group id: $AUTH_GROUP_ID.

[0094] Step S27: Record the first identity identification code and the second identity identification code, and store the first identity identification code and the second identity identification code persistently.

[0095] In this embodiment, after obtaining the first identity identification code and the second identity identification code, the first identity identification code and the second identity identification code are recorded, and then the first identity identification code and the second identity identification code are persistently stored.

[0096] Step S28: Create a first simple storage service user associated with the first authenticated user.

[0097] Furthermore, a first Simple Storage Service user associated with the first authenticated user is created. For example, create_user_s3_nas_mix($S3_USER_1,$AUTH_USER) is used to create an S3 user $S3_USER_1 associated with the first authenticated user.

[0098] Step S29: Create an independent second Simple Storage Service user, use the second Simple Storage Service user to create a target bucket, and then upload the target object to the target bucket.

[0099] In this embodiment, after creating a first Simple Storage Service user associated with the first authenticated user, a second independent Simple Storage Service user is created. This second Simple Storage Service user is then used to create a target bucket, and the target object is then uploaded to the target bucket. For example, an independent S3 user 2 ($S3_USER_2) is created using create_user_s3_nas_mix($S3_USER_2,NONE). Then, using $S3_USER_2's key pair, bucket $BUCKET_1 is created using create_bucket($BUCKET_1,$S3_USER_2_KEY). Finally, object $OBJECT_1 is uploaded to bucket $BUCKET_1 using s3_put_object($OBJECT_1,$BUCKET_1).

[0100] Step S210: Create a namespace using the first authenticated user group and the first authenticated user, and create an NFS share using the namespace to share the directory of the distributed storage cluster externally.

[0101] Next, use the persistent $AUTH_USER and $AUTH_GROUP to create the namespace $SPACE_NAME through create_namespace($SPACE_NAME,$AUTH_USER,$AUTH_GROUP). Then, use set_nfs_share($SPACE_NAME) to create an NFS share for the created namespace $SPACE_NAME to share the directory $SPACE_NAME in the cluster externally.

[0102] Step S211: using the first authentication user group and the first identity identification code, a second authentication user group having the same name as the first authentication user group is created.

[0103] In this embodiment, after creating a namespace using the first authentication user group and the first authentication user, and using the namespace to create an NFS share, the first authentication user group and the first identity code can be used to create a second authentication user group with the same name as the first authentication user group. For example, on the client, group_add($AUTH_GROUP,$AUTH_GROUP_ID) is used to create a Linux local user group $CLIENT_GROUP that can be bound to the cluster $AUTH_GROUP.

[0104] Step S212: using the second authentication user group and the second identity identification code, create a second authentication user with the same name as the first authentication user, and assign the second authentication user to the second authentication user group.

[0105] Furthermore, a second authentication user with the same name as the first authentication user is created using the second authentication user group and the second identity code, and then the second authentication user is assigned to the second authentication user group. For example, a Linux local user $CLIENT_USER that can be bound to the cluster $AUTH_USER is created using user_add($AUTH_USER,$AUTH_USER_ID), and then the $CLIENT_USER is assigned to the $CLIENT_GROUP.

[0106] Step S213: Mount the shared directory on the client, and write the file to be shared into the shared directory.

[0107] For example, use nfs_mount($NFS_MOUNT_DIR,$VIRTUAL_IP,$NAME_SPACE) to mount the shared directory $NAME_SPACE on the client. Then, log in to the client using $CLIENT_USER and write $FILE_2 to the mounted directory $NFS_MOUNT_DIR. Set the group ownership of $FILE_2 to $CLIENT_USER. Next, use the command set_name_sapce_acl("PWR",$NAME_SPACE) to set the namespace's ACL permissions to public read and write.

[0108] Step S214: The target object in the target bucket is copied and transferred to the namespace through the second authenticated user, and the file to be shared is copied and transferred from the namespace to the target bucket using the first authenticated user.

[0109] For example, s3_copy_object($S3_USER_2,$BUCKET_1,$OBJECT_1,$NAME_SPACE) is then used to copy and transfer the above-mentioned file to be shared from the target bucket to the above-mentioned namespace, and name_spae_copy_file($S3_USER_1,$NAME_SPACE,$FILE_2,$BUCKET_1) is used to copy and transfer the above-mentioned file to be shared from the command space to the above-mentioned target bucket.

[0110] Furthermore, check_object_to_nas() is used to check $OBJECT_1 in the client's shared directory $NFS_MOUNT_DIR. The content of $OBJECT_1 is read as "12345". If "67890" is appended to $OBJECT_1 successfully, the program returns 1; otherwise, it returns 0. On the cluster side, $S3_USER_2 is used to download and read the content from the bucket through check_nas_to_object(). If the content is "67890", the program returns 1 for success; otherwise, it returns 0 for an error.

[0111] It can be seen that the embodiment of the present application proposes a method that supports the mutual conversion of files between shared directories and buckets, which can integrate the file scenario and the object scenario, and enable the files in the shared directory and the bucket to be converted and operated on each other, such as enabling Linux users in the file scenario to operate objects from the bucket, and S3 users in the object scenario to operate objects from the shared directory.

[0112] The present application embodiment discloses a specific file conversion system applied to a distributed storage cluster, see Figure 3As shown, the system specifically includes: GateWay service processing module, NFS service processing module, virtual IP processing module, file user module, S3 and file user association module, S3 user module, S3 bucket and object module, namespace and NFS sharing module, client user and authenticated user binding module, NFS mount write module, S3 bucket and file directory permission setting module, shared directory and bucket file transfer module, and file conversion self-check module. 13 modules.

[0113] For details, see Figure 4 As shown, first use verify_all_node_s3_gateway_service() to determine whether the gateway services of all nodes in the distributed storage cluster are enabled. If not all are enabled, it will prompt that all gateway object services need to be enabled; then, see Figure 5 As shown, use verify_nfs_service() to verify whether the NFS file service of all nodes in the cluster is enabled. If not all are enabled, the user is prompted to enable them all; then refer to Figure 6 As shown, use get_ctdb_vir_ip_list() to obtain the cluster's 10G CTDB virtual IP for subsequent NFS mounting; then see Figure 7 As shown, use create_authenticated_user_group($AUTH_USER,$AUTH_GROUP) to create the local authentication user group and authentication user of Linux, record the authentication user ID: $AUTH_USER_ID and the authentication user group ID: $AUTH_GROUP_ID, and save them persistently; for further information, see Figure 8 As shown, use create_user_s3_nas_mix($S3_USER_1,$AUTH_USER) to create an S3 user $S3_USER_1 associated with the authentication user, and then use create_user_s3_nas_mix($S3_USER_2,NONE) to create an independent S3 user 2, namely $S3_USER_2; then, see Figure 9As shown in the following example, use the key pair of S3 user 2 to create bucket $BUCKET_1 through create_bucket($BUCKET_1,$S3_USER_2_KEY), then use the key pair of $S3_USER_2 to create bucket $BUCKET_1 through create_bucket($BUCKET_1,$S3_USER_2_KEY), and then use s3_put_object($OBJECT_1,$BUCKET_1) to upload object $OBEJCT_1 to bucket $BUCKET_1. The content of $OBJECT_1 is "12345". For more information, see Figure 10 As shown, use the persistent $AUTH_USER and $AUTH_GROUP to create the namespace $SPACE_NAME through create_namespace($SPACE_NAME,$AUTH_USER,$AUTH_GROUP), and then use set_nfs_share($SPACE_NAME) to create an NFS share for the created namespace $SPACE_NAME to share the cluster directory $SPACE_NAME externally; then, see Figure 11 As shown, on the client, create a Linux local user group $CLIENT_GROUP that can be bound to the cluster $AUTH_GROUP through group_add($AUTH_GROUP,$AUTH_GROUP_ID), and create a Linux local user $CLIENT_USER that can be bound to the cluster $AUTH_USER through user_add($AUTH_USER,$AUTH_USER_ID); for further information, see Figure 12 As shown, mount the $NAME_SPACE shared directory on the client through nfs_mount($NFS_MOUNT_DIR,$VIRTUAL_IP,$NAME_SPACE), then log in to the client using $CLIENT_USER, and write $FILE_2 to the mounted directory $NFS_MOUNT_DIR, while ensuring that the file group of $FILE_2 is $CLIENT_USER; then, see Figure 13 As shown, use s3_set_bucket_acl("PWR",$BUCKET_1) to set the bucket's ACL permissions to public read and write, and use set_name_sapce_acl("PWR",$NAME_SPACE) to set the namespace's ACL permissions to public read and write; see also Figure 14As shown, use s3_copy_object($S3_USER_2,$BUCKET_1,$OBJECT_1,$NAME_SPACE) to copy the file to be shared from the bucket to the namespace, and then use name_spae_copy_file($S3_USER_1,$NAME_SPACE,$FILE_2,$BUCKET_1) to copy the file to be shared from the namespace to the bucket; finally, see Figure 15 As shown in the following example, check_object_to_nas() is used to see $OBJECT_1 in the shared directory $NFS_MOUNT_DIR on the client. The content of $OBJECT_1 is "12345". If "54321" is appended to $OBJECT_1 successfully, the program returns 1; otherwise, it returns 0. On the cluster side, $S3_USER_2 is used to download and read from the bucket through check_nas_to_object(). If the content is "67890", the program returns 1 for success; otherwise, it returns 0 with an error.

[0114] This application proposes a system that supports file conversion between shared directories and buckets. This system integrates file and object scenarios, enabling conversion and manipulation of files within shared directories and buckets. This system enables distributed storage systems to support operations such as file conversion between directories and buckets, improving system robustness, usability, and functionality, thereby enhancing the competitiveness of mass storage automation platforms.

[0115] Correspondingly, the embodiment of the present application also discloses a file conversion device, which is applied to a distributed storage cluster, see Figure 16 As shown, the device includes:

[0116] A first creation module 11 is configured to create a first authentication user group and a first authentication user belonging to the first authentication user group;

[0117] A second creation module 12 is configured to create a first simple storage service user associated with the first authentication user;

[0118] A third creation module 13 is configured to create an independent second simple storage service user and create a target bucket using the second simple storage service user;

[0119] An upload module 14 is configured to upload a target object to the target bucket;

[0120] A fourth creation module 15 is configured to create a namespace using the first authenticated user group and the first authenticated user, and to create an NFS share using the namespace to share the directory of the distributed storage cluster externally;

[0121] a fifth creating module 16, configured to create a second authentication user group and a second authentication user capable of being bound with the first authentication user group and the first authentication user respectively at a client;

[0122] a distributing module 17, configured to distribute the second authentication user into the second authentication user group;

[0123] a shared directory mounting module 18, configured to mount a shared directory at the client and write a to-be-shared file in the shared directory;

[0124] a file converting module 19, configured to copy and transmit the target object in the target bucket to the namespace by the second authentication user, and copy and transmit the to-be-shared file from the namespace to the target bucket by the first authentication user.

[0125] The specific working procedures of the above modules can refer to the corresponding content disclosed in the foregoing embodiments, and will not be described here in detail.

[0126] It can be seen that in the embodiments of the present application, the first authentication user group is first created, and then the first authentication user belonging to the first authentication user group is created, the first authentication user associated with the first authentication user is then created, the independent second authentication user is created, and the target bucket is created by using the second authentication user, the target object is uploaded into the target bucket, the namespace is created by using the first authentication user group and the first authentication user, the NFS share is created by using the namespace, and the directory of the distributed storage cluster is shared externally, the second authentication user group and the second authentication user capable of being bound with the first authentication user group and the first authentication user are then created at the client, the second authentication user is distributed into the second authentication user group, the shared directory is mounted at the client, and the to-be-shared file is written in the shared directory, and finally the target object in the target bucket is copied and transmitted to the namespace by the second authentication user, and the to-be-shared file is copied and transmitted from the namespace to the target bucket by the first authentication user, so that the file scenario and the object scenario are integrated, the shared directory and the file in the bucket are converted and operated, the user of the file scenario can operate the object from the bucket, the user of the object scenario can operate the object from the shared directory, and the robustness, ease of use and richness of functions of the distributed system are improved.

[0127] In some specific embodiments, the shared directory mounting module 18 can specifically include:

[0128] a login unit, configured to log in the client by the second authentication user;

[0129] A shared directory mounting unit, configured to mount a shared directory on the client and write files to be shared into the shared directory;

[0130] The permission setting unit is used to set the access control list permissions of the target bucket and the namespace to public read and write.

[0131] In some specific embodiments, before the first creation module 11, the following steps may also be included:

[0132] A determination unit, configured to determine whether the gateway service of all nodes in the distributed storage cluster is enabled;

[0133] The gateway service activation unit is configured to generate prompt information for activating all gateway object services if the gateway services of all nodes in the distributed storage cluster are not activated, so as to activate all gateway object services.

[0134] In some specific embodiments, the file conversion device may further include:

[0135] A verification unit, configured to verify whether the NFS file service of all nodes in the distributed storage cluster is enabled;

[0136] The NFS file service enabling unit is configured to generate prompt information for enabling all NFS file services if the NFS file services of all nodes in the distributed storage cluster are not enabled, so as to enable all NFS file services.

[0137] In some specific embodiments, after the first creation module 11, the following steps may also be included:

[0138] an identity identification code obtaining unit, configured to obtain the identity identification codes of the first authenticated user group and the first authenticated user, respectively, to obtain a first identity identification code and a second identity identification code;

[0139] The identity identification code recording unit is used to record the first identity identification code and the second identity identification code, and store the first identity identification code and the second identity identification code persistently.

[0140] In some specific embodiments, the fifth creation module 16 may specifically include:

[0141] an authentication user group creating unit, configured to create a second authentication user group having the same name as the first authentication user group by using the first authentication user group and the first identity identification code;

[0142] The authentication user creating unit is configured to create a second authentication user having the same name as the first authentication user by using the second authentication user group and the second identity identification code.

[0143] In some specific embodiments, the file conversion module 19 may further include:

[0144] a content reading unit, configured to read the content of the target object in the shared directory of the client to obtain first content;

[0145] a first content verification unit, configured to verify the first content;

[0146] A first prompt information generating unit is configured to generate prompt information indicating that the bucket is successfully converted to a shared directory if the verification passes;

[0147] a content downloading and reading unit, configured to download and read the content of the to-be-shared file from the target bucket through the second authenticated user to obtain second content;

[0148] A second content verification unit, configured to verify the second content;

[0149] The second prompt information generating unit is used to generate prompt information that the shared directory is successfully converted into a bucket if the verification is passed.

[0150] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 17 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.

[0151] Figure 17 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the file conversion method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0152] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0153] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0154] The operating system 221 is used to manage and control the hardware devices on the electronic device 20 and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of implementing the file conversion method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program capable of implementing other specific tasks.

[0155] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when executed by a processor, the computer program implements the aforementioned method for file conversion. The specific steps of this method can be referred to the corresponding contents disclosed in the aforementioned embodiments and will not be repeated here.

[0156] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0157] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0158] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0159] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0160] The above is a detailed introduction to the file conversion method, device, equipment and storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. At the same time, for those skilled in the art, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A method for mutual conversion of files, characterized in that: Applicable to distributed storage clusters, including: Creating a first authentication user group, and creating a first authentication user belonging to the first authentication user group; Creating a first simple storage service user associated with the first authenticated user; Create an independent second Simple Storage Service user, use the second Simple Storage Service user to create a target bucket, and then upload the target object to the target bucket; Creating a namespace using the first authenticated user group and the first authenticated user, and creating an NFS share using the namespace to share the directory of the distributed storage cluster externally; creating, on the client side, a second authentication user group and a second authentication user that can be bound to the first authentication user group and the first authentication user, respectively, and assigning the second authentication user to the second authentication user group; Mount a shared directory on the client and write the file to be shared into the shared directory; The target object in the target bucket is copied and transferred to the namespace by the second authenticated user, and the file to be shared is copied and transferred from the namespace to the target bucket by the first authenticated user. Mounting a shared directory on the client and writing a file to be shared in the shared directory includes: Log in to the client through the second authenticated user, mount a shared directory on the client, and write the file to be shared in the shared directory; set the access control list permissions of the target bucket and the namespace to public read and write.

2. The file conversion method according to claim 1, wherein: Before creating the first authentication user group, the method further includes: Determine whether the gateway service of all nodes in the distributed storage cluster is enabled; If the gateway services of all nodes in the distributed storage cluster are not enabled, a prompt message for enabling all gateway object services is generated to enable all gateway object services.

3. The file conversion method according to claim 2, characterized in that: Also includes: Verify that the NFS file service of all nodes in the distributed storage cluster is enabled; If the NFS file services of all nodes in the distributed storage cluster are not enabled, prompt information for enabling all the NFS file services is generated to enable all the NFS file services.

4. The file conversion method according to claim 1, wherein: After creating the first authentication user belonging to the first authentication user group, the method further includes: Obtaining the identity identification codes of the first authenticated user group and the first authenticated user respectively to obtain a first identity identification code and a second identity identification code; The first identity identification code and the second identity identification code are recorded and stored persistently.

5. The file conversion method according to claim 4, characterized in that: The step of creating, on the client side, a second authentication user group and a second authentication user that can be bound to the first authentication user group and the first authentication user, respectively, comprises: creating a second authentication user group having the same name as the first authentication user group using the first authentication user group and the first identity identification code; A second authentication user having the same name as the first authentication user is created using the second authentication user group and the second identity identification code.

6. The method for mutual conversion of files according to any one of claims 1 to 5, characterized in that: After using the first authenticated user to copy and transfer the to-be-shared file from the namespace to the target bucket, the method further includes: Reading the content of the target object in the shared directory of the client to obtain first content, and verifying the first content. If the verification passes, generating a prompt message indicating that the bucket has been successfully converted to the shared directory; The second authenticated user downloads and reads the content of the file to be shared from the target bucket to obtain the second content, and verifies the second content. If the verification passes, a prompt message is generated that the shared directory is successfully converted to the bucket.

7. A file conversion device, characterized in that: Applicable to distributed storage clusters, including: A first creation module, configured to create a first authentication user group and a first authentication user belonging to the first authentication user group; A second creation module is configured to create a first simple storage service user associated with the first authentication user; A third creation module is configured to create an independent second simple storage service user and create a target bucket using the second simple storage service user; An upload module, configured to upload a target object to the target bucket; a fourth creation module, configured to create a namespace using the first authenticated user group and the first authenticated user, and to create an NFS share using the namespace to share the directory of the distributed storage cluster externally; A fifth creation module, configured to create, on the client side, a second authentication user group and a second authentication user that can be bound to the first authentication user group and the first authentication user; an allocating module, configured to allocate the second authenticated user to the second authenticated user group; A shared directory mounting module, used to mount a shared directory on the client and write files to be shared in the shared directory; A file conversion module is configured to copy and transfer the target object in the target bucket to the namespace using the second authenticated user, and simultaneously copy and transfer the file to be shared from the namespace to the target bucket using the first authenticated user; The shared directory mounting module includes: A login unit, configured to log in to the client through the second authenticated user; A shared directory mounting unit, configured to mount a shared directory on the client and write files to be shared into the shared directory; The permission setting unit is used to set the access control list permissions of the target bucket and the namespace to public read and write.

8. An electronic device, characterized in that: The method comprises a processor and a memory; wherein, when the processor executes the computer program stored in the memory, the method for converting files into each other as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that Used to store computer programs; wherein, when the computer program is executed by a processor, it implements the file conversion method according to any one of claims 1 to 6.

Images