A data storage management method and related components

Abstract

This invention discloses a data storage management method and related components, relating to the field of distributed storage management. It obtains the identity identifier corresponding to the data to be stored and generates a unique version identifier for this version. When it is determined that an internal multi-version summary file exists, a new actual storage file is created to store the data to be stored. Still based on distributed file storage, it realizes multi-version data storage corresponding to the identity identifier. Furthermore, when it is determined that a corresponding externally viewable file exists, the latest version attribute information stored in the externally viewable file is changed to the storage attribute information corresponding to the data to be stored. The externally viewable file serves as the actual display file, while the internal multi-version summary file is not the actual display file. This design of multi-version file storage is imperceptible to the user. While facilitating user operation, it enriches the product functionality of the unstructured converged and interoperable storage system, improving the product's usability and competitiveness.

Description

Technical Field

[0001] This invention relates to the field of distributed storage management technology, and in particular to a data storage management method and related components. Background Technology

[0002] Distributed storage systems that support multi-protocol interoperability are also known as distributed unstructured converged storage systems. They achieve interoperability between S3 (Simple Storage Service), NAS (Network Attached Storage), and HDFS (Hadoop Distributed File System) protocols, allowing multiple protocols to share and access data simultaneously. For example, files written via the NAS protocol can be accessed via the S3 or HDFS protocol, and data written via S3 can also be accessed via the NAS or HDFS protocol.

[0003] In existing technologies, common implementation schemes for distributed unstructured converged and interoperable storage systems mainly involve using a distributed storage system as a foundation to achieve interoperability between S3 and NAS. Specifically, data is stored in the underlying layer as file storage, but simultaneously accepts data write and read operations in S3 object storage mode. This is achieved by establishing a mapping relationship between object storage and file storage. Buckets in object storage are essentially directories in file storage. In other words, this mapping relationship maps users in object storage to users in file storage, buckets in object storage to directories in file storage, and objects in object storage to files in file storage, thus achieving interoperability between object storage and file storage in terms of users and storage structure. Please refer to [reference needed]. Figure 1 , Figure 1 This is a schematic diagram of the mapping relationship of a distributed unstructured converged and interoperable storage system in the prior art. Based on the establishment of the mapping relationship, permission interoperability is realized.

[0004] However, current distributed unstructured converged and interoperable storage systems lack a metadata pool for object storage in their converged and interoperable scenarios. The underlying layer is a file system, and the way file system stores and manages metadata is completely different from that of object storage. The foundation is file storage, which only supports uploading and downloading ordinary files. Files with the same name uploaded to a bucket will be overwritten, and object storage does not have a recycle bin function. That is, in the final underlying file corresponding to the bucket, only the latest version of the file with the same name is saved. This makes it impossible to recover data after some abnormal operations, causing inconvenience to users and resulting in the lack of S3 semantic functions.

[0005] Therefore, how to provide a solution to the above-mentioned technical problems is a problem that those skilled in the art need to solve at present. Summary of the Invention

[0006] The purpose of this invention is to provide a data storage management method and related components. While still based on distributed file storage, it realizes multi-version data storage corresponding to identity identifiers, avoiding direct overwriting of files with the same identity identifier and only retaining the latest version of the file. It supports S3 multi-version functionality. Moreover, the design of multi-version file storage is imperceptible to users. On the basis of user convenience, it enriches the product functions of unstructured converged and interoperable storage system and improves the ease of use and competitiveness of the product.

[0007] To address the aforementioned technical problems, this invention provides a data storage management method applied to a distributed unstructured converged and interoperable storage system. The data storage management method includes:

[0008] Obtain the identity identifier corresponding to the data to be stored;

[0009] Generate a unique version identifier corresponding to the identity identifier for this version;

[0010] Based on the identity identifier, determine whether there is an internal multi-version summary file corresponding to the identity identifier and used for non-actual display, wherein the internal multi-version summary file includes at least one actual storage file of a historical version;

[0011] If so, based on the identity identifier and the unique version identifier, a new actual storage file is created in the internal multi-version summary file to store the data to be stored in the new actual storage file;

[0012] Determine whether there exists an external viewable file that corresponds to the internal multi-version summary file and is actually used for display;

[0013] If so, change the latest version attribute information stored in the externally viewable file to the storage attribute information corresponding to the data to be stored.

[0014] Preferably, the identity identifier includes a first bucket identifier and a first object identifier;

[0015] When it is determined that there is no internal multi-version summary file corresponding to the identity identifier and which is not actually displayed, the following applies:

[0016] Create a new file named "First Bucket Identifier - First Object Identifier" and use it as an internal multi-version summary file that is not actually displayed.

[0017] Based on the first object identifier and the unique version identifier, a current actual storage file is created in the newly created internal multi-version summary file so that the data to be stored is stored in the current actual storage file;

[0018] The new file is named after the first object identifier and is used as an externally viewable file for actual display;

[0019] The storage attribute information corresponding to the data to be stored is stored as the latest version attribute information in the newly created externally viewable file.

[0020] Preferably, the identity identifier includes a first object identifier; the data storage management method further includes:

[0021] Determine the current time value;

[0022] Based on the identity identifier and the unique version identifier, a new actual storage file is created in the internal multi-version summary file, including:

[0023] Create a new actual storage file in the internal multi-version summary file;

[0024] The filename of the new actual storage file is set to the time value - the first object identifier - the unique version identifier.

[0025] Preferably, the order of the actual stored files in the internal multi-version summary file is determined based on the time value in their respective file names;

[0026] The data storage management method further includes:

[0027] Obtain the enumeration identifiers that represent the summarization of multiple versions of data;

[0028] Based on the enumeration identifier, determine the first target internal multi-version summary file corresponding to the enumeration identifier;

[0029] Control the output of all actual stored files in the multi-version summary file within the first target.

[0030] Preferably, the file name of the internal multi-version summary file is determined based on the corresponding second object identifier and storage location; the file name of the actual stored file is determined based on the second object identifier and the corresponding unique version identifier.

[0031] The data storage management method further includes:

[0032] Retrieve the information of the first action object that represents the target processing action to be performed on the specified data version;

[0033] Based on the file names of each internal multi-version summary file and the file names of each actual storage file, determine the target actual storage file corresponding to the first action object information;

[0034] The target processing action is performed on the actual storage file of the target.

[0035] Preferably, when the target processing action is deletion;

[0036] After performing the target processing action on the target actual storage file, the method further includes:

[0037] Determine whether the storage status of the multi-version summary file within the second target to which the actual target storage file belongs is empty;

[0038] If so, delete the multi-version summary file inside the second target;

[0039] Delete the externally viewable file of the first target that corresponds to the multi-version summary file inside the second target.

[0040] Preferred options also include:

[0041] Retrieve information about deleted objects whose deletion action was performed without specifying a data version;

[0042] Based on the file names of each of the aforementioned internal multi-version summary files, determine the third target internal multi-version summary file corresponding to the deleted object information;

[0043] Create an empty version file within the multi-version summary file of the third target;

[0044] Delete the externally viewable file of the second target that corresponds to the internal multi-version summary file of the third target.

[0045] Preferred options also include:

[0046] Retrieve information about the download object that represents a download action performed without specifying a data version;

[0047] Based on the file names of the internal multi-version summary files, determine the third target externally viewable file corresponding to the download object information;

[0048] The actual storage file located based on the latest version attribute information stored in the externally viewable file of the third target is taken as the target file to be downloaded corresponding to the download object information.

[0049] To address the aforementioned technical problems, this invention also provides a distributed unstructured converged and interoperable storage system, comprising:

[0050] Memory, used to store computer programs;

[0051] A processor is configured to implement the steps of the data storage management method as described above when executing the computer program.

[0052] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium, comprising:

[0053] The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the data storage management method described above.

[0054] This application provides a data storage management method and related components. It obtains an identity identifier corresponding to the data to be stored and generates a unique version identifier for the current version. When it is determined that an internal multi-version summary file corresponding to the identity identifier exists, a new actual storage file is further created within it to store the data to be stored. The corresponding internal multi-version summary file includes at least one historical version's actual storage file. Thus, while still using a distributed file storage method as its foundation, it achieves multi-version data storage corresponding to the identity identifier, avoiding direct overwriting of the same identity identifier file and retaining only the latest version file. This avoids the loss of S3 semantic functionality, facilitates remedial handling of abnormal operations, and supports S3 multi-version functionality. Furthermore, when it is determined that an externally viewable file corresponding to the internal multi-version summary file exists, the latest version attribute information stored in the externally viewable file is changed to the storage attribute information corresponding to the data to be stored. The externally viewable file is the file actually displayed to the user, while the internal multi-version summary file is not actually displayed to the user. This makes the multi-version file storage design imperceptible to the user; the user still perceives the latest version file information. This enhances the usability and competitiveness of the unstructured converged and interoperable storage system while facilitating user use. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the prior art and embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic diagram illustrating the mapping relationship of a distributed unstructured converged and interoperable storage system in the prior art;

[0057] Figure 2 A flowchart of a data storage management method provided by the present invention;

[0058] Figure 3 This is a schematic diagram illustrating the implementation of S3 and NAS protocol permission interoperability in a distributed unstructured converged interoperable storage system provided by the present invention.

[0059] Figure 4 This is a schematic diagram of the structure of a distributed unstructured converged and interoperable storage system provided by the present invention;

[0060] Figure 5 This is a schematic diagram of the structure of a computer-readable storage medium provided by the present invention. Detailed Implementation

[0061] The core of this invention is to provide a data storage management method and related components. While still based on distributed file storage, it realizes multi-version data storage corresponding to identity identifiers, avoiding direct overwriting of files with the same identity identifier and only retaining the latest version of the file. It supports S3 multi-version functionality. Moreover, the design of multi-version file storage is imperceptible to users. On the basis of user convenience, it enriches the product functions of unstructured converged and interoperable storage system and improves the ease of use and competitiveness of the product.

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Please refer to Figure 1 , Figure 2 and Figure 3 , Figure 1 This is a schematic diagram illustrating the mapping relationship of a distributed unstructured converged and interoperable storage system in the prior art. Figure 2 A flowchart of a data storage management method provided by the present invention. Figure 3 This is a schematic diagram illustrating the implementation of S3 and NAS protocol permission interoperability in a distributed unstructured converged interoperable storage system provided by the present invention.

[0064] A distributed storage system that supports multi-protocol interoperability is also called a distributed unstructured converged interoperable storage system. In this embodiment, considering the common implementation schemes of distributed unstructured converged interoperable storage systems in the prior art, the main approach is to use a distributed storage system as a foundation to achieve interoperability between S3 and NAS. Specifically, this involves establishing a mapping relationship between object storage and file storage, which can be referred to... Figure 1 , Figure 1A schematic diagram of the above mapping relationship is provided, where the tenant name is the username, and the bucket in object storage is essentially a file directory in the underlying file storage. For example, the bucket identifier (bucket1) corresponds to the bucket name and the corresponding bucket1 file in the file storage. The object identifier "object3" in the bucket is actually in the "object3" subfile under the bucket1 file directory in the file storage. The object identifiers in other buckets are similarly represented, and will not be elaborated here. However, due to the convergence design rules of this distributed unstructured converged storage system, its converged interoperability scenario does not have a metadata pool for object storage. The underlying layer is a file system, and the metadata storage and management methods of the file system are completely different from those of object storage. The underlying layer is a file storage method, which only supports the upload and download of ordinary files. That is, in the final underlying file, only the latest version of the file with the same name is saved. This leads to the inability to recover data after some abnormal operations, causing inconvenience to users and resulting in the loss of S3 semantic functionality. To solve the above technical problems, this application provides a data storage management method to avoid the loss of S3 semantic functionality. While still using distributed file storage as the underlying layer, it realizes multi-version data storage corresponding to identity identifiers.

[0065] This data storage management method is applied to distributed unstructured converged and interoperable storage systems, including:

[0066] S11: Obtain the identity identifier corresponding to the data to be stored;

[0067] Specifically, S3 is essentially an object storage service; the aforementioned distributed unstructured converged and interoperable storage system is the storage system based on a distributed storage system, enabling interoperability between S3 and NAS / HDFS. It supports multiple protocols simultaneously sharing and accessing the same data. For example, files written via the NAS protocol can subsequently be manipulated via the S3 or HDFS protocol, and data written via S3 can also be manipulated via the NAS or HDFS protocol. Please refer to [reference needed]. Figure 3 , Figure 3This diagram illustrates the implementation of S3 and NAS protocol permission interoperability in a distributed unstructured converged storage system provided by the present invention. It shows that the S3 Client can use the S3 Gateway (object gateway) and the corresponding Liblcfsfs.so component to convert files uploaded via the S3 object protocol into files stored in the storage pool according to file storage methods. Metadata is uniformly managed by MDS (Meta Data). When downloading an object via the S3 object protocol, a request to retrieve the corresponding file data from the file storage system is generated, and the retrieved data is returned to the user. This ultimately enables data upload and download under the S3 object protocol. Similarly, the NAS Client can use the corresponding NAS Server to call the corresponding Liblcfsfs.so component to upload and download data under different data transmission protocols. These different transmission protocols include, but are not limited to, NFS (Network File System) and CIFS (Common Internet File System).

[0068] Obtain the identity identifier sent by the client corresponding to the data to be stored in the current version. The identity identifier includes a bucket identifier and an object identifier. The bucket identifier can be a bucket name (in this application, since the base of the distributed unstructured converged interoperable storage system is a distributed file storage method, the bucket name can be understood as the file directory name that is mapped to it). Then, based on the bucket name and the preset name-ino number correspondence, determine the bucket ino number used as a unique identifier for the bucket. The object identifier includes, but is not limited to, the object name.

[0069] It should also be noted that before step S11, it is possible to determine whether the client has enabled the multi-version bucket function (in this application, the multi-version bucket function refers to scenarios where distributed object storage overwrites with the same name, or where objects are restored after accidental deletion by the user; simply put, it can save multiple versions of data for files with the same name, and set corresponding processing logic for the first and subsequent uploads of these multiple versions of data, for the deletion and download of stored data of a specified version, and for the deletion and download of stored data of an unspecified version, which will be described in detail in the following embodiments). If yes, it means that the data to be stored will be processed according to the multi-version bucket data upload logic, and thus proceed to step S11; if no, it means that the data to be stored is just a normal object upload, and it can be processed directly according to the normal object upload process.

[0070] S12: Generate a unique version identifier corresponding to the identity identifier for this version;

[0071] Specifically, the unique version identifier can be randomly generated by the aforementioned object gateway, but the unique version identifier is not repeated, and different versions of the same object name have different unique version identifiers.

[0072] S13: Based on the identity identifier, determine whether there is an internal multi-version summary file that corresponds to the identity identifier and is used for non-actual display. The internal multi-version summary file includes at least one actual storage file of a historical version. If so, proceed to S14.

[0073] S14: Based on the identity identifier and unique version identifier, create a new actual storage file in the internal multi-version summary file to store the data to be stored in the new actual storage file.

[0074] Specifically, the internal multi-version summary file is a folder specifically used to store the actual storage files of each version corresponding to the identity identifier. This summary file is not actually displayed and is not presented to the outside world, meaning that the user is unaware of it. The file name of this summary file can be bucket identifier-object name (specifically, in the form of bucket ino number + object name). Of course, in order to indicate that it is a multi-version file, the file name of this summary file can be bucket identifier-object name-version, where version is only used to identify that the file is a multi-version file.

[0075] The identity identifier includes a bucket identifier (such as a bucket name, which uniquely identifies the bucket number) and an object identifier (such as an object name). Based on this, combined with the file name of the internal multi-version summary file, a search can be performed to determine whether an internal multi-version summary file corresponding to the identity identifier exists. If so, it means that the identity identifier is not uploading data for the first time, and the searched internal multi-version summary file includes at least one historical version's actual storage file, that is, it includes at least one version of historical data. Therefore, a new actual storage file can be created directly in the internal multi-version summary file to store the data to be stored. The file name of the new actual storage file can be the object name in the identity identifier + the generated unique version number, or it can be the current time value corresponding to the current time + the object name in the identity identifier + the generated unique version number, as described in the following embodiment. The specifics will not be elaborated here.

[0076] S15: Determine if there exists an external viewable file that corresponds to the internal multi-version summary file and is used for actual display; if so, proceed to S16;

[0077] S16: Change the latest version attribute information stored in the externally viewable file to the storage attribute information corresponding to the data to be stored.

[0078] Specifically, in order to make the storage of multiple versions seamless for users, based on step S15, it is further determined whether there is a corresponding externally viewable file. Only the externally viewable file is the file that is actually displayed to the user and can be viewed by the user on the client. The file name of the externally viewable file includes, but is not limited to, object identifiers, such as object names; identity identifiers include object names. Therefore, a search can be performed based on the identity identifiers to determine whether there is an externally viewable file that corresponds to the internal multi-version summary file and is used for actual display.

[0079] If so, the latest version attribute information stored in the externally viewable file will be changed to the storage attribute information corresponding to the data to be stored. The latest version attribute information here includes, but is not limited to, the location information (such as access path) of the data to be stored, so as to locate the data to be stored based on the location information. Of course, other metadata information of the data to be stored can also be added according to actual needs. No special limitation is made here, and it depends on actual needs.

[0080] If not, since there is an internal multi-version summary file corresponding to the identity identifier, it means that the previously externally viewable file corresponding to the identity identifier may have been deleted due to the data deletion action without specifying the data version as described in the following embodiments. Therefore, an externally viewable file with the file name of the object name in the identity identifier can be recreated, and the storage attribute information of the data to be stored can be stored in it as the latest version attribute information.

[0081] In addition, after the above steps are completed, a success signal indicating that the multi-version data upload was successful can be sent to the client.

[0082] In summary, this application provides a data storage management method that obtains the identity identifier corresponding to the data to be stored and generates a unique version identifier for the current version. When it is determined that an internal multi-version summary file corresponding to the identity identifier exists, a new actual storage file is further created within it to store the data to be stored. The corresponding internal multi-version summary file includes at least one historical version's actual storage file. Thus, while still based on a distributed file storage method, multi-version data storage corresponding to the identity identifier is realized, avoiding direct overwriting of the same identity identifier file and retaining only the latest version file. This avoids the loss of S3 semantic functionality, facilitates remedial handling of abnormal operations, and supports S3 multi-version functionality. Furthermore, when it is determined that an externally viewable file corresponding to the internal multi-version summary file exists, the latest version attribute information stored in the externally viewable file is changed to the storage attribute information corresponding to the data to be stored. The externally viewable file is the file actually displayed to the user, while the internal multi-version summary file is not the file actually displayed to the user. This makes the design of multi-version file storage imperceptible to the user, and the user still perceives the latest version file information. While facilitating user use, this method enriches the product functionality of the unstructured converged and interoperable storage system, improving the product's usability and competitiveness.

[0083] Based on the above embodiments:

[0084] In one preferred embodiment, the identity identifier includes a first bucket identifier and a first object identifier;

[0085] When determining that there is no internal multi-version summary file corresponding to the identity identifier and used for non-actual display, including:

[0086] Create a new file named First Bucket Identifier - First Object Identifier, intended for use as an internal multi-version summary file that is not actually displayed.

[0087] Based on the first object identifier and the unique version identifier, the current actual storage file is created in the newly created internal multi-version summary file so that the data to be stored can be stored in the current actual storage file.

[0088] The new file is named the first object identifier and is used for the actual display of the externally viewable file;

[0089] Store the storage attribute information corresponding to the data to be stored as the latest version attribute information in a newly created externally viewable file.

[0090] This embodiment provides processing steps when it is determined that there is no internal multi-version summary file corresponding to the identity identifier and used for non-actual display. Specifically, since the same object name may store multiple versions of data in different buckets, but only one internal multi-version summary file corresponding to the object name exists in the same bucket, that is, all version files of the object name in the bucket are stored in the corresponding internal multi-version summary file's actual storage files. Therefore, the identity identifier may include a first object identifier (such as the first object name) and a first bucket identifier (such as the first bucket name). When it is determined that there is no internal multi-version summary file corresponding to the first object identifier, it indicates that the data to be stored is uploaded for the first time. The newly created file is named "First Bucket Identifier - First Object Identifier" and is used for non-actual display as an internal multi-version summary file. Preferably, the first bucket identifier uploaded by the user on the client can be the first bucket name. The first bucket identifier in the file name of the internal multi-version summary file can further use the first bucket ino number corresponding to the first bucket name (different buckets have different ino numbers) to better conform to the internal operating logic of the computer.

[0091] The data to be stored is stored in the newly created internal multi-version summary file. Since this is the first upload, there is naturally no corresponding external viewable file. Therefore, a new external viewable file named after the first object identifier is created, such as an external viewable file named after the first object name. The storage attribute information corresponding to the data to be stored is stored as the latest version attribute information in the newly created external viewable file for easy viewing later.

[0092] In a preferred embodiment, the identity identifier includes a first object identifier; the data storage management method further includes:

[0093] Determine the current time value;

[0094] Based on the identity identifier and unique version identifier, a new actual storage file is created in the internal multi-version summary file, including:

[0095] Create a new actual storage file within the internal multi-version summary file;

[0096] Set the filename of the new actual storage file to Time Value - First Object Identifier - Unique Version Identifier.

[0097] In this embodiment, the inventors further considered that in the prior art, the file saving method in a distributed unstructured converged and interoperable storage system cannot be associated with the order of uploaded data. Therefore, this application provides an execution logic for multi-version stored data to achieve time-ordered execution according to the upload order, so as to facilitate subsequent saving and downloading, and make the storage more standardized. The file name of the internal multi-version summary file is named as First Bucket Ino Number + First Object Name, or First Bucket Ino Number + First Object Name + Version. The file name of the actual stored file included can be Time Value - First Object Identifier - Unique Version Identifier. Files in the same folder in the file storage system are ordered. Since the Time Value comes first and is used as the beginning of the file name, the order of each actual stored file is the order of the Time Value, that is, it achieves time-ordered execution according to the upload order. The first actual stored file in the internal multi-version summary file is the latest version file.

[0098] In a preferred embodiment, the order of the actual stored files in the internal multi-version summary file is determined based on the time value in their respective file names;

[0099] Data storage management methods also include:

[0100] Obtain the enumeration identifiers that represent the summarization of multiple versions of data;

[0101] Based on the enumeration identifier, determine the first target internal multi-version summary file corresponding to the enumeration identifier;

[0102] Control the output of all actual stored files in the multi-version summary file within the first target.

[0103] This embodiment provides the execution steps for multi-version data aggregation. Specifically, it obtains an enumeration identifier sent by the client to represent multi-version data aggregation. This enumeration identifier may only include object identifiers (such as object names). In this case, the multi-version data aggregation involves the stored data corresponding to the object identifier in each bucket, thereby obtaining the multi-version aggregation files within each first target corresponding to the object identifier in each bucket. It then controls the output of all actual stored files in the multi-version aggregation files within each first target and sends them to the client to complete the data aggregation. Since the arrangement order of the actual stored files in the multi-version aggregation files within each first target is chronologically ordered according to the upload order, it is convenient for subsequent viewing. Of course, the enumeration identifier may also include object identifiers (such as object names) + bucket identifiers (such as bucket names). In this case, the multi-version data aggregation involves the stored data corresponding to the object identifier in the specified bucket identifier, thereby obtaining the multi-version aggregation files within the first target corresponding to the object identifier in the bucket identifier, and outputting them to the client.

[0104] It should also be noted that this method can also obtain the enumeration identifier for summarizing ordinary file data, and then execute the ordinary file enumeration process. The data format of the target data of each actual storage file output to the client is not particularly limited here, and can be converted according to the agreed format before being fed back to the client. In addition, if it is determined based on the enumeration identifier that the first target internal multi-version summary file corresponding to the enumeration identifier does not exist, an error can be reported. The specific settings can be set according to actual needs.

[0105] In a preferred embodiment, the file name of the internal multi-version summary file is determined based on the corresponding second object identifier and storage location; the file name of the actual stored file is determined based on the second object identifier and the corresponding unique version identifier.

[0106] Data storage management methods also include:

[0107] Retrieve the information of the first action object that represents the target processing action to be performed on the specified data version;

[0108] Based on the file names of each internal multi-version summary file and the file names of each actual stored file, determine the target actual stored file corresponding to the first action object information;

[0109] Perform target processing actions on the target's actual stored file.

[0110] In this embodiment, the execution logic for performing target processing actions on a specified data version is given. Specifically, the target processing actions include, but are not limited to, deletion and download. In the distributed unstructured fused and interoperable storage system, the file name of each internal multi-version summary file is determined based on its corresponding second object identifier and storage location. Specifically, the storage location is the second bucket identifier (such as the second bucket ino number) that represents its actual storage location. The second object identifier includes, but is not limited to, the second object name. The file name of the internal multi-version summary file can be the second bucket ino number - the second object name. The file name of the actual stored file in the internal multi-version summary file is determined based on the second object identifier and the corresponding unique version identifier. Preferably, the file name of the actual stored file can be the time value - the second object identifier - the unique version identifier.

[0111] The first action object information includes the second bucket name, the second object identifier (such as the second object name), and the unique version identifier. Based on the second bucket name and the preset name-ino number correspondence, the corresponding second bucket ino number can be determined. Then, based on the second bucket ino number and the second object identifier, the target internal multi-version summary file is determined. Based on the obtained unique version identifier, the target actual storage file in the target internal multi-version summary file is found.

[0112] When the target processing action is deletion, the target's actual storage file is deleted. It can be understood that when the target's actual storage file is the latest version of the storage file, since the filenames of other actual storage files in the same folder are named as time value-second object identifier-unique version identifier, the latest actual storage file after the target's actual storage file is deleted can be determined based on the names of each actual storage file, and its corresponding storage attribute information is updated to the corresponding externally viewable file as the latest version attribute information; in addition, after the deletion action is completed, a signal of successful execution can be sent to the client.

[0113] When the target processing action is download, the stored data in the actual storage file of the target is downloaded and fed back to the client.

[0114] It should also be noted that if it is determined based on the first action object information that the target actual storage file does not exist, an error can be reported; the method can also obtain action object information representing the download or deletion of ordinary objects, and then execute according to the ordinary object download or deletion process.

[0115] As can be seen, by setting the above execution logic in the distributed unstructured converged and interoperable storage system, the deletion and download actions of specified data versions are realized, which is beneficial for practical applications.

[0116] As a preferred embodiment, when the target processing action is deletion;

[0117] After performing target processing actions on the actual target storage file, the process also includes:

[0118] Determine whether the storage status of the multi-version summary file within the second target to which the actual target file belongs is empty;

[0119] If so, delete the multi-version summary file within the second target;

[0120] Delete the externally viewable file in the first target that corresponds to the multi-version summary file inside the second target.

[0121] In this embodiment, when the target processing action is deletion, if the storage status of the second target internal multi-version summary file to which the target actual storage file belongs is empty after the target actual storage file is deleted, it means that there is no data of any version inside the second target internal multi-version summary file. Therefore, the second target internal multi-version summary file is deleted, and the first target external viewable file corresponding to the second target internal multi-version summary file is also deleted, saving storage space and facilitating storage management.

[0122] In addition, after deleting the multiple version summary files within the second target and the externally viewable files of the first target, a signal indicating successful deletion can be sent to the client.

[0123] As a preferred embodiment, it also includes:

[0124] Retrieve information about deleted objects whose deletion action was performed without specifying a data version;

[0125] Based on the file names of each internal multi-version summary file, determine the third target internal multi-version summary file corresponding to the deleted object information;

[0126] Create an empty version file within the multi-version summary file of the third target;

[0127] Delete the externally viewable file in the second target that corresponds to the multi-version summary file inside the third target.

[0128] In this embodiment, the inventors further considered that in practical applications, users may accidentally trigger data deletion. Based on the premise that users are only aware of externally viewable files, this application further provides the execution logic for performing deletion actions without specifying a data version, effectively solving the problem of data being unrecoverable after accidental deletion. Specifically, the filename of the externally viewable file corresponding to the internal multi-version summary file is determined based on a second object identifier. The filename of the externally viewable file can be the second object identifier, such as a second object name.

[0129] The deleted object information includes the second bucket name and the second object identifier. Based on the correspondence between the second bucket name and the preset name-ino number, the corresponding second bucket ino number can be determined. Then, based on the second bucket ino number, the second object identifier, and the file names of each internal multi-version summary file, the third target internal multi-version summary file is determined. An empty version file is created in the third target internal multi-version summary file. The empty version file is only used to record the deletion of the data version without specifying it. It does not store any data, and its file name includes the time value corresponding to the current time. Based on the file names of each externally viewable file, the second target externally viewable file corresponding to the deleted object information is determined and deleted. Thus, for the user, it is believed that the file corresponding to the deleted object information has been deleted, but in fact, the historical data of each version corresponding to the deleted object information is still stored in the third target internal multi-version summary file, and can be accessed and retrieved later by downloading the specified data version.

[0130] In addition, after creating the empty version file and deleting the externally viewable file of the second target, a signal indicating successful deletion can be sent to the client.

[0131] As a preferred embodiment, it also includes:

[0132] Retrieve information about the download object that represents a download action performed without specifying a data version;

[0133] Based on the file names of the various internal multi-version summary files, determine the third target externally viewable file corresponding to the download object information;

[0134] The actual storage file located based on the latest version attribute information stored in the externally viewable file of the third target is used as the target file to be downloaded corresponding to the download object information.

[0135] This embodiment provides the execution logic for performing a download action on a specified data version, that is, directly providing the latest version of the data. Specifically, the download object information includes the second bucket name and the second object identifier. Based on the second bucket name and the preset name-ino number correspondence, the corresponding second bucket ino number can be determined. Then, based on the second bucket ino number, the second object identifier, and the file name of each internal multi-version summary file, the target internal multi-version summary file is determined. The third target externally viewable file in the target internal multi-version summary file is determined. The third target externally viewable file stores the latest version attribute information, which includes location information. The actual storage file located based on the attribute information is taken as the target file to be downloaded corresponding to the download object information. Thus, the data in the actual storage file is fed back to the client, thereby realizing the download of data without specifying the data version in the distributed unstructured converged interoperable storage system and supporting the S3 multi-version function.

[0136] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of a distributed unstructured converged and interoperable storage system provided by the present invention.

[0137] This distributed, unstructured, converged, and interoperable storage system includes:

[0138] Memory 21 is used to store computer programs;

[0139] The processor 22 is used to implement the steps of the data storage management method as described above when executing a computer program.

[0140] For a description of the distributed unstructured converged and interoperable storage system provided in this invention, please refer to the embodiments of the data storage management method described above; further details will not be repeated here.

[0141] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer-readable storage medium provided by the present invention.

[0142] The computer-readable storage medium 3 includes:

[0143] The computer-readable storage medium 3 stores a computer program 31, which, when executed by a processor, implements the steps of the data storage management method as described above.

[0144] For a description of the computer-readable storage medium 3 provided in this invention, please refer to the embodiments of the data storage management method described above, which will not be repeated here.

[0145] The present invention also provides a data storage management system, comprising:

[0146] The first acquisition unit is used to acquire the identity identifier corresponding to the data to be stored;

[0147] The first generation unit is used to generate a unique version identifier corresponding to the identity identifier in this version.

[0148] The first determination unit is used to determine, based on the identity identifier, whether there exists an internal multi-version summary file corresponding to the identity identifier and used for non-actual display, wherein the internal multi-version summary file includes at least one actual storage file of a historical version; if yes, the first creation unit is triggered; if no, the second creation unit is triggered.

[0149] The first newly created unit is used to create a new actual storage file in the internal multi-version summary file based on the identity identifier and the unique version identifier, so as to store the data to be stored into the new actual storage file;

[0150] The second judgment unit is used to determine whether there is an externally viewable file that corresponds to the internal multi-version summary file and is actually displayed; if so, the first update unit is triggered.

[0151] The first update unit is used to change the latest version attribute information stored in the externally viewable file to the storage attribute information corresponding to the data to be stored.

[0152] In a preferred embodiment, the identity identifier includes a first bucket identifier and a first object identifier;

[0153] The second newly created unit is used to create an internal multi-version summary file named "first bucket identifier - first object identifier" and used for non-actual display.

[0154] The third new unit is used to create a current actual storage file in the newly created internal multi-version summary file based on the first object identifier and the unique version identifier, so as to store the data to be stored into the current actual storage file;

[0155] The fourth new unit is used to create an externally viewable file named after the first object identifier and used for actual display.

[0156] The second update unit is used to store the storage attribute information corresponding to the data to be stored as the latest version attribute information into the newly created externally viewable file.

[0157] In a preferred embodiment, the identity identifier includes a first object identifier;

[0158] The data storage management system further includes:

[0159] The first determining unit is used to determine the current time value;

[0160] The first newly created unit includes:

[0161] The fifth new unit is used to create a new actual storage file in the internal multi-version summary file;

[0162] The file name setting unit is used to set the file name of the new actual storage file to be the time value - the first object identifier - the unique version identifier.

[0163] In a preferred embodiment, the order of the actual stored files in the internal multi-version summary file is determined based on the time value in their respective file names;

[0164] The data storage management system further includes:

[0165] The second acquisition unit is used to acquire the enumeration identifier that represents the aggregation of multi-version data.

[0166] The second determining unit is used to determine, based on the enumeration identifier, the first target internal multi-version summary file corresponding to the enumeration identifier;

[0167] The first output unit is used to control the output of all actual stored files in the multi-version summary file inside the first target.

[0168] In a preferred embodiment, the file name of the internal multi-version summary file is determined based on the corresponding second object identifier and storage location; the file name of the actual stored file is determined based on the second object identifier and the corresponding unique version identifier.

[0169] The data storage management system further includes:

[0170] The third acquisition unit is used to acquire the first action object information that represents the target processing action to be performed on the specified data version;

[0171] The third determining unit is used to determine the target actual storage file corresponding to the first action object information based on the file name of each of the internal multi-version summary files and the file name of each of the actual storage files.

[0172] The first execution unit is used to perform the target processing action on the target actual storage file.

[0173] In a preferred embodiment, when the target processing action is deletion, the data storage management system further includes:

[0174] The third judgment unit is used to determine whether the storage status of the internal multi-version summary file of the second target to which the actual storage file of the target is located is empty; if so, the first deletion unit is triggered.

[0175] The first deletion unit is used to delete the multi-version summary file inside the second target;

[0176] The second deletion unit is used to delete the externally viewable file of the first target that corresponds to the multi-version summary file inside the second target.

[0177] In a preferred embodiment, the data storage management system further includes:

[0178] The fourth acquisition unit is used to acquire information about deleted objects that represent deletion actions performed without specifying a data version;

[0179] The fourth determining unit is used to determine the third target internal multi-version summary file corresponding to the deleted object information based on the file name of each of the internal multi-version summary files;

[0180] The sixth newly created unit is used to create an empty version file in the multi-version summary file within the third target;

[0181] The third deletion unit is used to delete the externally viewable file of the second target that corresponds to the internal multi-version summary file of the third target.

[0182] In a preferred embodiment, the data storage management system further includes:

[0183] The fifth acquisition unit is used to acquire information about download objects that represent download actions performed without specifying a data version;

[0184] The fifth determining unit is used to determine the third target externally viewable file corresponding to the download object information based on the file name of each of the internal multi-version summary files;

[0185] The positioning output unit is used to locate the actual storage file based on the latest version attribute information stored in the externally viewable file of the third target, and use it as the target file to be downloaded corresponding to the download object information.

[0186] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatuses disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to in the method section. Relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0187] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. The foregoing description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data storage management method, characterized in that, This method is applied to a distributed unstructured converged interoperable storage system. The distributed unstructured converged interoperable storage system is a storage system based on a distributed storage system that enables interoperability between S3 and NAS / HDFS. It supports multiple protocols simultaneously sharing and interoperating with each other. The data storage management method includes: Determine whether the client has enabled the bucket multi-version feature. The bucket multi-version feature is for scenarios where distributed object storage uses the same name to overwrite, or where objects are restored after being accidentally deleted by the user. If so, obtain the identity identifier corresponding to the data to be stored; the identity identifier includes a first bucket identifier and a first object identifier; Generate a unique version identifier corresponding to the identity identifier for this version; Based on the identity identifier, determine whether there is an internal multi-version summary file corresponding to the identity identifier and used for non-actual display, wherein the internal multi-version summary file includes at least one actual storage file of a historical version; If so, based on the identity identifier and the unique version identifier, a new actual storage file is created in the internal multi-version summary file to store the data to be stored in the new actual storage file; the current time value is determined, and the file name of the new actual storage file is set to the time value - the first object identifier - the unique version identifier; Determine whether there exists an external viewable file that corresponds to the internal multi-version summary file and is actually used for display; If so, change the latest version attribute information stored in the externally viewable file to the storage attribute information corresponding to the data to be stored; The filename of the internal multi-version summary file is determined based on the corresponding second object identifier and storage location; the filename of the actual stored file is determined based on the second object identifier and the corresponding unique version identifier; the data storage management method further includes: Retrieve the information of the first action object that represents the target processing action to be performed on the specified data version; Based on the file names of each internal multi-version summary file and the file names of each actual storage file, determine the target actual storage file corresponding to the first action object information; The target processing action is performed on the actual storage file of the target.

2. The data storage management method as described in claim 1, characterized in that, When it is determined that there is no internal multi-version summary file corresponding to the identity identifier and which is not actually displayed, the following applies: Create a new file named "First Bucket Identifier - First Object Identifier" and use it as an internal multi-version summary file that is not actually displayed. Based on the first object identifier and the unique version identifier, a current actual storage file is created in the newly created internal multi-version summary file so that the data to be stored is stored in the current actual storage file; The new file is named after the first object identifier and is used as an externally viewable file for actual display; The storage attribute information corresponding to the data to be stored is stored as the latest version attribute information in the newly created externally viewable file.

3. The data storage management method as described in claim 1, characterized in that, The order of the actual stored files in the internal multi-version summary file is determined based on the time value in their respective file names. The data storage management method further includes: Obtain the enumeration identifiers that represent the summarization of multiple versions of data; Based on the enumeration identifier, determine the first target internal multi-version summary file corresponding to the enumeration identifier; Control the output of all actual stored files in the multi-version summary file within the first target.

4. The data storage management method as described in claim 1, characterized in that, When the target processing action is deletion; After performing the target processing action on the target actual storage file, the method further includes: Determine whether the storage status of the multi-version summary file within the second target to which the actual target storage file belongs is empty; If so, delete the multi-version summary file inside the second target; Delete the externally viewable file of the first target that corresponds to the multi-version summary file inside the second target.

5. The data storage management method as described in claim 1, characterized in that, Also includes: Retrieve information about deleted objects whose deletion action was performed without specifying a data version; Based on the file names of each of the aforementioned internal multi-version summary files, determine the third target internal multi-version summary file corresponding to the deleted object information; Create an empty version file within the multi-version summary file of the third target; Delete the externally viewable file of the second target that corresponds to the internal multi-version summary file of the third target.

6. The data storage management method as described in claim 1, characterized in that, Also includes: Retrieve information about the download object that represents a download action performed without specifying a data version; Based on the file names of the internal multi-version summary files, determine the third target externally viewable file corresponding to the download object information; The actual storage file located based on the latest version attribute information stored in the externally viewable file of the third target is taken as the target file to be downloaded corresponding to the download object information.

7. A distributed unstructured converged and interoperable storage system, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data storage management method as described in any one of claims 1 to 6 when executing the computer program.

8. A computer-readable storage medium, characterized in that, include: The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data storage management method as described in any one of claims 1 to 6.

Images