Data management method, electronic equipment, computer program and storage medium

Abstract

The invention discloses a data management method, electronic equipment, a computer program and a storage medium, and is applied to the technical field of storage. When data is written into the directory added with the acceleration mark, file metadata of a tree structure and object metadata of a key-value pair structure are generated for the directory; when an object access request which is sent by the client and points to the directory added with the acceleration mark is received, obtaining object metadata of the directory to which the object access request points, and feeding back read data contents to the client according to the object metadata; and when a file access request which is sent by a client and points to any catalog is received, obtaining file metadata of the catalog to which the file access request points, and feeding back the read data content to the client according to the file metadata. By applying the scheme of the invention, the access efficiency can be effectively improved, in addition, acceleration can be performed as required, and resource waste is avoided.

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular to a data management method, electronic equipment, computer program, and storage medium. Background Art

[0002] Both file storage and object storage are unstructured storage, but they employ different storage concepts. In file storage, file metadata follows a traditional tree-like organizational structure. As scale increases, performance scalability is limited, resulting in low access efficiency. Object storage, on the other hand, uses a flat storage structure, offering strong scalability and high access efficiency. Currently, some storage vendors have developed unstructured converged solutions that allow a single piece of data to be accessed using both file and object protocols, ensuring compatibility. For users, unstructured convergence can save storage space and the time required to convert data storage formats.

[0003] Current unstructured convergence solutions use the file system as the foundation and implement object storage protocols on top. However, the performance ceiling of this solution depends on the underlying file system, and access efficiency also depends on the underlying file system.

[0004] In summary, how to effectively realize the unstructured integration of file storage and object storage and improve access efficiency is a technical problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0005] The purpose of the present invention is to provide a data management method, electronic device, computer program and storage medium to effectively realize the unstructured integration of file storage and object storage and improve access efficiency.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a data management method, comprising:

[0008] Add acceleration mark to the specified directory;

[0009] When data is written into the directory with the acceleration mark added, file metadata in a tree structure and object metadata in a key-value pair structure are generated for the directory;

[0010] When receiving an object access request sent by a client and pointing to the directory with the acceleration mark added, obtaining object metadata of the directory pointed to by the object access request;

[0011] Reading the data content pointed to by the object metadata from a storage device based on the acquired object metadata, and feeding back the read data content to the client;

[0012] When receiving a file access request directed to any directory from a client, obtaining file metadata of the directory directed to by the file access request;

[0013] The data content pointed to by the file metadata is read from the storage device based on the acquired file metadata, and the read data content is fed back to the client.

[0014] In one embodiment, adding an acceleration mark to a specified directory includes:

[0015] Receive acceleration mark adding instruction;

[0016] Add an acceleration mark to the directory specified by the acceleration mark adding instruction.

[0017] In one embodiment, it further includes:

[0018] Whenever a trigger condition is met, obtaining the data access heat of each directory in the storage device;

[0019] Add an acceleration tag to each directory whose data access popularity meets the popularity requirement.

[0020] In one embodiment, when data is written to the directory with the acceleration mark added, file metadata in a tree structure and object metadata in a key-value pair structure are generated for the directory, including:

[0021] When data is written into the directory with the acceleration mark added, file metadata in a tree structure is generated for the directory, and transaction logs are recorded;

[0022] Based on the transaction log, object metadata in a key-value pair structure is generated for the directory, and before the object metadata is generated, modification of the corresponding file metadata is prohibited.

[0023] In one embodiment, it further includes:

[0024] When receiving a rename instruction for a directory with an acceleration mark added thereto, rename the directory and update the file metadata and the object metadata of the directory;

[0025] When a deletion instruction for a directory with an acceleration mark added thereto is received, the directory is deleted, and the file metadata and the object metadata of the directory are deleted.

[0026] In one embodiment, it further includes:

[0027] The object metadata is stored in a distributed key-value database.

[0028] In one embodiment, it further includes:

[0029] receiving an object access request sent by a client and directed to a directory to which no acceleration mark is added;

[0030] Converting the object access request into a file access request;

[0031] Acquire file metadata of a directory not marked with an acceleration based on the converted file access request;

[0032] The data content pointed to by the file metadata is read from the storage device based on the acquired file metadata, and the read data content is fed back to the client.

[0033] In a second aspect, the present invention provides an electronic device, comprising:

[0034] memory for storing computer programs;

[0035] A processor is used to execute the computer program to implement the steps of the data management method as described above.

[0036] In a third aspect, the present invention provides a computer program product, comprising a computer program, which implements the steps of the data management method as described above when executed by a processor.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the data management method as described above are implemented.

[0038] With the technical solution provided by the embodiment of the application, the file system is also taken as the base, but an acceleration mechanism is set up to effectively improve the access efficiency. Specifically, in order to support file protocol access and object protocol access and guarantee compatibility, the object access request sent by the client can be received and processed, and the file access request sent by the client can also be received and processed. When the file access request sent by the client and pointing to any directory is received, the file metadata of the directory pointed to by the file access request can be acquired, and then the data content pointed to by the file metadata is read from the storage device, so as to be fed back to the client. When the object access request sent by the client and pointing to the directory added with the acceleration mark is received, the object metadata of the directory pointed to by the object access request is acquired. Since the object metadata is a flat key-value pair structure, the operation complexity is lower than that of the file metadata in a tree structure, and therefore, based on the acquired object metadata, the data content pointed to by the object metadata can be efficiently read from the storage device, so as to be fed back to the client. It can be seen that if the client reads the directory added with the acceleration mark through the object access request, since the application scheme generates two sets of metadata (file metadata and object metadata) for the directory, the client can efficiently read the required data content based on the object metadata, and the access efficiency is improved. In addition, it needs to be explained that in order to save the memory resources and avoid the waste of resources caused by the full-amount maintenance of the two sets of metadata, the application scheme is accelerated on demand, that is, not all directories are added with the acceleration mark, but only the specified directories are added with the acceleration mark, and therefore, the application scheme is used for the access acceleration of the specified directories.

[0039] In summary, the application scheme can effectively realize the unstructured fusion of file storage and object storage, and when the directory added with the acceleration mark is accessed as an object, the access efficiency can be effectively improved. In addition, the application scheme is used for the access acceleration of the specified directories, and therefore, the acceleration is on demand, the memory resources are saved, and the waste of resources caused by the full-amount maintenance of the two sets of metadata is avoided. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only show some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without any creative effort.

[0041] Figure 1 The data management method provided by the embodiment of the application is provided in the following.

[0042] Figure 2A schematic diagram of the architecture of a storage system in a specific embodiment of the present invention;

[0043] Figure 3 A schematic structural diagram of an electronic device provided in a specific embodiment of the present invention;

[0044] Figure 4 This is a schematic structural diagram of a computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0046] To help those skilled in the art better understand the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific embodiments. It is apparent that the embodiments described are only a portion of the present invention, not all of the embodiments. All other embodiments derived by those skilled in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0047] Please refer to Figure 1 , Figure 1 This is a flowchart of a data management method provided in a specific embodiment of the present invention. The data management method may include the following steps:

[0048] Step S101: Add an acceleration mark to a specified directory.

[0049] In order to avoid the waste of resources caused by maintaining two sets of metadata, save memory resources, and achieve on-demand acceleration, in this application scheme, not all directories are marked with acceleration tags, but acceleration tags are added to directories with acceleration requirements, that is, acceleration tags are added to designated directories.

[0050] Of course, you can set and adjust which directories require acceleration based on actual needs. For example, you can add acceleration tags to data in important directories and frequently accessed directories. Adding an acceleration tag to a specific directory also adds an acceleration tag to a specific namespace (bucket), for example, setting the acceleration tag to Accelerated=true.

[0051] In a specific embodiment of the present invention, adding an acceleration mark to a specified directory may specifically include:

[0052] Receive an acceleration mark adding instruction;

[0053] Adds an acceleration tag to the directory specified by the acceleration tag add instruction.

[0054] In this embodiment, an acceleration mark adding instruction can be received, and it can be understood that the directory specified by the acceleration mark adding instruction, that is, the directory that the staff believes needs to be accelerated, such as some directories of higher importance or directories with higher access frequency, can all be used as designated directories to add acceleration marks.

[0055] Furthermore, it is understood that staff can also, based on actual needs, remove the acceleration flags from directories that have already been marked as accelerated. For example, they can receive an acceleration flag removal instruction and remove the acceleration flags from the directories specified by the acceleration flag removal instruction. Of course, if the acceleration flag of a directory is removed, the corresponding object metadata can also be deleted.

[0056] In a specific embodiment of the present invention, it may further include:

[0057] Whenever a trigger condition is met, the data access heat of each directory in the storage device is obtained;

[0058] Add an acceleration tag to each directory whose data access popularity meets the popularity requirement.

[0059] This implementation allows for the automatic addition of acceleration tags to directories with high access popularity, in addition to proactively adding acceleration tags to certain directories upon staff request. This effectively reduces staff workload. Furthermore, this implementation considers that highly accessed directories are more likely to be accessed in the near future than less popular directories. Therefore, adding acceleration tags to such directories improves overall system access efficiency and performance.

[0060] Specifically, whenever a trigger condition is met, the data access popularity of each directory in the storage device is obtained. The specific content of the trigger condition can be set according to actual needs. For example, it can be set to trigger periodically, that is, the data access popularity of each directory is periodically obtained. Of course, in other implementations, other triggering methods can be set according to actual needs, which does not affect the implementation of the present invention.

[0061] When obtaining the data access heat of each directory in the storage device, the specific parameter indicators used to measure the data access heat can also be set and adjusted according to actual needs, as long as they can reflect the hot and cold conditions of data access. For example, in one implementation, for a directory, the number of times the directory is accessed within the most recent first time period can be used as the data access heat of the directory, which is relatively simple and convenient to implement. In other implementations, a more complex and accurate data access heat measurement method can be set according to actual needs.

[0062] The heat requirement can be adaptively set based on the measurement method of data access heat. For example, as described above, the number of directory accesses in the most recent first time period is used as the data access heat of the directory, and the heat requirement can be set to a fixed access count threshold. When the number of directory accesses in the most recent first time period exceeds this fixed access count threshold, it can be determined that the data access heat meets the heat requirement, and then an acceleration mark is added to the directory. On the contrary, when the access count threshold is not exceeded, it can be determined that the data access heat does not meet the heat requirement, and there is no need to add an acceleration mark to the directory.

[0063] Furthermore, in actual applications, when the data access heat of a certain directory does not meet the heat requirement, the acceleration mark that has been added to the directory can be cleared, thereby realizing the dynamic update of the directory with the acceleration mark. In addition, it should be pointed out that since the solution of the present application supports adding acceleration marks to the directories specified by the staff, if the solution for dynamically updating the directories with acceleration marks described herein is adopted, the acceleration marks of the directories specified by the staff will generally not be cleared. For example, for the directories specified by the staff, not only can the acceleration mark be added, but also a prompt mark can be added to indicate that "it will not be cleared due to data access heat". So that according to the above solution, when the directories with acceleration marks are dynamically updated, the acceleration marks of the directories specified by the staff will not be cleared due to data access heat. For example, in actual applications, for the acceleration directories specified by the staff, it can be set so that the acceleration marks of such directories can only be cleared in a specific way. For example, only when the acceleration mark clearing instruction sent by the staff is received, the prompt mark will be ignored and the acceleration mark of such directories will be cleared according to the acceleration mark clearing instruction. It can be seen that in this implementation, dynamic updates of directories with acceleration marks are implemented to ensure that directories with high data access popularity can be marked with acceleration, and directories with low data access popularity can have their acceleration marks cleared, which is beneficial to improving the overall access efficiency of the system, and the dynamic update may not affect the acceleration directories specified by the staff.

[0064] Step S102: When data is written into a directory with an acceleration mark added, file metadata in a tree structure and object metadata in a key-value pair structure are generated for the directory.

[0065] When data is written to a directory with an acceleration mark added, the present application scheme sets up a dual-write mechanism for metadata, which can generate two sets of metadata synchronously, that is, it can generate file metadata for file storage and object metadata for object storage.

[0066] File metadata needs to maintain a tree structure. For example, the tree-structured file metadata for a directory is represented as / bucket / dir / object. Object metadata, on the other hand, uses a flat key-value pair structure. The key represents the path information—the complete tree-structured file metadata—while the value represents the mapping between the path information and the specific storage location. For example, the key for the key-value pair object metadata for a directory is "bucket / dir / object," and the value is "location pointer, attribute."

[0067] It is understandable that both directory-based file metadata and directory-based object metadata can determine the specific location of the data content in the directory on the storage device, effectively locating the data content in the directory. However, in terms of access efficiency, using object metadata will have much higher access efficiency. For example, in the above example, when reading the data content in the directory from the storage device based on the directory-based file metadata, the first step is to obtain the file metadata bucket, then obtain the dir, and then obtain the object. The operation complexity is 3. After obtaining the tree-structured file metadata, based on the relevant mapping relationship, the specific location of the data content in the directory on the storage device can be determined, thereby performing data reading.

[0068] In the above example, when reading the data content of a directory from the storage device based on the directory's object metadata, the key is obtained, that is, the complete content of "bucket / dir / object" is directly obtained. The operation complexity is fixed at 1. Based on the value, the specific location of the data content in the directory on the storage device can be determined, and the data can be read.

[0069] It can be seen that the operational complexity of data reading through file metadata is O(n), and the operational complexity of data reading through object metadata is O(1). Here, O is the operational complexity, and the value of n depends on the tree structure of the file metadata. The more complex the tree structure, that is, the more levels it has, the larger the value of n, which makes the operational complexity greater and the access efficiency lower.

[0070] It should also be noted that for directories with acceleration tags, tree-structured file metadata and object metadata need to be generated and stored for them. However, for directories without acceleration tags, since this application solution is based on the file system, tree-structured file metadata needs to be generated and stored for directories without acceleration tags. In other words, regardless of whether the acceleration tag is added, the file metadata of the directory needs to be maintained, and for directories with acceleration tags, the object metadata of the directory needs to be generated and stored.

[0071] In a specific embodiment of the present invention, it may further include:

[0072] Object metadata is stored in a distributed key-value database.

[0073] In the present application, the generated object metadata needs to be stored. In this implementation, the object metadata is stored in a distributed, high-performance key-value database, which achieves decoupling from the underlying file system and ensures the reliability of the object metadata in a distributed scenario. For example, in one scenario, there are 10 storage nodes in the distributed system. Each of these 10 storage nodes is equipped with a key-value database, and each key-value database can be used to maintain all object metadata. Therefore, when the key-value database of any storage node is damaged, the key-value database of any other normal storage node can be used to repair its own key-value database, which is highly reliable. Moreover, when any storage node receives an object access request, it can use the local key-value database to easily obtain the object metadata of the directory pointed to by the object access request.

[0074] Step S103: when receiving an object access request sent by the client and pointing to a directory with an acceleration mark, obtaining object metadata of the directory pointed to by the object access request.

[0075] In the solution of the present application, an object access request sent by a client using an object protocol can be received. For example, the object access request sent by the client can usually be received through an object gateway.

[0076] See Figure 2, is an architectural diagram of the storage system in a specific implementation of the present application. The storage pool can be composed of several storage devices, and with the file system as the base, a NAS (Network Attached Storage) service that supports access to the file system through a file protocol is set up, which can specifically support multiple file protocols such as NFS, CIFS, IKC, etc. Therefore, both client 1 and client 3 can access the file system through the file protocol. For example, client 1 specifically accesses the file system through NFS (Network File System), while client 3 accesses the file system through CIFS (Common Internet File System).

[0077] Figure 2 An object gateway that supports accessing the file system using an object protocol is also set up. Specifically, the object gateway S3 that can provide object storage services can shield the heterogeneity of the underlying storage through a standardized interface (S3 API), so that upper-layer applications do not need to care whether the underlying storage is file storage / block storage or object storage, and can achieve compatibility. Therefore, the client does not need to develop adaptation logic separately for different types of storage. In other words, Figure 2 Both client 2 and client 4 can access the file system through the object protocol.

[0078] also Figure 2 A KVDB (key-value database) is also shown in the figure to store object metadata through a key-value database.

[0079] When an object access request is received from a client, it is necessary to determine whether the directory pointed to by the object access request is a directory with an acceleration mark added. If so, it means that the object metadata of the directory is stored, so that the flattened object metadata can be directly used to complete the data reading. Therefore, the object metadata of the directory pointed to by the object access request can be obtained at this time.

[0080] Step S104: based on the acquired object metadata, the data content pointed to by the object metadata is read from the storage device, and the read data content is fed back to the client.

[0081] When an object access request sent by a client pointing to a directory with an acceleration mark is received, the object metadata of the directory pointed to by the object access request can be obtained, and the Value of the object metadata can be obtained based on the Key of the object metadata, and then the data content can be located according to the Value of the object metadata, so that the data content pointed to by the object metadata can be read from the storage device, that is, the data content pointed to by the object access request sent by the client is read, and finally, the read data content can be fed back to the client. And from the above description, it can be seen that the process of obtaining the Value of the object metadata based on the Key of the object metadata, no matter how complex the path under the file system is, the operation complexity of the process is fixed at 1, which effectively guarantees the data reading efficiency of the solution of this application.

[0082] Step S105: When a file access request directed to any directory is received from the client, file metadata of the directory pointed to by the file access request is obtained.

[0083] The storage system of the present application supports client access using the object protocol and also supports client access using the file protocol. It is understandable that when a file access request is received from a client using the file protocol, there is no need to pay attention to whether the directory pointed to by the file access request is a directory with an acceleration mark added. Access can be performed uniformly in the manner of the file system. Specifically, according to the received file access request, the file metadata of the directory pointed to by the file access request can be directly obtained. The content of the file metadata effectively reflects the path information of the directory, and the specific location of the relevant data on the hard disk can be determined based on this.

[0084] Step S106: Based on the acquired file metadata, the data content pointed to by the file metadata is read from the storage device, and the read data content is fed back to the client.

[0085] The obtained file metadata effectively reflects the path information of the directory pointed to by the file access request, and can be used to determine the specific location of the relevant data in the hard disk. Therefore, based on the obtained file metadata, the data content pointed to by the file metadata can be read from the storage device. Finally, the read data content can be fed back to the client. For example, if the received Figure 2 In the example, client 1 sends a file access request pointing to a certain directory, and the read data content can be fed back to client 1 at this time.

[0086] In a specific embodiment of the present invention, it may further include:

[0087] receiving an object access request sent by a client and directed to a directory to which no acceleration mark is added;

[0088] convert the object access request into a file access request;

[0089] obtain file metadata of the directory without the acceleration mark based on the converted file access request;

[0090] read data content pointed by the file metadata from the storage device based on the obtained file metadata, and feed back the read data content to the client.

[0091] As described above, when receiving the object access request sent by the client, it is needed to judge whether the directory pointed by the object access request is the directory with the acceleration mark, if the directory pointed by the object access request is not the directory with the acceleration mark, in the scheme of the present application, the data required by the client should also be fed back, of course, since it is not the directory with the acceleration mark, the effect of improving the access efficiency cannot be achieved.

[0092] In this embodiment, the situation that the client sends the object access request pointing to the directory without the acceleration mark can be effectively coped with, at this time, the object access request needs to be converted into a file access request, for example, the object access request is converted into a file access request through the object gateway, after the conversion is completed, the file metadata of the corresponding directory can be obtained based on the converted file access request, the subsequent principle is consistent with step S105, that is, according to the obtained file metadata, the specific position of the related data in the hard disk can be determined, then the data content pointed by the file metadata can be read from the storage device, finally the read data content can be fed back to the client, for example, the read data content is fed back to the client through the object gateway.

[0093] As can be seen, in this embodiment, the situation that the client sends the object access request pointing to the directory without the acceleration mark can be effectively coped with, so that in the scheme of the present application, whether the client sends the object access request or the file access request, whether it points to the directory with the acceleration mark or the directory without the acceleration mark, the access can be effectively completed and the data required by the client can be fed back. Of course, for the situation that the client sends the object access request and points to the directory with the acceleration mark, the access efficiency can be greatly improved, and through tests in actual application, the delay can be reduced by 30% to 70%.

[0094] In a specific embodiment of the present application, when data is written into the directory with the acceleration mark, the tree structure file metadata and the key-value pair structure object metadata are generated for the directory, which can include:

[0095] When data is written into the directory with the acceleration mark, the tree structure file metadata is generated for the directory, and the transaction log is recorded;

[0096] Based on the transaction log, the object metadata in the key-value pair structure is generated for the directory, and modification of the corresponding file metadata is prohibited before the object metadata is generated.

[0097] When data is written into the directory added with the acceleration mark, the dual-write mechanism of the metadata set according to the scheme of the present application is required to synchronously generate two sets of metadata, and in this implementation, the transaction log (WAL) is used to effectively ensure the atomicity of dual-write, that is, to ensure the consistency of the file metadata and the object metadata. The consistency described herein refers to the fact that the two can point to the same data content in the storage location.

[0098] Specifically, when data is written into the directory, the file metadata in the tree structure is first generated for the directory, and it can be understood that there is a time interval between the generation of the file metadata and the generation of the object metadata. If the file metadata is changed during this period, an error is likely to occur, resulting in inconsistency between the file metadata and the object metadata.

[0099] In this implementation, the transaction log is recorded immediately when the file metadata in the tree structure is generated. It can be understood that the object metadata is generated according to the transaction log, and the corresponding file metadata is prohibited from being modified before the object metadata is generated, so that the generated object metadata is consistent with the corresponding file metadata, ensuring the atomicity of dual-write.

[0100] Further, in a specific implementation of the present application, the current version number of the file metadata is recorded each time the file metadata is changed, thereby realizing conflict detection. Of course, if the directory is added with the acceleration mark, the corresponding object metadata needs to be updated, and the current version number of the file metadata is copied. When a file access request pointing to any directory sent by the client is received, or when an object access request pointing to any directory sent by the client is received, it can be determined whether there is a conflict by judging whether the version number in the file access request or the object access request is consistent with the current version number of the file metadata or the current version number of the object metadata. When the version numbers are inconsistent, it can be considered as an abnormal situation, and the client is prompted.

[0101] In a specific implementation of the present application, the following can also be included:

[0102] When a rename instruction for the directory added with the acceleration mark is received, the directory is renamed, and the file metadata and the object metadata of the directory are updated;

[0103] When a delete instruction for the directory added with the acceleration mark is received, the directory is deleted, and the file metadata and the object metadata of the directory are deleted.

[0104] In actual application, the metadata synchronization service can listen to file system events and update object metadata in the key-value pair database in real time. Specifically, when receiving a renaming instruction for a directory with the acceleration mark added, the directory needs to be renamed, at this time, the path information of the directory is changed, thus the file metadata of the directory after renaming needs to be updated, and the corresponding object metadata also needs to be updated. Of course, if the received is a renaming instruction for a directory without the acceleration mark added, only the directory needs to be renamed, and the file metadata of the directory needs to be updated.

[0105] When receiving a deletion instruction for a directory with the acceleration mark added, the directory needs to be deleted, and the file metadata and object metadata of the directory also need to be deleted. Of course, if the received is a deletion instruction for a directory without the acceleration mark added, only the directory needs to be deleted, and the file metadata of the directory needs to be deleted.

[0106] The technical solution provided by the embodiment of the application is also based on a file system, but an acceleration mechanism is set to effectively improve the access efficiency. Specifically, in order to support file protocol access and object protocol access and guarantee compatibility, the object access request sent by the client can be received and processed, and the file access request sent by the client can also be received and processed. When receiving the file access request sent by the client and pointing to any directory, the file metadata of the directory pointed to by the file access request can be obtained, and then the data content pointed to by the file metadata is read from the storage device, so as to be fed back to the client. When receiving the object access request sent by the client and pointing to a directory with the acceleration mark added, the object metadata of the directory pointed to by the object access request is obtained. Since the object metadata is a flat key-value pair structure, the operation complexity is lower than that of the tree-shaped file metadata, thus based on the obtained object metadata, the data content pointed to by the object metadata can be efficiently read from the storage device, so as to be fed back to the client. It can be seen that if the client reads the directory with the acceleration mark added through the object access request, since the application generates two sets of metadata (file metadata and object metadata) for it, the client can efficiently read the required data content based on the object metadata, and the access efficiency is improved. In addition, it needs to be explained that in order to save memory resources and avoid resource waste caused by full maintenance of two sets of metadata, the application is accelerated on demand, that is, not all directories have the acceleration mark added, but only specified directories have the acceleration mark added, thus the application is used for access acceleration of the specified directories.

[0107] In conclusion, the scheme can effectively realize the unstructured fusion of file storage and object storage, and when the object is accessed for the directory added with the acceleration mark, the access efficiency can be effectively improved. In addition, the scheme is for the access acceleration of the specified directory, and thus the acceleration can be performed on demand, the memory resources are saved, and the waste of resources caused by the full maintenance of two sets of metadata is avoided.

[0108] Corresponding to the above method embodiments, the embodiments of the application further provide an electronic device, a computer readable storage medium and a computer program product, which can be mutually corresponding with the above.

[0109] Referring to Figure 3 As shown in the figure, the device can include:

[0110] The memory 301 is configured to store a computer program.

[0111] The processor 302 is configured to execute the computer program to implement the steps of the data management method in any of the above embodiments.

[0112] The computer program product includes computer programs / instructions, which, when executed by the processor, implement the steps of the data management method in any of the above embodiments.

[0113] Referring to Figure 4 The computer readable storage medium 40 stores a computer program 41, and the computer program 41, when executed by the processor, implements the steps of the data management method in any of the above embodiments. The computer readable storage medium 40 mentioned herein includes RAM (Random Access Memory, Random Access Memory), memory, ROM (Read-Only Memory, Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory, Electrically Erasable Programmable Read-Only Memory), register, hard disk, removable disk, or any other form of storage medium known in the technical field.

[0114] It is also important to note that the use of relational terms such as first and second, and the like, are used solely to distinguish one from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0115] Those skilled in the art will further appreciate that the units and algorithms described in connection with the examples disclosed herein can be embodied directly in hardware, in software, or in a combination of the two. For the sake of brevity, descriptions of these conventional implementations are not included herein. Those skilled in the art will appreciate that the embodiments described herein can be practiced with the aid of one or more computers or processors, which can be hand-held, notebook, desktop, or mainframe. In general, the computer or processor will have a memory that can include, without limitation, local memory and / or remote memory. The memory will also include media, such as removable media or media that is internal; media that is volatile, non-volatile, or transitory; and media that is shared, dedicated, or otherwise. Also, the memory can include mass storage, volatile storage, or both. In general, the computer or processor will also include input means, standard input means, and output means, which can include peripheral devices, such as a keyboard or mouse, and / or a display. In general, the computer or processor will further include communication means, such as a modem or network interface. Those skilled in the art will further recognize that at least some of the units and algorithms described herein can be implemented with the aid of one or more computers or processors, which can be hand-held, notebook, desktop, or mainframe. In general, the computer or processor will have a memory that can include, without limitation, local memory and / or remote memory. The memory will also include media, such as removable media or media that is internal; media that is volatile, non-volatile, or transitory; and media that is shared, dedicated, or otherwise. Also, the memory can include mass storage, volatile storage, or both. In general, the computer or processor will also include input means, standard input means, and output means, which can include peripheral devices, such as a keyboard or mouse, and / or a display. In general, the computer or processor will further include communication means, such as a modem or network interface. Those skilled in the art will further recognize that at least some of the units and algorithms described herein can be implemented with the aid of one or more computers or processors, which can be hand-held, notebook, desktop, or mainframe. In general, the computer or processor will have a memory that can include, without limitation, local memory and / or remote memory. The memory will also include media, such as removable media or media that is internal; media that is volatile, non-volatile, or transitory; and media that is shared, dedicated, or otherwise. Also, the memory can include mass storage, volatile storage, or both. In general, the computer or processor will also include input means, standard input means, and output means, which can include peripheral devices, such as a keyboard or mouse, and / or a display. In general, the computer or processor will further include communication means, such as a modem or network interface. Those skilled in the art will further recognize that at least some of the units and algorithms described herein can be implemented with the aid of one or more computers or processors, which can be hand-held, notebook, desktop, or mainframe. In general, the computer or processor will have a memory that can include, without limitation, local memory and / or remote memory. The memory will also include media, such as removable media or media that is internal; media that is volatile, non-volatile, or transitory; and media that is shared, dedicated, or otherwise. Also, the

Claims

1. A data management method, characterized in that: include: Add acceleration mark to the specified directory; When data is written into the directory with the acceleration mark added, file metadata in a tree structure and object metadata in a key-value pair structure are generated for the directory; When receiving an object access request sent by a client and pointing to the directory with the acceleration mark added, obtaining object metadata of the directory pointed to by the object access request; Reading the data content pointed to by the object metadata from a storage device based on the acquired object metadata, and feeding back the read data content to the client; When receiving a file access request sent by a client pointing to any directory, obtaining file metadata of the directory pointed to by the file access request; The data content pointed to by the file metadata is read from the storage device based on the acquired file metadata, and the read data content is fed back to the client.

2. The data management method according to claim 1, wherein: Add acceleration tags to the specified directory, including: Receive acceleration mark adding instruction; Add an acceleration mark to the directory specified by the acceleration mark adding instruction.

3. The data management method according to claim 2, characterized in that: Also includes: Whenever a trigger condition is met, obtaining the data access heat of each directory in the storage device; Add an acceleration tag to each directory whose data access popularity meets the popularity requirement.

4. The data management method according to claim 1, wherein: When data is written into the directory with the acceleration mark added, file metadata in a tree structure and object metadata in a key-value pair structure are generated for the directory, including: When data is written into the directory with the acceleration mark added, file metadata in a tree structure is generated for the directory, and transaction logs are recorded; Based on the transaction log, object metadata in a key-value pair structure is generated for the directory, and before the object metadata is generated, modification of the corresponding file metadata is prohibited.

5. The data management method according to claim 1, wherein: Also includes: When receiving a rename instruction for a directory with an acceleration mark added, rename the directory and update the file metadata and the object metadata of the directory; When a deletion instruction for a directory with an acceleration mark added is received, the directory is deleted, and the file metadata and the object metadata of the directory are deleted.

6. The data management method according to claim 1, wherein: Also includes: The object metadata is stored in a distributed key-value database.

7. The data management method according to any one of claims 1 to 6, characterized in that: Also includes: receiving an object access request sent by a client and directed to a directory to which no acceleration mark is added; Converting the object access request into a file access request; Acquire file metadata of a directory not marked with an acceleration based on the converted file access request; The data content pointed to by the file metadata is read from the storage device based on the acquired file metadata, and the read data content is fed back to the client.

8. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the data management method according to any one of claims 1 to 7.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the data management method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data management method according to any one of claims 1 to 7 are implemented.

Images