Data Pool Management Method, Computer Device, Storage Medium, and Program Product

Abstract

The present application relates to a data pool management method, a computer device, a storage medium and a program product, belonging to the field of distributed storage technology. The method includes: when receiving a first file truncation request corresponding to a target file, sending a first space application request to a first metadata server; receiving a first target space allocated according to the first space application request, and defining the first target space as a first file header; adding the first file header to the first file truncation request to generate a second file truncation request, and sending the second file truncation request to a second metadata server; receiving an adjustment result obtained by adjusting a second file header of the target file and the content in the second file header according to the second file truncation request; and releasing a corresponding storage space according to the adjustment result. The present application reduces resource occupation and improves the processing efficiency of truncate requests and the reliability of the distributed storage system.

Description

Technical Field

[0001] This application relates to the field of distributed storage technologies, and particularly to a data pool management method, a computer device, a storage medium, and a program product. Background Art

[0002] In Ceph (an open-source distributed storage system), the OSD (Object Storage Daemon) is a key component responsible for managing the actual data storage. Each OSD node is responsible for managing a part of the storage space and collaborates through the RADOS (Reliable Autonomic Distributed Object Store) layer to ensure functions such as data distribution, replication, and recovery. Among them, for the management of the data storage space, the OSD must rely on the RADOS layer to implement. When executing the file truncate process, the OSD is used to process the block content that needs to be released by the background file itself, and the Client (Ceph client) and MDS (Metadata Server) only need to update the file metadata content (file size and time).

[0003] The OSD manages the data pool space for the entire Ceph cluster operation, that is, it can implement large and comprehensive functions such as object storage, block storage, and file storage. However, in the scenario where the user only uses file storage and does not require object storage, if the OSD is still used for management, not only must the RADOS component be configured, but the OSD also needs to manage the corresponding metadata pool, resulting in a multi-layer processing requirement for the entire environment. Moreover, the file storage still needs to provide the MDS service, making the current storage process overly complicated. When truncating a file, the OSD actually divides a block to be released in 4M size. When a file is very large, all 4M-sized blocks need to be found and released, resulting in a low processing speed for the truncate request. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a data pool management method, a computer device, a storage medium, and a program product that can improve the processing efficiency of truncate requests.

[0005] In a first aspect, a data pool management method is provided. The method includes:

[0006] When receiving a first file truncate request corresponding to a target file, sending a first space application request to a first metadata server, where the file truncate request includes the truncated file size, and the space application request includes the applied space size;

[0007] Receive the first target space allocated by the first metadata server according to the first space application request, and define the first target space as the first file header;

[0008] Add the first file header to the first file truncation request to generate a second file truncation request, and send the second file truncation request to the second metadata server;

[0009] Receive the adjustment result obtained by the second metadata server adjusting the second file header of the target file and the content in the second file header according to the second file truncation request, where the file header is bound to the file and is used to store the basic information of the file;

[0010] Release the corresponding storage space according to the adjustment result.

[0011] Optionally, before receiving the first file truncation request corresponding to the target file, it further includes:

[0012] Receive the target file creation request;

[0013] When receiving the target file creation request, send a second space application request to the first metadata server;

[0014] Receive the second target space allocated by the first metadata server according to the second space application request, and define the second target space as the second file header;

[0015] Save the basic information of the target file to the second file header, bind the second file header that has stored the basic information of the target file to the target file, and save them to the distributed storage system client and the second metadata server respectively.

[0016] Optionally, before receiving the first file truncation request corresponding to the target file, it further includes:

[0017] Based on the server, send the first file truncation request corresponding to the target file to the distributed storage system client.

[0018] Optionally, after the first metadata server allocates the first target space for the distributed storage system client according to the first space application request, it further includes:

[0019] In response to the completion of the allocation of the first target space, return the first space application request to the distributed storage system client based on the first metadata server;

[0020] When the distributed storage system client receives the first space application request returned by the first metadata server, define the first target space as the first file header.

[0021] Optionally, after defining the first target space as the first file header, it further includes:

[0022] Empty the content stored in the second file header of the distributed storage system client.

[0023] Optionally, before adjusting the second file header of the target file and the content in the second file header according to the second file truncation request, it further includes:

[0024] Obtain the truncated file size and the first file header in the second file truncation request;

[0025] Based on the truncated file size, determine a target adjustment mechanism to adjust the second file header of the target file and the content in the second file header based on the target adjustment mechanism and the first file header.

[0026] Optionally, determining the target adjustment mechanism based on the truncated file size includes:

[0027] In response to the truncated file size being the target value, determine the target adjustment mechanism as the first target adjustment mechanism;

[0028] In response to the truncated file size not being the target value, determine the target adjustment mechanism as the second target adjustment mechanism.

[0029] Optionally, adjusting the second file header of the target file and the content in the second file header based on the first target adjustment mechanism and the first file header includes:

[0030] Use the first file header to replace the second file header of the target file as the current file header of the target file, add the data block corresponding to the second file header to the release queue, and at the same time, return the second file truncation request to the distributed storage system client.

[0031] Optionally, before adjusting the second file header of the target file and the content in the second file header based on the second target adjustment mechanism and the first file header, it further includes:

[0032] Based on the background thread, read the data block corresponding to the second file header and write the data block corresponding to the second file header to the object storage to the metadata pool;

[0033] In response to writing the object being completed, generate a record log, and the record log includes the object name.

[0034] Optionally, adjusting the second file header of the target file and the content in the second file header based on the second target adjustment mechanism and the first file header includes:

[0035] Based on the background thread, read the data block corresponding to the second file header;

[0036] Divide the data blocks corresponding to the second file header into data blocks to be written back and data blocks to be released according to the truncated file size;

[0037] Write the data blocks to be written back to the second file header, add the data blocks to be released to the release queue, and at the same time, return a second file truncation request to the distributed storage system client.

[0038] Optionally, in response to a failure of the second metadata server during the process of writing back data blocks, it further includes:

[0039] Perform a failure recovery on the second metadata server;

[0040] In response to the failure of the second metadata server being recovered, read the target record log;

[0041] According to the object name in the target record log, read the data block corresponding to the object name, and rewrite the data block corresponding to the object name to the second file header.

[0042] Optionally, it further includes:

[0043] Detect whether there is a need to save the data blocks to be written back based on the first file header;

[0044] In response to the non-existence of the need to save the data blocks to be written back based on the first file header, add the first file header to the release queue.

[0045] Optionally, it further includes:

[0046] In response to the distributed storage system client receiving the second file truncation request returned by the second metadata server, determine that the second file truncation request has been processed;

[0047] In response to the second file truncation request being processed, send the adjustment result of the second file header to multiple distributed storage system clients based on the second metadata server;

[0048] When multiple distributed storage system clients receive the adjustment result of the second file header, synchronously adjust the second file header and the content in the second file header saved in the corresponding distributed storage system clients.

[0049] Optionally, according to the adjustment result, releasing the corresponding storage space includes:

[0050] Obtain the release queue;

[0051] Determine the storage space occupied by the data blocks and / or the first file header in the release queue, and release the storage space.

[0052] In a second aspect, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0053] When receiving a first file truncation request corresponding to a target file, send a first space application request to a first metadata server. The file truncation request includes a truncated file size, and the space application request includes an applied space size.

[0054] Receive a first target space allocated by the first metadata server according to the first space application request, and define the first target space as a first file header.

[0055] Add the first file header to the first file truncation request to generate a second file truncation request, and send the second file truncation request to a second metadata server.

[0056] Receive an adjustment result obtained by the second metadata server adjusting a second file header of the target file and the content in the second file header according to the second file truncation request. The file header is bound to the file and is used to store basic information of the file.

[0057] Release the corresponding storage space according to the adjustment result.

[0058] In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0059] When receiving a first file truncation request corresponding to a target file, send a first space application request to a first metadata server. The file truncation request includes a truncated file size, and the space application request includes an applied space size.

[0060] Receive a first target space allocated by the first metadata server according to the first space application request, and define the first target space as a first file header.

[0061] Add the first file header to the first file truncation request to generate a second file truncation request, and send the second file truncation request to a second metadata server.

[0062] Receive an adjustment result obtained by the second metadata server adjusting a second file header of the target file and the content in the second file header according to the second file truncation request. The file header is bound to the file and is used to store basic information of the file.

[0063] Release the corresponding storage space according to the adjustment result.

[0064] Fourthly, a computer program product is provided. The computer program product includes a computer program which, when executed by a processor, implements the following steps:

[0065] When receiving a first file truncation request corresponding to a target file, send a first space application request to a first metadata server. The file truncation request includes a truncated file size, and the space application request includes an applied space size.

[0066] Receive a first target space allocated by the first metadata server according to the first space application request, and define the first target space as a first file header.

[0067] Add the first file header to the first file truncation request to generate a second file truncation request, and send the second file truncation request to a second metadata server.

[0068] Receive an adjustment result obtained by the second metadata server adjusting a second file header of the target file and the content in the second file header according to the second file truncation request. The file header is bound to the file and is used to store basic information of the file.

[0069] Release the corresponding storage space according to the adjustment result.

[0070] The above data pool management method, computer device, storage medium and program product. The method includes: when receiving a first file truncation request corresponding to a target file, send a first space application request to a first metadata server. The file truncation request includes a truncated file size, and the space application request includes an applied space size. Receive a first target space allocated by the first metadata server according to the first space application request, and define the first target space as a first file header. Add the first file header to the first file truncation request to generate a second file truncation request, and send the second file truncation request to a second metadata server. Receive an adjustment result obtained by the second metadata server adjusting a second file header of the target file and the content in the second file header according to the second file truncation request. The file header is bound to the file and is used to store basic information of the file. Release the corresponding storage space according to the adjustment result. In the present application, in the case of only applying the file storage scenario, by using MDS (metadata server) to manage the data pool to implement the file truncate action, not only the code architecture is reduced, the resource occupation is reduced, but also the overall process is simplified, making the truncate process execute faster, improving the processing efficiency of the truncate request. In addition, by MDS processing the entire truncate request, data consistency can be guaranteed, and the reliability of the distributed storage system is improved. Description of the Drawings

[0071] Figure 1Schematic flowchart of a data pool management method in an embodiment;

[0072] Figure 2 Schematic flowchart of a file adjustment process of a data pool management method in an embodiment;

[0073] Figure 3 Schematic flowchart of another file adjustment process of a data pool management method in an embodiment;

[0074] Figure 4 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0075] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0076] It should be understood that in the description of the present application, unless otherwise clearly required by the context, the words such as "including" and "comprising" throughout the specification shall be interpreted in an inclusive sense rather than an exclusive or exhaustive sense; that is, the meaning of "including but not limited to".

[0077] It should also be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0078] It should be noted that the terms "S1", "S2", etc. are only used for the purpose of step description and do not particularly refer to the order or sequence. Nor are they used to limit the present application. They are only used to conveniently describe the method of the present application and cannot be understood as indicating the sequence of steps. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement them. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present application.

[0079] In an embodiment, as Figure 1 shown, a data pool management method is provided, including the following steps:

[0080] S1: When receiving a first file truncation request corresponding to a target file, send a first space application request to a first metadata server, where the file truncation request includes a truncated file size, and the space application request includes an applied space size.

[0081] It should be noted that the first file truncation request corresponding to the target file is received through a distributed storage system client. The distributed storage system client refers to the Ceph Client. Ceph is an open-source distributed storage system designed to provide high-performance, reliable, and scalable storage solutions. Ceph can provide object storage, block storage, and file system storage services simultaneously, making it a very flexible choice suitable for various application scenarios, including cloud computing, data centers, and large-scale data storage, etc.; the metadata server refers to the MDS. The MDS node is responsible for managing the metadata of the Ceph file system, that is, the directory tree structure and attribute information of the file system. The MDS nodes cooperate with each other to provide metadata services and support the efficient access of large-scale file systems; the first metadata server generally refers to the master node MDS0, which can be used to allocate space; the file truncation request refers to the truncate request. "Truncate" is an operation usually used in the file system. When a file is truncated, its content is reduced to the specified size. If the specified size is less than the current size of the file, the data beyond the specified size will be deleted. If the specified size is greater than the current size of the file, the file size will increase, but the newly added part will be filled with zeros or remain undefined; the truncated file size is the above-mentioned file specified size, which is determined according to requirements; the space application request refers to the request for applying for allocated space. The size of the applied space can be determined according to actual needs, and its preferred value is 4k.

[0082] In some specific embodiments, before receiving the first file truncation request corresponding to the target file, the method further includes:

[0083] Based on the distributed storage system client, receive a target file creation request to create a target file for storing file data and file metadata. The target file creation request is sent by a server;

[0084] When creating the target file, the distributed storage system client sends a second space application request to the first metadata server, where the second space application request also includes the size of the applied space. The size of the applied space can be determined according to actual needs, and its preferred value is 4k;

[0085] The first metadata server allocates a second target space for the distributed storage system client according to the second space application request. After receiving the second target space, the distributed storage system client defines the second target space as the second file header. Here, the file header refers to the "header", which is a data structure mainly used to store the basic information content of the file, such as the file modification time, file size, and space map (data structure). Since the final size of each file cannot be determined, a map mapping is used to store the corresponding allocated space blocks. When writing data later, for example, when appending 1M of data to a file, if the file only has a 4k header at this time, it is necessary to apply to MDS0 for more than 1M of space to store the data to be written by the Client. At this time, this newly allocated space is written into the corresponding space map, and when reading and writing the file later, the corresponding space block can be found from the space map for reading and writing, thus realizing the file reading and writing of the Client. The basic information data of the file is stored in the second file header here;

[0086] Save the basic information of the target file to the second file header, bind the second file header that has stored the basic information of the target file to the target file, and save them to the distributed storage system client and the second metadata server respectively. Here, the second metadata server is the MDS node corresponding to the distributed storage system client, which can be MDS0 or other nodes.

[0087] Specifically, when the Client creates a file, it actively sends a request to MDS0 to apply for a 4k-sized space. After MDS0 allocates the corresponding space and returns the request, when the Client obtains this 4k-sized space, it uses it as the header of the newly created file, binds the header of the newly created file to the newly created file, and saves them to the Ceph Client and the MDS node corresponding to the Ceph Client at the same time.

[0088] In some specific embodiments, before receiving the first file truncation request corresponding to the target file, the method further includes:

[0089] Based on the server, send the first file truncation request corresponding to the target file to the distributed storage system client. Here, as Figure 2 and Figure 3 shown, the truncate request is sent by the server to the distributed storage system client and processed by the distributed storage system client and the MDS node.

[0090] In the above embodiments, the corresponding file header is constructed when creating a file, so that when a file truncation request is received, the space occupied by the target file can be directly adjusted through the generated spare file header, thereby improving the execution speed of the truncate request.

[0091] S2: Receive the first target space allocated by the first metadata server according to the first space application request, and define the first target space as the first file header.

[0092] It should be noted that the first metadata server allocates the first target space for the distributed storage system client according to the first space application request and sends it to the distributed storage system client. The first file header is a spare header; the first target space is generally a 4k space, which is a data structure, and the basic information data of the file is not stored in the first target space.

[0093] In some specific embodiments, after the first metadata server allocates the first target space for the distributed storage system client according to the first space application request, the method further includes:

[0094] In response to the completion of the allocation of the first target space, return the first space application request to the distributed storage system client based on the first metadata server;

[0095] When the distributed storage system client receives the first space application request returned by the first metadata server, it defines the first target space as the first file header.

[0096] In some specific embodiments, after defining the first target space as the first file header, the method further includes:

[0097] Empty the content stored in the second file header of the distributed storage system client.

[0098] Specifically, when the Client responds to the truncate request, that is, when truncating the target file, it actively sends a request to MDS0 to apply for a 4k-sized space. After MDS0 allocates the corresponding space and returns the request, after the Client obtains this 4k-sized space, it uses it as the spare header of the target file and empties the content of the header of the target file itself at the Client side. Since the header of the target file has been emptied, the corresponding data block management is also gone, and at this time, the target file cannot be read and written on the current Client.

[0099] In the above embodiments, a spare file header for adjusting the occupied space of the target file is generated through the applied space, and the content in the second file header stored in the client of the distributed storage system is cleared, so as to avoid data inconsistency problems caused by file modification at the Client side during the file truncation request processing.

[0100] S3: Add the first file header to the first file truncation request to generate a second file truncation request, and send the second file truncation request to the second metadata server.

[0101] It should be noted that the second file truncation request includes the first file header, that is, the spare header. The second metadata server is the MDS node corresponding to the client of the distributed storage system, which can be MDS0 or other nodes and is used to manage metadata information.

[0102] Specifically, after clearing the content of the original header of the target file, a truncate request is sent to the MDS, and the spare header data just applied is carried in the truncate request.

[0103] In the above embodiments, by adding the spare header to the file truncation request and sending it to the MDS, it is ensured that the MDS can manage the data pool, and then the file truncate action is realized, reducing the code architecture and simplifying the overall process, thereby reducing resource occupation.

[0104] S4: Receive the adjustment result obtained by the second metadata server according to the second file truncation request for adjusting the second file header of the target file and the content in the second file header, where the file header is bound to the file and is used to store the basic information of the file.

[0105] It should be noted that the adjustment result is received by the client of the distributed storage system. Generally, there are two cases for truncating the file size, namely specifying the size as 0 and not 0, and there are corresponding adjustment mechanisms for each case to adjust the second file header of the target file and the content in the second file header.

[0106] In some specific embodiments, before adjusting the second file header of the target file and the content in the second file header according to the truncated file size and the first file header, the method further includes:

[0107] Obtain the truncated file size and the first file header in the second file truncation request;

[0108] Based on the truncated file size, determine the target adjustment mechanism, so as to adjust the second file header of the target file and the content in the second file header based on the target adjustment mechanism and the first file header.

[0109] In some specific embodiments, determining a target adjustment mechanism based on a truncated file size includes:

[0110] In response to the truncated file size being a target value, determining that the target adjustment mechanism is a first target adjustment mechanism, where the target value is 0 as described above, that is, when the specified size of the truncated file is 0, the second file header of the target file and the content in the second file header are adjusted through the first target adjustment mechanism and the first file header;

[0111] In response to the truncated file size not being the target value, determining that the target adjustment mechanism is a second target adjustment mechanism, that is, when the specified size of the truncated file is not 0, the second file header of the target file and the content in the second file header are adjusted through the second target adjustment mechanism and the first file header.

[0112] In some specific embodiments, as Figure 2 shown, adjusting the second file header of the target file and the content in the second file header based on the first target adjustment mechanism and the first file header includes:

[0113] Using the first file header to replace the second file header of the target file as the current file header of the target file, adding the data block corresponding to the second file header to the release queue, and at the same time, returning a second file truncation request to the distributed storage system client.

[0114] Specifically, when the MDS receives a truncate request, it determines that the specified size of the truncated file is 0, that is, all the file data block contents can be released, then stores the current header of the target file in the release queue of the file, updates the spare header in the truncate request to the header of the file. At this time, the content in this header is empty and there is no data in it. At this time, the truncate request can be returned, determining that the truncate request has been processed. The release queue will be slowly released by the background thread of the MDS itself later, without blocking the current truncate request and being able to respond quickly.

[0115] In the above embodiment, when the specified size of the truncated file is 0, by directly replacing the header to respond to the file truncate request, the processing speed of the truncate request is improved.

[0116] In some specific embodiments, before adjusting the second file header of the target file and the content in the second file header based on the second target adjustment mechanism and the first file header, the method further includes:

[0117] Based on a background thread, read the data blocks corresponding to the second file header and write the data blocks corresponding to the second file header to the object storage into the metadata pool, where the data blocks in the file header are data blocks generated based on a map mapping and are used to map to the data content of the target file;

[0118] In response to completing the writing of the object, generate a record log, and the record log includes the object name.

[0119] In some specific embodiments, such as Figure 3 As shown, adjusting the second file header of the target file and the content in the second file header based on the second target adjustment mechanism and the first file header includes:

[0120] Based on a background thread, read the data blocks corresponding to the second file header;

[0121] According to the truncated file size, divide the data blocks corresponding to the second file header into data blocks that need to be written back and data blocks that need to be released;

[0122] Write back the data blocks that need to be written back to the second file header, add the data blocks that need to be released to the release queue, and at the same time, return a second file truncation request to the distributed storage system client.

[0123] In some specific embodiments, in response to a failure occurring in the second metadata server during the process of writing back data blocks, the method further includes:

[0124] Perform a failure recovery on the second metadata server;

[0125] In response to the failure of the second metadata server being recovered, read the target record log;

[0126] According to the object name in the target record log, read the data blocks corresponding to the object name and rewrite the data blocks corresponding to the object name back to the second file header.

[0127] In some specific embodiments, the method further includes:

[0128] Detect whether there is a need to save the data blocks that need to be written back based on the first file header;

[0129] In response to there being no need to save the data blocks that need to be written back based on the first file header, add the first file header to the release queue.

[0130] Specifically, when the MDS receives a truncate request and determines that the truncate size is non-zero, that is, only the data block space corresponding to the specified size needs to be released. At this time, the data block content in the original header of the target file needs to be read out and divided into two parts. One part is the data block within the specified size, and this part of the data block still needs to be used continuously. The other part is the data block content outside the specified size, and this part is to be added to the release queue for release. Based on this, the data blocks to be retained and continued to be used are written back to the header, and the data blocks to be released are added to the release queue. However, if the MDS fails during the write-back process, the header of the target file is not written with the data blocks completely, resulting in a scenario of data inconsistency. Based on this, before writing back the retained data blocks to the header, all the original data blocks of the target file are first read out and written into an object and stored in the metadata pool. This object is used to record all the data blocks before the file is truncated. After the object is written, a journal (log) is recorded. This log carries the name of the object. If the MDS fails during the process of writing back the data blocks to the header, at this time, the header of the target file is in an abnormal state, that is, the content in the header is inaccurate. After the MDS failure is recovered and the log is played back, the recorded journal is read out. According to the object name recorded in the log, the object content is read, that is, all the data blocks in the original header of the target file. The read all data blocks are written back to the header again. At this time, the file truncate request has not been processed, and the target file is still in the initial state, and the original data blocks are still intact, thus avoiding the scenario of data inconsistency. On this basis, the file truncate request is continued to be responded to. This is the corresponding processing method when the MDS fails. If the MDS does not fail, after recording the log, the write-back process is continued.

[0131] Furthermore, when the processing of the write-back process is completed, there may be a need to use 4k space as a map to record continuous data blocks, that is, when the original header of the target file cannot store all the data blocks that need to be written back. At this time, the spare header carried in the truncate request is used as a newly inserted map. If a 4k space map is not needed, the spare header in the truncate request is also added to the release queue to complete the space release.

[0132] Further, in the process of reading data blocks from the original header and writing the data blocks back to the header, a disk read / write process needs to occur. This operation is a blocking operation. If it is directly executed in the message thread of the MDS, it will block all message processing. Based on this, when a request to truncate to a specified size is recognized, a background thread is started to read or write the corresponding data block of the header, and the message thread directly exits. At this time, it is necessary to ensure the lock occupancy of the target file. When the background thread completes the corresponding operation, the request is retried. The MDS reprocesses the request to truncate to the specified size and determines whether the action of reading or writing the header has been executed according to the flag bit specified by the background thread. When the write-back is completed, the truncate request flag is marked as processed, and after logging, a response is sent to the Client, that is, a second file truncation request is returned to the distributed storage system client.

[0133] In some specific embodiments, the method further includes:

[0134] In response to the distributed storage system client receiving the second file truncation request returned by the second metadata server, it is determined that the second file truncation request has been processed;

[0135] In response to the second file truncation request having been processed, based on the second metadata server, the adjustment results of the second file header are sent to multiple distributed storage system clients, where the adjustment results include the adjustment results corresponding to the first target adjustment mechanism and the second target adjustment mechanism;

[0136] When multiple distributed storage system clients receive the adjustment results of the second file header, they synchronously adjust the second file header and the content in the second file header saved in the corresponding distributed storage system clients.

[0137] Specifically, after the truncate request is returned, in the truncate operation, the MDS will pass the updated header of the target file to each Client through the message MClientCap (managing the permissions and capabilities of the client). After receiving this message, the Client will update the file size, time, and header according to the content passed in the message. At this time, the header corresponding to each Client is the updated header. For example, in the case of adjusting the file header using the first target adjustment mechanism, that is, the spare header generated based on the applied 4k space when sending the truncate request has been converted into the actual header of the target file corresponding to each Client, and the original header of the target file has been added to the release queue by the MDS, and the content in the header will be gradually released in the background. Thus, the processing of the request to truncate to 0 is completed, realizing the replacement of the header. There is no data block in the new header, and the original header is added to the release queue, and the MDS releases the data block space in the background. Similarly, in the case of adjusting the file header using the second target adjustment mechanism, in the subsequent process, the MDS will pass the updated header of the target file to each Client through the message MClientCap in the truncate operation. After receiving this message, the Client will update the file size, time, and header according to the content passed in the message. Thus, the processing of the request to truncate to the specified size is completed.

[0138] In the above embodiment, the file truncate operation is completed by writing back the header. For the truncate file request of the specified size, a new journal is recorded to ensure the integrity of the file data block in case of a failure, while improving the processing speed of the truncate request and ensuring the data consistency of the file.

[0139] S5: Release the corresponding storage space according to the adjustment result.

[0140] It should be noted that the storage space is the storage space corresponding to the header and / or data block in the release queue.

[0141] In some specific embodiments, releasing the corresponding storage space according to the adjustment result includes:

[0142] Obtain the release queue;

[0143] Determine the storage space occupied by the data block and / or the first file header in the release queue, and release the storage space, where the storage space is released by the MDS background.

[0144] In the above embodiments, the MDS background releases the corresponding occupied space, thereby avoiding blocking the current truncate request and improving the response speed of the truncate request.

[0145] In some specific embodiments, the method further includes:

[0146] Obtain the sizes of the storage spaces to be released corresponding to the target release queue at multiple time nodes within multiple time periods, where the time periods can be set according to actual needs, such as one week, etc.;

[0147] In response to the size of the storage space to be released being greater than a first preset threshold, mark the time nodes corresponding to the size of the storage space to be released, where the first preset threshold can be set according to actual needs;

[0148] Obtain the number of times the target time node is marked within multiple time periods, and determine the ratio of the number of times marked to the number of time periods;

[0149] In response to the ratio being greater than a second preset threshold, determine the number of threads of the target release queue, including: calculating the quotient between the size of the storage space to be released corresponding to the target release queue and a third preset threshold. If the mantissa of the quotient is greater than 0.5, round up the quotient; if the mantissa of the quotient is less than or equal to 0.5, round down the quotient to generate the number of threads, where the second preset threshold and the third preset threshold can be set according to actual needs;

[0150] In response to the time node within the current time period being marked, based on the corresponding number of threads, start multiple threads to simultaneously release the storage space to be released. If the mantissa of the quotient is less than or equal to 0.5, allocate the tasks corresponding to the extra storage space to be released to any thread for processing.

[0151] In the above embodiments, when the storage space to be released is large, the tasks corresponding to the storage space to be released are processed simultaneously by multiple threads, improving the release efficiency of the storage space and further reducing resource occupation.

[0152] In the above data pool management method, the method includes: when receiving a first file truncation request corresponding to a target file, sending a first space application request to a first metadata server, where the file truncation request includes a truncated file size, and the space application request includes an applied space size; receiving a first target space allocated by the first metadata server according to the first space application request, and defining the first target space as a first file header; adding the first file header to the first file truncation request to generate a second file truncation request, and sending the second file truncation request to a second metadata server; receiving an adjustment result obtained by the second metadata server adjusting a second file header of the target file and the content in the second file header according to the second file truncation request, where the file header is bound to the file and is used to store basic information of the file; according to the adjustment result, releasing the corresponding storage space. In the case of only applying the file storage scenario in this application, the OSD and RADOS layers in the Ceph architecture are removed, avoiding implementing the file truncate action through the OSD, and implementing the file truncate action by using the MDS (metadata server) to manage the data pool. This not only reduces the code architecture and resource occupancy, but also simplifies the overall process. The message thread occupancy time of the MDS is shorter, making the truncate process execute faster and improving the processing efficiency of the truncate request. In addition, by using the MDS to process the entire truncate request, data consistency can be guaranteed, and the reliability of the distributed storage system is improved.

[0153] It should be understood that although Figures 1-3 the steps in the flowchart of Figures 1-3 are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover,

[0154] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as Figure 4As shown in the figure. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it realizes a data pool management method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad set on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0155] Those skilled in the art can understand that Figure 4 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0156] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0157] S1: When receiving a first file truncation request corresponding to a target file, send a first space application request to a first metadata server. Among them, the file truncation request includes the truncated file size, and the space application request includes the requested space size;

[0158] S2: Receive the first target space allocated by the first metadata server according to the first space application request, and define the first target space as the first file header;

[0159] S3: Add the first file header to the first file truncation request to generate a second file truncation request, and send the second file truncation request to a second metadata server;

[0160] S4: Receive the adjustment result obtained by the second metadata server according to the second file truncation request for adjusting the second file header of the target file and the content in the second file header. Among them, the file header is bound to the file and is used to store the basic information of the file;

[0161] S5: Release the corresponding storage space according to the adjustment result.

[0162] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0163] Receive a target file creation request;

[0164] When receiving the target file creation request, send a second space application request to the first metadata server;

[0165] Receive the second target space allocated by the first metadata server according to the second space application request, and define the second target space as the second file header;

[0166] Save the basic information of the target file into the second file header, bind the second file header storing the basic information of the target file to the target file, and save them to the distributed storage system client and the second metadata server respectively.

[0167] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0168] Based on the server, send a first file truncation request corresponding to the target file to the distributed storage system client.

[0169] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0170] In response to the completion of the allocation of the first target space, return a first space application request to the distributed storage system client based on the first metadata server;

[0171] When the distributed storage system client receives the first space application request returned by the first metadata server, define the first target space as the first file header.

[0172] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0173] Empty the content stored in the second file header of the distributed storage system client.

[0174] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0175] Obtain the truncated file size and the first file header in the second file truncation request;

[0176] Based on the truncated file size, determine the target adjustment mechanism, and adjust the second file header of the target file and the content in the second file header based on the target adjustment mechanism and the first file header.

[0177] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0178] In response to the truncated file size being the target value, determine that the target adjustment mechanism is the first target adjustment mechanism;

[0179] In response to the truncated file size not being the target value, determine that the target adjustment mechanism is the second target adjustment mechanism.

[0180] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0181] Use the first file header to replace the second file header of the target file as the current file header of the target file, add the data block corresponding to the second file header to the release queue, and at the same time, return a second file truncation request to the distributed storage system client.

[0182] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0183] Based on the background thread, read the data block corresponding to the second file header and write the data block corresponding to the second file header to the object storage into the metadata pool;

[0184] In response to writing the object being completed, generate a record log, and the record log includes the object name.

[0185] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0186] Based on the background thread, read the data block corresponding to the second file header;

[0187] According to the truncated file size, divide the data block corresponding to the second file header into data blocks that need to be written back and data blocks that need to be released;

[0188] Write the data blocks that need to be written back to the second file header, add the data blocks that need to be released to the release queue, and at the same time, return a second file truncation request to the distributed storage system client.

[0189] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0190] Perform fault recovery on the second metadata server;

[0191] In response to the fault of the second metadata server being recovered, read the target record log;

[0192] According to the object name in the target record log, read the data block corresponding to the object name and rewrite the data block corresponding to the object name back to the second file header.

[0193] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0194] Detect whether there is a need to write back data blocks based on the first file header;

[0195] In response to the non-existence of the need to write back data blocks based on the first file header, add the first file header to the release queue.

[0196] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0197] In response to the distributed storage system client receiving the second file truncation request returned by the second metadata server, determine that the second file truncation request has been processed;

[0198] In response to the completion of the processing of the second file truncation request, send the adjustment result of the second file header to multiple distributed storage system clients based on the second metadata server;

[0199] When multiple distributed storage system clients receive the adjustment result of the second file header, synchronously adjust the second file header and the content in the second file header saved in the corresponding distributed storage system clients.

[0200] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0201] Obtain the release queue;

[0202] Determine the storage space occupied by the data blocks and / or the first file header in the release queue, and release the storage space.

[0203] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0204] S1: When receiving the first file truncation request corresponding to the target file, send a first space application request to the first metadata server, where the file truncation request includes the truncated file size, and the space application request includes the requested space size;

[0205] S2: Receive the first target space allocated by the first metadata server according to the first space application request, and define the first target space as the first file header;

[0206] S3: Add the first file header to the first file truncation request to generate a second file truncation request, and send the second file truncation request to the second metadata server;

[0207] S4: Receive the adjustment result obtained by the second metadata server adjusting the second file header and the content in the second file header of the target file according to the second file truncation request, where the file header is bound to the file and is used to store the basic information of the file;

[0208] S5: Release the corresponding storage space according to the adjustment result.

[0209] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0210] Receive a target file creation request;

[0211] When receiving the target file creation request, send a second space application request to the first metadata server;

[0212] Receive the second target space allocated by the first metadata server according to the second space application request, and define the second target space as the second file header;

[0213] Save the basic information of the target file into the second file header, bind the second file header storing the basic information of the target file to the target file, and save them to the distributed storage system client and the second metadata server respectively.

[0214] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0215] Based on the server, send a first file truncation request corresponding to the target file to the distributed storage system client.

[0216] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0217] In response to the completion of the allocation of the first target space, return a first space application request to the distributed storage system client based on the first metadata server;

[0218] When the distributed storage system client receives the first space application request returned by the first metadata server, define the first target space as the first file header.

[0219] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0220] Empty the content stored in the second file header of the distributed storage system client.

[0221] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0222] Obtain the truncated file size and the first file header in the second file truncation request;

[0223] Based on the truncated file size, determine the target adjustment mechanism, and based on the target adjustment mechanism and the first file header, adjust the second file header of the target file and the content in the second file header.

[0224] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0225] In response to the truncated file size being the target value, determine that the target adjustment mechanism is the first target adjustment mechanism;

[0226] In response to the truncated file size not being the target value, determine that the target adjustment mechanism is the second target adjustment mechanism.

[0227] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0228] Use the first file header to replace the second file header of the target file as the current file header of the target file, add the data block corresponding to the second file header to the release queue, and at the same time, return a second file truncation request to the distributed storage system client.

[0229] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0230] Based on the background thread, read the data block corresponding to the second file header and write the data block corresponding to the second file header to the object storage in the metadata pool;

[0231] In response to writing the object being completed, generate a record log, and the record log includes the object name.

[0232] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0233] Based on the background thread, read the data block corresponding to the second file header;

[0234] According to the truncated file size, divide the data block corresponding to the second file header into data blocks that need to be written back and data blocks that need to be released;

[0235] Write back the data blocks that need to be written back to the second file header, add the data blocks that need to be released to the release queue, and at the same time, return a second file truncation request to the distributed storage system client.

[0236] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0237] Perform a fault recovery on the second metadata server;

[0238] In response to the fault of the second metadata server being recovered, read the target record log;

[0239] According to the object name in the target record log, read the data block corresponding to the object name and rewrite the data block corresponding to the object name back to the second file header.

[0240] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0241] Detect whether there is a need to write back data blocks based on the first file header;

[0242] In response to the non-existence of the need to write back data blocks based on the first file header, add the first file header to the release queue.

[0243] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0244] In response to the distributed storage system client receiving the second file truncation request returned by the second metadata server, determine that the second file truncation request has been processed;

[0245] In response to the completion of the processing of the second file truncation request, send the adjustment result of the second file header to multiple distributed storage system clients based on the second metadata server;

[0246] When multiple distributed storage system clients receive the adjustment result of the second file header, synchronously adjust the second file header and the content in the second file header saved in the corresponding distributed storage system clients.

[0247] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0248] Obtain the release queue;

[0249] Determine the storage space occupied by the data blocks and / or the first file header in the release queue, and release the storage space.

[0250] In one embodiment, a computer program product is provided, the computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0251] S1: When receiving a first file truncation request corresponding to a target file, send a first space application request to the first metadata server, where the file truncation request includes the truncated file size, and the space application request includes the requested space size;

[0252] S2: Receive the first target space allocated by the first metadata server according to the first space application request, and define the first target space as the first file header;

[0253] S3: Add the first file header to the first file truncation request to generate a second file truncation request, and send the second file truncation request to the second metadata server;

[0254] S4: Receive the adjustment result obtained by the second metadata server by adjusting the second file header of the target file and the content in the second file header according to the second file truncation request, where the file header is bound to the file and is used to store the basic information of the file;

[0255] S5: Release the corresponding storage space according to the adjustment result.

[0256] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0257] Receive a target file creation request;

[0258] When receiving the target file creation request, send a second space application request to the first metadata server;

[0259] Receive the second target space allocated by the first metadata server according to the second space application request, and define the second target space as the second file header;

[0260] Save the basic information of the target file to the second file header, bind the second file header storing the basic information of the target file to the target file, and save them to the distributed storage system client and the second metadata server respectively.

[0261] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0262] Based on the server, send a first file truncation request corresponding to the target file to the distributed storage system client.

[0263] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0264] In response to the completion of the allocation of the first target space, return the first space application request to the distributed storage system client based on the first metadata server;

[0265] When the distributed storage system client receives the first space application request returned by the first metadata server, define the first target space as the first file header.

[0266] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0267] Empty the content stored in the second file header of the distributed storage system client.

[0268] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0269] Obtain the truncated file size and the first file header in the second file truncation request;

[0270] Determine a target adjustment mechanism based on the truncated file size, and adjust the second file header of the target file and the content in the second file header based on the target adjustment mechanism and the first file header.

[0271] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0272] In response to the truncated file size being the target value, determine that the target adjustment mechanism is the first target adjustment mechanism;

[0273] In response to the truncated file size not being the target value, determine that the target adjustment mechanism is the second target adjustment mechanism.

[0274] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0275] Use the first file header to replace the second file header of the target file as the current file header of the target file, add the data block corresponding to the second file header to the release queue, and at the same time, return a second file truncation request to the distributed storage system client.

[0276] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0277] Based on a background thread, read the data block corresponding to the second file header, and write the data block corresponding to the second file header to the object storage into the metadata pool;

[0278] In response to completing the writing of the object, generate a record log, and the record log includes the object name.

[0279] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0280] Based on a background thread, read the data block corresponding to the second file header;

[0281] According to the truncated file size, divide the data block corresponding to the second file header into data blocks that need to be written back and data blocks that need to be released;

[0282] Write back the data blocks that need to be written back to the second file header, add the data blocks that need to be released to the release queue, and at the same time, return a second file truncation request to the distributed storage system client.

[0283] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0284] Perform a failure recovery on the second metadata server;

[0285] In response to the failure of the second metadata server being recovered, read the target record log;

[0286] Read the data block corresponding to the object name from the target record log, and rewrite the data block corresponding to the object name back to the second file header.

[0287] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0288] Detect whether there is a need to save the data block to be rewritten based on the first file header;

[0289] In response to the non-existence of the need to save the data block to be rewritten based on the first file header, add the first file header to the release queue.

[0290] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0291] In response to the distributed storage system client receiving the second file truncation request returned by the second metadata server, determine that the second file truncation request has been processed;

[0292] In response to the completion of the processing of the second file truncation request, send the adjustment result of the second file header to multiple distributed storage system clients based on the second metadata server;

[0293] When multiple distributed storage system clients receive the adjustment result of the second file header, synchronously adjust the second file header and the content in the second file header saved in the corresponding distributed storage system clients.

[0294] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0295] Obtain the release queue;

[0296] Determine the storage space occupied by the data block and / or the first file header in the release queue, and release the storage space.

[0297] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0298] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0299] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application.

Claims

1. A data pool management method, characterized in that, The method includes: When receiving a first file truncation request corresponding to a target file, sending a first space application request to a first metadata server, where the file truncation request includes a truncated file size, and the space application request includes an applied space size; Receiving a first target space allocated by the first metadata server according to the first space application request, and defining the first target space as a first file header; Adding the first file header to the first file truncation request to generate a second file truncation request, and sending the second file truncation request to a second metadata server; Receiving an adjustment result obtained by the second metadata server adjusting a second file header of the target file and the content in the second file header according to the second file truncation request, where the file header is bound to the file and is used to store basic information of the file, the second file header is a second target space allocated by the first metadata server according to a second space application request, the basic information is used to describe attribute information of the file, and the second space application request is a request sent to the first metadata server when receiving a target file creation request; Releasing the corresponding storage space according to the adjustment result.

2. The data pool management method according to claim 1, wherein Before receiving the first file truncation request corresponding to the target file, the method further includes: Receiving a target file creation request; When receiving the target file creation request, sending a second space application request to the first metadata server; Receiving a second target space allocated by the first metadata server according to the second space application request, and defining the second target space as a second file header; Saving the basic information of the target file to the second file header, binding the second file header storing the basic information of the target file to the target file, and saving them to a distributed storage system client and the second metadata server respectively.

3. The data pool management method according to claim 1, characterized in that Before receiving the first file truncation request corresponding to the target file, the method further includes: Based on the server, sending the first file truncation request corresponding to the target file to the distributed storage system client.

4. The data pool management method according to claim 1, wherein After the first metadata server allocates a first target space according to the first space application request, the method further includes: In response to the completion of the allocation of the first target space, returning the first space application request to the distributed storage system client based on the first metadata server; When receiving the first space application request returned by the first metadata server, the distributed storage system client defines the first target space as a first file header.

5. The data pool management method according to claim 2 or 4, characterized in that After defining the first target space as a first file header, the method further includes: Clearing the content stored in the second file header of the distributed storage system client.

6. The data pool management method according to claim 1, characterized in that Before adjusting the second file header of the target file and the content in the second file header according to the second file truncation request, the method further includes: Obtaining the truncated file size and the first file header in the second file truncation request; Based on the truncated file size, determine a target adjustment mechanism to adjust the second file header of the target file and the content in the second file header based on the target adjustment mechanism and the first file header.

7. The data pool management method according to claim 6, wherein Determining a target adjustment mechanism based on the truncated file size includes: In response to the truncated file size being a target value, determine that the target adjustment mechanism is a first target adjustment mechanism; In response to the truncated file size not being the target value, determine that the target adjustment mechanism is a second target adjustment mechanism.

8. The data pool management method according to claim 7, wherein Adjusting the second file header of the target file and the content in the second file header based on the first target adjustment mechanism and the first file header includes: Use the first file header to replace the second file header of the target file as the current file header of the target file, add the data block corresponding to the second file header to the release queue, and at the same time, return the second file truncation request to the distributed storage system client.

9. The data pool management method according to claim 7, wherein Before adjusting the second file header of the target file and the content in the second file header based on the second target adjustment mechanism and the first file header, the method further includes: Based on a background thread, read the data block corresponding to the second file header and write the data block corresponding to the second file header to the object storage metadata pool; In response to writing the object being completed, generate a record log, and the record log includes the object name.

10. The data pool management method according to claim 7, wherein, Adjusting the second file header of the target file and the content in the second file header based on the second target adjustment mechanism and the first file header includes: Based on a background thread, read the data block corresponding to the second file header; According to the truncated file size, divide the data block corresponding to the second file header into data blocks to be written back and data blocks to be released; Write back the data blocks to be written back to the second file header, add the data blocks to be released to the release queue, and at the same time, return the second file truncation request to the distributed storage system client.

11. The data pool management method according to claim 10, characterized in that, In response to the second metadata server failing during the process of writing back data blocks, the method further includes: Perform fault recovery on the second metadata server; In response to the second metadata server's fault being recovered, read the target record log; According to the object name in the target record log, read the data block corresponding to the object name and rewrite the data block corresponding to the object name back to the second file header.

12. The data pool management method according to claim 10, wherein The method further includes: Detect whether there is a need to save the data blocks to be written back based on the first file header; In response to there being no need to save the data blocks to be written back based on the first file header, add the first file header to the release queue.

13. The data pool management method according to claim 8 or 10, characterized in that, The method further includes: In response to the distributed storage system client receiving the second metadata server returning the second file truncation request, determine that the second file truncation request has been processed; In response to the second file truncation request having been processed, send the adjustment result of the second file header to multiple distributed storage system clients based on the second metadata server. When the multiple distributed storage system clients receive the adjustment result of the second file header, they synchronously adjust the second file header and the content in the second file header saved in the corresponding distributed storage system clients.

14. The data pool management method according to claim 8 or 10 or 12, characterized in that Releasing the corresponding storage space according to the adjustment result includes: Obtaining the release queue; Determining the storage space occupied by the data blocks and / or the first file header in the release queue, and releasing the storage space.

15. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 14.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method according to any one of claims 1 to 14.

17. A computer program product, characterized in that, The computer program product includes a computer program, and when the computer program is executed by the processor, it implements the method according to any one of claims 1 to 14.

Images