Garbage collection method, distributed storage system and computer readable storage medium

Abstract

The invention provides a garbage collection method, a distributed storage system and a computer readable storage medium. The distributed storage system comprises a server and a client, the server accesses a storage device comprising a plurality of erasure units, the client comprises first maintenance information used for indicating a plurality of data units and the erasure units to which the data units belong, and the server comprises second maintenance information used for indicating the erasure units; the method comprises the following steps: a client determines a first data deletion request comprising an identifier of at least one to-be-deleted data unit based on first maintenance information, and sends the first data deletion request; the server adds an identifier of at least one to-be-deleted data unit in the received first data deletion request into a data unit identifier set; and based on the second maintenance information and the data unit identifier set, indicating the storage device to recycle the plurality of to-be-deleted data units in the target erasure unit. The data units in the erasing unit are recycled in a centralized mode, write amplification is reduced, and the recycling efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and in particular to a garbage collection method, a distributed storage system, and a computer-readable storage medium. Background Art

[0002] A distributed storage system can divide all data into fixed sizes, and each part can be called a data unit. In the process of distributed block storage, the data in the data unit will be continuously updated. After the data is updated, the old version of the data becomes garbage data. In the process of deleting the data unit by the client of the distributed storage system, the hard disk performs garbage collection (GC) on the data unit to be deleted, and the hard disk performs GC relocation according to a certain granularity. GC relocation requires GC relocation of valid data at the granularity of the recycle, and deletes invalid data. However, the amount of garbage in the data unit maintained by the client is not the amount of garbage at the granularity of the hard disk recycle, resulting in that in the process of GC, in addition to GC relocation, the hard disk may also contain other data that needs to be relocated, so that the GC process inside the hard disk will cause write amplification (the data actually written by the hard disk is multiple times the data written by the client). The write amplification coefficient is defined as the data written by the medium / the data written by the client. Assuming that the write amplification coefficient is 2.5, one possible situation corresponding to it is: the client writes 1MB of data, and the SSD writes 2.5MB of data to the SSD medium. The internal writing of 2.5MB of data to the SSD includes: writing 1MB of client data; 1.5MB of GC relocation data; and additional media reads to enable the client to write 1.5MB of data.

[0003] However, write amplification will affect the performance of the distributed storage system and reduce the life of the hard disk. Therefore, a garbage collection method that can reduce write amplification is urgently needed. Summary of the invention

[0004] The embodiments of the present application provide a garbage collection method, a distributed storage system, and a computer-readable storage medium. The hard disk exposes the information of the erase unit to the client. Through the cooperation between the client and the server, the server can centrally recycle multiple data units to be deleted in the erase unit, thereby reducing the data that needs to be additionally relocated, reducing write amplification, reducing the time spent on GC, and improving GC efficiency.

[0005] In a first aspect, an embodiment of the present application provides a garbage collection method, which is applied to a distributed storage system, the distributed storage system includes a client and a server, the server accesses a storage device, the storage device includes multiple erasure units, the client includes first maintenance information, the first maintenance information is used to indicate multiple data units and the erasure units to which the multiple data units belong, the server includes second maintenance information, the second maintenance information is used to indicate multiple erasure units, and the method includes:

[0006] The client determines at least one data unit to be deleted from multiple data units based on first maintenance information; the client sends a first data deletion request, which includes an identifier of at least one data unit to be deleted; the server receives the first data deletion request, and adds the identifier of at least one data unit to be deleted to a data unit identifier set; the server instructs the storage device to recycle multiple data units to be deleted in the target erase unit based on second maintenance information and the data unit identifier set.

[0007] In this solution, the hard disk exposes the information of the erase unit to the client. Through the cooperation between the client and the server, the server can centrally recycle multiple data units to be deleted in the erase unit, thereby reducing the data that needs to be relocated additionally, reducing write amplification, reducing the time spent on GC, and improving GC efficiency.

[0008] In a possible implementation, the data unit identifier set also includes an identifier of at least one data unit to be deleted included in the second data deletion request sent by the client. It should be noted that the second data deletion request indicates the data unit to be deleted determined last time in the first data deletion request.

[0009] In this solution, the server selects multiple data units to be deleted located in the same erase unit from the multiple data units to be deleted indicated by the data deletion requests sent to the client multiple times, and collects them collectively, thereby reducing the amount of data processed by the hard disk during the GC process to a certain extent, reducing the time spent on GC, and improving GC efficiency.

[0010] In one possible implementation, the first maintenance information includes one or more of the following information: identification of multiple data units to be deleted, the amount of garbage in the multiple data units to be deleted, identification of erasure units to which the multiple data units to be deleted belong, or the amount of garbage in the erasure units to which the multiple data units to be deleted belong.

[0011] In a possible implementation manner, the second maintenance information includes one or more of the following information:

[0012] Identifications of multiple erase units, amounts of garbage in the multiple erase units, and identifications of data units included in the multiple erase units.

[0013] In an example of this implementation, the client determines at least one data unit to be deleted from multiple data units based on the first maintenance information, including: the client determines at least one data unit to be deleted from multiple data units with the largest amount of garbage based on the amount of garbage in the multiple data units.

[0014] In an example of this implementation method, the client determines at least one data unit to be deleted from multiple data units based on the first maintenance information, including: the client determines at least one data unit to be deleted from multiple data units located in the same erasure unit based on the identifier of the erasure unit to which the multiple data units belong.

[0015] In an example of this implementation, the client determines at least one data unit to be deleted from multiple data units based on the first maintenance information, including: the client determines at least one data unit to be deleted from the erasure unit with the largest amount of garbage based on the amount of garbage in the erasure units to which the multiple data units belong.

[0016] In one possible implementation, the server instructs the storage device to recycle multiple data units to be deleted in the target erasure unit based on the second maintenance information and the data unit identification set, including: the server instructs the storage device to recycle multiple data units to be deleted in the target erasure unit having the largest number of data units to be deleted among the multiple erasure units based on the identifications of the data units included in the multiple erasure units and the data unit identification set.

[0017] Exemplarily, the identifier of each data unit in the target erasure unit is located in the data unit identifier set.

[0018] In this solution, the server selects the erasure unit containing the most data units to be deleted for erasure, thereby reducing the amount of data relocated during the hard disk GC process to a certain extent.

[0019] In one possible implementation, the server instructs the storage device to recycle multiple data units to be deleted in a target erase unit based on the second maintenance information and a set of data unit identifiers, including: the server instructs the storage device to recycle multiple data units to be deleted in a target erase unit having the largest amount of garbage among the multiple erase units based on the amount of garbage in the multiple erase units and a set of data unit identifiers.

[0020] Exemplarily, the identifier of each data unit in the target erasure unit is located in the data unit identifier set.

[0021] In this solution, the server selects the erasure unit with the largest amount of garbage for erasure, thereby reducing the amount of data moved during the hard disk GC process to a certain extent.

[0022] In a second aspect, an embodiment of the present application provides a garbage collection method, which is applied to a client of a distributed storage system, wherein the client accesses a storage device through a server of the distributed storage system, wherein the storage device includes multiple erasure units, and the client includes first maintenance information, wherein the first maintenance information is used to indicate multiple data units and the erasure units to which the multiple data units belong, and the server includes second maintenance information, wherein the second maintenance information is used to indicate multiple erasure units, and the method includes:

[0023] Based on the first maintenance information, at least one data unit to be deleted is determined from multiple data units; a first data deletion request is sent to a server, the first data deletion request includes an identifier of at least one data unit to be deleted, so that the server adds the identifier of the at least one data unit to be deleted to a data unit identifier set; based on the second maintenance information and the data unit identifier set, a storage device is instructed to recycle multiple data units to be deleted in a target erase unit.

[0024] In this solution, the hard disk exposes the information of the erase unit to the client, and the client considers the garbage amount of the erase unit and the data unit to determine the data unit to be deleted; the server adds the identifier of the data unit to be deleted to the data unit identifier set, and centrally recycles multiple data units in the erase unit, thereby reducing the data that needs to be additionally relocated, reducing write amplification, reducing the time spent on GC, and improving GC efficiency.

[0025] In a possible implementation, the data unit identifier set also includes an identifier of at least one data unit to be deleted included in the second data deletion request sent by the client. It should be noted that the second data deletion request indicates the data unit to be deleted determined last time in the first data deletion request.

[0026] In this solution, the server selects multiple data units to be deleted in the same erasing unit from among the multiple data units to be deleted indicated by the data deletion requests sent multiple times to the client for centralized recycling, thereby reducing the amount of data processed by the hard disk during the GC process to a certain extent, reducing the time spent on GC, and improving GC efficiency.

[0027] In one possible implementation, the first maintenance information includes one or more of the following information: identification of multiple data units to be deleted, the amount of garbage in the multiple data units to be deleted, identification of erasure units to which the multiple data units to be deleted belong, or the amount of garbage in the erasure units to which the multiple data units to be deleted belong.

[0028] In an example of this implementation, the client determines at least one data unit to be deleted from multiple data units based on the first maintenance information, including: the client determines at least one data unit to be deleted from multiple data units with the largest amount of garbage based on the amount of garbage in the multiple data units.

[0029] In an example of this implementation method, the client determines at least one data unit to be deleted from multiple data units based on the first maintenance information, including: the client determines at least one data unit to be deleted from multiple data units located in the same erasure unit based on the identifier of the erasure unit to which the multiple data units belong.

[0030] In an example of this implementation, the client determines at least one data unit to be deleted from multiple data units based on the first maintenance information, including: the client determines at least one data unit to be deleted from the erasure unit with the largest amount of garbage based on the amount of garbage in the erasure units to which the multiple data units belong.

[0031] In a third aspect, an embodiment of the present application provides a garbage collection method, which is applied to a server of a distributed storage system, the server accesses a storage device, the storage device includes multiple erase units, the distributed storage system also includes a client, the client includes first maintenance information, the first maintenance information is used to indicate multiple data units and the erase units to which the multiple data units belong, the server includes second maintenance information, the second maintenance information is used to indicate multiple erase units, and the method includes:

[0032] Receive a first data deletion request sent by a client, add the identifier of at least one data unit to be deleted to a data unit identifier set; based on the second maintenance information and the data unit identifier set, instruct the storage device to recycle multiple data units to be deleted in the target erasure unit.

[0033] In this solution, the server adds the identifiers of several data units to be deleted sent by the client to the data unit identifier set, so that based on the data unit identifier set and the data units in the erase unit, a target erase unit containing multiple data units to be deleted can be selected for garbage collection.

[0034] In a possible implementation, the data unit identifier set also includes an identifier of at least one data unit to be deleted included in the second data deletion request sent by the client. It should be noted that the second data deletion request indicates the data unit to be deleted determined last time in the first data deletion request.

[0035] In this solution, the server selects multiple data units to be deleted located in the same erase unit from the multiple data units to be deleted indicated by the data deletion requests sent to the client multiple times, and collects them collectively, thereby reducing the amount of data processed by the hard disk during the GC process to a certain extent, reducing the time spent on GC, and improving GC efficiency.

[0036] In a possible implementation manner, the second maintenance information includes one or more of the following information:

[0037] Identifications of multiple erase units, amounts of garbage in the multiple erase units, and identifications of data units included in the multiple erase units.

[0038] In one possible implementation, the server instructs the storage device to recycle multiple data units to be deleted in the target erasure unit based on the second maintenance information and the data unit identification set, including: the server instructs the storage device to recycle multiple data units to be deleted in the target erasure unit having the largest number of data units to be deleted among the multiple erasure units based on the identifications of the data units included in the multiple erasure units and the data unit identification set.

[0039] Exemplarily, the identifier of each data unit in the target erasure unit is located in the data unit identifier set.

[0040] In this solution, the server selects the erasure unit containing the most data units to be deleted for erasure, thereby reducing the amount of data relocated during the hard disk GC process to a certain extent.

[0041] In one possible implementation, the server instructs the storage device to recycle multiple data units to be deleted in a target erase unit based on the second maintenance information and a set of data unit identifiers, including: the server instructs the storage device to recycle multiple data units to be deleted in a target erase unit having the largest amount of garbage among the multiple erase units based on the amount of garbage in the multiple erase units and a set of data unit identifiers.

[0042] Exemplarily, the identifier of each data unit in the target erasure unit is located in the data unit identifier set.

[0043] In this solution, the server selects the erasure unit with the largest amount of garbage for erasure, thereby reducing the amount of data moved during the hard disk GC process to a certain extent.

[0044] In a fourth aspect, an embodiment of the present application provides a distributed storage system, which may include a client and a server; the server accesses a storage device, the storage device includes multiple erasure units, the client includes first maintenance information, the first maintenance information is used to indicate multiple data units and the erasure units to which the multiple data units belong, and the server includes second maintenance information, the second maintenance information is used to indicate the multiple erasure units;

[0045] The client is configured to determine at least one data unit to be deleted from a plurality of data units based on the first maintenance information;

[0046] The client is used to send a first data deletion request, where the first data deletion request includes an identifier of at least one data unit to be deleted;

[0047] The server is configured to receive a first data deletion request and add an identifier of at least one data unit to be deleted to a data unit identifier set;

[0048] The service end is used to instruct the storage device to recycle multiple data units to be deleted in the target erasure unit based on the second maintenance information and the data unit identification set.

[0049] In this solution, the hard disk exposes the information of the erase unit to the client. Through the cooperation between the client and the server, the server can centrally recycle multiple data units to be deleted in the erase unit, thereby reducing the amount of data processed by the hard disk during the GC process to a certain extent, reducing the time spent on GC, and improving GC efficiency.

[0050] In a possible implementation, the data unit identifier set also includes an identifier of at least one data unit to be deleted included in the second data deletion request sent by the client. It should be noted that the second data deletion request indicates the data unit to be deleted determined last time in the first data deletion request.

[0051] In this solution, the server selects multiple data units to be deleted located in the same erase unit from the multiple data units to be deleted indicated by the data deletion requests sent to the client multiple times, and collects them collectively, thereby reducing the amount of data processed by the hard disk during the GC process to a certain extent, reducing the time spent on GC, and improving GC efficiency.

[0052] In one possible implementation, the first maintenance information includes one or more of the following information: identification of multiple data units to be deleted, the amount of garbage in the multiple data units to be deleted, identification of erasure units to which the multiple data units to be deleted belong, or the amount of garbage in the erasure units to which the multiple data units to be deleted belong.

[0053] In a possible implementation manner, the second maintenance information includes one or more of the following information:

[0054] Identifications of multiple erase units, amounts of garbage in the multiple erase units, and identifications of data units included in the multiple erase units.

[0055] In an example of this implementation, the client is specifically configured to: determine at least one data unit to be deleted from a plurality of data units with the largest amount of garbage based on the amount of garbage in the plurality of data units.

[0056] In an example of this implementation, the client is specifically configured to: determine at least one data unit to be deleted from multiple data units located in the same erasure unit based on identifiers of erasure units to which the multiple data units belong.

[0057] In an example of this implementation, the client is specifically configured to: determine at least one data unit to be deleted from an erasure unit having the largest amount of garbage based on the amount of garbage in the erasure units to which the multiple data units belong.

[0058] In one possible implementation, the server is specifically used to: based on the identifiers of the data units included in the multiple erasure units and the data unit identifier sets, instruct the storage device to recycle multiple data units to be deleted in the target erasure unit with the most data units to be deleted among the multiple erasure units.

[0059] Exemplarily, the identifier of each data unit in the target erasure unit is located in the data unit identifier set.

[0060] In this solution, the server selects the erasure unit containing the most data units to be deleted for erasure, thereby reducing the amount of data relocated during the hard disk GC process to a certain extent.

[0061] In a possible implementation, the server is specifically configured to: based on the garbage amount of multiple erase units and a set of data unit identifiers, instruct the storage device to recycle multiple data units to be deleted in a target erase unit with the largest garbage amount among the multiple erase units.

[0062] Exemplarily, the identifier of each data unit in the target erasure unit is located in the data unit identifier set.

[0063] In this solution, the server selects the erasure unit with the largest amount of garbage for erasure, thereby reducing the amount of data moved during the hard disk GC process to a certain extent.

[0064] In the fifth aspect, an embodiment of the present application provides a garbage collection device, comprising: at least one memory for storing programs; at least one processor for executing the programs stored in the memory, when the program stored in the memory is executed, the processor is used to execute the method provided in the first aspect, execute the method provided in the second aspect, or execute the method provided in the third aspect.

[0065] In a sixth aspect, an embodiment of the present application provides a garbage collection device, characterized in that the device runs computer program instructions to execute the method provided in the first aspect, the method provided in the second aspect, or the method provided in the third aspect. Exemplarily, the device can be a chip or a processor.

[0066] In one example, the device may include a processor, which may be coupled to a memory, read instructions in the memory and execute the method provided in the first aspect, the method provided in the second aspect, or the method provided in the third aspect according to the instructions. The memory may be integrated in a chip or a processor, or may be independent of the chip or the processor.

[0067] In the seventh aspect, an embodiment of the present application provides a computer storage medium, which stores instructions. When the instructions are executed on a computer, the computer executes the method provided in the first aspect, the method provided in the second aspect, or the method provided in the third aspect.

[0068] In an eighth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method provided in the first aspect, the method provided in the second aspect, or the method provided in the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is a system architecture diagram of a distributed storage system provided in an embodiment of the present application;

[0070] Figure 2a This is a schematic diagram of the structure of the distributed storage system provided in the embodiment of the present application. Figure 1 ;

[0071] Figure 2b This is a second structural diagram of the distributed storage system provided in an embodiment of the present application;

[0072] Figure 3a yes Figure 1 The structure of the hard disk provided in Figure 1 ;

[0073] Figure 3b yes Figure 1The structural diagram of the hard disk provided in 2;

[0074] Figure 4 It is a flowchart of a garbage collection method provided in an embodiment of the present application;

[0075] Figure 5 yes Figure 4 Schematic diagram of the process of step 440;

[0076] Figure 6 is a schematic diagram of a scenario of a garbage collection method provided in an embodiment of the present application;

[0077] Figure 7a This is a schematic diagram of a scenario in which a client and a server interact in a garbage collection method provided in an embodiment of the present application;

[0078] Figure 7b It is a schematic diagram of a scenario in which a server controls a hard disk in a garbage collection method provided in an embodiment of the present application;

[0079] Figure 8 is a structural example diagram of a distributed storage system provided in an embodiment of the present application;

[0080] Fig. 9 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0081] Fig.10 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0082] Fig.11 yes Fig.10 A schematic diagram of a cluster of computing devices is provided. DETAILED DESCRIPTION

[0083] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0084] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0085] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple terminals refers to two or more terminals.

[0086] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0087] Hereinafter, some terms used in this embodiment are explained. It should be noted that these explanations are for the convenience of understanding by those skilled in the art and do not constitute a limitation on the scope of protection claimed by the present invention.

[0088] Solid-state drive (Solid-state drive or Solid-state disk, referred to as SSD): is a non-volatile storage device that mainly uses flash memory (NAND Flash) as permanent memory.

[0089] Hard Disk Drive (HDD): A mechanical hard disk is a traditional ordinary hard disk, which is mainly composed of: disk, head, disk shaft and control motor, head controller, data converter, interface, cache, etc. The head can move along the radius of the disk, and with the high-speed rotation of the disk at several thousand revolutions per minute, the head can be positioned at the specified position of the disk to read and write data.

[0090] Logical Block Address (LBA): It is a common mechanism for describing the block where data is located on a computer storage device, generally used in auxiliary memory devices such as hard disks. LBA can refer to the address of a data block or the data block pointed to by an address.

[0091] Data unit: It is the smallest unit for data storage by the client of the distributed storage system.

[0092] Reclaim Unit (RU): is the minimum granularity of recycled data.

[0093] Block: The smallest unit of erasing and writing of SSD disk. Each Block can include several Pages. Page is the smallest data writing unit of SSD.

[0094] Garbage Collection (GC): Delete old data while retaining the latest data. GC can be a basic function of a hard disk. GC will operate on the EU in the hard disk to recycle space. Currently, GC is a basic function of SSD. The block in the SSD is used as the erasure unit. GC will erase the block in the SSD to recycle space.

[0095] Figure 1 FIG. 1 shows an example diagram of the architecture of a distributed storage system used in an embodiment of the present invention. Figure 1 As shown, the distributed storage system includes a client 101 , a server 102 , and a hard disk 103 .

[0096] The client 101 is used to determine a data operation request, which may be a request generated by processing a data access request from an external source, or a request generated by processing a data access request generated within a distributed storage system. Exemplarily, the processing may include metadata management, data deduplication, data compression, data verification, virtualized storage space, and address translation.

[0097] The server 102 is used to receive a data operation request from a client, and based on the data operation request, perform an operation on the hard disk 103. Exemplarily, the operation performed on the hard disk 103 may include reading data from the hard disk 103, writing data to the hard disk 103, or deleting data from the hard disk 103. In a specific implementation, the server 102 may process the data operation request through a local storage engine to perform the operation on the hard disk 103.

[0098] Among them, the hard disk 103 is used to provide storage resources, such as storing data. It can be a disk or other types of storage media, such as a solid state disk (SSD) or a shingled magnetic recording hard disk. It should be noted that the hard disk 103 is only an example of a non-volatile storage device and does not constitute a specific limitation. In practical applications, other non-volatile storage devices can also be used. The embodiment of the present application takes the hard disk 103 as an example to illustrate the solution.

[0099] For the hard disk 103, the hard disk 103 is divided into multiple erase units (EU). It should be noted that EU is the minimum granularity of recycling. Specifically, the hard disk 103 performs an erase operation with EU as the granularity. More specifically, the data in the entire EU will be erased at one time. The hard disk 103 has a variety of data storage modes. Under different data storage modes, EU can have other names, or have different meanings, or different space sizes. In addition, EU is the name of the granularity of recycling in the embodiment of the present application, and does not have a special meaning. It is only for the convenience of description. In some possible implementations, EU can also be called various names such as recycling basic unit, basic recycling unit, etc., and the embodiment of the present application does not make specific limitations on this.

[0100] In one possible way, a data storage method (Flexible Direct Placement, FDP) can be used. FDP exposes some internal information of the hard disk 103 to the client 101, including the reclaim unit (RU) and the reclaim group (RG). In this scenario, EU is one RU. RU is an erased unit of the hard disk 103. For example, when the hard disk 103 is an SSD, one RU is one Block. RG is a performance isolation unit, which includes multiple RUs. Figure 3a As shown, the client 101 carries the RG identifier (called RGID) and the RU identifier (called PID) when writing data to the hard disk 103, so that the data is written into the RU unit in the specific RG through the RUH corresponding to the PID. In addition, the client 101 can also query the internal garbage amount of the RU and RG.

[0101] In another possible way, a multi-stream data storage method can be used. Multi-stream is a hint exposed to the outside by the hard disk 103. Its use method is: divide the life cycle of data, and divide different life cycles into different streams (erase units EU of the hard disk 103), so as to achieve the desired arrangement of data inside the hard disk 103. Figure 3b When deleting data, since each stream is a continuous area, it is possible to invalidate the continuous area inside the hard disk 103. In a multi-stream scenario, the EU can be a stream.

[0102] In one possible way, a data storage method of a zoned namespace (Zoned Namespace, ZNS) can be used. ZNS divides the storage area inside the entire hard disk 103 into multiple zones (erase units EU of the hard disk 103). Each zone manages a segment of LBA and only allows sequential writing. Similar to multi-stream, ZNS also expects applications to divide data into life cycles, and assign different life cycles to different zones. Since each zone is a continuous area, the continuous area inside the hard disk 103 can be invalidated when deleted. In the ZNS scenario, EU can be a zone.

[0103] In the related art, data will be continuously updated during the distributed storage process. After the data is updated, the old version of the data becomes garbage data; distributed storage can use block storage. For block storage, all data are divided into blocks of fixed size, and each block (which can be referred to as a data unit in the embodiment of the present application) is assigned a number for addressing. Taking a mechanical hard disk as an example, a block is a sector, and a sector is 512 bytes or 4K bytes. Some hard disks use a number composed of a cylinder-head-sector number (CylindeHead-Sector, CHS) for addressing, and some hard disks use a logical block number (Logical Block Addressing, LBA) for addressing. Of course, in addition to hard disks, there are other block devices, such as floppy disks of different specifications, optical disks of various specifications, magnetic tapes, etc. In the scenario of block storage using LBA addressing, for client 101, the definition of garbage data is for data units. For example, if data is repeatedly written to an object, the data unit to which the object belongs will accumulate garbage data. For the hard disk 103, the definition of garbage data is for the logical block address (LBA); for example, overwriting of the same logical block address (LBA) and abnormal data units appearing inside will accumulate garbage data.

[0104] In the related art, during the process of deleting a data unit by the client 101 of the distributed storage system, the hard disk 103 performs GC relocation on the data unit to be deleted. GC relocation can be understood as the process of autonomous GC relocation of the hard disk. GC relocation will retain the valid data recognized in the hard disk 103 and delete the junk data. During the GC process, before the hard disk 103 performs GC relocation, it is necessary to write the valid data of the data unit to be deleted. The server 102 of the distributed storage system determines that the data in the EU does not need to be deleted, thereby ensuring data security. Since the client 101 and the hard disk 103 have different definitions of junk data, the data unit maintained by the client is not the EU inside the hard disk; and the GC relocation of the hard disk 103 is erased in units of EU. Since the data unit and the EU are not compatible, the server 102 of the distributed storage system will determine that the EU contains data that does not need to be deleted. Only after these data are migrated can the EU be GCed, thereby ensuring data security. Therefore, the GC process of the hard disk 103 will cause write amplification. The write amplification coefficient is defined as data written by the medium / data written by the client. Assuming the write amplification factor is 2.5, a possible situation is: the client writes 1MB of data, and the SSD writes 2.5MB of data to the SSD media. The SSD's internal writing of 2.5MB of data includes: writing 1MB of client data; 1.5MB of GC relocation data; and additional media reads that allow the client to write 1.5MB of data.

[0105] In the related art, in order to reduce write amplification, it is considered that the amount of valid data of the data unit to be deleted cannot be adjusted and controlled, but the amount of data that needs to be relocated in the EU can be adjusted and controlled; for example, a multi-stream data storage method can be adopted. In the multi-stream storage method, when the client 101 deletes the data unit, it deletes it based on the granularity of the life cycle. Since the data of one life cycle is stored in one stream, the stream in the hard disk 103 is composed of continuous areas, and the data is sequentially written into the continuous areas in the stream. In this way, the EU of the hard disk 103 and the data unit that the client needs to delete are adapted; therefore, before the GC relocation of the data unit to be deleted in the hard disk 103, after the valid data of the data unit to be deleted is written, no additional media reading is required, thereby reducing the amount of data relocated in the hard disk 103 during the GC process and reducing write amplification. The principle of reducing write amplification by the ZNS data storage method is similar.

[0106] However, the division of the life cycle is very complex, and the embodiments of the present application provide other garbage collection methods that can reduce write amplification.

[0107] In actual applications, users access the distributed storage system through the client 101. In some possible scenarios, users access the distributed storage system through terminal devices. In other possible scenarios, the distributed storage system is deployed in the cloud, and users access the cloud management platform through terminal devices to access the client 101, thereby accessing the distributed storage system. Among them, the terminal device can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. Exemplary embodiments of the terminal devices involved in the embodiments of the present invention include but are not limited to electronic devices equipped with iOS, Android, Windows, Harmony OS or other operating systems. The embodiments of the present invention do not specifically limit the type of electronic device. Among them, the cloud management platform can be a separate electronic device or a device integrated in an electronic device. The embodiments of the present invention do not limit the specific deployment method and deployment location of the cloud management platform.

[0108] In a feasible scenario, the client 101, the server 102 and the hard disk 103 are deployed in an integrated manner. In this scenario, the distributed storage system 100 includes an application server cluster. Figure 2a As shown, the application server cluster includes one or more application servers 110 ( Figure 1 Two application servers 110 are shown in the figure, but not limited to two application servers 110), and each application server 110 can communicate with each other. The application server 110 is a device that has both computing and storage capabilities, such as a server, a desktop computer, etc. In terms of software, each application server 110 has an operating system. A virtual machine 104 can be created on the application server 110, and the computing resources required by the virtual machine 104 come from the local processor 111 and memory 112 of the application server 110, while the storage resources required by the virtual machine 104 can come from the local hard disk 103 of the application server 110 or from the hard disk 103 in other application servers 110. In addition, various application programs can be run in the virtual machine 104, and the user can trigger a read / write data request through the application program in the virtual machine 104. In one example, the virtual machine 104 can serve as the client 101.

[0109] In hardware, such as Figure 2aAs shown, the application server 110 includes at least a processor 111, a memory 112, a network card 113 and a hard disk 103. The processor 111, the memory 112, the network card 113 and the hard disk 103 are connected via a bus. Among them, the processor 111 and the memory 112 are used to provide computing resources. Specifically, the processor 111 is a central processing unit (CPU) for processing data access requests from outside the application server 110, or requests generated inside the application server 110. Exemplarily, when the processor 111 receives a write data request sent by a user, the data in these write data requests will be temporarily saved in the memory 112. When the total amount of data in the memory 112 reaches a certain threshold, the processor 111 sends the data stored in the memory 112 to the hard disk 103 for persistent storage. In addition, the processor 111 is also used for data calculation or processing, such as metadata management, deduplication, data compression, data verification, virtualized storage space and address conversion. Figure 2a Only one CPU 111 is shown in the figure. In actual applications, there are often multiple CPUs 111, wherein one CPU 111 has one or more CPU cores. This embodiment does not limit the number of CPUs and the number of CPU cores.

[0110] The memory 112 refers to an internal memory that directly exchanges data with the processor. It can read and write data at any time and at a very fast speed. It serves as a temporary data storage for the operating system or other running programs. The memory includes at least two types of memories. For example, the memory can be either a random access memory or a read-only memory (ROM). For example, the random access memory is a dynamic random access memory (DRAM) or a storage class memory (SCM). DRAM is a semiconductor memory, and like most random access memories (RAM), it is a volatile memory device. SCM is a composite storage technology that combines the characteristics of traditional storage devices and memory. Storage class memory can provide faster read and write speeds than hard disks, but is slower than DRAM in terms of computing speed and cheaper than DRAM in terms of cost. However, DRAM and SCM are only exemplary descriptions in this embodiment, and the memory can also include other random access memories, such as static random access memories (SRAM). As for the read-only memory, for example, it can be a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), etc. In addition, the memory 113 can also be a dual in-line memory module or a dual-line memory module (DIMM), that is, a module composed of a dynamic random access memory (DRAM), or a solid-state drive (SSD). In actual applications, a plurality of memories 112 and different types of memories 112 can be configured in the computing node 110. This embodiment does not limit the number and type of memory 112. In addition, the memory 112 can be configured to have a power-saving function. The power-saving function means that when the system loses power and then powers on again, the data stored in the memory 112 will not be lost. A memory with a power-saving function is called a non-volatile memory.

[0111] The network card 113 is used to communicate with other application servers 110 .

[0112] In a feasible scenario, the client 101, the server 102 and the hard disk 103 are deployed separately. In this scenario, the distributed storage system 100 includes a computing node cluster and a storage node cluster. Figure 2bAs shown, the computing node cluster includes one or more computing nodes 120 ( Figure 2b Two computing nodes 120 are shown in the figure, but are not limited to two computing nodes 120), each computing node 110 can communicate with each other, and the computing node 110 deploys the client 101. The computing node 120 is a computing device, such as a server, a desktop computer, or a controller of a storage array. In terms of hardware, Figure 2b As shown, the computing node 120 at least includes a processor 121, a memory 122, and a network card 123. The details of the processor 121 refer to the above description of the processor 111 and are not repeated here. The details of the memory 122 refer to the above description of the memory 112 and are not repeated here.

[0113] The network card 123 is used to communicate with the storage node 130. For example, when the total amount of data in the memory 122 reaches a certain threshold, the computing node 120 can send a request to the storage node 130 through the network card 123 to store the data persistently. In addition, the computing node 120 can also include a bus for communication between the components inside the computing node 120. Figure 2b The main function of the computing node 120 in the system is computing services. When storing data, remote storage can be used to achieve persistent storage. Therefore, it has less local storage than a conventional server, thereby saving costs and space. However, this does not mean that the computing node 120 cannot have local storage. In actual implementation, the computing node 120 can also have a small number of hard disks built in or externally connected.

[0114] Any computing node 120 can access any storage node 130 in the storage node cluster through the network. The storage node cluster includes multiple storage nodes 130 ( Figure 2b Two storage nodes 130 are shown in the figure, but are not limited to two storage nodes 130). The storage node 110 deploys the server 102, and a storage node 130 includes one or more controllers 131 (deployed with the server 102), a network card 132 and multiple hard disks 103. The network card 132 is used to communicate with the computing node 120. The hard disk 103 is used to store data. The controller 131 is used to write data to the hard disk 103, or read or delete data from the hard disk 103 according to the read / write data request sent by the computing node 120. In the process of reading and writing data, the controller 131 needs to convert the address carried in the read / write data request into an address that can be recognized by the hard disk. It can be seen that the controller 131 also has some simple computing functions.

[0115] The network may be a wired network or a wireless network. For example, the wired network may be a cable network, an optical fiber network, a digital data network (DDN), etc., and the wireless network may be a telecommunication network, an internal network, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a ZigBee network (ZigBee), a mobile phone (Global System for Mobile Communications, GSM) network, etc. or any combination thereof. It will be appreciated that the network may use any known network communication protocol to achieve communication between different client layers and gateways. The network communication protocol may be various wired or wireless communication protocols, such as Ethernet, universal serial bus (USB), Firewire, global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), new radio (NR) and other communication protocols.

[0116] In practical applications, the control unit 131 may have various forms. In one case, the control unit 131 includes a CPU and a memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store data to be written to the hard disk 103, or read from the hard disk 103 to be sent to the computing node 120. In another case, the control unit 131 is a programmable electronic component, such as a data processing unit (DPU). The DPU has the versatility and programmability of the CPU, but is more specialized and can run efficiently on network data packets, storage requests or analysis requests. The DPU is distinguished from the CPU by a greater degree of parallelism (needing to process a large number of requests). Optionally, the DPU here can also be replaced by a processing chip such as a graphics processing unit (GPU), an embedded neural network processor (nEUral-network processing units, NPU). Generally, the number of control units 131 can be one, or two or more. When the storage node 130 includes at least two control units 131, there is an ownership relationship between the hard disk 103 and the control unit 131, and each controller can only access the hard disk that belongs to it, so this often involves forwarding read / write data requests between the control units 131, resulting in a longer data access path. In addition, if the storage space is insufficient, when adding a new hard disk 103 to the storage node 130, it is necessary to rebind the ownership relationship between the hard disk 103 and the controller 131, which is complicated and leads to poor scalability of the storage space.

[0117] Therefore, in another embodiment, the functions of the control unit 131 can be offloaded to the network card 132. In other words, the storage node 130 does not have a controller 131 inside, but the network card 132 completes data reading and writing, address conversion and other computing functions. At this time, the network card 132 is an intelligent network card. It can include a CPU and a memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store data to be written to the hard disk 103, or read from the hard disk 103 to be sent to the computing node 120. It can also be a programmable electronic component, such as a data processing unit (DPU). There is no ownership relationship between the network card 123 and the hard disk 103 in the storage node 130. The network card 132 can access any hard disk 103 in the storage node 130, so it is more convenient to expand the hard disk when the storage space is insufficient.

[0118] It should be understood that the distributed storage system applicable to the embodiment of the present invention may also be applicable to Figure 1 The embodiments of the present invention do not limit other distributed storage systems than the distributed storage system shown.

[0119] Next, in conjunction with the distributed storage system described above, a garbage collection method provided by an embodiment of the present invention is introduced in detail.

[0120] In a specific implementation, for the hard disk 103, EU can be a block, a stream or a zone; the disk 103 exposes information to the outside (referred to as externally exposed information in the embodiment of the present application), for example, the externally exposed information may include the garbage amount of the EU (for the convenience of description and distinction, it can be referred to as the second garbage amount). The second garbage amount of the EU represents the proportion of the garbage data contained in the hard disk 103 in the EU to the capacity of the EU.

[0121] Correspondingly, the client 101 has maintenance information (for the convenience of description and distinction, this embodiment of the application refers to first maintenance information). Figure 1 , Figure 3a , Figure 3b As shown, the first maintenance information is used to indicate the capacity of the data unit (for the convenience of description and distinction, it can be called Plog), the garbage amount of Plog (for the convenience of description and distinction, it can be called the first garbage amount), the identifier of the hard disk 103 (referred to as HDID in the embodiment of the present application), the identifier of the EU (referred to as EUID in the embodiment of the present application), the data unit identifier (referred to as PlogID in the implementation of the present application), and the second garbage amount of EU.

[0122] Correspondingly, the server 102 has maintenance information (for the convenience of description and distinction, the embodiment of the present application is referred to as second maintenance information), based on the externally exposed information of the hard disk 103, such as Figure 1 , Figure 3a , Figure 3b As shown, the second maintenance information can be used to record the EUID, PlogID, and second garbage amount of the EU. Furthermore, it can also include information such as the capacity of the EU and the remaining capacity (remaining available capacity).

[0123] It should be noted that the first garbage amount and the second garbage amount are only named for the convenience of description and distinction and do not have any special meaning. The first garbage amount can also be expressed as the garbage amount of Plog, and the second garbage amount can also be expressed as the garbage amount of EU.

[0124] Figure 4 1 is a flow chart of the garbage collection method provided in the embodiment of the present application. The present embodiment can be applied to a distributed storage system, and specifically to a computing device cluster. The present embodiment of the present application takes the storage device as a hard disk 103 for garbage collection as an example. Figure 4 As shown, the garbage collection method provided in the embodiment of the present application includes at least the following steps:

[0125] Step 410 : The client 101 determines at least one Plog to be deleted from the multiple Plogs in the hard disk 103 based on its own first maintenance information.

[0126] It should be noted that Plog is the smallest unit for data storage by the client 101. Plog can be understood as the Plog used by the client 101 for data storage, or can be understood as the Plog whose first garbage volume exceeds the first garbage volume threshold. In specific implementation, if the client 101 feels that the storage space of the hard disk 103 is insufficient, or the garbage volume of some Plogs is large, the deletion of Plog will be triggered.

[0127] According to a possible situation, in step 410, the principle of selecting the Plog to be deleted includes the following two aspects:

[0128] On the one hand, when the amount of valid data of the Plog to be deleted is as large as possible, the second garbage amount of the EU to which the Plog to be deleted belongs is as large as possible, wherein the amount of valid data of the Plog to be deleted is equal to the difference between the capacity of the Plog to be deleted (the size of one Plog) and the first garbage amount of the Plog to be deleted. In this way, the less valid data of the Plog to be deleted and the larger the second garbage amount of the EU to which it belongs, the less valid data of the Plog to be deleted and the less valid data in the EU that needs to be relocated inside the hard disk 103, which can reduce the data relocation overhead inside the hard disk 103 to a certain extent.

[0129] On the second aspect, when the second garbage volume of EU is as large as possible, EU contains as many Plogs to be deleted as possible and / or the amount of valid data of Plogs to be deleted as possible. In this way, the more Plogs to be deleted contained in EU, the fewer Plogs that need to be relocated and do not need to be deleted in EU inside hard disk 103, which can reduce the data relocation overhead inside hard disk 103 to a certain extent; the larger the amount of valid data of Plogs to be deleted contained in EU, the less valid data of Plogs to be deleted that need to be relocated inside hard disk 103, which can reduce the data relocation overhead inside hard disk 103 to a certain extent.

[0130] Among them, the case where the amount of valid data of the Plog to be deleted is as large as possible may include that the amount of valid data of the Plog to be deleted is greater than a preset valid data amount threshold (in some possible cases, it can be called the first threshold), or, it is the minimum value among the amounts of valid data of the Plog to be deleted.

[0131] Among them, the situation where the second garbage amount of the EU is as large as possible may include the second garbage amount of the EU being greater than a preset second garbage amount threshold (in some possible cases, it may be called the second threshold), or the EU with the largest second garbage amount among the EU to which the Plog to be deleted belongs.

[0132] Among them, the situation where the EU contains as many Plogs to be deleted as possible may include that each Plog in the EU is a Plog to be deleted, or the number of Plogs to be deleted contained in the EU is greater than or equal to a data unit number threshold (in some possible cases, it can be called a third threshold).

[0133] It should be noted that, in the scenario where Plog is located in multiple EUs, the embodiment of the present application does not need to consider the number of EUs to which Plog belongs, but only needs to consider the second garbage amount of a single EU and the Plog to be deleted contained in the EU.

[0134] In this case, a feasible implementation method is that the EU to which the Plog to be deleted belongs satisfies the following conditions:

[0135] The second garbage amount of the EU to which the Plog to be deleted belongs is greater than or equal to the garbage amount threshold; or, the EU to which the Plog to be deleted belongs is the EU with the second largest garbage amount among the EUs to which the Plog belongs.

[0136] And / or, the amount of valid data of the Plog to be deleted satisfies any of the following conditions:

[0137] The data volume of the valid data of the Plog to be deleted is less than or equal to the valid data volume threshold; or, the data volume of the valid data of the Plog to be deleted is the minimum value of the data volumes of the valid data of multiple Plogs.

[0138] In the first example, there are multiple Plogs to be deleted, the multiple Plogs to be deleted are located in the same EU, and the second garbage amount of the EU to which the multiple Plogs to be deleted belong is greater than the garbage amount threshold.

[0139] In the second example, multiple Plogs to be deleted are located in the same EU, which is the EU with the second largest amount of garbage among the EUs to which the multiple Plogs belong.

[0140] In the third example, the amount of valid data of the Plog to be deleted is less than or equal to the valid data amount threshold.

[0141] In the fourth example, the multiple Plogs to be deleted include a Plog having the smallest amount of valid data among the multiple Plogs to be deleted.

[0142] In the fifth example, the above first example or second example is combined with the third example or fourth example.

[0143] It should be noted that the above examples are only partial examples and do not constitute a specific limitation. In practical applications, the method of determining the Plog to be deleted from the Plog can be set in combination with the actual situation.

[0144] Correspondingly, there may be multiple ways to determine the Plog to be deleted in step 410.

[0145] In a feasible implementation, correspondingly, step 410 may include:

[0146] The client 101 determines at least one Plog to be deleted from the multiple Plogs with the largest first garbage amount based on the first garbage amount of the multiple Plogs. For example, a preset number of Plogs with the largest first garbage amount can be used as the Plogs to be deleted. The preset number can be set in combination with actual needs, for example, 3. For another example, multiple Plogs with a first garbage amount threshold value or greater can be used as the Plogs to be deleted.

[0147] In a feasible implementation, correspondingly, step 410 may include:

[0148] The client 101 determines at least one Plog to be deleted from the multiple Plogs located in the same EU based on the EUID of the EU to which the multiple Plogs belong. Exemplarily, each of the multiple Plogs located in the same EU can be used as a Plog to be deleted. In some possible cases, each of the multiple EUs may contain multiple Plogs. In this case, each of the multiple Plogs in each EU can be used as a Plog to be deleted.

[0149] In a feasible implementation, correspondingly, step 410 may include:

[0150] The client 101 determines at least one Plog to be deleted from the EU with the second largest amount of garbage based on the second garbage amount of the EU to which the multiple Plogs belong. Exemplarily, each Plog in the EU with the second largest amount of garbage can be used as a Plog to be deleted. There can be one or more Plogs in the EU with the second largest amount of garbage. In some possible situations, there may be multiple EUs, each of which contains multiple Plogs. In this case, each Plog in the multiple Plogs in the EU with the second largest amount of garbage in the multiple EUs can be used as a Plog to be deleted.

[0151] In a possible example of this implementation, step 410 may include: the client 101 determines several EUs containing at least 2 Plogs based on the EUs to which multiple Plogs belong and the second garbage amount of the EUs, and when there is only one EU, each Plog in the EU is used as a Plog to be deleted; when there are multiple EUs, the EU with the largest second garbage amount can be selected, and each Plog in the EU with the largest second garbage amount can be used as a Plog to be deleted.

[0152] In a feasible implementation, step 410 may include: the client 101 determines at least one Plog to be deleted from multiple Plogs with the smallest amount of valid data based on the first garbage amount and capacity of multiple Plogs. For example, a preset number of Plogs with the smallest amount of valid data may be used as Plogs to be deleted. The preset number may be set in combination with actual needs, such as 3. For another example, multiple Plogs with a valid data amount threshold value or less may be used as Plogs to be deleted.

[0153] In a feasible implementation, step 410 may include: the client 101 determines the EU to which the Plog with the smallest amount of valid data belongs based on the first garbage amount, capacity and EU to which the multiple Plogs belong, and takes each Plog in the EU as a Plog to be deleted.

[0154] In a feasible implementation, in step 410, the garbage collection overhead of the Plog can be measured to achieve the selection of the Plog to be deleted. In this case, step 410 may include: for each Plog in multiple Plogs, based on the amount of valid data of the Plog and the second garbage amount of the EU to which the Plog belongs, determining the overhead measurement value corresponding to the Plog; the amount of valid data is the difference between the capacity of the Plog and the first garbage amount; based on the overhead measurement value corresponding to each Plog, determining at least one Plog to be deleted. The smaller the amount of valid data and the larger the amount of second garbage, the smaller the overhead measurement value, otherwise the larger.

[0155] In specific implementation, a metric function can be constructed through the capacity of Plog, the first garbage amount and the second garbage amount of the EU where it is located; subsequently, the function value calculated by the metric function can be used as the overhead metric value.

[0156] For example, the metric function can be expressed by the following formula (1):

[0157] y=k1(P size –P gsize )+k2EU gsize (1)

[0158] Among them, k1 and k2 are constants, which can be set according to actual needs, for example, k1+k2 equals 1. For example, if the effective data volume of Plog is emphasized, k1 can be greater than k2; if the second garbage volume of EU to which Plog belongs is emphasized, k2 can be greater than k1.

[0159] For example, when the client 101 knows the capacity EU size In the case of , the metric function can be expressed by the following formula (2):

[0160] y=k1(P size –P gsize )+k2(EU size –EU gsize )(2)

[0161] For example, the client 101 may know the capacity EU of EU size , the time for the client to read or write a unit capacity from the hard disk 103: Net_time, and the data migration overhead per unit capacity within the hard disk 103: HD_time, then the metric function can be expressed by the following formula (3):

[0162] y=k1(P size –P gsize )*Net_time*2+k2(EU size –EU gsize )*HD_time*2(3)

[0163] It should be noted that the above metric function is only an example and does not constitute a specific limitation.

[0164] There are multiple ways to determine the data unit identifier of the Plog to be deleted based on the overhead metric value corresponding to each Plog. A larger overhead metric value indicates a larger overhead.

[0165] In an optional manner, the EU to which the Plog corresponding to the minimum cost metric value belongs is determined; and each Plog in the EU is used as a Plog to be deleted.

[0166] In another optional manner, the EU to which the Plog corresponding to the cost metric value that is less than or equal to the cost metric threshold belongs is determined; and each Plog in the EU is taken as a Plog to be deleted.

[0167] In yet another optional manner, a Plog corresponding to a cost metric value that is less than or equal to a cost metric threshold is determined as the Plog to be deleted.

[0168] Step 420: The client 101 sends a first data deletion request to the server 102, where the first data deletion request includes a PlogID of at least one Plog to be deleted.

[0169] Step 430 : The server 102 adds the PlogID of at least one Plod to be deleted to the data unit identification set corresponding to the hard disk 103 .

[0170] Exemplarily, the data unit identification set may be expressed as a deletion linked list, and the Plog indicated by the data unit identification set needs to be marked for deletion. If necessary, the latest data of the Plog marked for deletion may be restored later.

[0171] In actual applications, the data unit identification set is used to store the PlogID of at least one data unit to be deleted included in the data deletion requests sent multiple times by the client 101. Exemplarily, the data unit identification set also includes the identification of at least one data unit to be deleted included in the second data deletion request sent by the client. It should be noted that the second data deletion request indicates the data unit to be deleted determined in the last data deletion request.

[0172] Step 440 : The server 102 instructs the hard disk 103 to recycle multiple Plogs to be deleted in the target EU based on the second maintenance information and the data unit identification set corresponding to the hard disk 103 .

[0173] In specific implementation, in step 440, the server 102 detects whether the hard disk 103 has a target EU based on the second maintenance information and the data unit identification set corresponding to the hard disk 103. If so, the hard disk 103 is instructed to recycle multiple Plogs to be deleted in the target EU.

[0174] It should be noted that there can be one or more target EUs, and this embodiment of the present application does not make any specific limitation on this, and the specific number can be determined based on actual needs.

[0175] According to a feasible implementation, step 440 may include: the server 102 determines the target EU with the most Plogs to be deleted from the multiple EUs based on the PlogIDs and data unit identifiers of the Plogs included in the multiple EUs. It should be noted that the more Plogs to be deleted, the fewer Plogs that need to be migrated and not deleted in the target EU before the hard disk 103 performs GC migration on the target EU, and the less migration overhead.

[0176] Exemplarily, the PlogID of each Plog in the target erase unit is located in the data unit identification set.

[0177] According to a feasible implementation, step 440 may include the server 102 determining the target EU with the largest second garbage amount from the multiple EUs based on the PlogID and data unit identifier set of the Plog included in the multiple EUs. It should be noted that the larger the second garbage amount of the EU, the less valid data recognized by the hard disk 103 in the target EU during the hard disk 103's GC relocation of the target EU, and the less the relocation overhead.

[0178] Exemplarily, the PlogID of each Plog in the target EU is located in the data unit identification set, and the second garbage amount is the largest.

[0179] Exemplarily, the PlogIDs of at least two Plogs in the target EU are located in the data unit identification set, and the second one has the largest amount of garbage. For the above two implementations, in one possible scenario S1, the target EU can be the first EU, or the first EU with the highest amount of garbage among multiple first EUs, and the PlogID of each Plog in each first EU is located in the data unit identification set. It should be noted that the first EU is only named for the convenience of description and distinction, and has no special meaning.

[0180] In specific implementation, in some possible scenarios, when the hard disk 103 does not issue an alarm for insufficient storage space, the server 102 detects whether there is a first EU among the several EUs where the Plog to be deleted indicated by the data unit identification set is located, based on the PlogID of the Plog to be deleted indicated by the data unit identification set and the EU to which the Plog belongs. If not, the server 102 continues to receive the PlogID of the Plog to be deleted sent by the client 101. If yes, when one first EU is detected, the first EU is used as the target EU; when multiple first EUs are detected, the first EU with the second largest amount of garbage is used as the target EU.

[0181] In one possible scenario S2, the target EU is the second EU with the largest amount of second garbage among multiple second EUs; wherein each second EU contains at least 2 Plogs whose PlogID is located in the data unit identifier set. In some possible scenarios, the amount of second garbage in each second EU is greater than or equal to the second garbage amount threshold; or, the storage space proportion of the Plogs whose PlogID in the second EU is located in the data unit identifier set is greater than or equal to the storage space threshold. It should be noted that the second EU is named only for the convenience of description and distinction and has no special meaning. In specific implementation, in some possible scenarios, when the hard disk 103 does not issue an alarm for insufficient storage space, the server 102 detects whether there is a second EU among the several EUs to which the Plog to be deleted indicated by the data unit identification set belongs, based on the PlogID of the Plog to be deleted indicated by the data unit identification set and the EU to which the Plog belongs. If not, the server 102 continues to receive the PlogID of the Plog to be deleted sent by the client 101. If so, when it is detected that there is only one second EU, it is determined as the target EU; when it is detected that there are multiple second EUs, the second EU with the largest amount of second garbage is taken as the target EU.

[0182] In some other possible situations, the hard disk 103 issues an alarm of insufficient storage space. In this case, the server 102 detects whether there is a first EU. If so, when one first EU is detected, the first EU is used as the target EU; when multiple first EUs are detected, the first EU with the second largest amount of garbage is used as the target EU; if not, it detects whether there is a second EU. If detected, when only one second EU is detected, it is determined as the target EU; when multiple second EUs are detected, the second EU with the second largest amount of garbage is used as the target EU. If it is detected that there are no first EU and second EU, the second EU with the second largest amount of garbage can be directly determined as the target EU.

[0183] There may be the following two situations in the process of recycling multiple PIogs to be deleted in the target EU.

[0184] In a possible case, the PlogID of each Plog stored in the target EU is in the data unit identifier set. The server 102 can instruct the storage device to relocate the valid data recognized by the hard disk 103 in the target EU to other EUs and erase the target EU.

[0185] In another possible situation, the target EU stores a valid Plog (PlogID is in the data unit identifier set). The server 102 can instruct the storage device to relocate the valid Plog data in the target EU to other EUs, and then migrate the valid data recognized by the hard disk 103 to other EUs, and then erase the target EU.

[0186] like Figure 5 As shown, in Figure 4 Based on the steps 410 to 440 shown in the figure, in the embodiment of the present application, step 440 includes the following steps:

[0187] Step 441a: The server 102 determines the garbage collection command corresponding to the target EU under the condition that the PlogID of each Plog stored in the target EU is located in the data unit identifier set; the garbage collection command includes the EUID of the target EU.

[0188] Step 441b1: When the target EU stores a valid Plog, the server 102 determines a read instruction corresponding to the valid Plog; the PlogID of the valid Plog is a PlogID other than the PlogIDs in the data unit identifier set.

[0189] Step 441b2: the server 102 sends a read instruction to the hard disk 103.

[0190] Step 441b3: The first disk 103 sends the valid Plog read out based on the read instruction to the server 102.

[0191] Step 441b4, the server 102 sends a write instruction to the hard disk 103.

[0192] Step 441b5: The first disk 103 writes the valid Plog into the EU based on the write instruction.

[0193] In actual applications, the first disk 103 may write the valid Plog into the third EU.

[0194] Step 441b6: The first disk 103 sends a write success notification to the server 102.

[0195] Step 441b7, the server 102 determines the garbage collection command corresponding to the target EU; the garbage collection command includes the EUID of the target EU.

[0196] Step 442 , the server 102 sends a garbage collection command corresponding to the target EU to the hard disk 103 .

[0197] Step 443: The hard disk 103 performs garbage collection on the target EU based on the garbage collection command.

[0198] In one possible scenario, the garbage collection process for the target EU includes: migrating valid data recognized by the hard disk 103 in the target EU to other EUs, such as an empty EU, which can be called a fourth EU, and erasing the target EU.

[0199] Step 444 , the hard disk 103 sends an erasure success notification to the server 102 .

[0200] Step 445 , the server 102 deletes the PlogID in the data unit identifier set if the Plog corresponding to the PlogID is only located in the target EU.

[0201] It should be noted that Plog can be located in several EUs, for example, in one EU or in multiple EUs. For PlogID in the data unit identifier set, if the Plog corresponding to PlogID is also located in an EU other than the target EU, the PlogID needs to be retained so that the other EUs where the PlogID is located can be erased later to achieve phased deletion.

[0202] In this solution, the client 101 considers the Plog and garbage amount in the EU exposed by the hard disk when selecting the Plog to be deleted. The server 102 adds the PlogID of the Plog to be deleted sent by the client 101 to the data unit identification set, and performs garbage collection on the EU of multiple Plogs to be deleted indicated by the data unit identification set, thereby reducing the data that needs to be additionally relocated, reducing write amplification, and reducing data relocation overhead.

[0203] Based on this, the technical ideas for deriving the garbage collection method proposed in the embodiment of the present application are described.

[0204] The current cost of recycling one Plog is the cost of recycling one Plog by the client 101 and the cost of recycling EU inside the hard disk 103: (P_size–P_gsize)*Net_time*2+(EU_size–EU_gsize)*HD_time*2.

[0205] Wherein, P_size represents the size of Plog; P_gsize represents the amount of garbage in Plog; Net_time represents the time it takes for the client to read or write a unit of capacity from the hard disk 103. Figure 2a In the integrated deployment scenario shown in the figure, Net_time can be equal to HD_time; in the distributed storage system using Figure 2b In the scenario of separate deployment shown, Net_time can represent the sum of the network transmission speed per unit capacity and HD_time; EU_size represents the capacity of EU; EU_gsize represents the valid data of EU; HD_time represents the data migration overhead per unit capacity within hard disk 103, that is, the time taken by hard disk 103 to read or write data per unit capacity.

[0206] It should be noted that the cost of client 101 recycling a plog includes: the cost of reading valid data of plog: (P_size–P_gsize)*Net_time; the cost of writing valid data of plog to new plog: (P_size–P_gsize)*Net_time; therefore, the cost of client 101 recycling a Plog is (P_size–P_gsize)*Net_time*2. The cost of recycling EU inside hard disk 103 is: the cost of reading valid data of EU: (EU_size–EU_gsize)*HD_time; the cost of writing valid data of EU: (EU_size–EU_gsize)*HD_time; therefore, the cost of recycling EU inside hard disk 103 is (EU_size–EU_gsize)*HD_time*2.

[0207] The embodiment of the present application reduces the data size of valid data in multiple plogs (P_size-P_gsize) and the data size of valid data in one EU (EU_size-EU_gsize), so that when garbage collection occurs in the client 101, the data migration overhead in the hard disk 103 is as small as possible.

[0208] It should be noted that the valid data in the EU may include data recognized as valid in the hard disk 103. In the scenario where the EU includes Plogs other than the Plog to be deleted, the valid data in the EU may also include Plogs that do not need to be deleted. Therefore, the more valid data in the EU, the more valid data that needs to be read and written from the EU in the hard disk 103, and the greater the data migration overhead. In order to reduce write amplification, on the one hand, it is necessary to reduce the valid data of multiple Plogs as much as possible, and on the other hand, it is necessary to reduce the valid data of the EU as much as possible.

[0209] Specifically, when selecting a Plog to be deleted, the client 101 will consider the following: reduce EU_size-EU_gsize while minimizing P_size-P_gsize. For example, multiple Plogs to be deleted are located in the same EU, and the garbage volume of the EU is high. Alternatively, select one or more Plogs in the EU with a high garbage volume for garbage collection.

[0210] Furthermore, the server 102 writes the PlogID of the Plog to be deleted sent by the client 101 into the data unit identification set. When the garbage collection command (for example, the Trim command) is actually issued, it tries to find all the Plogs in an EU with a large amount of garbage among all the PlogIDs in the data unit identification set for garbage collection, thereby further reducing the overhead of erasing data. If no such Plogs can be found, it tries to find more Plogs in an EU with a large amount of garbage among all the PlogIDs in the data unit identification set for garbage collection.

[0211] Based on the garbage collection method provided above, the specific application of the garbage collection method is explained. Figure 6 A flowchart of a specific application of a garbage collection method provided for the implementation of the present invention.

[0212] like Figure 6 As shown, for example, it is assumed that garbage collection is performed on hard disk 103: HD1.

[0213] For the client 101, assume that the PlogIDs of multiple Plogs are: Plog1, Plog2, Plog3, Plog6, Plog7, Plog8; based on the first maintenance information, determine that the Plogs corresponding to Plog1, Plog2, and Plog3 are located in the same EU1, and the amount of garbage in EU1 is also relatively high, for example, higher than the preset second garbage amount threshold (also referred to as the first threshold); in addition, (P_size–P_gsize) of Plog6 is the minimum value among (P_size–P_gsize) of Plog, therefore, the PlogIDs of the Plogs to be deleted can be determined: Plog1, Plog2, Plog3, and Plog6; then, the PlogIDs of the Plogs to be deleted: Plog1, Plog2, Plog3, and Plog6 are sent to the server 102.

[0214] For the server 102, it is assumed that the PlogID in the data unit identification set is: Plog4, Plog5, Plog10. It is known that the second garbage volume of EU1 is R1, and the PlogIDs of the Plogs included are: Plog1, Plog2, Plog3; the second garbage volume of EU2 is R2, and the PlogIDs of the Plogs included are: Plog4, Plog5, Plog6; the second garbage volume of EU3 is R3, and the PlogIDs of the Plogs included are: Plog7, Plog8, Plog9; the second garbage volume of EU4 is R4, and the PlogIDs are: Plog10, Plog11, Plog12.

[0215] In one possible scenario, Figure 7aAs shown, it is assumed that HD1 does not issue an alarm of insufficient storage space before writing the PlogIDs of the Plogs to be deleted: Plog1, Plog2, Plog3, Plog6 into the data unit identification set. After the server 102 writes the PlogIDs of the Plogs to be deleted: Plog1, Plog2, Plog3, Plog6 into the data unit identification set, the PlogIDs in the data unit identification set are: Plog1, Plog2, Plog3, Plog4, Plog5, Plog6, Plog10. The Plogs corresponding to Plog1, Plog2, and Plog3 are located in EU1, and are adapted to the PlogIDs of all Plogs stored in EU1. Plog4, Plog5, and Plog6 are located in the same EU2, and are adapted to the PlogIDs of all Plogs stored in EU2. If the second garbage amount R2 of EU2 is greater than the second garbage amount R1 of EU1, HD1 is controlled to perform garbage collection on EU2. For example, HD1 migrates valid data in EU2 to EU5, and then erases EU2.

[0216] In one possible scenario, Figure 7b As shown, it is assumed that before the PlogIDs of the Plogs to be deleted: Plog1, Plog2, Plog3, and Plog6 are written into the data unit identification set, HD1 issues an alarm for insufficient storage space. The Plogs corresponding to Plog4 and Plog5 are located in EU2, and the Plog corresponding to Plog10 is located in EU4. The second garbage volume R2 of EU2 is less than the second garbage volume R4 of EU4, then HD1 is controlled to perform garbage collection on EU4. Since EU4 contains data that does not need to be recycled, in order to ensure data security, the server 102 controls HD1 to write the Plogs corresponding to Plog11 and Plog12 to EU5; subsequently, HD1 moves the valid data in EU4 to EU6, and then erases EU4.

[0217] It should be noted that in the embodiment of the present application, the hard disk 103 is merely an example of a storage device and does not constitute a specific limitation. Any storage device that can be divided into multiple EUs, with EU as the recycling granularity, and has a garbage collection function uses the garbage collection method provided in the embodiment of the present application.

[0218] It is worth noting that the garbage collection method provided in the embodiment of the present application does not describe migrating the valid data in the Plog to be deleted to other EUs. In fact, before the hard disk 103 performs garbage collection, the valid data in the Plog to be deleted has been migrated to other EUs.

[0219] The present application also provides a distributed storage system, the distributed storage system includes a client and a server, the server accesses a storage device, the storage device includes multiple erasure units, the client includes first maintenance information, the first maintenance information is used to indicate multiple data units and the erasure units to which the multiple data units belong, the server includes second maintenance information, the second maintenance information is used to indicate multiple erasure units; Figure 8 As shown, including:

[0220] The client is configured to determine at least one data unit to be deleted from a plurality of data units based on the first maintenance information;

[0221] The client is used to send a first data deletion request, where the first data deletion request includes an identifier of at least one data unit to be deleted;

[0222] The server is configured to receive a first data deletion request and add an identifier of at least one data unit to be deleted to a data unit identifier set;

[0223] The service end is used to instruct the storage device to recycle multiple data units to be deleted in the target erasure unit based on the second maintenance information and the data unit identification set.

[0224] Among them, the client and the server can be implemented by software or hardware. For example, the implementation of the client is introduced below by taking the client as an example. Similarly, the implementation of the server and can refer to the implementation of the client.

[0225] As an example of a software functional unit, a module may include a code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the client may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Generally, a region may include multiple AZs.

[0226] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0227] As an example of a hardware functional unit, a client may include at least one computing device, such as a server, etc. Alternatively, the client may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0228] The multiple computing devices included in the client can be distributed in the same region or in different regions. The multiple computing devices included in the client can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the client can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0229] It should be noted that, in other embodiments, the client can be used to execute any step in the garbage collection method, and the server can be used to execute any step in the garbage collection method. The steps that the client and the server are responsible for implementing can be specified as needed. The full functions of the distributed storage system can be realized by implementing different steps in the garbage collection method by the client and the server respectively.

[0230] The present application also provides a computing device 900. Fig. 9As shown, the computing device 900 includes: a bus 902, a processor 904, a memory 906, and a communication interface 908. The processor 904, the memory 906, and the communication interface 908 communicate through the bus 902. The computing device 900 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 900.

[0231] The bus 902 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 The bus 902 may include a path for transmitting information between various components of the computing device 900 (eg, the memory 906, the processor 904, and the communication interface 908).

[0232] The processor 904 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0233] The memory 906 may include a volatile memory, such as a random access memory (RAM). The processor 904 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0234] The memory 906 stores executable program codes, and the processor 904 executes the executable program codes to respectively implement the aforementioned server and / or client functions, thereby implementing the garbage collection method. That is, the memory 906 stores instructions for executing the garbage collection method.

[0235] The communication interface 908 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 900 and other devices or communication networks.

[0236] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0237] like Fig.10 As shown, the computing device cluster includes at least one computing device 900. The memory 906 in one or more computing devices 900 in the computing device cluster may store the same instructions for executing the garbage collection method.

[0238] In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also store partial instructions for executing the garbage collection method. In other words, the combination of one or more computing devices 900 may jointly execute instructions for executing the garbage collection method.

[0239] It should be noted that the memory 906 in different computing devices 900 in the computing device cluster can store different instructions, which are used to execute part of the functions of the garbage collection device of the server or client. That is, the instructions stored in the memory 906 in different computing devices 900 can implement the functions of one or more modules in the client and the server.

[0240] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.11 A possible implementation is shown. Fig.11 As shown, two computing devices 900A and 900B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 906 in the computing device 900A stores instructions for executing the functions of the client. At the same time, the memory 906 in the computing device 900B stores instructions for executing the functions of the server.

[0241] Fig.11 The connection method between the computing device clusters shown may be that considering that the garbage collection method provided in the present application requires the client and the server to interact with each other, it is considered to hand over the functions implemented by the client to the computing device 900A for execution.

[0242] It should be understood that Fig.11 The functions of the computing device 900A shown in FIG. 9A may also be completed by multiple computing devices 900. Similarly, the functions of the computing device 900B may also be completed by multiple computing devices 900.

[0243] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device is caused to perform a garbage collection method.

[0244] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data center or other non-volatile data storage device containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute a garbage collection method.

[0245] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0246] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0247] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0248] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0249] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0250] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0251] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

Claims

1. A garbage collection method, characterized in that: The method is applied to a distributed storage system, the distributed storage system includes a client and a server, the server accesses a storage device, the storage device includes multiple erasure units, the client includes first maintenance information, the first maintenance information is used to indicate multiple data units and the erasure units to which the multiple data units belong, the server includes second maintenance information, the second maintenance information is used to indicate the multiple erasure units, the method includes: The client determines, based on the first maintenance information, at least one data unit to be deleted from the multiple data units; The client sends a first data deletion request, where the first data deletion request includes an identifier of the at least one data unit to be deleted; The server receives the first data deletion request, and adds the identifier of the at least one data unit to be deleted to the data unit identifier set; The service end instructs the storage device to recycle multiple to-be-deleted data units in the target erasure unit based on the second maintenance information and the data unit identification set.

2. The method according to claim 1, characterized in that The data unit identification set also includes an identification of at least one data unit to be deleted included in the second data deletion request sent by the client.

3. The method according to claim 1 or 2, characterized in that: The first maintenance information includes one or more of the following information: The identifiers of the multiple data units to be deleted, the garbage amounts of the multiple data units to be deleted, the identifiers of the erasure units to which the multiple data units to be deleted belong, or the garbage amounts of the erasure units to which the multiple data units to be deleted belong.

4. The method according to any one of claims 1 to 3, characterized in that: The second maintenance information includes one or more of the following information: The identifiers of the multiple erasure units, the garbage amounts of the multiple erasure units, and the identifiers of the data units included in the multiple erasure units.

5. The method according to claim 1 or 4, characterized in that: The client determines, based on the first maintenance information, at least one data unit to be deleted from the multiple data units, including: The client determines, based on the garbage amounts of the multiple data units, the at least one data unit to be deleted from the multiple data units with the largest amount of garbage; or The client determines, based on the identifier of the erasure unit to which the multiple data units belong, the at least one data unit to be deleted from the multiple data units located in the same erasure unit; or The client determines the at least one data unit to be deleted from the erasure unit having the largest amount of garbage based on the amount of garbage in the erasure units to which the multiple data units belong.

6. The method according to claim 1 or 5, characterized in that: The server instructs the storage device to recycle a plurality of to-be-deleted data units in a target erasure unit based on the second maintenance information and the data unit identifier set, including: The server instructs the storage device to recycle multiple data units to be deleted in a target erasure unit having the most data units to be deleted in the multiple erasure units based on the identifiers of the data units included in the multiple erasure units and the data unit identifier set; or The server instructs the storage device to recycle multiple data units to be deleted in a target erasure unit having the largest amount of garbage among the multiple erasure units based on the amount of garbage in the multiple erasure units and the data unit identification set.

7. A distributed storage system, characterized in that: The distributed storage system includes a server and a client; the server accesses a storage device, the storage device includes a plurality of erasure units, the client includes first maintenance information, the first maintenance information is used to indicate a plurality of data units and the erasure units to which the plurality of data units belong, and the server includes second maintenance information, the second maintenance information is used to indicate the plurality of erasure units; The client is configured to determine at least one data unit to be deleted from the plurality of data units based on the first maintenance information; The client is configured to send a first data deletion request, wherein the first data deletion request includes an identifier of the at least one data unit to be deleted; The server is configured to receive the first data deletion request and add the identifier of the at least one data unit to be deleted to the data unit identifier set; The service end is used to instruct the storage device to recycle multiple to-be-deleted data units in the target erasure unit based on the second maintenance information and the data unit identification set.

8. The distributed storage system according to claim 7, characterized in that: The data unit identification set also includes an identification of at least one data unit to be deleted included in the second data deletion request sent by the client.

9. The distributed storage system according to claim 7 or 8, characterized in that: The first maintenance information includes one or more of the following information: The identifiers of the multiple data units to be deleted, the garbage amounts of the multiple data units to be deleted, the identifiers of the erasure units to which the multiple data units to be deleted belong, or the garbage amounts of the erasure units to which the multiple data units to be deleted belong.

10. The distributed storage system according to any one of claims 7 to 9, characterized in that: The second maintenance information includes one or more of the following information: identifiers of the multiple erasure units, garbage amounts of the multiple erasure units, and identifiers of data units included in the multiple erasure units.

11. The distributed storage system according to claim 7 or 10, characterized in that: The client is specifically used for: Based on the garbage amounts of the multiple data units, determining the at least one data unit to be deleted from the multiple data units with the largest garbage amount; or, Determining the at least one data unit to be deleted from the multiple data units located in the same erasure unit based on the identifiers of the erasure units to which the multiple data units belong; or, Based on the garbage amounts of the erasure units to which the multiple data units belong, the at least one data unit to be deleted is determined from the erasure unit having the largest garbage amount.

12. The distributed storage system according to claim 7 or 11, characterized in that: The server is specifically used for: Based on the identifiers of the data units included in the multiple erasure units and the data unit identifier set, instruct the storage device to recycle multiple data units to be deleted in a target erasure unit having the most data units to be deleted in the multiple erasure units; or Based on the garbage amounts of the multiple erasure units and the data unit identification set, the storage device is instructed to recycle multiple data units to be deleted in a target erasure unit having the largest garbage amount among the multiple erasure units.

13. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

14. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 6.

15. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

Images