Garbage collection method, distributed storage system and computer-readable storage medium

Abstract

Provided in the present application are a garbage collection method, a distributed storage system and a computer-readable storage medium. The distributed storage system comprises a server and a client, which access a storage device comprising a plurality of erase units, wherein the client comprises first maintenance information for indicating a plurality of data units and erase units to which the data units belong, and the server comprises second maintenance information for indicating the plurality of erase units. The method comprises: on the basis of first maintenance information, a client determining a first data deletion request comprising an identifier of at least one data unit to be deleted, and sending the first data deletion request; a server adding the identifier of said at least one data unit in the received first data deletion request to a data unit identifier set; and on the basis of second maintenance information and the data unit identifier set, instructing a storage device to perform collection on a plurality of data units to be deleted in a target erase unit. By means of performing centralized collection on the plurality of data units in the erase unit, a write amplification is reduced, and the collection efficiency is improved.

Description

Garbage collection method, distributed storage system, and computer-readable storage medium

[0001] This application claims priority to the Chinese patent applications filed with the State Intellectual Property Office of China on June 8, 2023, with application number 202310678002.5 and application name “A method, device and other equipment for deleting data”, and filed with the State Intellectual Property Office of China on September 26, 2023, with application number 202311256325.1 and application name “Garbage collection method, distributed storage system and computer-readable storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of storage technology, and in particular to a garbage collection method, a distributed storage system, and a computer-readable storage medium. Background Art

[0003] Distributed storage systems can divide all data into fixed-size units, each of which can be called a data unit. During distributed block storage, data within a data unit is continuously updated. After the data is updated, the previous version of the data becomes garbage data. When a client in a distributed storage system deletes a data unit, the hard drive performs garbage collection (GC) on the data unit to be deleted. GC relocation is performed at a specific granularity within the hard drive. GC relocation requires relocating valid data at the reclaimed granularity and deleting invalid data. However, the amount of garbage in the data unit maintained by the client is not the same as the amount of garbage at the granularity collected by the hard drive. As a result, the hard drive may contain other data that needs to be relocated in addition to the data that needs to be relocated during the GC process. This can lead to write amplification (the actual data written to the hard drive is multiple times the data written by the client). The write amplification factor is defined as data written to the media / data written by the client. Assuming a write amplification factor of 2.5, one possible scenario is: the client writes 1MB of data, and the SSD writes 2.5MB of data to the SSD media. The SSD's internal writing of 2.5MB of data includes: writing 1MB of client data; 1.5MB of GC relocation data; and additional media reads to enable the client to write 1.5MB of data.

[0004] However, write amplification will affect the performance of distributed storage systems and reduce the life of hard disks. Therefore, a garbage collection method that can reduce write amplification is urgently needed.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a garbage collection method, a distributed storage system, and a computer-readable storage medium. The hard disk exposes information about the erase unit to the client. Through the coordinated cooperation between the client and the server, the server can centrally recycle multiple data units to be deleted in the erase unit, thereby reducing the data that needs to be additionally relocated, reducing write amplification, reducing the time spent on GC, and improving GC efficiency.

[0007] In a first aspect, an embodiment of the present application provides a garbage collection method, which is applied to a distributed storage system. The distributed storage system includes a client and a server, the server accesses a storage device, the storage device includes multiple erase units, the client includes first maintenance information, the first maintenance information is used to indicate multiple data units and the erase units to which the multiple data units belong, the server includes second maintenance information, the second maintenance information is used to indicate the multiple erase units, and the method includes:

[0008] The client determines at least one data unit to be deleted from multiple data units based on the first maintenance information; the client sends a first data deletion request, which includes the identifier of the at least one data unit to be deleted; the server receives the first data deletion request and adds the identifier of the at least one data unit to be deleted to the data unit identifier set; the server instructs the storage device to recycle the multiple data units to be deleted in the target erasure unit based on the second maintenance information and the data unit identifier set.

[0009] In this solution, the hard drive exposes the erase unit information to the client. Through the coordinated cooperation between the client and the server, the server can centrally recycle multiple data units to be deleted in the erase unit, thereby reducing the need for additional data relocation, reducing write amplification, reducing the time spent on GC, and improving GC efficiency.

[0010] In one possible implementation, the data unit identifier set also includes the identifier of at least one data unit to be deleted included in the second data deletion request sent by the client. It should be noted that the second data deletion request indicates the data unit to be deleted determined last in the first data deletion request.

[0011] In this solution, the server selects multiple data units to be deleted located in the same erase unit from the multiple data units to be deleted indicated by the data deletion requests sent to the client multiple times, and collects them collectively, thereby reducing the amount of data processed by the hard disk during the GC process to a certain extent, reducing the time spent on GC, and improving GC efficiency.

[0012] In one possible implementation, the first maintenance information includes one or more of the following information: identification of multiple data units to be deleted, the amount of garbage in multiple data units to be deleted, identification of erasure units to which multiple data units to be deleted belong, or the amount of garbage in erasure units to which multiple data units to be deleted belong.

[0013] In a possible implementation, the second maintenance information includes one or more of the following information:

[0014] Identifications of multiple erasure units, amounts of garbage in the multiple erasure units, and identifications of data units included in the multiple erasure units.

[0015] In an example of this implementation method, the client determines at least one data unit to be deleted from multiple data units based on the first maintenance information, including: the client determines at least one data unit to be deleted from multiple data units with the largest amount of garbage based on the amount of garbage in the multiple data units.

[0016] In an example of this implementation method, the client determines at least one data unit to be deleted from multiple data units based on the first maintenance information, including: the client determines at least one data unit to be deleted from multiple data units located in the same erasure unit based on the identifier of the erasure unit to which the multiple data units belong.

[0017] In an example of this implementation method, the client determines at least one data unit to be deleted from multiple data units based on the first maintenance information, including: the client determines at least one data unit to be deleted from the erasure unit with the largest amount of garbage based on the amount of garbage in the erasure units to which the multiple data units belong.

[0018] In one possible implementation, the server instructs the storage device to recycle multiple data units to be deleted in the target erasure unit based on the second maintenance information and the data unit identification set, including: the server instructs the storage device to recycle multiple data units to be deleted in the target erasure unit having the largest number of data units to be deleted in the multiple erasure units based on the identifications of the data units included in the multiple erasure units and the data unit identification set.

[0019] Exemplarily, the identifier of each data unit in the target erasure unit is located in the data unit identifier set.

[0020] In this solution, the server selects the erasure unit containing the most data units to be deleted for erasure, thereby reducing the amount of data moved during the hard disk GC process to a certain extent.

[0021] In one possible implementation, the server instructs the storage device to recycle multiple data units to be deleted in the target erasure unit based on the second maintenance information and the data unit identification set, including: the server instructs the storage device to recycle multiple data units to be deleted in the target erasure unit with the largest amount of garbage among the multiple erasure units based on the amount of garbage in the multiple erasure units and the data unit identification set.

[0022] Exemplarily, the identifier of each data unit in the target erasure unit is located in the data unit identifier set.

[0023] In this solution, the server selects the erase unit with the largest amount of garbage to erase, thereby reducing the amount of data moved during the hard disk GC process to a certain extent.

[0024] In a second aspect, an embodiment of the present application provides a garbage collection method, which is applied to a client of a distributed storage system, wherein the client accesses a storage device through a server of the distributed storage system, wherein the storage device includes multiple erase units, the client includes first maintenance information, and the first maintenance information is used to indicate multiple data units and the erase units to which the multiple data units belong, and the server includes second maintenance information, and the second maintenance information is used to indicate the multiple erase units. The method includes:

[0025] Based on the first maintenance information, at least one data unit to be deleted is determined from multiple data units; a first data deletion request is sent to the server, and the first data deletion request includes the identifier of the at least one data unit to be deleted, so that the server adds the identifier of the at least one data unit to be deleted to the data unit identifier set; based on the second maintenance information and the data unit identifier set, the storage device is instructed to recycle the multiple data units to be deleted in the target erasure unit.

[0026] In this solution, the hard drive exposes erase unit information to the client. The client considers the amount of garbage in the erase unit and data unit and determines the data unit to be deleted. The server adds the identifier of the data unit to be deleted to the data unit identifier set and centrally recycles multiple data units in the erase unit, thereby reducing the amount of data that needs to be relocated, reducing write amplification, shortening the time spent on garbage collection, and improving garbage collection efficiency.

[0027] In one possible implementation, the data unit identifier set also includes the identifier of at least one data unit to be deleted included in the second data deletion request sent by the client. It should be noted that the second data deletion request indicates the data unit to be deleted determined last in the first data deletion request.

[0028] In this solution, the server selects multiple data units to be deleted in the same erasure unit from among the multiple data units to be deleted indicated by the data deletion requests sent to the client multiple times, and collects them collectively, thereby reducing the amount of data processed by the hard disk during the GC process to a certain extent, reducing the time spent on GC, and improving GC efficiency.

[0029] In one possible implementation, the first maintenance information includes one or more of the following information: identification of multiple data units to be deleted, the amount of garbage in multiple data units to be deleted, identification of erasure units to which multiple data units to be deleted belong, or the amount of garbage in erasure units to which multiple data units to be deleted belong.

[0030] In an example of this implementation method, the client determines at least one data unit to be deleted from multiple data units based on the first maintenance information, including: the client determines at least one data unit to be deleted from multiple data units with the largest amount of garbage based on the amount of garbage in the multiple data units.

[0031] In an example of this implementation method, the client determines at least one data unit to be deleted from multiple data units based on the first maintenance information, including: the client determines at least one data unit to be deleted from multiple data units located in the same erasure unit based on the identifier of the erasure unit to which the multiple data units belong.

[0032] In an example of this implementation method, the client determines at least one data unit to be deleted from multiple data units based on the first maintenance information, including: the client determines at least one data unit to be deleted from the erasure unit with the largest amount of garbage based on the amount of garbage in the erasure units to which the multiple data units belong.

[0033] In a third aspect, an embodiment of the present application provides a garbage collection method, which is applied to a server of a distributed storage system, wherein the server accesses a storage device, the storage device includes multiple erase units, and the distributed storage system further includes a client, the client includes first maintenance information, the first maintenance information is used to indicate multiple data units and the erase units to which the multiple data units belong, and the server includes second maintenance information, the second maintenance information is used to indicate the multiple erase units, and the method includes:

[0034] Receive a first data deletion request sent by the client, add the identifier of at least one data unit to be deleted to the data unit identifier set; based on the second maintenance information and the data unit identifier set, instruct the storage device to recycle multiple data units to be deleted in the target erasure unit.

[0035] In this solution, the server adds the identifiers of several data units to be deleted sent by the client to the data unit identifier set, so that based on the data unit identifier set and the data units in the erasure unit, the target erasure unit containing multiple data units to be deleted can be selected for garbage collection.

[0036] In one possible implementation, the data unit identifier set also includes the identifier of at least one data unit to be deleted included in the second data deletion request sent by the client. It should be noted that the second data deletion request indicates the data unit to be deleted determined last in the first data deletion request.

[0037] In this solution, the server selects multiple data units to be deleted located in the same erase unit from the multiple data units to be deleted indicated by the data deletion requests sent to the client multiple times, and collects them collectively, thereby reducing the amount of data processed by the hard disk during the GC process to a certain extent, reducing the time spent on GC, and improving GC efficiency.

[0038] In a possible implementation, the second maintenance information includes one or more of the following information:

[0039] Identifications of multiple erasure units, amounts of garbage in the multiple erasure units, and identifications of data units included in the multiple erasure units.

[0040] In one possible implementation, the server instructs the storage device to recycle multiple data units to be deleted in the target erasure unit based on the second maintenance information and the data unit identification set, including: the server instructs the storage device to recycle multiple data units to be deleted in the target erasure unit having the largest number of data units to be deleted in the multiple erasure units based on the identifications of the data units included in the multiple erasure units and the data unit identification set.

[0041] Exemplarily, the identifier of each data unit in the target erasure unit is located in the data unit identifier set.

[0042] In this solution, the server selects the erasure unit containing the most data units to be deleted for erasure, thereby reducing the amount of data moved during the hard disk GC process to a certain extent.

[0043] In one possible implementation, the server instructs the storage device to recycle multiple data units to be deleted in the target erasure unit based on the second maintenance information and the data unit identification set, including: the server instructs the storage device to recycle multiple data units to be deleted in the target erasure unit with the largest amount of garbage among the multiple erasure units based on the amount of garbage in the multiple erasure units and the data unit identification set.

[0044] Exemplarily, the identifier of each data unit in the target erasure unit is located in the data unit identifier set.

[0045] In this solution, the server selects the erase unit with the largest amount of garbage to erase, thereby reducing the amount of data moved during the hard disk GC process to a certain extent.

[0046] In a fourth aspect, an embodiment of the present application provides a distributed storage system, which may include a client and a server; the server accesses a storage device, the storage device includes multiple erase units, the client includes first maintenance information, the first maintenance information is used to indicate multiple data units and the erase units to which the multiple data units belong, and the server includes second maintenance information, the second maintenance information is used to indicate the multiple erase units;

[0047] The client is configured to determine at least one data unit to be deleted from the plurality of data units based on the first maintenance information;

[0048] The client is configured to send a first data deletion request, where the first data deletion request includes an identifier of at least one data unit to be deleted;

[0049] The server is configured to receive the first data deletion request and add an identifier of at least one data unit to be deleted to a data unit identifier set;

[0050] The service end is used to instruct the storage device to recycle multiple to-be-deleted data units in the target erasure unit based on the second maintenance information and the data unit identifier set.

[0051] In this solution, the hard disk exposes the erase unit information to the client. Through the cooperation between the client and the server, the server can centrally recycle multiple data units to be deleted in the erase unit, thereby reducing the amount of data processed by the hard disk during the GC process to a certain extent, reducing the time spent on GC, and improving GC efficiency.

[0052] In one possible implementation, the data unit identifier set also includes the identifier of at least one data unit to be deleted included in the second data deletion request sent by the client. It should be noted that the second data deletion request indicates the data unit to be deleted determined last in the first data deletion request.

[0053] In this solution, the server selects multiple data units to be deleted located in the same erase unit from the multiple data units to be deleted indicated by the data deletion requests sent to the client multiple times, and collects them collectively, thereby reducing the amount of data processed by the hard disk during the GC process to a certain extent, reducing the time spent on GC, and improving GC efficiency.

[0054] In one possible implementation, the first maintenance information includes one or more of the following information: identification of multiple data units to be deleted, the amount of garbage in multiple data units to be deleted, identification of erasure units to which multiple data units to be deleted belong, or the amount of garbage in erasure units to which multiple data units to be deleted belong.

[0055] In a possible implementation, the second maintenance information includes one or more of the following information:

[0056] Identifications of multiple erasure units, amounts of garbage in the multiple erasure units, and identifications of data units included in the multiple erasure units.

[0057] In an example of this implementation, the client is specifically configured to: determine, based on the amount of garbage in the multiple data units, at least one data unit to be deleted from the multiple data units with the largest amount of garbage.

[0058] In an example of this implementation, the client is specifically configured to: determine at least one data unit to be deleted from multiple data units located in the same erasure unit based on identifiers of the erasure units to which the multiple data units belong.

[0059] In an example of this implementation, the client is specifically configured to: determine at least one data unit to be deleted from an erasure unit having the largest amount of garbage based on the amount of garbage in the erasure units to which the multiple data units belong.

[0060] In one possible implementation, the server is specifically used to: based on the identifiers of the data units included in multiple erasure units and the data unit identifier set, instruct the storage device to recycle multiple data units to be deleted in the target erasure unit with the most data units to be deleted among the multiple erasure units.

[0061] Exemplarily, the identifier of each data unit in the target erasure unit is located in the data unit identifier set.

[0062] In this solution, the server selects the erasure unit containing the most data units to be deleted for erasure, thereby reducing the amount of data moved during the hard disk GC process to a certain extent.

[0063] In a possible implementation, the server is specifically configured to: based on the garbage amount and data unit identification set of multiple erase units, instruct the storage device to recycle multiple data units to be deleted in a target erase unit with the largest garbage amount among the multiple erase units.

[0064] Exemplarily, the identifier of each data unit in the target erasure unit is located in the data unit identifier set.

[0065] In this solution, the server selects the erase unit with the largest amount of garbage to erase, thereby reducing the amount of data moved during the hard disk GC process to a certain extent.

[0066] In the fifth aspect, an embodiment of the present application provides a garbage collection device, comprising: at least one memory for storing programs; at least one processor for executing the programs stored in the memory; when the program stored in the memory is executed, the processor is used to execute the method provided in the first aspect, the method provided in the second aspect, or the method provided in the third aspect.

[0067] In a sixth aspect, embodiments of the present application provide a garbage collection device, characterized in that the device runs computer program instructions to execute the method provided in the first aspect, the method provided in the second aspect, or the method provided in the third aspect. Exemplarily, the device can be a chip or a processor.

[0068] In one example, the apparatus may include a processor, which may be coupled to a memory, read instructions from the memory, and execute the method provided in the first aspect, the method provided in the second aspect, or the method provided in the third aspect according to the instructions. The memory may be integrated into a chip or processor, or may be independent of the chip or processor.

[0069] In the seventh aspect, an embodiment of the present application provides a computer storage medium, which stores instructions. When the instructions are run on a computer, the computer executes the method provided in the first aspect, the method provided in the second aspect, or the method provided in the third aspect.

[0070] In an eighth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method provided in the first aspect, the method provided in the second aspect, or the method provided in the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] FIG1 is a system architecture diagram of a distributed storage system provided in an embodiment of the present application;

[0072] FIG2a is a structural diagram 1 of a distributed storage system provided in an embodiment of the present application;

[0073] FIG2 b is a second structural diagram of a distributed storage system provided in an embodiment of the present application;

[0074] FIG3 a is a first structural diagram of the hard disk provided in FIG1 ;

[0075] FIG3 b is a second structural diagram of the hard disk provided in FIG1 ;

[0076] FIG4 is a flow chart of a garbage collection method according to an embodiment of the present application;

[0077] FIG5 is a schematic diagram of the process of step 440 in FIG4 ;

[0078] FIG6 is a schematic diagram of a scenario of a garbage collection method provided in an embodiment of the present application;

[0079] FIG7 a is a schematic diagram of a scenario in which a client and a server interact in a garbage collection method provided in an embodiment of the present application;

[0080] FIG7 b is a schematic diagram of a scenario in which a server controls a hard disk in a garbage collection method provided in an embodiment of the present application;

[0081] FIG8 is a diagram illustrating a structure of a distributed storage system according to an embodiment of the present application;

[0082] FIG9 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0083] FIG10 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0084] FIG. 11 is a schematic diagram of the computing device cluster provided in FIG. 10 . DETAILED DESCRIPTION

[0085] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0086] In the description of the embodiments of the present application, words such as "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of the present application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0087] In the description of the embodiments of this application, the term "and / or" is simply a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent the following three situations: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "plurality" means two or more. For example, "multiple systems" refers to two or more systems, and "multiple terminals" refers to two or more terminals.

[0088] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly identifying the technical features being referred to. Thus, features specified as "first" or "second" may explicitly or implicitly include one or more of such features. The terms "include," "comprising," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0089] The following is an explanation of some of the terms used in this embodiment. It should be noted that these explanations are for the purpose of facilitating understanding by those skilled in the art and are not intended to limit the scope of protection claimed by the present invention.

[0090] Solid-state drive (SSD) is a non-volatile storage device that mainly uses NAND Flash as permanent memory.

[0091] A mechanical hard disk drive (HDD) is a traditional hard drive consisting primarily of a platter, a magnetic head, a spindle and control motor, a magnetic head controller, a data converter, an interface, and a cache. The magnetic head moves along the radius of the platter. With the platter spinning at several thousand revolutions per minute, the magnetic head can locate a specific location on the platter to read and write data.

[0092] Logical Block Address (LBA): A universal mechanism for describing the location of data blocks on computer storage devices, commonly used in secondary storage devices like hard drives. LBA can refer to either the address of a data block or the data block to which an address points.

[0093] Data unit: It is the smallest unit of data storage used by the client of a distributed storage system.

[0094] Reclaim Unit (RU): is the minimum granularity of recycled data.

[0095] Block: The smallest unit of data write in an SSD. Each block can include several pages. Page is the smallest unit of data write in an SSD.

[0096] Garbage Collection (GC): Deletes older data while retaining the latest data. GC can be a basic function of hard drives, operating on EUs in the hard drive to reclaim space. Currently, GC is a basic function of SSDs, with blocks in the SSD as the erase unit, and GC erases blocks in the SSD to reclaim space.

[0097] FIG1 shows an exemplary architecture diagram of a distributed storage system used in an embodiment of the present invention. As shown in FIG1 , the distributed storage system includes a client 101 , a server 102 , and a hard disk 103 .

[0098] The client 101 is used to determine a data operation request. The data operation request can be generated by processing an external data access request or a data access request generated within the distributed storage system. For example, the processing may include metadata management, data deduplication, data compression, data verification, virtualized storage space, and address translation.

[0099] The server 102 is configured to receive data operation requests from the client and, based on the data operation requests, perform operations on the hard disk 103. For example, the operations performed on the hard disk 103 may include reading data from the hard disk 103, writing data to the hard disk 103, or deleting data from the hard disk 103. In a specific implementation, the server 102 may process the data operation requests through a local storage engine to perform the operations on the hard disk 103.

[0100] The hard disk 103 is used to provide storage resources, such as storing data. It can be a magnetic disk or other type of storage medium, such as a solid-state drive (SSD) or a shingled magnetic recording hard disk. It should be noted that the hard disk 103 is merely an example of a non-volatile storage device and does not constitute a specific limitation. In actual applications, other non-volatile storage devices can also be used. The embodiments of this application use the hard disk 103 as an example to illustrate the solution.

[0101] For the hard disk 103, the hard disk 103 is divided into multiple erase units (EU). It should be noted that EU is the minimum granularity of recycling. Specifically, the hard disk 103 performs an erase operation with EU as the granularity. More specifically, the data in the entire EU will be erased at one time. The hard disk 103 has a variety of data storage methods. Under different data storage methods, EU can have other names, or have different meanings, or different space sizes. In addition, EU is the name of the granularity of recycling in the embodiment of the present application, and does not have a special meaning. It is only for the convenience of description. In some possible implementations, EU can also be referred to as various names such as recycling basic unit, basic recycling unit, etc., and the embodiment of the present application does not make specific limitations on this.

[0102] In one possible way, the data storage method of Flexible Direct Placement (FDP) can be adopted. FDP exposes some internal information of the hard disk 103 to the client 101, including the reclaim unit (RU) and the reclaim group (RG). In this scenario, EU is one RU. Among them, RU is an erased unit of the hard disk 103. For example, when the hard disk 103 is an SSD, one RU is one Block. RG is a performance isolation unit, which includes multiple RUs. As shown in Figure 3a, when the client 101 writes data to the hard disk 103, it carries the RG identifier (called RGID) and the RU identifier (called PID), so that the data is written to the RU unit in the specific RG through the RUH corresponding to the PID. In addition, the client 101 can also query the internal garbage amount of the RU and RG.

[0103] In another possible approach, a multi-stream data storage method can be used. Multi-stream is a hint exposed to the outside of the hard disk 103. Its use is as follows: divide the data life cycle, and assign different life cycles to different streams (the erase units EU of the hard disk 103), so that the data is arranged inside the hard disk 103 in the form shown in Figure 3b; when deleting data, since each stream is a continuous area, the continuous area inside the hard disk 103 can be invalidated. In the multi-stream scenario, EU can be a stream.

[0104] In one possible way, a data storage method of Zoned Namespace (ZNS) can be adopted. ZNS divides the storage area inside the entire hard disk 103 into multiple zones (erase units EU of the hard disk 103). Each zone manages a segment of LBA and only allows sequential writing. Similar to multi-streaming, ZNS also expects applications to divide data into life cycles, and assign different life cycles to different zones. Since each zone is a continuous area, the continuous area inside the hard disk 103 can be invalidated when deleted. In the ZNS scenario, EU can be a zone.

[0105] In the related art, data is constantly updated during the distributed storage process. After the data is updated, the old version of the data becomes garbage data. Distributed storage can use block storage. For block storage, all data is divided into blocks of fixed size, and each block (which can be referred to as a data unit in the embodiment of the present application) is assigned a number for addressing. Taking a mechanical hard disk as an example, a block is a sector, and a sector is 512 bytes or 4K bytes. Some hard disks use a number consisting of a cylinder-head-sector (CHS) number for addressing, and some hard disks use a logical block number (LBA) for addressing. Of course, in addition to hard disks, there are other block devices, such as floppy disks of different specifications, optical disks of various specifications, magnetic tapes, etc. In the scenario of block storage using LBA addressing, for client 101, the definition of garbage data is for data units. For example, if data is repeatedly written to an object, the data unit to which the object belongs will accumulate garbage data. For the hard disk 103, the definition of garbage data is based on the logical block address (LBA); for example, overwriting the same logical block address (LBA) and abnormal data units appearing inside will accumulate garbage data.

[0106] In the related art, during the process of deleting a data unit by the client 101 of the distributed storage system, the hard disk 103 performs GC relocation on the data unit to be deleted. GC relocation can be understood as the process of autonomous GC relocation of the hard disk. GC relocation will retain the valid data identified in the hard disk 103 and delete the junk data. During the GC process, before the hard disk 103 performs GC relocation, it is also necessary to write the valid data of the data unit to be deleted. The server 102 of the distributed storage system identifies the data in the EU that does not need to be deleted, thereby ensuring data security. Since the client 101 and the hard disk 103 have different definitions of junk data, the data unit maintained by the client is not the EU inside the hard disk; and the GC relocation of the hard disk 103 is erased in units of EU. Since the data unit and the EU are not compatible, the server 102 of the distributed storage system will identify the EU as containing data that does not need to be deleted. Only after these data are migrated can the EU be GCed, thereby ensuring data security. Therefore, the GC process of the hard disk 103 will cause write amplification. The write amplification coefficient is defined as data written by the medium / data written by the client. Assuming a write amplification factor of 2.5, a possible scenario is: the client writes 1MB of data, and the SSD internally writes 2.5MB of data to the SSD media. The SSD's internal writing of 2.5MB of data involves writing 1MB of client data, 1.5MB of GC relocation data, and additional media reads to allow the client to write 1.5MB of data.

[0107] In related technologies, in order to reduce write amplification, it is considered that the amount of valid data of the data unit to be deleted cannot be adjusted and controlled, but the amount of data that needs to be relocated in the EU can be adjusted and controlled; for example, a multi-stream data storage method can be adopted. In the multi-stream storage method, when the client 101 deletes the data unit, it deletes it based on the granularity of the life cycle. Since the data of one life cycle is stored in one stream, the stream in the hard disk 103 is composed of continuous areas, and the data is written sequentially to the continuous areas in the stream. In this way, the EU of the hard disk 103 and the data unit that the client needs to delete are adapted; therefore, before the GC relocation of the data unit to be deleted in the hard disk 103, after the valid data of the data unit to be deleted is written, no additional media reading is required, thereby reducing the amount of data relocated in the hard disk 103 during the GC process and reducing write amplification. The principle of reducing write amplification by the ZNS data storage method is similar.

[0108] However, the division of life cycles is very complex, and the embodiments of the present application provide other garbage collection methods that can reduce write amplification.

[0109] In actual applications, users access the distributed storage system through the client 101. In some possible scenarios, users access the distributed storage system through terminal devices. In other possible scenarios, the distributed storage system is deployed in the cloud, and users access the cloud management platform through terminal devices to access the client 101, thereby accessing the distributed storage system. Among them, the terminal device can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. Exemplary embodiments of the terminal devices involved in the embodiments of the present invention include but are not limited to electronic devices equipped with iOS, Android, Windows, Harmony OS or other operating systems. The embodiments of the present invention do not specifically limit the type of electronic device. Among them, the cloud management platform can be a separate electronic device or a device integrated into the electronic device. The embodiments of the present invention do not limit the specific deployment method and deployment location of the cloud management platform.

[0110] In one feasible scenario, the client 101, server 102, and hard disk 103 are deployed in an integrated manner. In this scenario, the distributed storage system 100 includes an application server cluster. As shown in FIG2a , the application server cluster includes one or more application servers 110 ( FIG1 shows two application servers 110, but is not limited to two application servers 110), and each application server 110 can communicate with each other. An application server 110 is a device that has both computing and storage capabilities, such as a server or desktop computer. In terms of software, each application server 110 has an operating system. A virtual machine 104 can be created on the application server 110. The computing resources required by the virtual machine 104 come from the local processor 111 and memory 112 of the application server 110, while the storage resources required by the virtual machine 104 can come from the local hard disk 103 of the application server 110 or from the hard disks 103 of other application servers 110. In addition, various applications can run in the virtual machine 104, and users can trigger read / write data requests through the applications in the virtual machine 104. In one example, the virtual machine 104 can serve as the client 101.

[0111] In terms of hardware, as shown in Figure 2a, the application server 110 includes at least a processor 111, memory 112, a network card 113, and a hard disk 103. The processor 111, memory 112, network card 113, and hard disk 103 are connected via a bus. The processor 111 and memory 112 are used to provide computing resources. Specifically, the processor 111 is a central processing unit (CPU) that processes data access requests from outside the application server 110 or requests generated within the application server 110. For example, when the processor 111 receives write data requests from users, it temporarily stores the data in these write data requests in the memory 112. When the total amount of data in the memory 112 reaches a certain threshold, the processor 111 sends the data stored in the memory 112 to the hard disk 103 for persistent storage. In addition, the processor 111 is also used for data calculation or processing, such as metadata management, data deduplication, data compression, data verification, virtualized storage space, and address translation. Figure 2a shows only one CPU 111. In actual applications, there are often multiple CPUs 111, wherein one CPU 111 has one or more CPU cores. This embodiment does not limit the number of CPUs or CPU cores.

[0112] Memory 112 refers to internal storage that directly exchanges data with the processor. It can read and write data at any time and at a high speed, and serves as temporary data storage for the operating system or other running programs. Memory includes at least two types of memory. For example, memory can be either random access memory (RAM) or read-only memory (ROM). For example, random access memory is dynamic random access memory (DRAM) or storage class memory (SCM). DRAM is a type of semiconductor memory that, like most random access memory (RAM), is a volatile memory device. SCM is a hybrid storage technology that combines the characteristics of traditional storage devices and memory. Storage class memory can provide faster read and write speeds than hard drives, but has slower computing speeds than DRAM and is also cheaper than DRAM. However, DRAM and SCM are merely exemplary in this embodiment; memory can also include other random access memories, such as static random access memory (SRAM). As for the read-only memory, for example, it can be a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), etc. In addition, the memory 113 can also be a dual in-line memory module or a dual-line memory module (DIMM), that is, a module composed of dynamic random access memory (DRAM), or a solid-state drive (SSD). In actual applications, multiple memories 112 and different types of memories 112 can be configured in the computing node 110. This embodiment does not limit the number and type of memory 112. In addition, the memory 112 can be configured to have a power-saving function. The power-saving function means that when the system loses power and then powers on again, the data stored in the memory 112 will not be lost. A memory with a power-saving function is called a non-volatile memory.

[0113] The network card 113 is used to communicate with other application servers 110 .

[0114] In a feasible scenario, the client 101, the server 102 and the hard disk 103 are deployed separately. In this scenario, the distributed storage system 100 includes a computing node cluster and a storage node cluster. As shown in Figure 2b, the computing node cluster includes one or more computing nodes 120 (two computing nodes 120 are shown in Figure 2b, but are not limited to two computing nodes 120). Each computing node 110 can communicate with each other, and the computing node 110 deploys the client 101. The computing node 120 is a computing device, such as a server, a desktop computer, or a controller of a storage array. In terms of hardware, as shown in Figure 2b, the computing node 120 includes at least a processor 121, a memory 122, and a network card 123. Among them, the details of the processor 121 refer to the description of the processor 111 above and will not be repeated here. The details of the memory 122 refer to the description of the memory 112 above and will not be repeated here.

[0115] The network card 123 is used to communicate with the storage node 130. For example, when the total amount of data in the memory 122 reaches a certain threshold, the computing node 120 can send a request to the storage node 130 through the network card 123 to perform persistent storage on the data. In addition, the computing node 120 may also include a bus for communication between the components inside the computing node 120. Functionally, since the main function of the computing node 120 in Figure 2b is computing business, remote storage can be used to achieve persistent storage when storing data, so it has less local storage than a conventional server, thereby achieving cost and space savings. However, this does not mean that the computing node 120 cannot have local storage. In actual implementation, the computing node 120 can also have a small number of hard disks built in, or a small number of external hard disks.

[0116] Any computing node 120 can access any storage node 130 in the storage node cluster through the network. The storage node cluster includes multiple storage nodes 130 (two storage nodes 130 are shown in Figure 2b, but are not limited to two storage nodes 130). The storage node 110 deploys a server 102, and a storage node 130 includes one or more controllers 131 (deployed with a server 102), a network card 132 and multiple hard disks 103. The network card 132 is used to communicate with the computing node 120. The hard disk 103 is used to store data. The controller 131 is used to write data to the hard disk 103, or read or delete data from the hard disk 103 according to the read / write data request sent by the computing node 120. In the process of reading and writing data, the controller 131 needs to convert the address carried in the read / write data request into an address that the hard disk can recognize. It can be seen that the controller 131 also has some simple computing functions.

[0117] The network may be a wired network or a wireless network. For example, the wired network may be a cable network, a fiber optic network, a digital data network (DDN), etc., and the wireless network may be a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a ZigBee network, a global system for mobile communications (GSM) network, etc., or any combination thereof. It will be understood that the network may use any known network communication protocol to implement communication between different client layers and gateways. The above-mentioned network communication protocol may be various wired or wireless communication protocols, such as Ethernet, universal serial bus (USB), Firewire, global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), new radio (NR) and other communication protocols.

[0118] In actual applications, the control unit 131 can have various forms. In one case, the control unit 131 includes a CPU and memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store data to be written to the hard disk 103, or data read from the hard disk 103 to be sent to the computing node 120. In another case, the control unit 131 is a programmable electronic component, such as a data processing unit (DPU). The DPU has the versatility and programmability of the CPU, but is more specialized and can run efficiently on network data packets, storage requests or analysis requests. The DPU is distinguished from the CPU by a greater degree of parallelism (needing to process a large number of requests). Optionally, the DPU here can also be replaced by a processing chip such as a graphics processing unit (GPU) or an embedded neural network processor (NPU). Generally, the number of control units 131 can be one, two or more. When a storage node 130 includes at least two control units 131, the hard disks 103 and the control units 131 have an ownership relationship. Each controller can only access its own hard disks. This often involves forwarding read / write data requests between the control units 131, resulting in a long data access path. Furthermore, if storage space is insufficient, adding a new hard disk 103 to the storage node 130 requires rebinding the ownership relationship between the hard disk 103 and the controller 131, which is complex and results in poor storage space scalability.

[0119] Therefore, in another embodiment, the functions of the control unit 131 can be offloaded to the network card 132. In other words, the storage node 130 does not have a controller 131 inside, but the network card 132 performs data reading and writing, address conversion and other computing functions. In this case, the network card 132 is an intelligent network card. It can include a CPU and memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store data to be written to the hard disk 103, or data read from the hard disk 103 to be sent to the computing node 120. It can also be a programmable electronic component, such as a data processing unit (DPU). There is no ownership relationship between the network card 123 and the hard disk 103 in the storage node 130. The network card 132 can access any hard disk 103 in the storage node 130, so it is more convenient to expand the hard disk when the storage space is insufficient.

[0120] It should be understood that the distributed storage system applicable to the embodiment of the present invention may also be applicable to other distributed storage systems besides the distributed storage system shown in FIG. 1 , and the embodiment of the present invention does not limit this.

[0121] Next, a garbage collection method provided by an embodiment of the present invention is introduced in detail in conjunction with the distributed storage system described above.

[0122] In a specific implementation, for the hard disk 103, an EU can be a block, a stream, or a zone; the disk 103 exposes information to the outside (referred to as externally exposed information in this embodiment of the application), for example, the externally exposed information may include the amount of garbage in the EU (for ease of description and distinction, it can be referred to as the second garbage amount). The second garbage amount of the EU represents the proportion of garbage data contained in the EU by the hard disk 103 in the EU's capacity.

[0123] Correspondingly, the client 101 has maintenance information (for the sake of ease of description and distinction, referred to as first maintenance information in this embodiment of the application). As shown in Figures 1, 3a, and 3b, the first maintenance information is used to indicate the capacity of the data unit (for the sake of ease of description and distinction, it can be referred to as Plog), the amount of garbage in the Plog (for the sake of ease of description and distinction, it can be referred to as the first amount of garbage), the identifier of the hard disk 103 (referred to as HDID in this embodiment of the application), the identifier of the EU (referred to as EUID in this embodiment of the application), the data unit identifier (referred to as PlogID in this embodiment of the application), and the second amount of garbage in the EU.

[0124] Correspondingly, for the server 102, there is maintenance information (for the sake of convenience of description and distinction, this embodiment of the application is referred to as the second maintenance information). Based on the externally exposed information of the hard disk 103, as shown in Figures 1, 3a, and 3b, the second maintenance information can be used to record the EUID, PlogID, and second garbage amount of the EU. Furthermore, it can also include the capacity of the EU, the remaining capacity (remaining available capacity) and other information.

[0125] It should be noted that the first garbage amount and the second garbage amount are named only for the convenience of description and distinction and do not have any special meaning. The first garbage amount can also be expressed as the garbage amount of Plog, and the second garbage amount can also be expressed as the garbage amount of EU.

[0126] FIG4 is a flow chart of a garbage collection method provided in an embodiment of the present application. This embodiment can be applied to a distributed storage system, specifically a computing device cluster. This embodiment of the present application uses a storage device, such as hard disk 103, to describe garbage collection. As shown in FIG4 , the garbage collection method provided in an embodiment of the present application includes at least the following steps:

[0127] Step 410 : The client 101 determines at least one Plog to be deleted from the multiple Plogs in the hard disk 103 based on its own first maintenance information.

[0128] It should be noted that a Plog is the smallest unit of data storage used by client 101. A Plog can be understood as a Plog used by client 101 for data storage, or as a Plog whose first garbage volume exceeds a first garbage volume threshold. In a specific implementation, if client 101 detects that the storage space on hard disk 103 is insufficient, or that some Plogs contain a large amount of garbage, deletion of the Plog will be triggered.

[0129] According to a possible scenario, in step 410, the principles for selecting the Plog to be deleted include the following two aspects:

[0130] First, when the amount of valid data in the Plog to be deleted is as large as possible, the second garbage amount of the EU to which the Plog to be deleted belongs is as large as possible. The amount of valid data in the Plog to be deleted is equal to the difference between the capacity of the Plog to be deleted (the size of one Plog) and the first garbage amount of the Plog to be deleted. In this way, the less valid data in the Plog to be deleted and the larger the second garbage amount of the EU to which it belongs, the less valid data of the Plog to be deleted and the less valid data within the EU that needs to be relocated within the hard disk 103, which can reduce the data relocation overhead within the hard disk 103 to a certain extent.

[0131] Secondly, when the EU's second garbage volume is as large as possible, the EU contains as many Plogs to be deleted as possible and / or the amount of valid data in the Plogs to be deleted as possible. Thus, the more Plogs to be deleted the EU contains, the fewer Plogs that need to be relocated from the EU within hard disk 103 that do not need to be deleted, which can reduce the data relocation overhead within hard disk 103 to a certain extent. The larger the amount of valid data in the Plogs to be deleted the EU contains, the less valid data in the Plogs to be deleted that needs to be relocated from hard disk 103, which can reduce the data relocation overhead within hard disk 103 to a certain extent.

[0132] Among them, the case where the amount of valid data of the Plog to be deleted is as large as possible may include that the amount of valid data of the Plog to be deleted is greater than a preset valid data amount threshold (in some possible cases, it can be called the first threshold), or, it is the minimum value among the amounts of valid data of the Plog to be deleted.

[0133] Among them, the situation where the second garbage amount of the EU is as large as possible may include the second garbage amount of the EU being greater than a preset second garbage amount threshold (in some possible cases, it may be called the second threshold), or the EU with the largest second garbage amount among the EU to which the Plog to be deleted belongs.

[0134] Among them, the situation where the EU contains as many Plogs to be deleted as possible may include that every Plog in the EU is a Plog to be deleted, or the number of Plogs to be deleted contained in the EU is greater than or equal to a data unit number threshold (in some possible cases, it can be called a third threshold).

[0135] It should be noted that, in the scenario where Plog is located in multiple EUs, the embodiment of the present application does not need to consider the number of EUs to which the Plog belongs, but only needs to consider the second garbage amount of a single EU and the Plog to be deleted contained in the EU.

[0136] In this case, a feasible implementation method is that the EU to which the Plog to be deleted belongs meets the following conditions:

[0137] The second garbage amount of the EU to which the Plog to be deleted belongs is greater than or equal to the garbage amount threshold; or, the EU to which the Plog to be deleted belongs is the EU with the second largest garbage amount among the EUs to which the Plog belongs.

[0138] And / or, the amount of valid data in the Plog to be deleted meets any of the following conditions:

[0139] The data volume of the valid data of the Plog to be deleted is less than or equal to the valid data volume threshold; or, the data volume of the valid data of the Plog to be deleted is the minimum value among the data volumes of the valid data of multiple Plogs.

[0140] In the first example, there are multiple Plogs to be deleted, the multiple Plogs to be deleted are located in the same EU, and the second garbage amount of the EU to which the multiple Plogs to be deleted belong is greater than the garbage amount threshold.

[0141] In the second example, multiple Plogs to be deleted are located in the same EU, which is the EU with the second largest amount of garbage among the EUs to which the multiple Plogs belong.

[0142] In the third example, the amount of valid data in the Plog to be deleted is less than or equal to the valid data amount threshold.

[0143] In the fourth example, the multiple Plogs to be deleted include a Plog having the smallest amount of valid data among the multiple Plogs to be deleted.

[0144] In the fifth example, the above first or second example is combined with the third or fourth example.

[0145] It should be noted that the above examples are only partial examples and do not constitute a specific limitation. In actual applications, the method of determining the Plog to be deleted from the Plog can be set in combination with actual conditions.

[0146] Correspondingly, there may be multiple ways to determine the Plog to be deleted in step 410 .

[0147] In a feasible implementation, correspondingly, step 410 may include:

[0148] Based on the first garbage amounts of the multiple Plogs, client 101 determines at least one Plog to be deleted from the multiple Plogs with the largest first garbage amount. For example, a preset number of Plogs with the largest first garbage amounts can be selected as the Plogs to be deleted. The preset number can be set based on actual needs, such as three. For another example, multiple Plogs with a garbage amount greater than or equal to a first garbage amount threshold can be selected as the Plogs to be deleted.

[0149] In a feasible implementation, correspondingly, step 410 may include:

[0150] The client 101 determines at least one Plog to be deleted from the multiple Plogs in the same EU based on the EUIDs of the EUs to which the multiple Plogs belong. For example, each of the multiple Plogs in the same EU can be used as a Plog to be deleted. In some possible scenarios, each of the multiple EUs may contain multiple Plogs. In this case, each of the multiple Plogs in each EU can be used as a Plog to be deleted.

[0151] In a feasible implementation, correspondingly, step 410 may include:

[0152] Based on the second garbage amount of the EUs to which the multiple Plogs belong, the client 101 determines at least one Plog to be deleted from the EU with the second largest garbage amount. For example, each Plog in the EU with the second largest garbage amount can be used as a Plog to be deleted. There can be one or more Plogs in the EU with the second largest garbage amount. In some possible situations, there may be multiple EUs, each of which contains multiple Plogs. In this case, each Plog in the multiple Plogs in the EU with the second largest garbage amount among the multiple EUs can be used as a Plog to be deleted.

[0153] In a possible example of this implementation, step 410 may include: the client 101 determines several EUs containing at least 2 Plogs based on the EUs to which multiple Plogs belong and the second garbage amount of the EUs. When there is only one EU, each Plog in the EU is used as a Plog to be deleted; when there are multiple EUs, the EU with the largest second garbage amount can be selected, and each Plog in the EU with the largest second garbage amount can be used as a Plog to be deleted.

[0154] In one feasible implementation, step 410 may include: client 101 determining, based on the first garbage amount and capacity of the multiple Plogs, at least one Plog to be deleted from the multiple Plogs with the smallest amount of valid data. For example, a preset number of Plogs with the smallest amount of valid data may be selected as the Plogs to be deleted. The preset number can be set based on actual needs, such as three. For another example, multiple Plogs with a valid data amount less than or equal to a threshold may be selected as the Plogs to be deleted.

[0155] In a feasible implementation, step 410 may include: the client 101 determines the EU to which the Plog with the smallest amount of valid data belongs based on the first garbage amount, capacity and EU to which the multiple Plogs belong, and sets each Plog in the EU as a Plog to be deleted.

[0156] In one feasible implementation, in step 410, the garbage collection overhead of a Plog can be measured to select the Plog to be deleted. In this case, step 410 may include: for each of the multiple Plogs, determining an overhead metric value corresponding to the Plog based on the amount of valid data in the Plog and the second amount of garbage in the EU to which the Plog belongs; the amount of valid data is the difference between the capacity of the Plog and the first amount of garbage; and determining at least one Plog to be deleted based on the overhead metric value corresponding to each Plog. The smaller the amount of valid data and the larger the second amount of garbage, the smaller the overhead metric value; otherwise, the larger the overhead metric value.

[0157] In specific implementation, a measurement function can be constructed through the capacity of Plog, the first garbage amount and the second garbage amount of the EU where it is located; subsequently, the function value calculated by the measurement function can be used as the overhead measurement value.

[0158] For example, the metric function can be expressed by the following formula (1): y=k1(P size –P gsize )+k2EU gsize (1)

[0159] Here, k1 and k2 are constants that can be set based on actual needs. For example, k1 + k2 equals 1. For example, if the effective data volume of Plog is emphasized, k1 can be set to be greater than k2; if the secondary garbage volume of the EU to which Plog belongs is emphasized, k2 can be set to be greater than k1.

[0160] For example, when the client 101 knows the capacity EU size In the case of , the metric function can be expressed by the following formula (2): y=k1(P size –P gsize)+k2(EU size –EU gsize ) (2)

[0161] For example, the client 101 may know the capacity EU of EU size , the time for the client to read or write a unit of capacity from the hard disk 103: Net_time, and the data migration overhead per unit of capacity within the hard disk 103: HD_time, then the metric function can be expressed by the following formula (3): y = k1 (P size –P gsize )*Net_time*2+k2(EU size –EU gsize )*HD_time*2 (3)

[0162] It should be noted that the above metric functions are merely examples and do not constitute a specific limitation.

[0163] There are multiple ways to determine the data unit identifier of the Plog to be deleted based on the overhead metric value corresponding to each Plog. A larger overhead metric value indicates a larger overhead.

[0164] In an optional manner, the EU to which the Plog corresponding to the minimum cost metric value belongs is determined; and each Plog in the EU is used as a Plog to be deleted.

[0165] In another optional manner, the EU to which the Plog corresponding to the cost metric value that is less than or equal to the cost metric threshold belongs is determined; and each Plog in the EU is treated as a Plog to be deleted.

[0166] In yet another optional manner, a Plog corresponding to a cost metric value that is less than or equal to a cost metric threshold is determined as the Plog to be deleted.

[0167] Step 420: The client 101 sends a first data deletion request to the server 102. The first data deletion request includes a PlogID of at least one Plog to be deleted.

[0168] Step 430 : The server 102 adds the PlogID of at least one Plod to be deleted to the corresponding data unit identifier set of the hard disk 103 .

[0169] Exemplarily, the data unit identification set may be represented as a deletion linked list. The Plog indicated by the data unit identification set needs to be marked for deletion. The latest data of the Plog marked for deletion may be restored later if necessary.

[0170] In actual applications, the data unit identification set is used to store the PlogID of at least one data unit to be deleted included in the data deletion requests sent multiple times by the client 101. Exemplarily, the data unit identification set also includes the identification of at least one data unit to be deleted included in the second data deletion request sent by the client. It should be noted that the second data deletion request indicates the data unit to be deleted determined in the last data deletion request.

[0171] Step 440 : The server 102 instructs the hard disk 103 to recycle the multiple Plogs to be deleted in the target EU based on the second maintenance information and the data unit identification set corresponding to the hard disk 103 .

[0172] In specific implementation, in step 440, the server 102 detects whether the hard disk 103 has the target EU based on the second maintenance information and the data unit identification set corresponding to the hard disk 103. If so, the hard disk 103 is instructed to recycle multiple Plogs to be deleted in the target EU.

[0173] It should be noted that there can be one or more target EUs, and this embodiment of the present application does not make any specific limitations on this, and the specific number can be determined based on actual needs.

[0174] According to one feasible implementation, step 440 may include: the server 102 determining a target EU with the most Plogs to be deleted from the multiple EUs based on the PlogIDs and data unit identifiers of the Plogs included in the multiple EUs. It should be noted that the more Plogs to be deleted, the fewer Plogs that need to be migrated and not deleted in the target EU before the hard disk 103 performs GC relocation on the target EU, and the lower the relocation overhead.

[0175] Exemplarily, the PlogID of each Plog in the target erase unit is located in the data unit identification set.

[0176] According to one feasible implementation, step 440 may include the server 102 determining a target EU with the largest second garbage volume from the multiple EUs based on the PlogIDs and data unit identifiers of the Plogs included in the multiple EUs. It should be noted that the larger the second garbage volume of the EU, the less valid data in the target EU is recognized internally by the hard disk 103 during GC relocation of the target EU by the hard disk 103, and the lower the relocation overhead.

[0177] Exemplarily, the PlogID of each Plog in the target EU is located in the data unit identification set, and the second garbage amount is the largest.

[0178] Exemplarily, the PlogIDs of at least two Plogs in the target EU are located in the data unit identifier set, and the second EU has the largest amount of junk. For the two aforementioned implementations, in one possible scenario S1, the target EU can be the first EU, or the first EU with the highest amount of junk among multiple first EUs, and the PlogID of each Plog in each first EU is located in the data unit identifier set. It should be noted that the first EU is simply a name given for ease of description and distinction and does not have any special meaning.

[0179] In specific implementation, in some possible scenarios, when the hard disk 103 does not issue an alarm for insufficient storage space, the server 102 detects whether there is a first EU among the several EUs where the Plog to be deleted indicated by the data unit identification set is located based on the PlogID of the Plog to be deleted indicated by the data unit identification set and the EU to which the Plog belongs. If not, it continues to receive the PlogID of the Plog to be deleted sent by the client 101. If yes, when one first EU is detected, the first EU is used as the target EU; when multiple first EUs are detected, the first EU with the second largest amount of garbage is used as the target EU.

[0180] In one possible scenario S2, the target EU is the second EU with the largest second garbage volume among multiple second EUs; wherein, each second EU contains at least 2 Plogs whose PlogIDs are located in the data unit identifier set. In some possible scenarios, the second garbage volume of each second EU is greater than or equal to the second garbage volume threshold; or, the storage space proportion of the Plogs whose PlogIDs in the second EU are located in the data unit identifier set is greater than or equal to the storage space threshold. It should be noted that the second EU is named only for the convenience of description and distinction and does not have any special meaning. In specific implementation, in some possible scenarios, when the hard disk 103 does not issue an alarm for insufficient storage space, the server 102 detects whether there is a second EU among the several EUs to which the Plog to be deleted indicated by the data unit identification set belongs based on the PlogID of the Plog to be deleted indicated by the data unit identification set and the EU to which the Plog belongs. If not, it continues to receive the PlogID of the Plog to be deleted sent by the client 101. If so, when it is detected that there is only one second EU, it is determined as the target EU; when it is detected that there are multiple second EUs, the second EU with the largest amount of second garbage is used as the target EU.

[0181] In other possible situations, the hard disk 103 issues an alarm for insufficient storage space. In this case, the server 102 detects whether a first EU exists. If so, when one first EU is detected, the first EU is used as the target EU; when multiple first EUs are detected, the first EU with the second largest amount of garbage is used as the target EU; if not, the server 102 detects whether a second EU exists. If a second EU is detected, when only one second EU is detected, it is determined as the target EU; when multiple second EUs are detected, the second EU with the second largest amount of garbage is used as the target EU. If neither the first EU nor the second EU exists, the second EU with the second largest amount of garbage is directly determined as the target EU.

[0182] There are two situations in the process of recycling multiple PIogs to be deleted in the target EU.

[0183] In one possible scenario, if the PlogID of each Plog stored in the target EU is in the data unit identifier set, the server 102 can instruct the storage device to relocate the valid data identified by the hard disk 103 in the target EU to other EUs and erase the target EU.

[0184] In another possible scenario, the target EU stores a valid Plog (PlogID is in the data unit identifier set). The server 102 can instruct the storage device to relocate the valid Plog data in the target EU to other EUs, then migrate the valid data identified by the hard disk 103 to other EUs, and then erase the target EU.

[0185] As shown in FIG5 , based on steps 410 to 440 shown in FIG4 , in the embodiment of the present application, step 440 includes the following steps:

[0186] Step 441a: The server 102 determines the garbage collection command corresponding to the target EU under the condition that the PlogID of each Plog stored in the target EU is in the data unit identifier set; the garbage collection command includes the EUID of the target EU.

[0187] Step 441b1: When the target EU stores a valid Plog, the server 102 determines a read instruction corresponding to the valid Plog; the PlogID of the valid Plog is a PlogID other than the PlogIDs in the data unit identifier set.

[0188] Step 441b2: The server 102 sends a read instruction to the hard disk 103.

[0189] Step 441b3: The first disk 103 sends the valid Plog read out based on the read instruction to the server 102.

[0190] Step 441b4: The server 102 sends a write instruction to the hard disk 103.

[0191] Step 441b5: The first disk 103 writes the valid Plog into the EU based on the write instruction.

[0192] In actual applications, the first disk 103 may write the valid Plog into the third EU.

[0193] Step 441b6: The first disk 103 sends a write success notification to the server 102.

[0194] Step 441b7: The server 102 determines the garbage collection command corresponding to the target EU; the garbage collection command includes the EUID of the target EU.

[0195] Step 442 : The server 102 sends a garbage collection command corresponding to the target EU to the hard disk 103 .

[0196] Step 443: The hard disk 103 performs garbage collection on the target EU based on the garbage collection command.

[0197] In one possible scenario, the garbage collection process for the target EU includes: migrating valid data identified by the hard disk 103 in the target EU to another EU, such as an empty EU, which can be called a fourth EU, and erasing the target EU.

[0198] Step 444 : The hard disk 103 sends an erasure success notification to the server 102 .

[0199] Step 445 : For a PlogID in the data unit identifier set, the server 102 deletes the PlogID if the Plog corresponding to the PlogID is only located in the target EU.

[0200] It should be noted that a Plog can be located in several EUs, for example, in one EU or in multiple EUs. Therefore, for a PlogID in the data unit identifier set, if the Plog corresponding to the PlogID is also located in an EU other than the target EU, the PlogID needs to be retained so that the other EUs where the PlogID is located can be erased later, achieving phased deletion.

[0201] In this solution, the client 101 considers the Plog and garbage amount in the EU exposed to the hard disk when selecting the Plog to be deleted. The server 102 adds the PlogID of the Plog to be deleted sent by the client 101 to the data unit identification set, and performs garbage collection on the EU containing multiple Plogs to be deleted indicated by the data unit identification set, thereby reducing the data that needs to be additionally relocated, reducing write amplification, and reducing data relocation overhead.

[0202] Based on this, the technical ideas for deriving the garbage collection method proposed in the embodiment of this application are described.

[0203] The current cost of recycling a Plog is the cost of recycling a Plog by the client 101 and the cost of recycling EUs within the hard disk 103: (P_size–P_gsize)*Net_time*2+(EU_size–EU_gsize)*HD_time*2.

[0204] Among them, P_size represents the size of Plog; P_gsize represents the amount of garbage in Plog; Net_time represents the time it takes for the client to read or write a unit capacity from the hard disk 103. In the scenario where the distributed storage system adopts the integrated deployment shown in Figure 2a, Net_time can be equal to HD_time; in the scenario where the distributed storage system adopts the separated deployment shown in Figure 2b, Net_time can represent the sum of the network transmission speed per unit capacity and HD_time; EU_size represents the capacity of EU; EU_gsize represents the valid data of EU; HD_time represents the data migration overhead per unit capacity within the hard disk 103, that is, the time it takes for the hard disk 103 to read or write data per unit capacity.

[0205] It should be noted that the cost of client 101 recycling a plog includes: the cost of reading the plog's valid data: (P_size – P_gsize) * Net_time; the cost of writing the plog's valid data to a new plog: (P_size – P_gsize) * Net_time; therefore, the cost of client 101 recycling a plog is (P_size – P_gsize) * Net_time * 2. The cost of recycling EUs within hard disk 103 is: the cost of reading EU valid data: (EU_size – EU_gsize) * HD_time; the cost of writing EU valid data: (EU_size – EU_gsize) * HD_time; therefore, the cost of recycling EUs within hard disk 103 is (EU_size – EU_gsize) * HD_time * 2.

[0206] The embodiment of the present application reduces the data size of valid data in multiple plogs (P_size-P_gsize) and the data size of valid data in an EU (EU_size-EU_gsize), so that the client 101 can minimize the data migration overhead in the hard disk 103 when garbage collection occurs.

[0207] It should be noted that valid data within an EU can include data deemed valid within hard drive 103. In scenarios where an EU includes Plogs other than those to be deleted, valid data within the EU can also include Plogs that do not need to be deleted. Therefore, the more valid data in an EU, the more valid data that needs to be read from and written to the EU within hard drive 103, resulting in greater data migration overhead. To reduce write amplification, it is necessary to minimize the valid data in multiple Plogs and the valid data in the EU.

[0208] Specifically, when selecting Plogs to delete, client 101 considers the following: While minimizing P_size – P_gsize, reduce EU_size – EU_gsize. For example, if multiple Plogs to be deleted are located in the same EU with a high garbage volume, one or more Plogs in a EU with a high garbage volume may be selected for garbage collection.

[0209] Furthermore, the server 102 writes the PlogID of the Plog to be deleted sent by the client 101 into the data unit identification set. When the garbage collection command (for example, the Trim command) is actually issued, it tries to find all the Plogs in the EU with a large amount of garbage among all the PlogIDs in the data unit identification set for garbage collection, thereby further reducing the overhead of erasing data; if it cannot be found, it tries to find more Plogs in the EU with a large amount of garbage among all the PlogIDs in the data unit identification set for garbage collection.

[0210] Based on the garbage recycling method provided above, a specific application of the garbage recycling method is described. FIG6 is a flow chart of a specific application of a garbage recycling method provided by an embodiment of the present invention.

[0211] As shown in FIG6 , for example, it is assumed that garbage collection is performed on the hard disk 103: HD1.

[0212] For the client 101, it is assumed that the PlogIDs of multiple Plogs are: Plog1, Plog2, Plog3, Plog6, Plog7, Plog8; based on the first maintenance information, it is determined that the Plogs corresponding to Plog1, Plog2, and Plog3 are located in the same EU1, and the amount of garbage in EU1 is also relatively high, for example, higher than the preset second garbage amount threshold (also referred to as the first threshold); in addition, (P_size–P_gsize) of Plog6 is the minimum value among (P_size–P_gsize) of Plog, therefore, the PlogIDs of the Plogs to be deleted can be determined: Plog1, Plog2, Plog3, Plog6; then, the PlogIDs of the Plogs to be deleted: Plog1, Plog2, Plog3, Plog6 are sent to the server 102.

[0213] For server 102, assume that the PlogIDs in the data unit identifier set are: Plog4, Plog5, Plog10. It is known that the second garbage volume of EU1 is R1, and the PlogIDs of the Plogs included are: Plog1, Plog2, Plog3; the second garbage volume of EU2 is R2, and the PlogIDs of the Plogs included are: Plog4, Plog5, Plog6; the second garbage volume of EU3 is R3, and the PlogIDs of the Plogs included are: Plog7, Plog8, Plog9; the second garbage volume of EU4 is R4, and the PlogIDs of the Plogs included are: Plog10, Plog11, Plog12.

[0214] In one possible scenario, as shown in Figure 7a, assuming HD1 does not issue an insufficient storage space warning before writing the PlogIDs of the Plogs to be deleted (Plog1, Plog2, Plog3, and Plog6) to the data unit identifier set, server 102 writes the PlogIDs of the Plogs to be deleted (Plog1, Plog2, Plog3, and Plog6) to the data unit identifier set. The PlogIDs in the data unit identifier set are: Plog1, Plog2, Plog3, Plog4, Plog5, Plog6, and Plog10. The Plogs corresponding to Plog1, Plog2, and Plog3 are located in EU1 and have the same PlogIDs as all Plogs stored in EU1. Plog4, Plog5, and Plog6 are located in the same EU2 and have the same PlogIDs as all Plogs stored in EU2. If EU2's second garbage volume R2 is greater than EU1's second garbage volume R1, HD1 is controlled to perform garbage collection on EU2. For example, HD1 migrates valid data in EU2 to EU5 and then erases EU2.

[0215] In one possible scenario, as shown in Figure 7b, assume that before writing the PlogIDs of the Plogs to be deleted (Plog1, Plog2, Plog3, and Plog6) to the data unit identifier set, HD1 issues an insufficient storage space alarm. The Plogs corresponding to Plog4 and Plog5 are located in EU2, and the Plog corresponding to Plog10 is located in EU4. Since EU2's second garbage volume R2 is less than EU4's second garbage volume R4, HD1 is controlled to perform garbage collection on EU4. Because EU4 contains data that does not need to be recycled, to ensure data security, server 102 controls HD1 to write the Plogs corresponding to Plog11 and Plog12 to EU5. Subsequently, HD1 relocates the valid data in EU4 to EU6 and then erases EU4.

[0216] It should be noted that in the embodiment of the present application, the hard disk 103 is merely an example of a storage device and does not constitute a specific limitation. Any storage device that can be divided into multiple EUs, with EU as the recycling granularity, and has a garbage collection function uses the garbage collection method provided in the embodiment of the present application.

[0217] It is worth noting that the garbage collection method provided in the embodiment of the present application does not describe migrating the valid data in the Plog to be deleted to other EUs. In fact, before the hard disk 103 performs garbage collection, the valid data in the Plog to be deleted has been migrated to other EUs.

[0218] The present application also provides a distributed storage system, which includes a client and a server. The server accesses a storage device, and the storage device includes multiple erasure units. The client includes first maintenance information, which is used to indicate multiple data units and the erasure units to which the multiple data units belong. The server includes second maintenance information, which is used to indicate the multiple erasure units. As shown in FIG8 , the system includes:

[0219] The client is configured to determine at least one data unit to be deleted from the plurality of data units based on the first maintenance information;

[0220] The client is configured to send a first data deletion request, where the first data deletion request includes an identifier of at least one data unit to be deleted;

[0221] The server is configured to receive the first data deletion request and add an identifier of at least one data unit to be deleted to a data unit identifier set;

[0222] The service end is used to instruct the storage device to recycle multiple to-be-deleted data units in the target erasure unit based on the second maintenance information and the data unit identifier set.

[0223] The client and the server can be implemented in software or hardware. For example, the client is used as an example to describe the implementation of the client. Similarly, the implementation of the server can refer to the implementation of the client.

[0224] As an example of a software functional unit, a module may include a code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the client may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0225] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0226] As an example of a hardware functional unit, a module may include at least one computing device, such as a server. Alternatively, the client may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0227] The multiple computing devices included in the client can be distributed in the same region or in different regions. They can be distributed in the same AZ or in different AZs. Similarly, they can be distributed in the same VPC or in multiple VPCs. These multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0228] It should be noted that in other embodiments, the client can be used to execute any step in the garbage collection method, and the server can be used to execute any step in the garbage collection method. The steps that the client and server are responsible for implementing can be specified as needed. The full functions of the distributed storage system can be realized by implementing different steps in the garbage collection method through the client and server respectively.

[0229] This application also provides a computing device 900. As shown in Figure 9, computing device 900 includes a bus 902, a processor 904, a memory 906, and a communication interface 908. Processor 904, memory 906, and communication interface 908 communicate with each other via bus 902. Computing device 900 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 900.

[0230] Bus 902 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG9 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 902 may include a path for transmitting information between various components of computing device 900 (e.g., memory 906, processor 904, and communication interface 908).

[0231] The processor 904 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0232] The memory 906 may include volatile memory, such as random access memory (RAM). The processor 904 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0233] The memory 906 stores executable program codes, and the processor 904 executes the executable program codes to implement the aforementioned server and / or client functions, thereby implementing the garbage collection method. That is, the memory 906 stores instructions for executing the garbage collection method.

[0234] The communication interface 908 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 900 and other devices or a communication network.

[0235] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0236] As shown in Figure 10, the computing device cluster includes at least one computing device 900. The memory 906 in one or more computing devices 900 in the computing device cluster may store the same instructions for executing the garbage collection method.

[0237] In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also store partial instructions for executing the garbage collection method. In other words, the combination of one or more computing devices 900 can jointly execute the instructions for executing the garbage collection method.

[0238] It should be noted that the memory 906 in different computing devices 900 in the computing device cluster can store different instructions, each for executing a portion of the functions of the garbage collection device on the server or client. In other words, the instructions stored in the memory 906 in different computing devices 900 can implement the functions of one or more modules on the client and server.

[0239] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network or a local area network, etc. FIG11 shows a possible implementation. As shown in FIG11 , two computing devices 900A and 900B are connected via a network. Specifically, the connection to the network is made via a communication interface in each computing device. In this type of possible implementation, the memory 906 in the computing device 900A stores instructions for executing the functions of the client. At the same time, the memory 906 in the computing device 900B stores instructions for executing the functions of the server.

[0240] The connection method between the computing device clusters shown in Figure 11 can be considered to be that the garbage collection method provided by this application requires the client and the server to interact with each other, so it is considered to hand over the functions implemented by the client to the computing device 900A for execution.

[0241] It should be understood that the functionality of the computing device 900A shown in FIG11 may also be implemented by multiple computing devices 900. Similarly, the functionality of the computing device 900B may also be implemented by multiple computing devices 900.

[0242] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the computer program product causes the at least one computing device to perform a garbage collection method.

[0243] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a non-volatile data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute a garbage collection method.

[0244] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0245] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0246] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of the present disclosure. In addition, the specific details disclosed above are merely illustrative and for ease of understanding, and are not restrictive. The above details do not limit the present disclosure to necessarily being implemented using the above specific details.

[0247] The block diagrams of the devices, devices, equipment, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0248] It should also be noted that in the apparatus, device, and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0249] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

[0250] It will be understood that the various numerical numbers involved in the embodiments of the present application are merely distinctions for the convenience of description and are not intended to limit the scope of the embodiments of the present application.

Claims

1. A garbage collection method, characterized in that: The method is applied to a distributed storage system, the distributed storage system includes a client and a server, the server is used to access a storage device, the storage device includes multiple erasure units, the client includes first maintenance information, the first maintenance information is used to indicate multiple data units and the erasure units to which the multiple data units belong, the server includes second maintenance information, the second maintenance information is used to indicate the multiple erasure units, the method includes: The client determines, based on the first maintenance information, at least one data unit to be deleted from the multiple data units; The client sends a first data deletion request, where the first data deletion request includes an identifier of the at least one data unit to be deleted; The server receives the first data deletion request, and adds the identifier of the at least one data unit to be deleted to the data unit identifier set; The service end instructs the storage device to recycle multiple to-be-deleted data units in the target erasure unit based on the second maintenance information and the data unit identification set.

2. The method according to claim 1, characterized in that: The data unit identification set also includes an identification of at least one data unit to be deleted included in the second data deletion request sent by the client.

3. The method according to claim 1 or 2, characterized in that: The first maintenance information includes one or more of the following information: The identifiers of the multiple data units to be deleted, the garbage amounts of the multiple data units to be deleted, the identifiers of the erasure units to which the multiple data units to be deleted belong, or the garbage amounts of the erasure units to which the multiple data units to be deleted belong.

4. The method according to any one of claims 1 to 3, characterized in that: The second maintenance information includes one or more of the following information: The identifiers of the multiple erasure units, the garbage amounts of the multiple erasure units, and the identifiers of the data units included in the multiple erasure units.

5. The method according to claim 1 or 4, characterized in that: The client determines, based on the first maintenance information, at least one data unit to be deleted from the multiple data units, including: The client determines, based on the garbage amounts of the multiple data units, the at least one data unit to be deleted from the multiple data units with the largest amount of garbage; or The client determines, based on the identifier of the erasure unit to which the multiple data units belong, the at least one data unit to be deleted from the multiple data units located in the same erasure unit; or The client determines the at least one data unit to be deleted from the erasure unit having the largest amount of garbage based on the amount of garbage in the erasure units to which the multiple data units belong.

6. The method according to claim 1 or 5, characterized in that: The server instructs the storage device to recycle a plurality of to-be-deleted data units in a target erasure unit based on the second maintenance information and the data unit identifier set, including: The server instructs the storage device to recycle multiple data units to be deleted in a target erasure unit having the most data units to be deleted in the multiple erasure units based on the identifiers of the data units included in the multiple erasure units and the data unit identifier set; or The server instructs the storage device to recycle multiple data units to be deleted in a target erasure unit having the largest amount of garbage among the multiple erasure units based on the amount of garbage in the multiple erasure units and the data unit identification set.

7. A distributed storage system, characterized in that: The distributed storage system includes a server and a client; the server is used to access a storage device, the storage device includes a plurality of erasure units, the client includes first maintenance information, the first maintenance information is used to indicate a plurality of data units and the erasure units to which the plurality of data units belong, the server includes second maintenance information, the second maintenance information is used to indicate the plurality of erasure units; The client is configured to determine at least one data unit to be deleted from the plurality of data units based on the first maintenance information; The client is configured to send a first data deletion request, wherein the first data deletion request includes an identifier of the at least one data unit to be deleted; The server is configured to receive the first data deletion request and add the identifier of the at least one data unit to be deleted to the data unit identifier set; The service end is used to instruct the storage device to recycle multiple to-be-deleted data units in the target erasure unit based on the second maintenance information and the data unit identification set.

8. The distributed storage system according to claim 7, characterized in that: The data unit identification set also includes an identification of at least one data unit to be deleted included in the second data deletion request sent by the client.

9. The distributed storage system according to claim 7 or 8, characterized in that: The first maintenance information includes one or more of the following information: The identifiers of the multiple data units to be deleted, the garbage amounts of the multiple data units to be deleted, the identifiers of the erasure units to which the multiple data units to be deleted belong, or the garbage amounts of the erasure units to which the multiple data units to be deleted belong.

10. The distributed storage system according to any one of claims 7 to 9, characterized in that: The second maintenance information includes one or more of the following information: identifiers of the multiple erasure units, garbage amounts of the multiple erasure units, and identifiers of data units included in the multiple erasure units.

11. The distributed storage system according to claim 7 or 10, characterized in that: The client is specifically used for: Based on the garbage amounts of the multiple data units, determining the at least one data unit to be deleted from the multiple data units with the largest garbage amount; or, Determining the at least one data unit to be deleted from the multiple data units located in the same erasure unit based on the identifiers of the erasure units to which the multiple data units belong; or, Based on the garbage amounts of the erasure units to which the multiple data units belong, the at least one data unit to be deleted is determined from the erasure unit having the largest garbage amount.

12. The distributed storage system according to claim 7 or 11, characterized in that: The server is specifically used for: Based on the identifiers of the data units included in the multiple erasure units and the data unit identifier set, instruct the storage device to recycle multiple data units to be deleted in a target erasure unit having the most data units to be deleted in the multiple erasure units; or Based on the garbage amounts of the multiple erasure units and the data unit identification set, the storage device is instructed to recycle multiple data units to be deleted in a target erasure unit having the largest garbage amount among the multiple erasure units.

13. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

14. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 6.

15. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

Images