Garbage collection method and apparatus

Abstract

The present application relates to the technical field of storage, and discloses a garbage collection method and apparatus. In the method, a client of a distributed storage system may send a garbage collection request to a server of the distributed storage system. The server may determine, on the basis of the garbage collection request, that a target unit data block is a valid data block, and send a migration command to a storage device storing the target unit data block, so that the storage device migrates the target unit data block from a first storage location to a second storage location. Since it is a server that migrates data between storage locations of storage devices accessed by the server, rather than a client migrating data between servers, the network bandwidth between the client and the server is reduced; moreover, since the storage device performs the migration operation, rather than the CPUs of the client and the server performing the migration operation, the CPU overhead between the client and the server is reduced, thereby improving the garbage collection efficiency.

Description

Garbage recycling method and device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on July 17, 2023, with application number 202310876345.2 and application name “A method, device and other equipment for garbage recycling”, the entire contents of which are incorporated by reference into this application; this application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on October 23, 2023, with application number 202311377544.5 and application name “A method and device for garbage recycling”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of storage technology, and in particular to a garbage collection method and device. Background Art

[0004] The garbage collection (GC) operation is an operation that reclaims used storage space when the available storage space in the storage space is small. In a distributed storage system, GC operations are generally performed uniformly by the client. For example, if the client stores the mapping relationship between data blocks and the storage devices accessed by each server, when performing a garbage collection operation, after the client determines that a data block stored on a storage device accessed by a server is a valid data block, it sends a read request to the server to read the valid data block stored on the storage device accessed by the server. After the valid data block is read by the client, the server sets the storage device as an available storage device. The client aggregates all the valid data blocks read out, and then scatters them again and writes them to the storage devices accessed by each server.

[0005] However, when garbage collection is performed uniformly by the client, data often needs to be moved across servers, resulting in high network bandwidth overhead between the client and server, which affects garbage collection efficiency. Furthermore, data movement often requires the central processing unit (CPU) of both the client and server to participate in reading and writing, resulting in high CPU overhead on both the client and server, which also affects garbage collection efficiency.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a garbage collection method and apparatus for improving garbage collection efficiency.

[0008] In a first aspect, an embodiment of the present application provides a garbage collection method, which can be applied to a server of a distributed storage system, wherein the server is used to access at least one storage device. The method includes: receiving a garbage collection request from a client of the distributed storage system, wherein the garbage collection request carries a target unit data block, wherein the target unit data block is a valid data block; the server can determine that the target unit data block is located at a first storage position of a first storage device, wherein the first storage device is one of the at least one storage devices; the server can determine a second storage position, wherein the second storage position is an idle position in the at least one storage device; the server can send a first move command to the first storage device, wherein the first move command is used to instruct the first storage device to move the target unit data block from the first storage position to the second storage position.

[0009] In the above scheme, the client can send a garbage collection request to a server. The server can determine that the target unit data block is a valid data block based on the garbage collection request, and send a move command to the storage device storing the target unit data block, so that the storage device moves the target unit data block from the first storage location to the second storage location. Since it is a server that moves data between the various storage locations of the storage devices it accesses, rather than the client moving data between the various servers, the network bandwidth overhead between the client and the server is reduced, and since it is the storage device that performs the move operation, rather than the CPU of the client and the server, the CPU overhead between the client and the server is reduced, and the garbage collection efficiency is improved.

[0010] In one possible implementation, the second storage location is located in a second storage device; the sending of the first move command to the first storage device includes: sending a second move command to the first storage device, the second move command including the identifier of the first storage device, the first storage location, the identifier of the second storage device, the second storage location, and the length of the target unit data block, the first storage location is used to indicate the storage location of the target unit data block in the first storage device before the move, and the second storage location is used to indicate the storage location of the target unit data block in the second storage device after the move.

[0011] In the above implementation, when the second storage location and the first storage location are storage locations of different storage devices, the server can send a cross-disk move command to the storage device that stores the target unit data block. Since the storage device performs the move operation, rather than the CPU of the client and server, the CPU overhead between the client and the server is reduced and the garbage collection efficiency is improved.

[0012] In one possible implementation, after sending the second move command to the first storage device, it includes: updating the first mapping relationship to a second mapping relationship, wherein the first mapping relationship is a mapping relationship between the target unit data block and the first storage position on the first storage device, and the second mapping relationship is a mapping relationship between the target unit data block and the second storage position on the second storage device.

[0013] In the above implementation method, the server can update the first mapping relationship between the target unit data block and the first storage position of the first storage device to the second mapping relationship between the target unit data block and the second storage position of the first storage device after the data is moved, so that the server can also quickly determine the storage position of the target unit data block according to the second mapping relationship after the data is moved, thereby improving the efficiency of data reading and moving.

[0014] In one possible implementation, the second storage location is located in the first storage device; the sending of the first move command to the first storage device includes: sending a third move command to the first storage device, the third move command including the first storage location, the second storage location, and the length of the target unit data block, the first storage location being used to indicate the storage location of the target unit data block in the first storage device before the move, and the second storage location being used to indicate the storage location of the target unit data block in the first storage device after the move.

[0015] In the above implementation, when the second storage location and the first storage location are storage locations of the same storage device, the server can send an on-disk move command to the storage device storing the target unit data block. Since the storage device performs the move operation, rather than the CPU of the client and server, the CPU overhead between the client and the server is reduced and the garbage collection efficiency is improved.

[0016] In one possible implementation, after sending the third move command to the first storage device, it includes: updating the first mapping relationship to a third mapping relationship, where the first mapping relationship is a mapping relationship between the target unit data block and the first storage position on the first storage device, and the third mapping relationship is a mapping relationship between the target unit data block and the second storage position on the first storage device.

[0017] In the above implementation method, the server can update the first mapping relationship between the target unit data block and the first storage position of the first storage device to a third mapping relationship between the target unit data block and the second storage position of the second storage device after the data is moved, so that the server can also quickly determine the storage position of the target unit data block according to the third mapping relationship after the data is moved, thereby improving the efficiency of data reading and moving.

[0018] In one possible implementation, the at least one storage device is a solid state drive (SSD); sending a first move command to the first storage device includes: sending the first move command to the first storage device through a non-volatile memory express (NVMe) protocol, wherein the first move command is used to instruct a controller of the first storage device to move the target unit data block from the first storage location to the second storage location.

[0019] In the above implementation, when the storage device is an SSD, the server can send a move command to the controller of the storage device through the NVMe protocol, so that the controller of the storage device performs the move operation. Since the move operation is not performed by the CPU of the client and the server, the CPU overhead between the client and the server is reduced and the garbage collection efficiency is improved.

[0020] In a second aspect, an embodiment of the present application further provides a garbage collection device, which can be applied to a server end of a distributed storage system, wherein the server end is used to access at least one storage device. The device has the function of implementing the method described in the first aspect or any possible design of the first aspect. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions, such as a communication module and a processing module.

[0021] a communication module, configured to receive a garbage collection request from a client of the distributed storage system, wherein the garbage collection request carries a target unit data block, and the target unit data block is a valid data block;

[0022] a processing module, configured to determine that the target unit data block is located at a first storage location of a first storage device, where the first storage device is one of the at least one storage device; and determine a second storage location, where the second storage location is an idle location in the at least one storage device;

[0023] The communication module is further configured to send a first move command to the first storage device, where the first move command is configured to instruct the first storage device to move the target unit data block from the first storage location to the second storage location.

[0024] In one possible implementation, the second storage location is located in the second storage device; the communication module is specifically used to: send a second move command to the first storage device, the second move command includes the identifier of the first storage device, the first storage location, the identifier of the second storage device, the second storage location, and the length of the target unit data block, the first storage location is used to indicate the storage location of the target unit data block in the first storage device before the move, and the second storage location is used to indicate the storage location of the target unit data block in the second storage device after the move.

[0025] In one possible implementation, the processing module is also used to: update the first mapping relationship to a second mapping relationship, where the first mapping relationship is a mapping relationship between the target unit data block and the first storage position on the first storage device, and the second mapping relationship is a mapping relationship between the target unit data block and the second storage position on the second storage device.

[0026] In one possible implementation, the second storage location is located in the first storage device; the communication module is specifically used to: send a third move command to the first storage device, the third move command includes the first storage location, the second storage location and the length of the target unit data block, the first storage location is used to indicate the storage location of the target unit data block in the first storage device before the move, and the second storage location is used to indicate the storage location of the target unit data block in the first storage device after the move.

[0027] In one possible implementation, the processing module is also used to: update the first mapping relationship to a third mapping relationship, where the first mapping relationship is a mapping relationship between the target unit data block and the first storage position on the first storage device, and the third mapping relationship is a mapping relationship between the target unit data block and the second storage position on the first storage device.

[0028] In one possible implementation, the at least one storage device is an SSD; the communication module is specifically used to: send the first move command to the first storage device through the NVME protocol, and the first move command is used to instruct the controller of the first storage device to move the target unit data block from the first storage location to the second storage location.

[0029] In a third aspect, embodiments of the present application further provide a computing device cluster comprising at least one computing device, each computing device comprising a processor and a memory. The processor is configured to support a server in executing the method of the first aspect or any possible design of the first aspect. The memory is coupled to the processor and stores program instructions and data necessary for the communication device. The communication device may also include a communication interface for communicating with other devices.

[0030] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium includes computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method of the first aspect or any possible design of the first aspect.

[0031] In a fifth aspect, an embodiment of the present application further provides a computer program product comprising instructions, which, when executed by a computing device cluster, enables the computing device cluster to execute the method of the first aspect or any possible design of the first aspect.

[0032] In a sixth aspect, an embodiment of the present application further provides a chip, which can be coupled to a memory and used to call a computer program stored in the memory to execute the method of the first aspect or any possible design of the first aspect.

[0033] The beneficial effects of the above-mentioned second to sixth aspects and their possible designs can refer to the description of the beneficial effects of the method described in the above-mentioned first aspect and any possible design thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is a schematic diagram of a distributed storage system;

[0035] FIG2 is a schematic diagram of a garbage collection scenario;

[0036] FIG3 is a flow chart of a garbage collection method;

[0037] FIG4 is a schematic diagram of another garbage collection scenario;

[0038] FIG5 is a schematic diagram of another garbage collection scenario;

[0039] FIG6 is a schematic diagram of the structure of an SSD;

[0040] FIG7 is a schematic diagram of the structure of a computing device;

[0041] FIG8 is a schematic diagram of the structure of a computing device cluster;

[0042] FIG9 is a schematic diagram of a connection method between computing device clusters. DETAILED DESCRIPTION

[0043] The present application provides a method and apparatus for garbage collection. The method and apparatus are based on the same concept. Since the method and apparatus solve similar problems, their implementations can refer to each other, and any repetitions will not be repeated.

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be further described in detail with reference to the accompanying drawings. Among them, in the description of the embodiments of the present application, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects before and after are in an "or" relationship. At least one involved in the embodiments of the present application refers to one or more; multiple refers to two or more. In addition, it should be understood that in the description of the embodiments of the present application, words such as "first" and "second" are only used to distinguish the purpose of description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0045] To facilitate understanding, exemplary descriptions of concepts related to this application are provided for reference.

[0046] 1) The storage system is primarily composed of a processor and memory. The number of processors and memory is not limited. The memory consists of a memory controller and storage units containing non-volatile media particles such as flash memory.

[0047] Optionally, the storage system may be a distributed storage system (DSS).

[0048] A distributed storage system consists of a client and multiple servers. The client manages the server, and the server stores data. The client in a distributed storage system is equivalent to the processor, while the server is equivalent to the memory.

[0049] The server, also known as a storage server or storage node, consists of multiple storage devices. These devices provide physical storage space for the server. These devices are equivalent to storage units in memory, and contain multiple storage locations. Alternatively, the storage devices can be disks.

[0050] A disk is a memory device that uses magnetic recording technology to store data. Disks are the primary storage medium for computers, capable of storing large amounts of binary data and maintaining data integrity even after power outages. Early computers used floppy disks (or floppy disks), while today's hard disks are the most common.

[0051] For example, FIG1 is a schematic diagram of a distributed storage system. As shown in FIG1 , the distributed storage system 100 includes a client 110 and multiple servers 120. FIG1 takes the distributed storage system 100 as an example, including four servers (respectively, server 1 to server 4). Each server 120 includes multiple storage devices. FIG1 takes the example, including server 1 including three storage devices (respectively, storage device A to storage device C), server 2 including three storage devices (respectively, storage device D to storage device F), server 3 including three storage devices (respectively, storage device G to storage device I), and server 4 including three storage devices (respectively, storage device J to storage device L).

[0052] Each storage device among storage devices A to storage devices M may be a hard disk drive (HDD) or an SSD, etc. An interface of each storage device among storage devices A to storage devices M may include a serial advanced technology attachment (SATA) interface, a small computer system interface (SCSI), a serial attached SCSI (SAS), a fiber channel (FC), or an interface supporting the NVMe protocol, etc.

[0053] 2) Unit data block (grain): A grain is stored in a storage location on a storage device on a server in a distributed storage system.

[0054] A stripe, also known as a data segment, consists of multiple grains, which are stored on multiple servers.

[0055] 3) Data processing requests, also referred to as input / output (IO) requests or IO requests, are requests sent by a processor in a storage system to a memory controller in the storage system to perform data processing operations. IO requests can include at least one of read requests, write requests, and erase requests.

[0056] Taking a distributed storage system as an example, a client can send an IO request to a server, and the server performs a corresponding IO operation based on the received IO request. The IO operation can include at least one of a read operation, a write operation, and an erase operation.

[0057] For example, a client can send write requests to multiple servers to store multiple grains included in a stripe on multiple servers. Correspondingly, a server can receive a write request from a client and store the grain indicated by the write request in a storage location on a storage device.

[0058] 4) Garbage collection (GC) operation is an operation to reclaim used storage space when the available storage space in the storage space is small. Its specific operations include: reading out the valid data in the selected storage location to be reclaimed and migrating it to the target storage location in the available storage space, and then erasing all data in the storage location to be reclaimed, thereby reclaiming the storage space corresponding to the storage location to be reclaimed through the GC operation.

[0059] In a distributed storage system, garbage collection is typically performed uniformly by the client. For example, if the client maintains a mapping between data blocks and the various storage devices on each server, during garbage collection, the client determines that a data block stored on a storage device on a server is valid. It then sends a read request to the server to retrieve the valid data block stored on the storage device on the server. After the client reads the valid data block, the server sets the storage device as an available storage device. The client aggregates all read valid data blocks and then breaks them up again, writing them to the various storage devices managed by the servers.

[0060] For example, FIG2 is a schematic diagram of a garbage collection scenario. As shown in FIG2 , a client sends a read request to server 1 and server 2, reading valid data block 1 stored on storage device A of server 1, valid data block 2 on storage device C of server 1, valid data block 3 on storage device D of server 2, and valid data block 4 on storage device F of server 2. Server 1 sets storage devices A and C as available storage devices, and server 2 sets storage device D as available storage device. The client aggregates the valid data blocks previously read, and then breaks them up and writes them to the various storage devices managed by server 1 and server 2, such as writing valid data block 1 to storage device A of server 1, valid data block 2 to storage device B of server 1, valid data block 3 to storage device C of server 1, and valid data block 4 to storage device D of server 2. Valid data block 2 was moved from storage device C on server 1 to storage device B on server 1, and valid data block 4 was moved from storage device F on server 2 to storage device D on server 2. This means valid data blocks 2 and 4 were not moved across servers. Valid data block 3 was moved from storage device D on server 2 to storage device C on server 1. This means valid data block 3 was moved across servers.

[0061] As can be seen, when garbage collection is performed uniformly by the client, data often moves across servers, resulting in high network bandwidth overhead between the client and server, affecting garbage collection efficiency. Furthermore, data movement often requires the client and server CPUs to participate in reading and writing, resulting in high CPU overhead on both the client and server, which also affects garbage collection efficiency.

[0062] In response to the problem of low garbage collection efficiency described above, an embodiment of the present application provides a garbage collection method. In this method, a client can send a garbage collection request to a server. The server can determine that the target unit data block is a valid data block based on the garbage collection request, and send a move command to the storage device storing the target unit data block, so that the storage device moves the target unit data block from the first storage location to the second storage location. Since it is a server that moves data between the various storage locations of the various storage devices it accesses, instead of the client moving data between the various servers, the network bandwidth overhead between the client and the server is reduced, and since it is the storage device that performs the move operation, instead of the CPU of the client and the server performing the move operation, the CPU overhead between the client and the server is reduced, and the garbage collection efficiency is improved.

[0063] Exemplarily, Figure 3 is a flow chart of a garbage collection method, which can be applied to the distributed storage system shown in Figure 1 or with a functional structure similar to that of Figure 1, and the embodiments of the present application are not specifically limited here. The specific process of the garbage collection method is described as follows. In the following introduction, the method is applied to the distributed storage system shown in Figure 1 as an example. For example, the client described below can be the client 110 in the system shown in Figure 1, and the server described below can be the server 120 in the system shown in Figure 1. In the embodiments of the present application, the distributed storage system is used as an example to introduce the execution subject of the method, or the execution subject of the method can also be other devices or systems, such as a chip, chip system or processor applied to a distributed storage system.

[0064] Step S301: The client sends a garbage collection request to the server. Correspondingly, the server receives the garbage collection request from the client.

[0065] In an embodiment of the present application, during garbage collection, after the client determines that the target unit data block is a valid data block, the client may send a garbage collection request to the server. In response, the server receives the garbage collection request from the client. The garbage collection request is used to request the server to perform garbage collection, and the garbage collection request may include the target unit data block, which is a valid data block.

[0066] Exemplarily, the client may store the mapping relationship between stripes and unit data blocks as shown in Table 1, and the mapping relationship between unit data blocks and the server as shown in Table 2.

[0067] Table 1

[0068] Table 2

[0069] If the client determines that grain 11, grain 13, grain 21, and grain 23 are valid data blocks, and grain 12 and grain 22 are invalid data blocks, then the client can send corresponding garbage collection requests to each server according to Table 2 above. For example, the client can send garbage collection request 1 to server 1, with garbage collection request 1 carrying grain 11 and grain 13; send garbage collection request 2 to server 2, with garbage collection request 2 carrying grain 21; and send garbage collection request 3 to server 3, with garbage collection request 3 carrying grain 23.

[0070] Step S302: The server determines that the target unit data block is located at a first storage location of a first storage device of the server.

[0071] In an embodiment of the present application, after a server receives a garbage collection request carrying a target unit data block from a client, the server may determine, based on a mapping relationship between unit data blocks and storage locations of storage devices, that the target unit data block is located at a first storage location of a first storage device on the server. The first storage device may be one of at least one storage device on the server.

[0072] Exemplarily, the server 1 may store a mapping relationship between unit data blocks and storage locations of the storage device as shown in Table 3.

[0073] Table 3

[0074] If server 1 receives garbage collection request 1 from client, and garbage collection request 1 carries grain 11 and grain 13, that is, grain 11 and grain 13 are valid data blocks, server 1 can determine, based on Table 3 above, that valid data block grain 11 is located at storage location 10 of storage device A of server 1, invalid data block grain 12 is located at storage location 20 of storage device B of server 1, and valid data block grain 13 is located at storage location 30 of storage device C of server 1.

[0075] Optionally, if the storage device A, the storage device B, and the storage device C are SSDs, the server 1 may store a mapping relationship between the unit data blocks and the storage locations of the storage devices as shown in Table 4.

[0076] Table 4

[0077] Table 4 indicates the location of grains stored on Server 1. This location includes the SSD the grain is located on and the offset of the grain on that SSD. For example, if grain 11 is located on SSD A, the offset of grain 11 on SSD A is sector 2. If one storage location consists of 16 sectors, then grain 1 is located from sector 2 to sector 18 on SSD 1. That is, the starting address of storage location 10 is sector 2, and the ending address of storage location 10 is sector 18.

[0078] Step S303: The server determines a second storage location.

[0079] In an embodiment of the present application, after the server determines that the target unit data block is located at the first storage location of the first storage device of the server, the server can determine the second storage location based on the mapping relationship between the unit data block and the storage location of the storage device. The second storage location can be any free location of any storage device among the at least one storage device of the server. The free location (also known as an available storage location) can be a location where no data block is stored or no valid data block is stored (i.e., an invalid data block is stored).

[0080] Optionally, the server can determine the second storage location based on the number of valid data blocks stored in the first storage device. Specifically, if the number of valid data blocks stored in the first storage device is less than the first threshold, the server can determine the second storage location to be any free location of the first storage device; if the number of valid data blocks stored in the first storage device is greater than or equal to the first threshold, the server can determine the second storage location to be any free location of the second storage device. The first threshold can be a default threshold or a threshold set by the user as needed, and the embodiments of the present application do not limit this.

[0081] Step S304: The server sends a first migration command to the first storage device.

[0082] In an embodiment of the present application, after the server determines the second storage location, it may send a first move command to the first storage device. The first move command is used to instruct the first storage device to move the target unit data block from the first storage location to the second storage location. Because the first storage device performs the move operation, rather than the CPUs of the client and server, this reduces CPU overhead between the client and server, and improves garbage collection efficiency.

[0083] Specifically, if the second storage location is located in the second storage device, the server can send a second move command (also known as a cross-disk move command, Move_Cross_Disk) to the first storage device. The second move command includes the identifier of the first storage device, the first storage location, the identifier of the second storage device, the second storage location, and the length of the target unit data block. The first storage location indicates the storage location of the target unit data block in the first storage device before the move. The second storage location indicates the storage location of the target unit data block in the second storage device after the move.

[0084] For example, FIG4 is a schematic diagram of another garbage collection scenario. As shown in FIG4 (1), storage device A includes four storage locations, namely storage location 10, storage location 11, storage location 12, and storage location 13. Storage device B includes four storage locations, namely storage location 20, storage location 21, storage location 22, and storage location 23. If server 1 receives a garbage collection request 1 from a client carrying grain 0, grain 1, grain 2, and grain 4, server 1 can determine that the valid data blocks stored in storage device A include grain 0 at storage location 11, grain 1 at storage location 12, and grain 2 at storage location 13 based on the mapping relationship between unit data blocks and storage locations of storage devices as shown in Table 5. The valid data blocks stored in storage device B include grain 4 at storage location 21; and the invalid data blocks stored in storage device B include grain 3 at storage location 20.

[0085] Table 5

[0086] As shown in (2) of Figure 4 , if server 1 determines to move grain 0 from storage location 11 to storage location 20, server 1 can send command 1 to storage device A, causing storage device A to move grain 0 from storage location 11 to storage location 20. Command 1 is the cross-disk move command Move_Cross_Disk(diskA, offset11, diskB, offset20, len0), where disk A is the identifier of storage device A, offset12 is the starting address of storage location 12, disk B is the identifier of storage device B, offset20 is the starting address of storage location 20, and len0 is the length of grain 0. After receiving command 1, storage device A executes the cross-device move, thereby moving grain 0 from storage location 11 to storage location 20.

[0087] After the server sends the second move command to the first storage device, the server may also update the first mapping relationship to a second mapping relationship. The first mapping relationship is a mapping relationship between the target unit data block and the first storage location on the first storage device. The second mapping relationship is a mapping relationship between the target unit data block and the second storage location on the second storage device.

[0088] For example, in the garbage collection scenario shown in FIG4 above, server 1 sends command 1 to storage device A, causing storage device A to move grain 0 from storage location 11 to storage location 20. Server 1 can then update the mapping relationship between the unit data block and the storage location of the storage device as shown in Table 5 above, and obtain the mapping relationship between the unit data block and the storage location of the storage device as shown in Table 6.

[0089] Table 6

[0090] Furthermore, when server 1 receives a write request from the client, server 1 can determine, based on Table 6, that storage locations 10 and 11 of storage device A store valid data blocks, that storage locations 12 and 13 of storage device A do not store data blocks or do not store valid data blocks, i.e., that storage locations 12 and 13 of storage device A are free locations, and that storage locations 20 and 21 of storage device B store valid data blocks, and that storage locations 22 and 23 of storage device B do not store data blocks or do not store valid data blocks, i.e., that storage locations 22 and 23 of storage device B are free locations. Therefore, server 1 can write the unit data block indicated by the write request to storage location 12 of storage device A, storage location 13 of storage device A, storage location 22 of storage device B, or storage location 23 of storage device B.

[0091] Specifically, if the second storage location is located in the first storage device, the server can send a third move command (also known as a Move_In_Disk command) to the first storage device. The third move command includes the first storage location, the second storage location, and the length of the target unit data block. The first storage location indicates the storage location of the target unit data block in the first storage device before the move. The second storage location indicates the storage location of the target unit data block in the first storage device after the move.

[0092] For example, FIG5 is a schematic diagram of another garbage collection scenario. As shown in FIG5 (1), storage device A includes four storage locations, namely storage location 10, storage location 11, storage location 12, and storage location 13. If server 1 receives garbage collection request 1 from client carrying grain 0 and grain 1, server 1 can determine that the valid data blocks stored in storage device A include grain 0 located at storage location 12 and grain 1 located at storage location 13 based on the mapping relationship between unit data blocks and storage locations of storage devices as shown in Table 7; and the invalid data blocks stored in storage device A include grain 2 located at storage location 11.

[0093] Table 7

[0094] As shown in (2) of Figure 5 , if server 1 determines to move grain 0 from storage location 12 to storage location 10, and to move grain 1 from storage location 13 to storage location 11, server 1 can send commands 1 and 2 to storage device A in sequence. Command 1 is the disk move command Move_In_Disk(offset12, offset10, len0), where offset12 is the starting address of storage location 12, offset10 is the starting address of storage location 10, and len0 is the length of grain 0. Command 2 is the disk move command Move_In_Disk(offset13, offset11, len1), where offset13 is the starting address of storage location 13, offset11 is the starting address of storage location 11, and len1 is the length of grain 1. After receiving command 1 and command 2, storage device A performs the intra-device move in sequence, thereby moving grain 0 from storage location 12 to storage location 10, and moving grain 1 from storage location 13 to storage location 11.

[0095] After the server sends the third move command to the first storage device, the server may also update the first mapping relationship to a third mapping relationship. The first mapping relationship is a mapping relationship between the target unit data block and the first storage location on the first storage device. The third mapping relationship is a mapping relationship between the target unit data block and the second storage location on the first storage device.

[0096] For example, in the garbage collection scenario shown in FIG5 , server 1 sends command 1 and command 2 to storage device A, causing storage device A to move grain 0 from storage location 12 to storage location 10, and grain 1 from storage location 13 to storage location 11. Server 1 can then update the mapping relationship between the unit data block and the storage location of the storage device as shown in Table 7 above, and obtain the mapping relationship between the unit data block and the storage location of the storage device as shown in Table 8.

[0097] Table 8

[0098] Furthermore, when server 1 receives a write request from the client, server 1 can determine, based on Table 8, that storage locations 10 and 11 of storage device A store valid data blocks, and that storage locations 12 and 13 of storage device A store no data blocks or no valid data blocks, i.e., storage locations 12 and 13 of storage device A are free locations. Therefore, server 1 can write the unit data block indicated by the write request into storage locations 12 and 13 of storage device A.

[0099] Optionally, if at least one storage device of the server is an SSD, the server may send a first move command to the first storage device via the NVME protocol, wherein the first move command is used to instruct a controller of the first storage device to move the target unit data block from the first storage location to the second storage location.

[0100] For example, Figure 6 is a schematic diagram of the structure of an SSD provided in an embodiment of the present application. The SSD includes flash memory and a main controller. The flash memory includes multiple flash memory chips for storing data. The main controller is the brain of the SSD and is responsible for complex tasks such as managing data storage and maintaining SSD performance and lifespan. The main controller is an embedded microchip that includes a processor. Its function is like a command center, issuing all operation requests for the SSD. For example, the processor can perform functions such as reading / writing data, garbage collection, and wear leveling through firmware in the buffer. The main controller also includes a host interface and several channel controllers. The host interface is used to communicate with the host. The host here can refer to any device such as a server, personal computer, or array controller. Through several channel controllers, the main controller can operate multiple flash memory chips in parallel, thereby increasing the underlying bandwidth. For example, assuming there are 8 channels between the main controller and the flash memory, the main controller can read and write data to the 8 flash memory chips in parallel through these 8 channels.

[0101] The first storage device and the second storage device may be the SSDs shown in FIG6 above, that is, the first storage device may be SSD1 and the second storage device may be SSD2. If the second storage location is located in the second storage device, the server may send a second move command to SSD1 via the NVME protocol, that is, the host interface of SSD1 is an NVME interface. The second move command is used to instruct the main controller of SSD1 to move the target unit data block from flash memory chip 1 of SSD1 to flash memory chip 2 of SSD2. After SSD1 receives the second move command via its host interface, the main controller of SSD1 may read the target unit data block stored in flash memory chip 1 via the channel controller corresponding to flash memory chip 1 and send a write request to SSD2 via the NVME protocol, that is, the host interface of SSD2 is an NVME interface. The write request is used to request the main controller of SSD2 to write the target unit data block to flash memory chip 2. After SSD2 receives the write request via its host interface, the main controller of SSD2 may write the target unit data block to flash memory chip 2 via the channel controller corresponding to flash memory chip 2.

[0102] If the second storage location is located in the first storage device, the server may send a third move command to SSD1 via the NVME protocol, wherein the third move command is used to instruct the main controller of SSD1 to move the target unit data block from the flash memory chip 1 of SSD1 to the flash memory chip 3 of SSD1. After SSD1 receives the second move command via the host interface of SSD1, the main controller of SSD1 may read the target unit data block stored in the flash memory chip 1 via the channel controller corresponding to the flash memory chip 1, and write the target unit data block to the flash memory chip 3 via the channel controller corresponding to the flash memory chip 3.

[0103] According to the above scheme, the client can send a garbage collection request to a server. The server can determine that the target unit data block is a valid data block based on the garbage collection request, and send a move command to the storage device storing the target unit data block, so that the storage device moves the target unit data block from the first storage location to the second storage location. Since it is a server that moves data between the various storage locations of the various storage devices it accesses, rather than the client moving data between the various servers, the network bandwidth overhead between the client and the server is reduced, and since it is the storage device that performs the move operation, rather than the CPU of the client and the server, the CPU overhead between the client and the server is reduced, and the garbage collection efficiency is improved.

[0104] Based on the above embodiments, the present application also provides a computing device. FIG7 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. The computing device 700 includes a bus 702, a processor 704, a memory 706, and a communication interface 708. The processor 704, the memory 706, and the communication interface 708 communicate with each other via the bus 702. The computing device 700 may be a server in a distributed storage system. It should be understood that the present application does not limit the number of processors and memories in the computing device 700.

[0105] Bus 702 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG7 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 702 may include a path for transmitting information between various components of computing device 700 (e.g., memory 706, processor 704, and communication interface 708).

[0106] The processor 704 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0107] The memory 706 may include volatile memory, such as random access memory (RAM), or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0108] The memory 706 stores executable program code, and the processor 704 executes the executable program code to implement the functions of the communication module and the processing module, thereby implementing the garbage collection method in the above method embodiment. In other words, the memory 706 stores instructions for executing the garbage collection method in the above method embodiment.

[0109] The communication interface 708 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 700 and other devices or a communication network.

[0110] Based on the above embodiments, the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server in a distributed storage system.

[0111] As shown in Figure 8, the computing device cluster includes at least one computing device 700. The memory 706 in one or more computing devices 700 in the computing device cluster may store the same instructions for executing the garbage collection method in the above method embodiment.

[0112] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the garbage collection method in the above method embodiment. In other words, the combination of one or more computing devices 700 can jointly execute the instructions for executing the garbage collection method in the above method embodiment.

[0113] It is understood that the memory 706 in different computing devices 700 in the computing device cluster can store different instructions, each for executing a portion of the functions of a server in the distributed storage system. In other words, the instructions stored in the memory 706 in different computing devices 700 can implement the functions of one or more modules in the communication module and the processing module.

[0114] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network (WAN) or a local area network (LAN), among others. FIG. 9 illustrates one possible implementation. As shown in FIG. 9 , two computing devices 700A and 700B are connected via a network. Specifically, each computing device is connected to the network via a communication interface within the computing device. In this type of possible implementation, the memory 706 in the computing device 700A stores instructions for executing the functions of the communication module. Simultaneously, the memory 706 in the computing device 700B stores instructions for executing the functions of the processing module.

[0115] It should be understood that the functionality of the computing device 700A shown in FIG9 may also be implemented by multiple computing devices 700. Similarly, the functionality of the computing device 700B may also be implemented by multiple computing devices 700.

[0116] Based on the above embodiments, embodiments of the present application further provide a computer-readable storage medium, which can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the garbage collection method in the above method embodiment.

[0117] Based on the above embodiments, embodiments of the present application further provide a computer program product comprising instructions. The computer program product may be software or a program product comprising instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device is caused to perform the garbage collection method of the above method embodiments.

[0118] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0119] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or box in the flow chart and / or block diagram, as well as the combination of the flow chart and / or box in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more flow charts and / or one or more boxes in the block diagram.

[0120] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0121] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0122] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A garbage collection method, characterized in that: The method is applied to a server of a distributed storage system, the server being used to access at least one storage device, and the method comprises: Receiving a garbage collection request from a client of the distributed storage system, wherein the garbage collection request carries a target unit data block, and the target unit data block is a valid data block; Determining that the target unit data block is located at a first storage location of a first storage device, where the first storage device is one of the at least one storage device; determining a second storage location, the second storage location being an empty location in the at least one storage device; A first move command is sent to the first storage device, where the first move command is used to instruct the first storage device to move the target unit data block from the first storage location to the second storage location.

2. The method according to claim 1, characterized in that The second storage location is located in a second storage device; and sending a first move command to the first storage device includes: A second move command is sent to the first storage device, wherein the second move command includes an identifier of the first storage device, the first storage position, an identifier of the second storage device, the second storage position, and a length of the target unit data block, wherein the first storage position is used to indicate a storage position of the target unit data block in the first storage device before the move, and the second storage position is used to indicate a storage position of the target unit data block in the second storage device after the move.

3. The method according to claim 2, characterized in that After sending the second move command to the first storage device, the method includes: The first mapping relationship is updated to a second mapping relationship, wherein the first mapping relationship is a mapping relationship between the target unit data block and the first storage position on the first storage device, and the second mapping relationship is a mapping relationship between the target unit data block and the second storage position on the second storage device.

4. The method according to claim 1, characterized in that: The second storage location is located in the first storage device; and sending a first move command to the first storage device includes: A third move command is sent to the first storage device, the third move command including the first storage position, the second storage position and the length of the target unit data block, the first storage position is used to indicate the storage position of the target unit data block in the first storage device before the move, and the second storage position is used to indicate the storage position of the target unit data block in the first storage device after the move.

5. The method according to claim 4, characterized in that After sending the third move command to the first storage device, the method includes: The first mapping relationship is updated to a third mapping relationship, wherein the first mapping relationship is a mapping relationship between the target unit data block and the first storage location on the first storage device, and the third mapping relationship is a mapping relationship between the target unit data block and the second storage location on the first storage device.

6. The method according to claim 1, characterized in that The at least one storage device is a solid state drive (SSD); and sending a first move command to the first storage device includes: The first move command is sent to the first storage device via the non-volatile memory standard NVME protocol, where the first move command is used to instruct a controller of the first storage device to move the target unit data block from the first storage location to the second storage location.

7. A garbage collection device, characterized in that: The device is applied to a server end of a distributed storage system, and the server end is used to access at least one storage device. The device includes: A communication module, configured to receive a garbage collection request from a client of the distributed storage system, wherein the garbage collection request carries a target unit data block, and the target unit data block is a valid data block; A processing module, configured to determine that the target unit data block is located at a first storage position of a first storage device, wherein the first storage device is one of the at least one storage device; and determine a second storage position, wherein the second storage position is an idle position in the at least one storage device; The communication module is further used to send a first move command to the first storage device, where the first move command is used to instruct the first storage device to move the target unit data block from the first storage location to the second storage location.

8. The device according to claim 7, characterized in that The second storage location is located in a second storage device; and the communication module is specifically configured to: A second move command is sent to the first storage device, wherein the second move command includes an identifier of the first storage device, the first storage position, an identifier of the second storage device, the second storage position, and a length of the target unit data block, wherein the first storage position is used to indicate a storage position of the target unit data block in the first storage device before the move, and the second storage position is used to indicate a storage position of the target unit data block in the second storage device after the move.

9. The device according to claim 8, characterized in that The processing module is further used for: The first mapping relationship is updated to a second mapping relationship, wherein the first mapping relationship is a mapping relationship between the target unit data block and the first storage position on the first storage device, and the second mapping relationship is a mapping relationship between the target unit data block and the second storage position on the second storage device.

10. The device according to claim 7, characterized in that The second storage location is located in the first storage device; the communication module is specifically used to: A third move command is sent to the first storage device, the third move command including the first storage position, the second storage position and the length of the target unit data block, the first storage position is used to indicate the storage position of the target unit data block in the first storage device before the move, and the second storage position is used to indicate the storage position of the target unit data block in the first storage device after the move.

11. The device according to claim 10, characterized in that The processing module is further used for: The first mapping relationship is updated to a third mapping relationship, wherein the first mapping relationship is a mapping relationship between the target unit data block and the first storage location on the first storage device, and the third mapping relationship is a mapping relationship between the target unit data block and the second storage location on the first storage device.

12. The device according to claim 7, characterized in that The at least one storage device is an SSD; and the communication module is specifically used for: The first move command is sent to the first storage device through the NVME protocol, and the first move command is used to instruct a controller of the first storage device to move the target unit data block from the first storage location to the second storage location.

13. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device includes a processor and a memory; the memory in the at least one computing device is used to store computer program instructions; The processor in the at least one computing device calls the computer program instructions stored in the memory to execute the method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device, the computing device performs the method according to any one of claims 1 to 6.

15. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device, the computing device is caused to perform the method as claimed in any one of claims 1 to 6.

Images