Garbage recycling method and device

Abstract

The invention discloses a garbage recycling method and device, and relates to the technical field of garbage recycling. In the method, a client of the distributed storage system can send a garbage collection request to a server of the distributed storage system. And the server can determine that the target unit data block is a valid data block according to the garbage collection request, and sends a moving command to a storage device storing the target unit data block, so that the storage device moves the target unit data block from a first storage position to a second storage position. According to the embodiment of the invention, the server side carries out data migration among the storage positions of the storage devices accessed by the server side rather than the client side carries out data migration among the server sides, so that the network bandwidth between the client side and the server side is reduced, and the migration operation is carried out by the storage devices, so that the data migration efficiency is improved. And the moving operation is not executed by the CPUs of the client and the server, so that the CPU overhead between the client and the server is reduced, and the garbage recycling efficiency is improved.

Description

[0001] This application requires to be submitted to the State Intellectual Property Office of the People's Republic of China on July 17, 2023, application number 202310876345.2, Chinese patent application entitled "A method, device and other equipment for garbage recycling" Priority, the entire contents of which are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of storage technology, and in particular to a garbage collection method and device. Background Art

[0003] The garbage collection (GC) operation is an operation to recycle used storage space when the available storage space in the storage space is small. In a distributed storage system, the GC operation is generally performed uniformly by the client. For example, if the client saves the mapping relationship between data blocks and the storage devices accessed by each server, when performing a garbage collection operation, after the client determines that a data block stored in a storage device accessed by a server is a valid data block, it sends a read request to the server to read out the valid data block stored on the storage device accessed by the server. After the valid data block is read out by the client, the server sets the storage device as an available storage device. The client aggregates all the valid data blocks read out, and then scatters them again and writes them to the storage devices accessed by each server.

[0004] However, when the client performs garbage collection operations uniformly, data is often moved across servers, resulting in high network bandwidth overhead between the client and server, affecting garbage collection efficiency. In addition, when data is moved, the central processing unit (CPU) of the client and server is often required to participate in reading and writing, resulting in high CPU overhead on the client and server, which also affects garbage collection efficiency. Summary of the invention

[0005] The embodiments of the present application provide a garbage collection method and device for improving garbage collection efficiency.

[0006] In a first aspect, an embodiment of the present application provides a garbage collection method, which can be applied to a server of a distributed storage system, wherein the server accesses at least one storage device. The method includes: receiving a garbage collection request from a client of the distributed storage system, wherein the garbage collection request carries a target unit data block, wherein the target unit data block is a valid data block; the server can determine that the target unit data block is located at a first storage position of a first storage device, wherein the first storage device is one of the at least one storage devices; the server can determine a second storage position, wherein the second storage position is an idle position in the at least one storage device; the server can send a first move command to the first storage device, wherein the first move command is used to instruct the first storage device to move the target unit data block from the first storage position to the second storage position.

[0007] In the above scheme, the client can send a garbage collection request to a server. The server can determine that the target unit data block is a valid data block based on the garbage collection request, and send a move command to the storage device storing the target unit data block, so that the storage device moves the target unit data block from the first storage location to the second storage location. Since it is a server that moves data between the various storage locations of the storage devices it accesses, instead of the client moving data between the various servers, the network bandwidth overhead between the client and the server is reduced, and since the storage device performs the move operation instead of the CPU of the client and the server, the CPU overhead between the client and the server is reduced, and the garbage collection efficiency is improved.

[0008] In one possible implementation, the second storage location is located in a second storage device; the sending of a first move command to the first storage device includes: sending a second move command to the first storage device, the second move command including an identifier of the first storage device, the first storage location, an identifier of the second storage device, the second storage location, and a length of the target unit data block, the first storage location being used to indicate a storage location of the target unit data block in the first storage device before the move, and the second storage location being used to indicate a storage location of the target unit data block in the second storage device after the move.

[0009] In the above implementation, when the second storage location and the first storage location are storage locations of different storage devices, the server can send a cross-disk move command to the storage device storing the target unit data block. Since the storage device performs the move operation instead of the CPU of the client and the server, the CPU overhead between the client and the server is reduced and the garbage collection efficiency is improved.

[0010] In a possible implementation, after sending the second move command to the first storage device, it includes: updating the first mapping relationship to a second mapping relationship, the first mapping relationship being a mapping relationship between the target unit data block and the first storage position on the first storage device, and the second mapping relationship being a mapping relationship between the target unit data block and the second storage position on the second storage device.

[0011] In the above implementation, after performing data migration, the server can update the first mapping relationship between the target unit data block and the first storage position of the first storage device to a second mapping relationship between the target unit data block and the second storage position of the first storage device, so that after performing data migration, the server can also quickly determine the storage position of the target unit data block according to the second mapping relationship, thereby improving data reading and migration efficiency.

[0012] In a possible implementation, the second storage location is located in the first storage device; the sending of the first move command to the first storage device includes: sending a third move command to the first storage device, the third move command including the first storage location, the second storage location and the length of the target unit data block, the first storage location is used to indicate the storage location of the target unit data block in the first storage device before the move, and the second storage location is used to indicate the storage location of the target unit data block in the first storage device after the move.

[0013] In the above implementation, when the second storage location and the first storage location are storage locations of the same storage device, the server can send an on-disk move command to the storage device storing the target unit data block. Since the storage device performs the move operation instead of the CPU of the client and the server, the CPU overhead between the client and the server is reduced and the garbage collection efficiency is improved.

[0014] In a possible implementation, after sending the third move command to the first storage device, it includes: updating the first mapping relationship to a third mapping relationship, the first mapping relationship being a mapping relationship between the target unit data block and the first storage location on the first storage device, and the third mapping relationship being a mapping relationship between the target unit data block and the second storage location on the first storage device.

[0015] In the above implementation, after performing data migration, the server can update the first mapping relationship between the target unit data block and the first storage position of the first storage device to a third mapping relationship between the target unit data block and the second storage position of the second storage device, so that after performing data migration, the server can also quickly determine the storage position of the target unit data block according to the third mapping relationship, thereby improving data reading and migration efficiency.

[0016] In one possible implementation, the at least one storage device is a solid state drive (SSD); sending a first move command to the first storage device includes: sending the first move command to the first storage device through a non-volatile memory express (NVMe) protocol, the first move command being used to instruct a controller of the first storage device to move the target unit data block from the first storage location to the second storage location.

[0017] In the above implementation, when the storage device is an SSD, the server can send a move command to the controller of the storage device through the NVMe protocol, so that the controller of the storage device performs the move operation. Since the move operation is not performed by the CPU of the client and the server, the CPU overhead between the client and the server is reduced and the garbage collection efficiency is improved.

[0018] In a second aspect, an embodiment of the present application further provides a garbage collection device, which can be applied to a server of a distributed storage system, and the server accesses at least one storage device. The device has the function of implementing the method described in the first aspect or any possible design of the first aspect, and the function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions, such as a communication module and a processing module.

[0019] A communication module, configured to receive a garbage collection request from a client of the distributed storage system, wherein the garbage collection request carries a target unit data block, and the target unit data block is a valid data block;

[0020] A processing module, configured to determine that the target unit data block is located at a first storage position of a first storage device, wherein the first storage device is one of the at least one storage device; and determine a second storage position, wherein the second storage position is an idle position in the at least one storage device;

[0021] The communication module is further used to send a first move command to the first storage device, where the first move command is used to instruct the first storage device to move the target unit data block from the first storage location to the second storage location.

[0022] In one possible implementation, the second storage location is located in a second storage device; the communication module is specifically used to: send a second move command to the first storage device, the second move command including an identifier of the first storage device, the first storage location, an identifier of the second storage device, the second storage location, and a length of the target unit data block, the first storage location is used to indicate the storage location of the target unit data block in the first storage device before the move, and the second storage location is used to indicate the storage location of the target unit data block in the second storage device after the move.

[0023] In a possible implementation, the processing module is also used to: update the first mapping relationship to a second mapping relationship, wherein the first mapping relationship is a mapping relationship between the target unit data block and the first storage position on the first storage device, and the second mapping relationship is a mapping relationship between the target unit data block and the second storage position on the second storage device.

[0024] In a possible implementation, the second storage location is located in the first storage device; the communication module is specifically used to: send a third move command to the first storage device, the third move command includes the first storage location, the second storage location and the length of the target unit data block, the first storage location is used to indicate the storage position of the target unit data block in the first storage device before the move, and the second storage location is used to indicate the storage position of the target unit data block in the first storage device after the move.

[0025] In a possible implementation, the processing module is also used to: update the first mapping relationship to a third mapping relationship, wherein the first mapping relationship is a mapping relationship between the target unit data block and the first storage position on the first storage device, and the third mapping relationship is a mapping relationship between the target unit data block and the second storage position on the first storage device.

[0026] In one possible implementation, the at least one storage device is an SSD; the communication module is specifically used to: send the first move command to the first storage device through the NVME protocol, and the first move command is used to instruct the controller of the first storage device to move the target unit data block from the first storage location to the second storage location.

[0027] In a third aspect, an embodiment of the present application further provides a computing device cluster, including at least one computing device, and the structure of each computing device includes a processor and a memory. The processor is configured to support the server to execute the above-mentioned first aspect or any possible design method of the above-mentioned first aspect. The memory is coupled to the processor, which stores the necessary program instructions and data of the communication device. The communication device may also include a communication interface for communicating with other devices.

[0028] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium includes computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method of the first aspect or any possible design of the first aspect.

[0029] In a fifth aspect, an embodiment of the present application further provides a computer program product comprising instructions, which, when executed by a computing device cluster, enables the computing device cluster to execute the method of the first aspect or any possible design of the first aspect.

[0030] In a sixth aspect, an embodiment of the present application further provides a chip, which can be coupled to a memory and used to call a computer program stored in the memory to execute the method of the first aspect or any possible design of the first aspect.

[0031] The beneficial effects of the above-mentioned second to sixth aspects and possible designs thereof can refer to the description of the beneficial effects of the method described in the above-mentioned first aspect and any possible design thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a schematic diagram of a distributed storage system;

[0033] Figure 2 A schematic diagram of a garbage collection scenario;

[0034] Figure 3 is a flowchart of a garbage collection method;

[0035] Figure 4 This is a schematic diagram of another garbage collection scenario;

[0036] Figure 5 This is a schematic diagram of another garbage collection scenario;

[0037] Figure 6 It is a structural diagram of an SSD;

[0038] Figure 7 A schematic diagram of the structure of a computing device;

[0039] Figure 8 It is a structural diagram of a computing device cluster;

[0040] Fig. 9 A schematic diagram of a connection method between computing device clusters. DETAILED DESCRIPTION

[0041] The embodiment of the present application provides a garbage collection method and device. The method and the device are based on the same concept. Since the method and the device solve the problem in a similar principle, the implementation of the method and the device can refer to each other, and the repeated parts will not be repeated.

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the embodiments of the present application will be further described in detail with reference to the accompanying drawings. Among them, in the description of the embodiments of the present application, the term "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects before and after are in an "or" relationship. At least one involved in the embodiments of the present application refers to one or more; multiple refers to two or more. In addition, it should be understood that in the description of the embodiments of the present application, words such as "first" and "second" are only used to distinguish the purpose of description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0043] To facilitate understanding, exemplary descriptions of concepts related to the present application are provided for reference.

[0044] 1) Storage system, mainly composed of processors and memories. The number of processors and memories is not limited. The memory is composed of a memory controller and a storage unit of non-volatile medium particles such as flash memory.

[0045] Optionally, the storage system may be a distributed storage system (DSS).

[0046] A distributed storage system consists of a client and multiple servers. The client is used to manage the server, and the server is used to store data. The client in a distributed storage system is equivalent to the processor in the storage system, and the server in a distributed storage system is equivalent to the memory in the storage system.

[0047] The server, also known as a storage server or storage node, is composed of multiple storage devices. The storage device provides physical storage space for the server. The storage device in the server is equivalent to a storage unit in a memory, and the storage device includes many storage locations. Optionally, the storage device can be a disk.

[0048] Disk refers to a storage device that uses magnetic recording technology to store data. Disk is the main storage medium of computers. It can store a large amount of binary data and keep the data intact even after a power outage. The disk used in early computers was a floppy disk (floppy disk for short), and the commonly used disk today is a hard disk (hard disk for short).

[0049] For example, Figure 1 A schematic diagram of a distributed storage system is shown in FIG. Figure 1 As shown, the distributed storage system 100 includes a client 110 and multiple server ends 120. Figure 1 In the example, a distributed storage system 100 includes four servers (server 1 to server 4). Each server 120 includes multiple storage devices. Figure 1 In the example, server 1 includes 3 storage devices (respectively storage device A to storage device C), server 2 includes 3 storage devices (respectively storage device D to storage device F), server 3 includes 3 storage devices (respectively storage device G to storage device I), and server 4 includes 3 storage devices (respectively storage device J to storage device L).

[0050] Each storage device in the storage device A to the storage device M may be a hard disk drive (HDD) and an SSD, etc. The interface of each storage device in the storage device A to the storage device M may include a serial advanced technology attachment (SATA) interface, a small computer system interface (SCSI), a serial attached SCSI interface (SAS), a fiber channel interface (FC), or an interface supporting the NVMe protocol, etc.

[0051] 2) Unit data block (grain), a grain is stored in a storage location of a storage device on a server in a distributed storage system.

[0052] A stripe, also known as a data segment, consists of multiple grains, which are stored on multiple servers.

[0053] 3) Data processing request, which may be referred to as input / output (IO) request, IO, hereinafter referred to as IO request, refers to a request sent by a processor in a storage system to a memory in the storage system to instruct a memory controller of the memory to perform a data processing operation. An IO request may include at least one of a read request, a write request, and an erase request.

[0054] Taking the storage system as a distributed storage system as an example, the client can send an IO request to the server, and the server performs a corresponding IO operation according to the received IO request. The IO operation may include at least one of a read operation, a write operation, and an erase operation.

[0055] For example, a client may send a write request to multiple servers to disperse and store multiple grains included in a stripe on multiple servers. Correspondingly, a server may receive a write request from a client and store a grain indicated by the write request in a storage location on a storage device.

[0056] 4) Garbage collection (GC) operation is an operation to reclaim used storage space when the available storage space in the storage space is small. Its specific operations include: reading out the valid data in the selected storage location to be reclaimed and migrating it to the target storage location in the available storage space, and then erasing all the data in the storage location to be reclaimed, thereby reclaiming the storage space corresponding to the storage location to be reclaimed through the GC operation.

[0057] In a distributed storage system, garbage collection operations are generally performed uniformly by the client. For example, if the client stores the mapping relationship between data blocks and storage devices of each server, when performing garbage collection operations, after the client determines that a data block stored in a storage device of a server is a valid data block, it sends a read request to the server to read out the valid data block stored on the storage device of the server. After the valid data block is read out by the client, the server sets the storage device as an available storage device. The client aggregates all the valid data blocks read out, and then scatters them again and writes them to the storage devices managed by each server.

[0058] For example, Figure 2 is a schematic diagram of a garbage collection scenario, such as Figure 2As shown, the client sends a read request to the server 1 and the server 2, and reads out the valid data block 1 stored on the storage device A of the server 1, the valid data block 2 on the storage device C of the server 1, the valid data block 3 on the storage device D of the server 2, and the valid data block 4 on the storage device F of the server 2. The server 1 sets the storage device A and the storage device C as available storage devices, and the server 2 sets the storage device D as an available storage device. The client aggregates the valid data blocks read previously, and then scatters them again and writes them to various storage devices managed by the server 1 and the server 2, such as writing the valid data block 1 to the storage device A of the server 1, writing the valid data block 2 to the storage device B of the server 1, writing the valid data block 3 to the storage device C of the server 1, and writing the valid data block 4 to the storage device D of the server 2. Among them, valid data block 2 is moved from the previous storage device C of server 1 to the storage device B of server 1, and valid data block 4 is moved from the previous storage device F of server 2 to the storage device D of server 2, that is, valid data block 2 and valid data block 4 are not moved across servers. Valid data block 3 is migrated from the previous storage device D of server 2 to the storage device C of server 1, that is, valid data block 3 is moved across servers.

[0059] It can be seen that when the client performs garbage collection operations uniformly, data is often moved across servers, resulting in high network bandwidth overhead between the client and the server, affecting garbage collection efficiency. In addition, when data is moved, the CPU of the client and the server is often required to participate in reading and writing, resulting in high CPU overhead on the client and the server, which also affects garbage collection efficiency.

[0060] In response to the problem of low garbage collection efficiency described above, an embodiment of the present application provides a garbage collection method. In this method, a client can send a garbage collection request to a server. The server can determine that the target unit data block is a valid data block based on the garbage collection request, and send a move command to the storage device storing the target unit data block, so that the storage device moves the target unit data block from a first storage location to a second storage location. Since it is a server that moves data between the various storage locations of the storage devices it accesses, rather than the client moving data between the various servers, the network bandwidth overhead between the client and the server is reduced, and since the storage device performs the move operation, rather than the CPU of the client and the server performing the move operation, the CPU overhead between the client and the server is reduced, and the garbage collection efficiency is improved.

[0061] For example, Figure 3 A schematic diagram of a waste collection method is shown in FIG. Figure 1 As shown or with Figure 1The distributed storage system with similar functional structure is not specifically limited in the present application embodiment. The specific process of the garbage collection method is described as follows. Figure 1 For example, the client described below may be Figure 1 The client 110 in the system shown in the figure, the server described below can be Figure 1 The server 120 in the system shown. In the embodiment of the present application, the distributed storage system is taken as an example to introduce the execution subject of the method, or the execution subject of the method can also be other devices or systems, such as chips, chip systems or processors applied to distributed storage systems.

[0062] Step S301: The client sends a garbage collection request to the server. Correspondingly, the server receives the garbage collection request from the client.

[0063] In an embodiment of the present application, when performing garbage collection, after the client determines that the target unit data block is a valid data block, the client can send a garbage collection request to the server. Correspondingly, the server receives the garbage collection request from the client. The garbage collection request is used to request the server to perform garbage collection, and the garbage collection request can carry the target unit data block, and the target unit data block is a valid data block.

[0064] Exemplarily, the client may store a mapping relationship between stripes and unit data blocks as shown in Table 1, and a mapping relationship between unit data blocks and the server as shown in Table 2.

[0065] stripe1 grain11 grain12 grain13 stripe2 grain21 grain22 grain23

[0066] Table 1

[0067] grain11 Server 1 grain12 Server 1 grain13 Server 1 grain21 Server 2 grain22 Server 3 grain23 Server 4

[0068] Table 2

[0069] If the client determines that grain11, grain13, grain21, and grain23 are valid data blocks, and grain12 and grain22 are invalid data blocks, then the client can send corresponding garbage collection requests to each server according to Table 2 above. For example, garbage collection request 1 is sent to server 1, and garbage collection request 1 carries grain11 and grain13; garbage collection request 2 is sent to server 2, and garbage collection request 2 carries grain21; garbage collection request 3 is sent to server 3, and garbage collection request 3 carries grain23.

[0070] Step S302: The server determines that the target unit data block is located at a first storage location of a first storage device of the server.

[0071] In an embodiment of the present application, after the server receives a garbage collection request carrying a target unit data block from a client, the server can determine that the target unit data block is located at a first storage location of a first storage device of the server based on a mapping relationship between the unit data block and the storage location of the storage device. The first storage device can be one of at least one storage device of the server.

[0072] Exemplarily, the server 1 may store a mapping relationship between a unit data block and a storage location of a storage device as shown in Table 3.

[0073] grain11 Storage device A, storage location 10 grain12 Storage device B, storage location 20 grain13 Storage device C, storage location 30

[0074] Table 3

[0075] If server 1 receives garbage collection request 1 from client, and garbage collection request 1 carries grain 11 and grain 13, that is, grain 11 and grain 13 are valid data blocks, server 1 can determine according to the above Table 3 that valid data block grain 11 is located at storage location 10 of storage device A of server 1, invalid data block grain 12 is located at storage location 20 of storage device B of server 1, and valid data block grain 13 is located at storage location 30 of storage device C of server 1.

[0076] Optionally, if the storage device A, the storage device B, and the storage device C are SSDs, the server 1 may store a mapping relationship between the unit data blocks and the storage locations of the storage devices as shown in Table 4.

[0077] grain11 SSD A, Sector 2 grain12 SSD B, sector 4 grain13 SSD C, sector 6

[0078] Table 4

[0079] Table 4 is used to indicate the location of grain storage on server 1, which includes the SSD where the grain is located and the offset of the grain on the SSD. For example, grain 11 is located on SSD A, and the offset of grain 11 on SSD A is sector 2. If 16 sectors are used as one storage location, grain 1 is located from sector 2 to sector 18 of SSD 1. That is, the starting address of storage location 10 is sector 2, and the ending address of storage location 10 is sector 18.

[0080] Step S303: The server determines a second storage location.

[0081] In an embodiment of the present application, after the server determines that the target unit data block is located at the first storage position of the first storage device of the server, the server can determine the second storage position based on the mapping relationship between the unit data block and the storage position of the storage device. The second storage position can be any free position of any storage device of at least one storage device of the server. The free position (also called an available storage position) can be a position where no data block is stored or no valid data block is stored (i.e., an invalid data block is stored).

[0082] Optionally, the server may determine the second storage location based on the number of valid data blocks stored in the first storage device. Specifically, if the number of valid data blocks stored in the first storage device is less than the first threshold, the server may determine the second storage location to be any free location of the first storage device; if the number of valid data blocks stored in the first storage device is greater than or equal to the first threshold, the server may determine the second storage location to be any free location of the second storage device. The first threshold may be a default threshold or a threshold set by the user as required, which is not limited in the embodiments of the present application.

[0083] Step S304: The server sends a first migration command to the first storage device.

[0084] In an embodiment of the present application, after the server determines the second storage location, the server may send a first move command to the first storage device. The first move command is used to instruct the first storage device to move the target unit data block from the first storage location to the second storage location. Since the first storage device performs the move operation, rather than the CPU of the client and the server, the CPU overhead between the client and the server is reduced, and the garbage collection efficiency is improved.

[0085] Specifically, if the second storage location is located in the second storage device, the server can send a second move command (also called a cross-disk move command Move_Cross_Disk) to the first storage device. The second move command includes the identifier of the first storage device, the first storage location, the identifier of the second storage device, the second storage location, and the length of the target unit data block. The first storage location is used to indicate the storage location of the target unit data block in the first storage device before the move. The second storage location is used to indicate the storage location of the target unit data block in the second storage device after the move.

[0086] For example, Figure 4 is a schematic diagram of another garbage collection scenario, such as Figure 4As shown in (1), storage device A includes 4 storage locations, namely storage location 10, storage location 11, storage location 12 and storage location 13. Storage device B includes 4 storage locations, namely storage location 20, storage location 21, storage location 22 and storage location 23. If server 1 receives garbage collection request 1 carrying grain0, grain1, grain2 and grain4 from the client, server 1 can determine that the valid data blocks stored in storage device A include: grain0 located at storage location 11, grain1 located at storage location 12, and grain2 located at storage location 13 according to the mapping relationship between unit data blocks and storage locations of storage devices as shown in Table 5. The valid data blocks stored in storage device B include: grain4 located at storage location 21; the invalid data blocks stored in storage device B include: grain3 located at storage location 20.

[0087] grain0 Storage device A, storage location 11 grain1 Storage device A, storage location 12 grain2 Storage device A, storage location 13 grain3 Storage device B, storage location 20 grain4 Storage device B, storage location 21

[0088] Table 5

[0089] like Figure 4 As shown in (2), if server 1 determines to move grain0 from storage location 11 to storage location 20, server 1 can send command 1 to storage device A, so that storage device A moves grain0 from storage location 11 to storage location 20. Command 1 is a cross-disk move command Move_Cross_Disk (diskA, offset11, diskB, offset20, len0), where disk A is the identifier of storage device A, offset12 is the starting address of storage location 12, disk B is the identifier of storage device B, ooffset20 is the starting address of storage location 20, and len0 is the length of grain0. After receiving command 1, storage device A executes the cross-device move, thereby moving grain0 from storage location 11 to storage location 20.

[0090] After the server sends the second move command to the first storage device, the server may also update the first mapping relationship to a second mapping relationship. The first mapping relationship is a mapping relationship between the target unit data block and the first storage location on the first storage device. The second mapping relationship is a mapping relationship between the target unit data block and the second storage location on the second storage device.

[0091] For example, in the above Figure 4In the garbage collection scenario shown, server 1 sends command 1 to storage device A, so that storage device A moves grain0 from storage location 11 to storage location 20. Server 1 can then update the mapping relationship between the unit data block and the storage location of the storage device as shown in Table 5 above, and obtain the mapping relationship between the unit data block and the storage location of the storage device as shown in Table 6.

[0092] grain0 Storage device B, storage location 20 grain1 Storage device A, storage location 10 grain2 Storage device A, storage location 11 grain4 Storage device B, storage location 21

[0093] Table 6

[0094] Further, when the server 1 receives the write request from the client, the server 1 can determine according to the above Table 6 that the storage location 10 and the storage location 11 of the storage device A store valid data blocks, the storage location 12 and the storage location 13 of the storage device A do not store data blocks or do not store valid data blocks, that is, the storage location 12 and the storage location 13 of the storage device A are free locations, and the storage location 20 and the storage location 21 of the storage device B store valid data blocks, the storage location 22 and the storage location 23 of the storage device B do not store data blocks or do not store valid data blocks, that is, the storage location 22 and the storage location 23 of the storage device B are free locations. Therefore, the server 1 can write the unit data block indicated by the write request to the storage location 12 of the storage device A, the storage location 13 of the storage device A, the storage location 22 of the storage device B, or the storage location 23 of the storage device B.

[0095] Specifically, if the second storage location is located in the first storage device, the server may send a third move command (also called the disk move command Move_In_Disk) to the first storage device. The third move command includes the first storage location, the second storage location, and the length of the target unit data block. The first storage location is used to indicate the storage location of the target unit data block in the first storage device before the move. The second storage location is used to indicate the storage location of the target unit data block in the first storage device after the move.

[0096] For example, Figure 5 is a schematic diagram of another garbage collection scenario, such as Figure 5 As shown in (1), storage device A includes four storage locations, namely storage location 10, storage location 11, storage location 12 and storage location 13. If server 1 receives garbage collection request 1 carrying grain0 and grain1 from the client, server 1 can determine that the valid data blocks stored in storage device A include: grain0 located at storage location 12, grain1 located at storage location 13; and the invalid data blocks stored in storage device A include: grain2 located at storage location 11 according to the mapping relationship between unit data blocks and storage locations of storage devices as shown in Table 7.

[0097] grain0 Storage device A, storage location 12 grain1 Storage device A, storage location 13 grain2 Storage device A, storage location 11

[0098] Table 7

[0099] like Figure 5 As shown in (2), if server 1 determines to move grain0 from storage location 12 to storage location 10, and to move grain1 from storage location 13 to storage location 11, server 1 can send command 1 and command 2 to storage device A in sequence. Command 1 is the disk move command Move_In_Disk(offset12, offset10, len0), where offset12 is the starting address of storage location 12, offset10 is the starting address of storage location 10, and len0 is the length of grain0. Command 2 is the disk move command Move_In_Disk(offset13, offset11, len1), where offset13 is the starting address of storage location 13, offset11 is the starting address of storage location 11, and len1 is the length of grain1. After receiving command 1 and command 2, storage device A executes the in-device move in sequence, thereby moving grain 0 from storage location 12 to storage location 10, and moving grain 1 from storage location 13 to storage location 11.

[0100] After the server sends the third move command to the first storage device, the server may also update the first mapping relationship to a third mapping relationship. The first mapping relationship is a mapping relationship between the target unit data block and the first storage location on the first storage device. The third mapping relationship is a mapping relationship between the target unit data block and the second storage location on the first storage device.

[0101] For example, in the above Figure 5 In the garbage collection scenario shown, server 1 sends command 1 and command 2 to storage device A, so that storage device A moves grain 0 from storage location 12 to storage location 10, and moves grain 1 from storage location 13 to storage location 11. Then, server 1 can update the mapping relationship between the unit data block and the storage location of the storage device as shown in Table 7 above, and obtain the mapping relationship between the unit data block and the storage location of the storage device as shown in Table 8.

[0102] grain0 Storage device A, storage location 10 grain1 Storage device A, storage location 11

[0103] Table 8

[0104] Further, when the server 1 receives the write request from the client, the server 1 can determine according to the above Table 8 that the storage location 10 and the storage location 11 of the storage device A store valid data blocks, and the storage location 12 and the storage location 13 of the storage device A do not store data blocks or do not store valid data blocks, that is, the storage location 12 and the storage location 13 of the storage device A are free locations. Therefore, the server 1 can write the unit data block indicated by the write request into the storage location 12 and the storage location 13 of the storage device A.

[0105] Optionally, if at least one storage device of the server is an SSD, the server may send a first move command to the first storage device via the NVME protocol, wherein the first move command is used to instruct the controller of the first storage device to move the target unit data block from the first storage location to the second storage location.

[0106] For example, Figure 6 A schematic diagram of the structure of an SSD provided in an embodiment of the present application, the SSD includes a flash memory and a main controller. The flash memory includes a plurality of flash memory chips for storing data. The main controller is the brain center of the SSD, responsible for some complex tasks, such as managing data storage, maintaining SSD performance and service life, etc. The main controller is an embedded microchip, which includes a processor, which functions like a command center, issuing all operation requests of the SSD. For example, the processor can perform functions such as reading / writing data, garbage collection and wear leveling through firmware in the buffer. The main controller also includes a host interface and several channel controllers. Among them, the host interface is used to communicate with the host. The host here can refer to any device such as a server, a personal computer or an array controller. Through several channel controllers, the main controller can operate multiple flash memory chips in parallel, thereby increasing the underlying bandwidth. For example, assuming that there are 8 channels between the main controller and the flash memory, the main controller reads and writes data to 8 flash memory chips in parallel through these 8 channels.

[0107] The first storage device and the second storage device may be the above Figure 6The SSD shown, that is, the first storage device can be SSD1 and the second storage device can be SSD2. If the second storage location is located in the second storage device, the server can send a second move command to SSD1 through the NVME protocol, that is, the host interface of SSD1 is the NVME interface. Among them, the second move command is used to instruct the main controller of SSD1 to move the target unit data block from the flash memory chip 1 of SSD1 to the flash memory chip 2 of SSD2. After SSD1 receives the second move command through the host interface of SSD1, the main controller of SSD1 can read the target unit data block stored in the flash memory chip 1 through the channel controller corresponding to the flash memory chip 1, and send a write request to SSD2 through the NVME protocol, that is, the host interface of SSD2 is the NVME interface. Among them. The write request is used to request the main controller of SSD2 to write the target unit data block to the flash memory chip 2. After SSD2 receives the write request through the host interface of SSD2, the main controller of SSD2 can write the target unit data block to the flash memory chip 2 through the channel controller corresponding to the flash memory chip 2.

[0108] If the second storage location is located in the first storage device, the server can send a third move command to SSD1 through the NVME protocol, wherein the third move command is used to instruct the main controller of SSD1 to move the target unit data block from the flash memory chip 1 of SSD1 to the flash memory chip 3 of SSD1. After SSD1 receives the second move command through the host interface of SSD1, the main controller of SSD1 can read the target unit data block stored in the flash memory chip 1 through the channel controller corresponding to the flash memory chip 1, and write the target unit data block into the flash memory chip 3 through the channel controller corresponding to the flash memory chip 3.

[0109] According to the above scheme, the client can send a garbage collection request to a server. The server can determine that the target unit data block is a valid data block based on the garbage collection request, and send a move command to the storage device storing the target unit data block, so that the storage device moves the target unit data block from the first storage location to the second storage location. Since it is a server that moves data between the various storage locations of the storage devices it accesses, rather than the client moving data between the various servers, the network bandwidth overhead between the client and the server is reduced, and since the storage device performs the move operation, rather than the CPU of the client and the server performing the move operation, the CPU overhead between the client and the server is reduced, and the garbage collection efficiency is improved.

[0110] Based on the above embodiments, the present application also provides a computing device, such as Figure 7As shown, it is a schematic diagram of the structure of a computing device provided in an embodiment of the present application, and the computing device 700 includes: a bus 702, a processor 704, a memory 706, and a communication interface 708. The processor 704, the memory 706, and the communication interface 708 communicate through the bus 702. The computing device 700 can be a server in a distributed storage system. It should be understood that the present application does not limit the number of processors and memories in the computing device 700.

[0111] The bus 702 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus 702 may include a path for transmitting information between various components of the computing device 700 (eg, the memory 706, the processor 704, and the communication interface 708).

[0112] The processor 704 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0113] The memory 706 may include a volatile memory, such as a random access memory (RAM). The memory 706 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0114] The memory 706 stores executable program codes, and the processor 704 executes the executable program codes to respectively implement the functions of the communication module and the processing module, thereby implementing the garbage collection method in the above method embodiment. That is, the memory 706 stores instructions for executing the garbage collection method in the above method embodiment.

[0115] The communication interface 708 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 700 and other devices or communication networks.

[0116] Based on the above embodiments, the embodiments of the present application further provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server in a distributed storage system.

[0117] like Figure 8 As shown, the computing device cluster includes at least one computing device 700. The memory 706 in one or more computing devices 700 in the computing device cluster may store the same instructions for executing the garbage collection method in the above method embodiment.

[0118] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the garbage collection method in the above method embodiment. In other words, the combination of one or more computing devices 700 can jointly execute the instructions for executing the garbage collection method in the above method embodiment.

[0119] It is understandable that the memory 706 in different computing devices 700 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of a server in the distributed storage system. That is, the instructions stored in the memory 706 in different computing devices 700 can implement the functions of one or more modules in the communication module and the processing module.

[0120] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig. 9 A possible implementation is shown. Fig. 9 As shown, two computing devices 700A and 700B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 706 in the computing device 700A stores instructions for executing the functions of the communication module. At the same time, the memory 706 in the computing device 700B stores instructions for executing the functions of the processing module.

[0121] It should be understood that Fig. 9 The functions of the computing device 700A shown in FIG. 7 may also be completed by multiple computing devices 700. Similarly, the functions of the computing device 700B may also be completed by multiple computing devices 700.

[0122] Based on the above embodiments, the embodiments of the present application also provide a computer-readable storage medium, which can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc. The computer-readable storage medium includes instructions that instruct the computing device to execute the garbage collection method in the above method embodiment.

[0123] Based on the above embodiments, the present application also provides a computer program product including instructions, which may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the garbage collection method in the above method embodiment.

[0124] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0125] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0126] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0128] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A garbage collection method, characterized in that: The method is applied to a server of a distributed storage system, the server accessing at least one storage device, and the method comprises: Receiving a garbage collection request from a client of the distributed storage system, wherein the garbage collection request carries a target unit data block, and the target unit data block is a valid data block; Determining that the target unit data block is located at a first storage location of a first storage device, where the first storage device is one of the at least one storage device; determining a second storage location, the second storage location being an empty location in the at least one storage device; A first move command is sent to the first storage device, where the first move command is used to instruct the first storage device to move the target unit data block from the first storage location to the second storage location.

2. The method according to claim 1, characterized in that: The second storage location is located in a second storage device; and sending a first move command to the first storage device includes: A second move command is sent to the first storage device, wherein the second move command includes an identifier of the first storage device, the first storage position, an identifier of the second storage device, the second storage position, and a length of the target unit data block, wherein the first storage position is used to indicate a storage position of the target unit data block in the first storage device before the move, and the second storage position is used to indicate a storage position of the target unit data block in the second storage device after the move.

3. The method according to claim 2, characterized in that After sending the second move command to the first storage device, the method includes: The first mapping relationship is updated to a second mapping relationship, wherein the first mapping relationship is a mapping relationship between the target unit data block and the first storage position on the first storage device, and the second mapping relationship is a mapping relationship between the target unit data block and the second storage position on the second storage device.

4. The method according to claim 1, characterized in that The second storage location is located in the first storage device; and sending a first move command to the first storage device includes: A third move command is sent to the first storage device, the third move command including the first storage position, the second storage position and the length of the target unit data block, the first storage position is used to indicate the storage position of the target unit data block in the first storage device before the move, and the second storage position is used to indicate the storage position of the target unit data block in the first storage device after the move.

5. The method according to claim 4, characterized in that After sending the third move command to the first storage device, the method includes: The first mapping relationship is updated to a third mapping relationship, wherein the first mapping relationship is a mapping relationship between the target unit data block and the first storage location on the first storage device, and the third mapping relationship is a mapping relationship between the target unit data block and the second storage location on the first storage device.

6. The method according to claim 1, characterized in that The at least one storage device is a solid state drive (SSD); and sending a first move command to the first storage device includes: The first move command is sent to the first storage device via the non-volatile memory standard NVME protocol, where the first move command is used to instruct a controller of the first storage device to move the target unit data block from the first storage location to the second storage location.

7. A garbage collection device, characterized in that: The device is applied to a server of a distributed storage system, the server accesses at least one storage device, and the device includes: A communication module, configured to receive a garbage collection request from a client of the distributed storage system, wherein the garbage collection request carries a target unit data block, and the target unit data block is a valid data block; A processing module, configured to determine that the target unit data block is located at a first storage position of a first storage device, wherein the first storage device is one of the at least one storage device; and determine a second storage position, wherein the second storage position is an idle position in the at least one storage device; The communication module is further used to send a first move command to the first storage device, where the first move command is used to instruct the first storage device to move the target unit data block from the first storage location to the second storage location.

8. The device according to claim 7, characterized in that The second storage location is located in a second storage device; and the communication module is specifically configured to: A second move command is sent to the first storage device, wherein the second move command includes an identifier of the first storage device, the first storage position, an identifier of the second storage device, the second storage position, and a length of the target unit data block, wherein the first storage position is used to indicate a storage position of the target unit data block in the first storage device before the move, and the second storage position is used to indicate a storage position of the target unit data block in the second storage device after the move.

9. The device according to claim 8, characterized in that The processing module is further used for: The first mapping relationship is updated to a second mapping relationship, wherein the first mapping relationship is a mapping relationship between the target unit data block and the first storage position on the first storage device, and the second mapping relationship is a mapping relationship between the target unit data block and the second storage position on the second storage device.

10. The device according to claim 7, characterized in that The second storage location is located in the first storage device; the communication module is specifically used to: A third move command is sent to the first storage device, the third move command including the first storage position, the second storage position and the length of the target unit data block, the first storage position is used to indicate the storage position of the target unit data block in the first storage device before the move, and the second storage position is used to indicate the storage position of the target unit data block in the first storage device after the move.

11. The device according to claim 10, characterized in that The processing module is further used for: The first mapping relationship is updated to a third mapping relationship, wherein the first mapping relationship is a mapping relationship between the target unit data block and the first storage location on the first storage device, and the third mapping relationship is a mapping relationship between the target unit data block and the second storage location on the first storage device.

12. The device according to claim 7, characterized in that The at least one storage device is an SSD; and the communication module is specifically used for: The first move command is sent to the first storage device through the NVME protocol, and the first move command is used to instruct a controller of the first storage device to move the target unit data block from the first storage location to the second storage location.

13. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device includes a processor and a memory; the memory in the at least one computing device is used to store computer program instructions; The processor in the at least one computing device calls the computer program instructions stored in the memory to execute the method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device, the computing device performs the method according to any one of claims 1 to 6.

15. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device, the computing device is caused to perform the method as claimed in any one of claims 1 to 6.

Images