A garbage collection method, device, equipment and medium of a solid state disk

Abstract

This invention provides a method, apparatus, device, and medium for garbage collection of a solid-state drive (SSD), belonging to the field of server technology. The method includes: dividing data in a distributed storage system into different types of data and assigning corresponding data tags to each type; writing the data in the distributed storage system to the SSD in a data stream manner according to the different data tags; and when the SSD receives a garbage collection operation command, controlling the SSD to perform garbage collection operations on the stored data in the SSD according to the different data streams. This method can write data with the same or similar lifecycles from the distributed storage system to the SSD, thus reducing the workload of garbage collection and significantly improving the performance of the SSD during garbage collection.

Description

Technical Field

[0001] This invention relates to the field of server technology, and in particular to a method, apparatus, device, and medium for garbage collection of solid-state drives. Background Technology

[0002] Solid State Drives (SSDs) are becoming the mainstream storage medium due to their advantages such as low power consumption, low noise, small size, light weight, and fast read / write speeds. Because SSDs store different types of data in a mixed manner, a single storage unit contains both cold and hot data. Since hot data is updated more frequently, during garbage collection (GC), the SSD needs to copy the infrequently updated cold data to other empty storage units and then erase the original storage unit containing the cold data. This causes the SSD to pause or slow down the host's input / output (I / O) operations during garbage collection, resulting in poor SSD performance. Currently, there is no effective solution to this technical problem. Summary of the Invention

[0003] The purpose of this invention is to provide a method, apparatus, device, and medium for garbage collection of solid-state drives (SSDs) to solve the problem of poor performance of SSDs when performing garbage collection operations in related technologies.

[0004] To address the aforementioned technical problems, this invention provides a method for garbage collection of a solid-state drive, comprising:

[0005] Divide the data in the distributed storage system into different types of data and set corresponding data labels for each type of data;

[0006] Data in the distributed storage system is written to the solid-state drive in a data stream manner according to different data tags;

[0007] When the solid-state drive receives a garbage collection operation command, it is controlled to perform garbage collection operations on the stored data in the solid-state drive according to the different data streams.

[0008] In one specific embodiment of this application, dividing the data in the distributed storage system into different types of data and setting corresponding data tags for each type of data includes:

[0009] The data in the distributed storage system is divided into cold data, hot data, metadata, and system data, and a first label, a second label, a third label, and a fourth label are set for the cold data, the hot data, the metadata, and the system data, respectively.

[0010] In one specific embodiment of this application, the step of dividing the data in the distributed storage system into cold data, hot data, metadata, and system data, and setting a first tag, a second tag, a third tag, and a fourth tag for the cold data, the hot data, the metadata, and the system data respectively, includes:

[0011] Real-time detection of whether the storage pool of the distributed storage system receives the target data sent by the user terminal;

[0012] If not, continue with the step of real-time detection of whether the storage pool of the distributed storage system has received the target data sent by the user terminal;

[0013] If so, the target data is differentiated by heat level to obtain the cold data and the hot data;

[0014] The cold data and the hot data are stored in a slow storage pool and a fast storage pool, respectively.

[0015] The first tag and the second tag are respectively set for the cold data and the hot data;

[0016] The cold data with the first tag and the hot data with the second tag are sent to the object storage device of the distributed storage system;

[0017] The object storage device is used to obtain data describing the cold data and the hot data to obtain the metadata, and the third tag is set for the metadata.

[0018] The cold data with the first tag, the hot data with the second tag, and the metadata with the third tag are sent to the storage device engine of the distributed storage system;

[0019] The storage device engine is used to identify the data generated by the distributed storage system as system data, and the fourth tag is set for the system data.

[0020] In one specific embodiment of this application, the step of distinguishing the target data by heat level to obtain the cold data and the hot data includes:

[0021] The target data is differentiated by popularity based on multiple data dimensions to obtain cold data and hot data; wherein, the data dimensions include: data access frequency, user name, and data storage path.

[0022] In one specific embodiment of this application, it further includes:

[0023] The configuration information related to the hot data and the cold data is stored in an object storage device mapping table;

[0024] The object storage device mapping table is sent to the object storage device via a monitor in the distributed storage system.

[0025] In one specific embodiment of this application, writing data from the distributed storage system to the solid-state drive in a data stream manner according to different data tags includes:

[0026] The storage device engine specifies a first data stream, a second data stream, a third data stream, and a fourth data stream for the cold data, the hot data, the metadata, and the system data;

[0027] The first data stream, the second data stream, the third data stream, and the fourth data stream are encapsulated using the driver layer of the distributed storage system, and then written to the solid-state drive.

[0028] In one specific embodiment of this application, the encapsulation of the first data stream, the second data stream, the third data stream, and the fourth data stream using the driver layer of the distributed storage system includes:

[0029] Based on the non-volatile memory host controller interface specification, and using the storage system development kit, the first data stream, the second data stream, the third data stream, and the fourth data stream are encapsulated.

[0030] To address the aforementioned technical problems, the present invention also provides a solid-state drive (SSD) garbage collection device, comprising:

[0031] The data partitioning module is used to divide the data in the distributed storage system into different types of data and set corresponding data labels for each type of data.

[0032] The data writing module is used to write data from the distributed storage system to the solid-state drive in the form of a data stream according to different data tags.

[0033] The garbage collection module is used to control the solid-state drive to perform garbage collection operations on the stored data in the solid-state drive according to the different data streams when the solid-state drive receives a garbage collection operation instruction.

[0034] To address the aforementioned technical problems, the present invention also provides a solid-state drive (SSD) waste recycling device, comprising:

[0035] Memory, used to store computer programs;

[0036] A processor is configured to implement the steps of a garbage collection method for a solid-state drive as disclosed above when executing the computer program.

[0037] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a garbage collection method for a solid-state drive as disclosed above.

[0038] In the solid-state drive garbage collection method provided by this invention, the data in the distributed storage system is first divided into different types of data, and a corresponding data tag is set for each type of data; then, the data in the distributed storage system is written to the solid-state drive in the form of a data stream according to the different data tags; when the solid-state drive receives a garbage collection operation instruction, the solid-state drive is controlled to perform garbage collection operation on the stored data in the solid-state drive according to the different data streams.

[0039] The beneficial effects of this invention are that the method provided by this invention is equivalent to writing data with the same or similar lifecycles in a distributed storage system to a solid-state drive (SSD). This reduces the workload of garbage collection during SSD operations, improving storage efficiency. Furthermore, in this configuration, the host computer can process multiple data streams in parallel, reducing the impact of SSD garbage collection on host performance and allowing the host to respond more quickly to user requests and applications. Therefore, this method can significantly improve the performance of SSDs during garbage collection.

[0040] Correspondingly, the solid-state drive waste recycling device, equipment, and storage medium provided by the present invention also have the above-mentioned beneficial effects. Attached Figure Description

[0041] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart illustrating a solid-state drive garbage collection method provided in an embodiment of the present invention;

[0043] Figure 2 This is a diagram illustrating the performance consumption of solid-state drives (SSDs) during garbage collection operations in related technologies.

[0044] Figure 3 This is a schematic diagram illustrating the classification of data in a distributed storage system according to an embodiment of the present invention.

[0045] Figure 4 This is a schematic diagram illustrating how data in a distributed storage system is written to a solid-state drive in the form of a data stream, as provided in an embodiment of the present invention.

[0046] Figure 5 This is a schematic diagram illustrating data interaction between management software, a monitor, and an object storage device provided in an embodiment of the present invention.

[0047] Figure 6 This is a structural diagram of a solid-state drive waste recycling device provided in an embodiment of the present invention;

[0048] Figure 7 This is a structural diagram of a solid-state drive (SSD) waste recycling device provided in an embodiment of the present invention;

[0049] Figure 8 This is a schematic diagram of the specific structure of a solid-state drive waste recycling device provided in an embodiment of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0051] The terms "comprising" and "having," and any variations thereof, in the specification and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may include steps or units not listed.

[0052] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0053] Please see Figure 1 , Figure 1 A flowchart of a solid-state drive garbage collection method provided in an embodiment of the present invention is shown, the method comprising:

[0054] Step S11: Divide the data in the distributed storage system into different types of data, and set corresponding data tags for each type of data;

[0055] Step S12: Write the data in the distributed storage system to the solid-state drive in the form of a data stream according to the different data tags;

[0056] Step S13: When the solid-state drive receives a garbage collection operation command, the solid-state drive is controlled to perform garbage collection operations on the stored data in the solid-state drive according to the different data streams.

[0057] To enable those skilled in the art to more clearly understand the implementation principle of this invention, a brief explanation of the existing garbage collection operation of solid-state drives (SSDs) is provided first. From the perspective of garbage collection principles, SSDs store different types of data in a mixed manner. This means that the same storage unit of an SSD may simultaneously store cold data and hot data. Since hot data is updated more frequently, when the SSD triggers garbage collection, it needs to copy the infrequently updated cold data to other empty storage units and then erase the original storage unit containing the cold data. This degrades the performance of the SSD. Please refer to [link to relevant documentation]. Figure 2 , Figure 2 This diagram illustrates the performance consumption of solid-state drives (SSDs) during garbage collection operations in related technologies. Figure 2 In the diagram, the red line represents the performance output of the solid-state drive (SSD), the blue line represents the performance consumption of the SSD during garbage collection, and the green line represents the total bandwidth of the SSD.

[0058] To improve the performance of solid-state drives (SSDs), this embodiment first requires dividing the data in the distributed storage system into different types and assigning corresponding data tags to each type. The data in the distributed storage system includes both data sent from user terminals to the distributed storage system and data generated by the distributed storage system itself.

[0059] Specifically, the data in the distributed storage system is divided into different types, and corresponding data labels are assigned to each type of data, including:

[0060] Based on the data's creation time, I / O size, and source, the data in the distributed storage system is divided into different types, and corresponding data tags are set for each type of data.

[0061] In practical applications, when dividing data in a distributed storage system into different types, this can be done based on creation time, I / O size, or data source, with each type assigned a corresponding data tag. This approach allows for centralized storage of data of the same type, optimizing storage performance and improving access speed for users accessing the same data type.

[0062] Of course, in practical applications, data in a distributed storage system can also be classified according to other attributes and characteristics, and corresponding data labels can be set for each type of data, which will not be elaborated here.

[0063] After dividing the data in the distributed storage system into different types of data and setting corresponding data tags for each type of data, it is also necessary to write the data in the distributed storage system to the solid-state drive in the form of data stream according to the different data tags. This is equivalent to writing data with the same or similar life cycle in the distributed storage system to the solid-state drive.

[0064] When a solid-state drive (SSD) receives a garbage collection command, it is controlled to perform garbage collection operations on the stored data according to the different data streams. It's conceivable that writing data with similar or identical lifecycles from a distributed storage system to the SSD can reduce write amplification (i.e., reducing the ratio between the actual amount of data written to the SSD and the amount of data requested to be written by the host). This reduction in write amplification extends the lifespan of the SSD and significantly improves its IOPS (Input / Output Operations Per Second).

[0065] When solid-state drives (SSDs) perform garbage collection in a multi-stream manner, they can process multiple data streams simultaneously, efficiently handling concurrent operations. In this configuration, while one data stream is performing garbage collection, the SSD can still provide I / O responses to other data streams on the host. This reduces the impact of SSD garbage collection on host I / O performance, thereby improving the overall performance of the distributed storage system. The specific advantages of this approach are as follows:

[0066] First, it improves the performance of solid-state drives (SSDs). SSDs can perform garbage collection operations efficiently in the background, and multi-stream SSDs can reduce performance flicker during garbage collection, providing more stable and consistent I / O performance.

[0067] Second, it improves the host's response speed to requests. Because solid-state drives (SSDs) reduce interference with host I / O operations during garbage collection, the host can respond to user requests and applications more quickly.

[0068] Third, it enhances the multitasking capabilities of solid-state drives (SSDs). Multi-stream technology allows SSDs to perform garbage collection while handling multiple concurrent I / O requests, which is highly beneficial for environments that require handling a large number of concurrent tasks. Furthermore, the parallel processing of multiple data streams by SSDs can more effectively handle concurrent I / O requests from different applications.

[0069] Fourth, improve the QoS (Quality of Service) of distributed storage systems. Multi-streaming technology can improve the service level between different applications or virtual machines, ensuring that critical applications receive sufficient I / O resources;

[0070] Fifth, it enhances the resource utilization of solid-state drives (SSDs). Multi-stream technology can utilize the resources in SSDs more efficiently, thereby improving the throughput and system response time of the entire distributed storage system.

[0071] The method provided in this embodiment is equivalent to writing data with the same or similar lifecycles in a distributed storage system to a solid-state drive (SSD). This reduces the workload of garbage collection on the SSD, improving storage efficiency. Furthermore, in this configuration, the host machine can process multiple data streams in parallel, minimizing the impact of SSD garbage collection on host performance and allowing for faster responses to user requests and applications. Therefore, this method significantly improves the performance of the SSD during garbage collection.

[0072] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. As a preferred implementation, the above steps include: dividing the data in the distributed storage system into different types of data and setting corresponding data tags for each type of data, including:

[0073] The data in the distributed storage system is divided into cold data, hot data, metadata, and system data, and a first label, a second label, a third label, and a fourth label are assigned to cold data, hot data, metadata, and system data, respectively.

[0074] In this embodiment, when dividing data in a distributed storage system into different types, the data can be categorized into cold data, hot data, metadata, and system data based on its popularity. Metadata has the highest popularity, followed by system data, hot data has a slightly lower popularity than system data, and cold data has the lowest popularity.

[0075] After dividing the data in a distributed storage system into cold data, hot data, metadata, and system data, it is necessary to assign a first label, a second label, a third label, and a fourth label to each of these categories. Specifically, the first label, second label, third label, and fourth label can be set to hot, cold, meta, and WAL (Write Ahead Logging), respectively.

[0076] After classifying data in a distributed storage system according to its popularity, data with the same or similar lifecycles can be written into the same erase unit of a solid-state drive (SSD). Furthermore, data with different popularity levels have different access patterns. Therefore, storing data with different popularity levels in SSDs not only reduces unnecessary data movement and copying, but also reduces performance loss during data migration.

[0077] Clearly, the technical solution provided in this embodiment can further improve the efficiency of solid-state drives when performing garbage collection operations.

[0078] As a preferred implementation, the above steps include: dividing the data in the distributed storage system into cold data, hot data, metadata, and system data, and assigning a first label, a second label, a third label, and a fourth label to the cold data, hot data, metadata, and system data respectively, including:

[0079] Real-time detection of whether the storage pool of the distributed storage system has received the target data sent by the user;

[0080] If not, continue with the step of real-time detection of whether the storage pool of the distributed storage system has received the target data sent by the user.

[0081] If so, the target data is differentiated by popularity to obtain cold data and hot data;

[0082] Cold data and hot data are stored in the slow storage pool and the fast storage pool, respectively.

[0083] Set a first label and a second label for cold data and hot data respectively;

[0084] Cold data with the first tag and hot data with the second tag are sent to the object storage device of the distributed storage system;

[0085] Use object storage devices to retrieve data that describes cold and hot data, obtain metadata, and set a third tag for the metadata;

[0086] The cold data with the first tag, the hot data with the second tag, and the metadata with the third tag are sent to the storage device engine of the distributed storage system.

[0087] The storage device engine is used to identify the data generated by the distributed storage system as system data and to set a fourth tag for the system data.

[0088] This embodiment details the data classification process in a distributed storage system. Please refer to [link / reference]. Figure 3 , Figure 3 This is a schematic diagram illustrating the classification of data in a distributed storage system according to an embodiment of the present invention. When classifying data in the distributed storage system, the first step is to detect in real-time whether the storage pool of the distributed storage system has received the target data sent by the user. If no target data is detected in the storage pool, the step of detecting whether the storage pool of the distributed storage system has received the target data sent by the user continues.

[0089] If the distributed storage system detects that the storage pool has received the target data sent by the user, it distinguishes the hotness of the target data to obtain cold data and hot data, and stores the cold data and hot data in the slow storage pool and the fast storage pool respectively. At the same time, it assigns a first tag and a second tag to the cold data and the hot data respectively.

[0090] In a distributed storage system, a slow storage pool refers to a storage medium with lower setup costs and lower usage frequency, while a fast storage pool refers to a storage medium with higher setup costs and faster access speeds. Generally speaking, slow storage pools are suitable for storing data that is not frequently accessed, while fast storage pools are suitable for storing data that is frequently accessed.

[0091] Then, the cold data with the first tag and the hot data with the second tag are sent to the object storage device (OSD) of the distributed storage system. When the object storage device receives the cold data with the first tag and the hot data with the second tag, it retrieves the data describing the cold and hot data, obtains metadata, and sets a third tag for the metadata. The data describing the cold and hot data includes: creation information, data storage location, data format, file name, and access permission information, etc.

[0092] Subsequently, the object storage device sends the cold data with the first tag, the hot data with the second tag, and the metadata with the third tag to the storage engine (BlueStore) of the distributed storage system. After receiving the cold data with the first tag, the hot data with the second tag, and the metadata with the third tag, the storage engine determines that the data generated by the distributed storage system is system data and sets a fourth tag for the system data.

[0093] System data refers to the data generated by the distributed storage system during its operation, including operating system information, application information, and system configuration information. System data is a crucial component of the distributed storage system and is key to ensuring its safe and stable operation.

[0094] Clearly, the technical solution provided in this embodiment can accurately classify data in a distributed storage system.

[0095] As a preferred implementation, the above steps, including writing data from the distributed storage system to the solid-state drive in a data stream manner according to different data tags, include:

[0096] The storage device engine specifies a first data stream, a second data stream, a third data stream, and a fourth data stream for cold data, hot data, metadata, and system data.

[0097] The first, second, third, and fourth data streams are encapsulated using the driver layer of the distributed storage system, and then written to the solid-state drive.

[0098] Please continue reading Figure 3 When writing cold data, hot data, metadata, and system data to the solid-state drive, the storage device engine specifies a first data stream, a second data stream, a third data stream, and a fourth data stream for the cold data, hot data, metadata, and system data, and uses the driver layer of the distributed storage system to encapsulate the first data stream, the second data stream, the third data stream, and the fourth data stream.

[0099] Once the driver layer of the distributed storage system has encapsulated the first, second, third, and fourth data streams, it will write them to the solid-state drive.

[0100] It should be noted that before writing the first, second, third, and fourth data streams to the solid-state drive (SSD), the driver layer of the distributed storage system will also check whether the SSD supports the directive feature, the enabled / disabled status of multiple streams, and query the multiple stream information. This multi-stream information query includes: the number of streams supported by the SSD, the number of available streams, whether they are shared, the granularity of stream writing, the granularity of stream allocation, and the number of exclusive streams, etc.

[0101] Clearly, the technical solution provided in this embodiment can accurately classify and write cold data, hot data, metadata, and system data into the solid-state drive.

[0102] As a preferred implementation, the above steps, including encapsulating the first data stream, the second data stream, the third data stream, and the fourth data stream using the driver layer of the distributed storage system, include:

[0103] Based on the non-volatile memory host controller interface specification, and using the storage system development kit, the first data stream, the second data stream, the third data stream, and the fourth data stream are encapsulated.

[0104] In this embodiment, when the driver layer of the distributed storage system encapsulates the first data stream, the second data stream, the third data stream, and the fourth data stream, it is based on the Non-Volatile Memory Express (NVMe) interface specification and uses the Storage Performance Development Kit (SPDK) to encapsulate the first data stream, the second data stream, the third data stream, and the fourth data stream.

[0105] Because SPDK provides a user-space NVMe driver that allows zero-copy and highly parallel access from user-space applications directly to the solid-state drive, and because the NVMe protocol also supports multiple queues and parallel operations, it can greatly improve the concurrency performance of distributed storage systems.

[0106] Furthermore, SPDK, through its custom user-space NVMe driver and other toolsets that replace the kernel protocol stack, can eliminate the frequent and cumbersome operations of copying data between user space and kernel space in host memory, thus greatly reducing the latency generated during data transmission and reception.

[0107] Combination Figure 3 As can be seen, when classifying data in a distributed storage system, the storage pool and object storage devices primarily use pooling and archiving strategies to distribute the target data. The storage device engine mainly organizes the target data and the data generated by the distributed storage system, and sends them to the driver layer in the form of data streams. When the driver layer receives the data sent by the storage device engine, it uses SPDK to encapsulate each data stream and writes each data stream to the solid-state drive (SSD) so that the SSD can perform garbage collection operations using multi-stream technology.

[0108] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating how data from a distributed storage system is written to a solid-state drive (SSD) in the form of a data stream, as provided in an embodiment of the present invention. When the distributed storage system receives data sent by a user, it issues instructions to the storage pool / object storage device within the system. The storage pool, object storage device, and storage device engine then categorize the data sent by the user and the data generated by the distributed storage system into metadata, system data, cold data, and hot data. During this process, the metadata and system data are stored in their respective databases, while the cold and hot data are stored on the storage device engine. Subsequently, the driver layer in the distributed storage system encapsulates the tagged metadata, system data, cold data, and hot data into a data stream and writes them to the SSD respectively.

[0109] Obviously, the technical solution provided in this embodiment can encapsulate the first data stream, the second data stream, the third data stream, and the fourth data stream more quickly and efficiently.

[0110] As a preferred implementation, the above step of: distinguishing the target data by heat level to obtain cold data and hot data includes:

[0111] The target data is differentiated by popularity based on multiple data dimensions to obtain cold data and hot data; the data dimensions include: data access frequency, user name, and data storage path.

[0112] In this embodiment, when differentiating target data by popularity to obtain cold data and hot data, the popularity of target data can be differentiated from several data dimensions, such as data access frequency, user name, and data storage path.

[0113] When using data access frequency to differentiate target data based on its access intensity, frequently accessed data and infrequently accessed data can be stored on different types of storage media. This not only reduces storage costs but also relatively improves data access efficiency. Furthermore, differentiating target data based on username and data storage path helps users better understand and control data access permissions, thereby enhancing data security.

[0114] Clearly, the technical solution provided in this embodiment can ensure the accuracy and reliability of classifying cold and hot data.

[0115] As a preferred embodiment, the above-mentioned solid-state drive garbage collection method further includes:

[0116] Store the configuration information related to hot and cold data in an object storage device mapping table;

[0117] The object storage device mapping table is sent to the object storage device through the monitor in the distributed storage system.

[0118] In this embodiment, in order to enable the object storage device to better classify cold data and hot data by attributes, the configuration information related to hot data and cold data is stored in the object storage device mapping table (OSDMap), and the object storage device mapping table is sent to the object storage device through the monitor in the distributed storage system.

[0119] Please see Figure 5 , Figure 5This diagram illustrates data interaction between management software, a monitor, and an object storage device, as provided in an embodiment of the present invention. When creating a storage pool, the management software in the distributed storage system categorizes target data sent by the user based on data popularity and sends the categorization results to the monitor. Upon receiving data from the management software, the monitor adds a popularity tag to the storage pool and stores the popularity tag in the Object Storage Device Map (OSDMap). Simultaneously, the monitor sends the OSDMap to the object storage device. Upon receiving the OSDMap, the object storage device assigns different popularity information to the data in the distributed storage system based on the data content stored in the OSDMap to ensure the smooth execution of subsequent processes.

[0120] Clearly, the technical solution provided in this embodiment enables object storage devices in a distributed storage system to fully understand the configuration information related to hot and cold data.

[0121] Please see Figure 6 , Figure 6 This is a structural diagram of a solid-state drive (SSD) garbage collection device provided in an embodiment of the present invention. The device includes:

[0122] The data partitioning module 21 is used to divide the data in the distributed storage system into different types of data and set corresponding data labels for each type of data.

[0123] Data writing module 22 is used to write data from the distributed storage system to the solid-state drive in the form of a data stream according to different data tags;

[0124] The garbage collection module 23 is used to control the solid-state drive to perform garbage collection operations on the stored data in the solid-state drive according to the different data streams when the solid-state drive receives a garbage collection operation instruction.

[0125] In one specific embodiment of this application, the data partitioning module 21 includes:

[0126] The data partitioning submodule is used to partition the data in the distributed storage system into cold data, hot data, metadata, and system data, and to set a first label, a second label, a third label, and a fourth label for the cold data, the hot data, the metadata, and the system data, respectively.

[0127] In one specific embodiment of this application, the data partitioning submodule includes:

[0128] The real-time detection unit is used to detect in real time whether the storage pool of the distributed storage system has received the target data sent by the user terminal;

[0129] The step jump unit is used to continue executing the step of real-time detection of whether the storage pool of the distributed storage system has received the target data sent by the user terminal when the determination result of the real-time detection unit is negative.

[0130] The data segmentation unit is used to distinguish the target data by heat level when the real-time detection unit determines that the result is yes, so as to obtain the hot data and the cold data.

[0131] A data storage unit is used to store the cold data and the hot data into a slow storage pool and a fast storage pool, respectively.

[0132] A tag setting unit is used to set the first tag and the second tag respectively for the cold data and the hot data;

[0133] A data sending unit is used to send the cold data with the first tag and the hot data with the second tag to the object storage device of the distributed storage system;

[0134] A data reading unit is used to acquire data describing the cold data and the hot data using the object storage device, obtain the metadata, and set the third tag for the metadata;

[0135] A tag sending unit is used to send the cold data with the first tag, the hot data with the second tag, and the metadata with the third tag to the storage device engine of the distributed storage system;

[0136] An engine partitioning unit is used to determine the data generated by the distributed storage system as system data using the storage device engine, and to set the fourth tag for the system data.

[0137] In one specific embodiment of this application, the data partitioning unit includes:

[0138] The data segmentation subunit is used to distinguish the target data by popularity based on multiple data dimensions to obtain the cold data and the hot data; wherein, the data dimensions include: data access frequency, user name, and data storage path.

[0139] In one specific embodiment of this application, it further includes:

[0140] An information storage unit is used to store configuration information related to the hot data and the cold data into an object storage device mapping table;

[0141] A mapping table sending unit is used to send the object storage device mapping table to the object storage device through a monitor in the distributed storage system.

[0142] In one specific embodiment of this application, the data writing module 22 includes:

[0143] A data stream designation unit is used to designate a first data stream, a second data stream, a third data stream, and a fourth data stream for the cold data, the hot data, the metadata, and the system data through the storage device engine;

[0144] The data writing unit is used to encapsulate the first data stream, the second data stream, the third data stream, and the fourth data stream using the driver layer of the distributed storage system, and write the first data stream, the second data stream, the third data stream, and the fourth data stream to the solid-state drive.

[0145] In one specific embodiment of this application, the data writing unit includes:

[0146] The data writing subunit is used to encapsulate the first data stream, the second data stream, the third data stream, and the fourth data stream using a storage system development kit based on the non-volatile memory host controller interface specification.

[0147] The solid-state drive (SSD) garbage collection device provided in this embodiment of the invention has the beneficial effects of the aforementioned solid-state drive garbage collection method.

[0148] For the method embodiments described above, please refer to [link / reference]. Figure 7 , Figure 7 This is a structural diagram of a solid-state drive (SSD) waste recycling device provided in an embodiment of the present invention. The device may include:

[0149] Memory 332 is used to store computer programs;

[0150] The processor 322 is used to implement the steps of the solid-state drive garbage collection method of the above method embodiment when executing a computer program.

[0151] For details, please refer to Figure 8 , Figure 8This is a schematic diagram illustrating the specific structure of a solid-state drive (SSD) garbage collection device according to an embodiment of the present invention. The SSD garbage collection device can vary significantly depending on its configuration or performance. It may include a processor (central processing unit, CPU) 322 (e.g., one or more processors) and a memory 332. The memory 332 stores one or more computer programs 342 or data 344. The memory 332 can be either temporary or persistent storage. The program stored in the memory 332 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the data processing device. Furthermore, the processor 322 may be configured to communicate with the memory 332 and execute the series of instruction operations stored in the memory 332 on the SSD garbage collection device 301.

[0152] The solid-state drive's garbage collection device 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.

[0153] The steps in the solid-state drive (SSD) garbage collection method described above can be implemented by the structure of the SSD garbage collection device.

[0154] Corresponding to the above method embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the following steps:

[0155] The data in the distributed storage system is divided into different types, and corresponding data tags are set for each type of data. The data in the distributed storage system is written to the solid-state drive (SSD) in the form of data streams according to the different data tags. When the SSD receives a garbage collection operation command, the SSD is controlled to perform garbage collection operations on the stored data in the SSD according to the different data streams.

[0156] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0158] Corresponding to the above method embodiments, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of a solid-state drive garbage collection method as disclosed above.

[0159] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatuses, devices, and computer-readable storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0160] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the technical solutions and core ideas of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for garbage collection of a solid-state drive, characterized in that, include: Divide the data in the distributed storage system into different types of data and set corresponding data labels for each type of data; Data in the distributed storage system is written to the solid-state drive in a data stream manner according to different data tags, so as to write data with the same or similar life cycle in the distributed storage system to the solid-state drive; When the solid-state drive receives a garbage collection operation command, the system controls the solid-state drive to perform garbage collection operations on the stored data in the solid-state drive according to the different data streams. The process of dividing data in a distributed storage system into different types and assigning corresponding data tags to each type of data includes: The data in the distributed storage system is divided into cold data, hot data, metadata, and system data, and a first label, a second label, a third label, and a fourth label are respectively assigned to the cold data, the hot data, the metadata, and the system data. The step of dividing the data in the distributed storage system into cold data, hot data, metadata, and system data, and assigning a first label, a second label, a third label, and a fourth label to the cold data, the hot data, the metadata, and the system data respectively, includes: Real-time detection of whether the storage pool of the distributed storage system receives the target data sent by the user terminal; If not, continue with the step of real-time detection of whether the storage pool of the distributed storage system has received the target data sent by the user terminal; If so, the target data is differentiated by heat level to obtain the cold data and the hot data; The cold data and the hot data are stored in a slow storage pool and a fast storage pool, respectively. The first tag and the second tag are respectively set for the cold data and the hot data; The cold data with the first tag and the hot data with the second tag are sent to the object storage device of the distributed storage system; The object storage device is used to obtain data describing the cold data and the hot data to obtain the metadata, and the third tag is set for the metadata. The cold data with the first tag, the hot data with the second tag, and the metadata with the third tag are sent to the storage device engine of the distributed storage system; The storage device engine is used to identify the data generated by the distributed storage system as system data, and the fourth tag is set for the system data.

2. The garbage collection method of claim 1, wherein, The step of distinguishing the target data by heat level to obtain the cold data and the hot data includes: The target data is differentiated by popularity based on multiple data dimensions to obtain cold data and hot data; wherein, the data dimensions include: data access frequency, user name, and data storage path.

3. The garbage collection method of claim 1, wherein, Also includes: The configuration information related to the hot data and the cold data is stored in an object storage device mapping table; The object storage device mapping table is sent to the object storage device via a monitor in the distributed storage system.

4. The garbage collection method of claim 1, wherein, The step of writing data from the distributed storage system to the solid-state drive in a data stream manner according to different data tags includes: The storage device engine specifies a first data stream, a second data stream, a third data stream, and a fourth data stream for the cold data, the hot data, the metadata, and the system data; The first data stream, the second data stream, the third data stream, and the fourth data stream are encapsulated using the driver layer of the distributed storage system, and then written to the solid-state drive.

5. The garbage collection method of claim 4, wherein, The encapsulation of the first data stream, the second data stream, the third data stream, and the fourth data stream using the driver layer of the distributed storage system includes: Based on the non-volatile memory host controller interface specification, and using the storage system development kit, the first data stream, the second data stream, the third data stream, and the fourth data stream are encapsulated.

6. A waste recycling device for a solid-state drive, characterized in that, include: The data partitioning module is used to divide the data in the distributed storage system into different types of data and set corresponding data labels for each type of data. The data writing module is used to write data from the distributed storage system to the solid-state drive in the form of a data stream according to different data tags, so as to write data with the same or similar life cycle from the distributed storage system to the solid-state drive. The garbage collection module is used to control the solid-state drive to perform garbage collection operations on the stored data in the solid-state drive according to the different data streams when the solid-state drive receives a garbage collection operation instruction; The data partitioning module includes: The data partitioning submodule is used to partition the data in the distributed storage system into cold data, hot data, metadata, and system data, and to set a first label, a second label, a third label, and a fourth label for the cold data, the hot data, the metadata, and the system data, respectively. The data partitioning submodule includes: The real-time detection unit is used to detect in real time whether the storage pool of the distributed storage system has received the target data sent by the user terminal; The step jump unit is used to continue executing the step of real-time detection of whether the storage pool of the distributed storage system has received the target data sent by the user terminal when the determination result of the real-time detection unit is negative. The data segmentation unit is used to distinguish the target data by heat level when the real-time detection unit determines that the result is yes, so as to obtain the hot data and the cold data. A data storage unit is used to store the cold data and the hot data into a slow storage pool and a fast storage pool, respectively. A tag setting unit is used to set the first tag and the second tag respectively for the cold data and the hot data; A data sending unit is used to send the cold data with the first tag and the hot data with the second tag to the object storage device of the distributed storage system; A data reading unit is used to acquire data describing the cold data and the hot data using the object storage device, obtain the metadata, and set the third tag for the metadata; A tag sending unit is used to send the cold data with the first tag, the hot data with the second tag, and the metadata with the third tag to the storage device engine of the distributed storage system; An engine partitioning unit is used to determine the data generated by the distributed storage system as system data using the storage device engine, and to set the fourth tag for the system data.

7. A solid-state drive (SSD) waste recycling device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of a garbage collection method for a solid-state drive as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a garbage collection method for a solid-state drive as described in any one of claims 1 to 5.

Images