NVMe storage device data placement methods, products, equipment and media

Abstract

This invention discloses a data placement method, product, device, and medium for NVMe storage devices, relating to the field of solid-state drive (SSD) storage technology. It is applied to a host connected to an NVMe storage device and includes: mounting a custom file system to a target directory to obtain a mount point directory; the custom file system is built based on the FUSE system framework, with each subdirectory of the mount point directory corresponding to a handle of the NVMe storage device; obtaining a current operation request; the current operation request represents a target operation to be performed on the current subdirectory; processing the current operation request using the custom file system to determine the type of the target operation and the target handle corresponding to the current subdirectory; if the type of the target operation is write, the NVMe storage device places the target data carried by the current operation request in the target recycling unit corresponding to the target handle. This improves the convenience of data placement and reduces the write amplification factor to enhance storage performance and extend the lifespan of the SSD.

Description

Technical Field

[0001] This invention relates to the field of solid-state drive storage technology, and in particular to data placement methods, products, devices and media for NVMe storage devices. Background Technology

[0002] In the field of solid-state drive (SSD) storage technology, with the increasing demand for data storage, SSD performance and lifespan have become critical issues. SSDs store data internally through flash memory chips, and their performance and lifespan are significantly affected by the garbage collection (GC) mechanism. When a user places data, the SSD needs to use GC to migrate valid data to new flash memory blocks and erase old blocks for reuse. This process increases the write amplification factor (WAF). Write amplification causes the actual amount of data written to the flash memory to far exceed the amount of data written by the user, not only reducing the SSD's write performance but also accelerating the wear and tear of the flash memory chips, thus shortening the SSD's lifespan.

[0003] Although technologies such as Flexible Data Placement (FDP) exist to optimize data storage location, it is difficult to achieve precise data placement through user-level control, and there are obvious shortcomings in balancing user operation convenience and storage performance optimization.

[0004] It is evident that improving the ease of data placement and reducing the write amplification factor to enhance storage performance and extend the lifespan of solid-state drives are problems that need to be addressed by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a data placement method, apparatus, device, and medium for NVMe storage devices, improving the convenience of data placement and reducing the write amplification factor to enhance storage performance and extend the lifespan of solid-state drives. The specific solution is as follows:

[0006] In a first aspect, the present invention discloses a data placement method for an NVMe storage device, applied to a host connected to an NVMe storage device, comprising:

[0007] A custom file system is mounted to the target directory to obtain a mount point directory; wherein, the custom file system is built based on the FUSE system framework, and the mount point directory includes subdirectories, each of which corresponds to a handle of the NVMe storage device;

[0008] Obtain the current operation request; wherein, the current operation request represents a target operation to be performed on the current subdirectory in the mount point directory;

[0009] The current operation request is processed using the custom file system to determine the type of the target operation and the target handle corresponding to the current subdirectory;

[0010] If the target operation is of type write, the NVMe storage device will place the target data carried by the current operation request in the target recycling unit corresponding to the target handle.

[0011] Optionally, the NVMe storage device data placement method further includes:

[0012] Establish a first mapping relationship between each subdirectory in the mount point directory and each handle identifier of the NVMe storage device; wherein each handle identifier corresponds to each handle of the NVMe storage device;

[0013] Accordingly, the current operation request is processed using the custom file system to determine the target handle corresponding to the current subdirectory, including:

[0014] Based on the first mapping relationship, and using the custom file system to process the current operation request, a target handle identifier corresponding to the current subdirectory is determined;

[0015] The target handle is locked from each of the handles according to the target handle identifier.

[0016] Optionally, the NVMe storage device data placement method further includes:

[0017] The FDP function parameters supported by the NVMe storage device are detected; wherein the FDP function parameters include the structural information of each durability group and the second mapping relationship between the reclaim unit handle and the handle identifier in the durability group, and the structural information of the durability group represents each reclaim unit contained in each reclaim group under the durability group.

[0018] Optionally, the current operation request is processed using the custom file system to determine the target handle corresponding to the current subdirectory, including:

[0019] The current operation request is processed using the custom file system to determine the target handle identifier corresponding to the current subdirectory;

[0020] According to the second mapping relationship, the target recycling unit handle corresponding to the target handle identifier is locked in each of the handles.

[0021] Optionally, the NVMe storage device places the target data carried by the current operation request in the target recycling unit corresponding to the target handle, including:

[0022] The NVMe storage device determines the target reclamation group pointed to by the target reclamation unit handle, determines the target reclamation unit in the target reclamation group, and places the target data carried by the current operation request in the target reclamation unit.

[0023] Optionally, obtaining the current operation request includes:

[0024] The FUSE kernel driver is used to encapsulate the original operation requests in a structured manner to obtain the structured encapsulated operation requests.

[0025] Each of the structured encapsulated operation requests is stored in the kernel queue;

[0026] Select the current operation request from the structured encapsulated operation requests in the kernel queue.

[0027] Optionally, selecting the current operation request from each of the structured, encapsulated operation requests in the kernel queue includes:

[0028] The user-mode daemon selects the current operation request from the structured, encapsulated operation requests in the kernel queue through the device file;

[0029] Accordingly, after the NVMe storage device places the target data carried by the current operation request into the target recycling unit corresponding to the target handle, it further includes:

[0030] The user-mode daemon is controlled to write the response result of the current operation request to the device file.

[0031] Optionally, the original operation request represents an application requesting to write target data to a subdirectory under the mount point directory through a standard file interface.

[0032] Optionally, after controlling the user-mode daemon to write the response result of the current operation request to the device file, the method further includes:

[0033] The FUSE kernel driver is controlled to obtain the response result written by the user-mode daemon from the device file;

[0034] The response result is returned to the application through the standard file interface to complete the input / output operation of the current operation request.

[0035] Optionally, before placing the target data carried by the current operation request in the target reclamation unit corresponding to the target handle, the NVMe storage device further includes:

[0036] Call the io_uring interface to submit the current operation request to the NVMe storage device;

[0037] Accordingly, controlling the user-mode daemon to write the response result of the current operation request to the device file includes:

[0038] The user-space daemon is controlled to write the response result of the current operation request returned by the NVMe storage device to the device file.

[0039] Optionally, the step of calling the io_uring interface to submit the current operation request to the NVMe storage device includes:

[0040] Create an io_uring circular buffer; wherein, the io_uring circular buffer includes a commit queue;

[0041] Retrieve idle request entries from the submission queue;

[0042] The opcode of the current operation request and the target handle identifier corresponding to the current subdirectory are filled into the idle request entry to obtain the submit request entry;

[0043] The submit queue is controlled to submit the submit request entry to the NVMe storage device.

[0044] Optionally, the io_uring circular buffer includes a completion queue;

[0045] The step of controlling the user-space daemon to write the response result of the current operation request returned by the NVMe storage device to the device file includes:

[0046] The completion queue is used to receive the response result of the current operation request returned by the NVMe storage device; wherein, the response result of the current operation request is a completion request entry encapsulated by the NVMe storage device using the completion status, status code and error information of the current operation request;

[0047] The user-mode daemon is controlled to write the response result of the current operation request to the device file.

[0048] Secondly, the present invention discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the aforementioned disclosed NVMe storage device data placement method.

[0049] Thirdly, the present invention discloses an electronic device, comprising:

[0050] Memory, used to store computer programs;

[0051] A processor for executing computer programs to implement the steps of the aforementioned disclosed NVMe storage device data placement method.

[0052] Fourthly, the present invention discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed NVMe storage device data placement method.

[0053] Therefore, this invention is applied to a host connected to an NVMe storage device, comprising: mounting a custom file system to a target directory to obtain a mount point directory; wherein the custom file system is built based on the FUSE system framework, the mount point directory includes subdirectories, each subdirectory corresponding to a handle of the NVMe storage device; obtaining a current operation request; wherein the current operation request represents a target operation on the current subdirectory in the mount point directory; processing the current operation request using the custom file system to determine the type of the target operation and the target handle corresponding to the current subdirectory; if the type of the target operation is a write type, the NVMe storage device places the target data carried by the current operation request in the target recycling unit corresponding to the target handle.

[0054] The beneficial effects are as follows: This invention mounts a custom file system based on the FUSE framework to the target directory, forming a subdirectory corresponding to the NVMe storage device handle. This allows the host to determine the target handle by parsing operation requests to the subdirectory of the mount point directory. During write operations, the target handle guides the NVMe device to accurately place data into the corresponding recycling unit, thereby reducing the write amplification factor, improving storage performance, and extending the lifespan of the solid-state drive. Furthermore, it eliminates the need for users to directly operate the complex FDP underlying interface. In other words, users only need to perform write operations on the file system directory, which is no different from operating a local hard drive directory, i.e., performing standard file operations. The host automatically converts standard file operation requests into control commands for the NVMe storage device handle through the custom file system, hiding the details of FDP technology and improving the convenience for users to place data. Attached Figure Description

[0055] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A flowchart illustrating a data placement method for an NVMe storage device provided in an embodiment of the present invention;

[0057] Figure 2 This is a comparative diagram illustrating the magnified advantages of flexible data placement in a specific embodiment of the present invention.

[0058] Figure 3 This is a schematic diagram of a specific NVMe storage device FDP functional unit provided in an embodiment of the present invention;

[0059] Figure 4 A flowchart illustrating a specific method for placing data on an NVMe storage device, as provided in this embodiment of the invention;

[0060] Figure 5 A specific initialization diagram is provided for an embodiment of the present invention;

[0061] Figure 6 This is a schematic diagram illustrating a specific mapping relationship between a handle identifier and a subdirectory, provided as an embodiment of the present invention.

[0062] Figure 7 This is a schematic diagram of a specific FUSE result provided in an embodiment of the present invention;

[0063] Figure 8 A specific data writing flowchart is provided for an embodiment of the present invention;

[0064] Figure 9 This is a schematic diagram of a data placement device for an NVMe storage device provided in an embodiment of the present invention;

[0065] Figure 10 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0067] In the field of solid-state drive (SSD) storage technology, with the increasing demand for data storage, SSD performance and lifespan have become critical issues. SSDs store data internally through flash memory chips, and their performance and lifespan are significantly affected by garbage collection mechanisms. When users move data, the SSD needs to use garbage collection (GC) to migrate valid data to new flash memory blocks and erase old blocks for reuse. This process increases the write amplification factor (WAF). Write amplification causes the actual amount of data written to the flash memory to far exceed the amount of data written by the user, not only reducing the SSD's write performance but also accelerating flash memory chip wear and shortening the SSD's lifespan.

[0068] While technologies such as flexible data placement exist to optimize data storage location, it is difficult to achieve precise data placement through user-level control, resulting in a significant deficiency in balancing user ease of operation with storage performance optimization.

[0069] The terms "comprising" and "having," and any variations thereof, in the specification and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may include steps or units not listed.

[0070] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0071] Next, we will describe in detail an NVMe storage device data placement scheme provided by an embodiment of the present invention. Figure 1 An embodiment of the present invention provides a method for placing data on an NVMe storage device, applied to a host connected to an NVMe storage device, comprising:

[0072] Step S11: Mount the custom file system to the target directory to obtain the mount point directory; wherein, the custom file system is built based on the FUSE system framework, and the mount point directory includes subdirectories, each of which corresponds to a handle of the NVMe storage device.

[0073] Solid-state drives (SSDs) are becoming increasingly popular in various storage systems due to their small size, fast access speed, and high data density. However, SSD performance and lifespan are significantly affected by garbage collection mechanisms. GC is a fundamental technology in SSDs, used to clean up invalid data to free up space, but this process leads to additional write operations, increasing the write amplification factor, thereby reducing the SSD's write performance and shortening its limited write lifespan. With the development of the NVMe (Non-Volatile Memory Express) standard, flexible data placement has been proposed as a new feature, aiming to reduce the overhead of GC, for example... Figure 2 The diagram illustrates a comparison of the advantages of flexible data placement. FDP allows the host system to more flexibly control the placement of data in the SSD, thereby optimizing data layout, reducing the generation of invalid data, and decreasing the frequency of garbage collection.

[0074] The host connects to the NVMe storage device. On the host side, a custom file system (Self-defined File System, i.e., SDFS) is built based on the FUSE system framework, and the custom file system is mounted to the target directory to obtain the mount point directory. The mount point directory includes subdirectories, and each subdirectory corresponds to a handle of the NVMe storage device. In other words, there is a mapping relationship between each subdirectory and each handle, so that different handles are mapped to the user layer in the form of directories.

[0075] In this embodiment, the method further includes: establishing a first mapping relationship between each subdirectory in the mount point directory and each handle identifier of the NVMe storage device; wherein each handle identifier corresponds to each handle of the NVMe storage device.

[0076] Specifically, a handle identifier is set for each handle, meaning there is a correspondence between handles and handle identifiers. Furthermore, a first mapping relationship is established between each subdirectory in the mount point directory and each handle identifier of the NVMe storage device. In this way, when a subdirectory is determined, the corresponding handle identifier can be determined based on the first mapping relationship, and thus the corresponding handle can be determined.

[0077] In this embodiment, the method further includes: detecting the FDP function parameters supported by the NVMe storage device; wherein the FDP function parameters include the structural information of each durability group and the second mapping relationship between the recycling unit handle and the handle identifier in the durability group, and the structural information of the durability group characterizes each recycling unit contained in each recycling group under the durability group.

[0078] The FDP function parameters supported by the NVMe storage device are detected. Specifically, the structural information of each durability group in the NVMe storage device is detected, as well as the second mapping relationship between the reclaim unit handles and handle identifiers in the durability group. The structural information of the durability group represents the reclaim units contained in each reclaim group under the durability group, for example... Figure 3The diagram shows a specific NVMe storage device FDP functional unit. The NVMe storage device is connected to the host and includes a controller, a namespace, and an endurance group (EG). The endurance group includes one or more reclaim groups (RG), and each reclaim group includes one or more reclaim units (RU). The reclaim unit handle (RUH) points to the reclaim unit in the corresponding reclaim group. There is a second mapping relationship between the placement handle and the reclaim unit handle. That is, there is a mapping relationship between the identity of the placement handle and the reclaim unit handle. In other words, determining the placement handle can determine the reclaim unit handle, and the reclaim unit in the reclaim group it points to can be determined based on the reclaim unit handle.

[0079] Step S12: Obtain the current operation request; wherein, the current operation request represents a target operation to be performed on the current subdirectory in the mount point directory.

[0080] The host obtains the current operation request, which represents the target operation to be performed on the current subdirectory in the mount point directory. The original operation request corresponding to the current operation request is the operation request initiated by the application to the host.

[0081] It is important to note that when the original operation requests after structured encapsulation are stored in the kernel queue, it is necessary to filter the current operation requests from the kernel queue. Specifically, the current operation requests can be filtered based on the storage time of each operation request in the kernel queue, or based on the processing priority of each operation request.

[0082] Step S13: Process the current operation request using the custom file system to determine the type of the target operation and the target handle corresponding to the current subdirectory.

[0083] The current operation request is processed using the custom file system to determine the type of the target operation, which specifically includes data write type, deletion type, and read type.

[0084] In this embodiment, the current operation request is processed using the custom file system to determine the target handle corresponding to the current subdirectory, including: based on the first mapping relationship, the current operation request is processed using the custom file system to determine the target handle identifier corresponding to the current subdirectory; and the target handle is locked from each of the handles according to the target handle identifier.

[0085] It is understandable that there is a first mapping relationship between each subdirectory in the mount point directory and each handle identifier of the NVMe storage device. The current operation request is to perform a target operation on the current subdirectory in the mount point directory. That is, the custom file system can determine the current subdirectory and then determine the target handle identifier corresponding to the current subdirectory based on the first mapping relationship. In this way, the target handle can be locked from each handle according to the target handle identifier.

[0086] In this embodiment, the current operation request is processed using the custom file system to determine the target handle corresponding to the current subdirectory. This includes: processing the current operation request using the custom file system to determine the target handle identifier corresponding to the current subdirectory; and locking the target recycling unit handle corresponding to the target handle identifier in each of the handles according to the second mapping relationship.

[0087] Furthermore, because there is a second mapping relationship between the reclaim unit handle and the handle identifier, when the current operation request is processed using a custom file system to lock the target handle identifier corresponding to the current subdirectory, the target reclaim unit handle corresponding to the target handle identifier can be locked in each handle according to the second mapping relationship. That is, the target reclaim unit handle is the target handle.

[0088] Step S14: If the type of the target operation is write, the NVMe storage device will place the target data carried by the current operation request in the target recycling unit corresponding to the target handle.

[0089] In this embodiment, the NVMe storage device places the target data carried by the current operation request into the target recycling unit corresponding to the target handle, including: the NVMe storage device determining the target recycling group pointed to by the target recycling unit handle, determining the target recycling unit in the target recycling group, and placing the target data carried by the current operation request into the target recycling unit.

[0090] If the target operation is a write operation, the NVMe storage device will place the target data carried by the current operation request in the target reclaim unit corresponding to the target handle. Specifically, after determining the target handle, the NVMe storage device determines the target reclaim group pointed to by the target reclaim unit handle, then determines the target reclaim unit in the target reclaim group, and then places the target data carried by the current operation request in the target reclaim unit.

[0091] After the NVMe storage device places the target data carried by the current operation request into the target reclaim unit corresponding to the target handle, the NVMe storage device returns the response result of the current operation request. The response result of the current operation request indicates whether the target data was successfully placed or not. Next, the user-space daemon writes the response result of the current operation request to the device file. The FUSE kernel driver obtains the response result from the device file and then returns the response result to the application through the standard file interface to complete the input / output operation of the current operation request.

[0092] Therefore, this invention is applied to a host connected to an NVMe storage device, comprising: mounting a custom file system to a target directory to obtain a mount point directory; wherein the custom file system is built based on the FUSE system framework, the mount point directory includes subdirectories, each subdirectory corresponding to a handle of the NVMe storage device; obtaining a current operation request; wherein the current operation request represents a target operation on the current subdirectory in the mount point directory; processing the current operation request using the custom file system to determine the type of the target operation and the target handle corresponding to the current subdirectory; if the type of the target operation is a write type, the NVMe storage device places the target data carried by the current operation request in the target recycling unit corresponding to the target handle.

[0093] The beneficial effects are as follows: This invention mounts a custom file system based on the FUSE framework to the target directory, forming a subdirectory corresponding to the NVMe storage device handle. This allows the host to determine the target handle by parsing operation requests to the subdirectory of the mount point directory. During write operations, the target handle guides the NVMe device to accurately place data into the corresponding recycling unit, thereby reducing the write amplification factor, improving storage performance, and extending the lifespan of the solid-state drive. Furthermore, it eliminates the need for users to directly operate the complex FDP underlying interface. In other words, users only need to perform write operations on the file system directory, which is no different from operating a local hard drive directory, i.e., performing standard file operations. The host automatically converts standard file operation requests into control commands for the NVMe storage device handle through the custom file system, hiding the details of FDP technology and improving the convenience for users to place data.

[0094] See Figure 4 This invention discloses a specific method for placing data on an NVMe storage device. Compared to the previous embodiment, this embodiment further explains and optimizes the technical solution. It is applied to a host connected to an NVMe storage device and includes:

[0095] Step S21: Mount the custom file system to the target directory to obtain the mount point directory; wherein, the custom file system is built based on the FUSE system framework, and the mount point directory includes subdirectories, each of which corresponds to a handle of the NVMe storage device.

[0096] On the host side, a custom file system based on FUSE is mounted to the target directory to obtain the mount point directory. By mapping different handle identifiers to the user level in the form of directories, explicit data placement functionality is implemented at the file system level, making it easier for users to use flexible data placement capabilities, thereby optimizing data access performance and storage management. Specifically, for example... Figure 5 The diagram illustrates a specific initialization process. A custom file system based on FUSE is mounted to the target directory to obtain the mount point directory. The FDP function parameters supported by the NVMe storage device are collected and detected to obtain the FDP capability set. The FDP function parameters of the FDP capability set include the structural information of each durability group and the second mapping relationship between the reclaim unit handle and the handle identifier in the durability group. The structural information of the durability group represents the reclaim units contained in each reclaim group under the durability group. Furthermore, the first mapping relationship between each subdirectory in the mount point directory and each handle identifier of the NVMe storage device is established. In other words, a corresponding subdirectory is created in the mount point directory for each handle identifier.

[0097] For example Figure 6 This diagram illustrates a specific mapping between handle identifiers and subdirectories. Based on a user-space file system framework, it's a user-defined file system whose main function is to identify the FPD function of the mounted NVMe device and the handle identifier corresponding to the device's RUH. These handle identifiers are presented to the user layer in directory form. When files in this directory undergo read, write, or delete operations, liburing-related interfaces are used to perform a series of operations, writing the data to a unique handle identifier bound to this folder. The mount point directory for this user-defined file system contains several subdirectories, each corresponding to a specific handle identifier. User programs can write data to these subdirectories through standard file operation interfaces.

[0098] Step S22: Use the FUSE kernel driver to encapsulate the original operation request in a structured manner to obtain each structured encapsulated operation request; store each structured encapsulated operation request in the kernel queue; select the current operation request from each structured encapsulated operation request in the kernel queue; wherein, the current operation request represents a target operation on the current subdirectory in the mount point directory.

[0099] For example Figure 7 The diagram illustrates a specific FUSE result. When an application interacts with a mounted custom file system, the operating system forwards these raw operation requests to the FUSE kernel driver. When the FUSE kernel driver receives the raw operation requests, it encapsulates them in a structured manner to obtain the structured operation requests. Then, it stores the structured operation requests in the kernel queue. At this point, the application that initiated the request enters a waiting state until it receives a response.

[0100] The current operation request is selected from the structured and encapsulated operation requests in the kernel queue. It can be understood that the current operation request represents a target operation on the current subdirectory in the mount point directory. Furthermore, when selecting the current operation request from the structured and encapsulated operation requests in the kernel queue, it can be done by first filtering according to the processing priority from highest to lowest, and then filtering according to the order in which the operation requests were stored in the kernel queue. Specifically, the processing priority of each structured and encapsulated operation request in the kernel queue is determined. If the processing priorities of each structured and encapsulated operation request are different, the current operation request is selected from the kernel queue in descending order of processing priority. If the processing priorities of each structured and encapsulated operation request are the same, the time when each structured and encapsulated operation request was stored in the kernel queue is determined, and the current operation request is selected from the kernel queue in the order in which it was stored. In this way, it is ensured that operation requests with higher processing priority are processed first, and it is also ensured that operations generated earlier can be processed earlier. In other words, this scheduling strategy, through a two-layer filtering mechanism of priority + time order, ensures that high-priority tasks are executed first while taking into account the fairness of tasks with the same priority. It satisfies real-time requirements and avoids task starvation. By dynamically balancing the response speed of critical tasks and the overall system throughput, it effectively improves resource utilization and optimizes system performance.

[0101] In this embodiment, selecting the current operation request from each of the structured encapsulated operation requests in the kernel queue includes: controlling the user-mode daemon to select the current operation request from each of the structured encapsulated operation requests in the kernel queue through the device file.

[0102] like Figure 7As shown, when a structured encapsulated operation request is detected in the kernel queue, the user-space daemon process, namely the user-space SDFS daemon process, is woken up. Then, the user-space daemon process is controlled to select the current operation request from the structured encapsulated operation requests in the kernel queue through the device file (i.e., / dev / fuse).

[0103] In this embodiment, the original operation request represents an application requesting to write target data to a subdirectory under the mount point directory via a standard file interface. The application initiates the original operation request to request the writing of target data to a subdirectory under the mount point directory via the standard file interface. In other words, SDFS exposes the FDP SSD as a file directory to user space, allowing users to easily group similar data, thereby managing data storage more efficiently, reducing the difficulty of using NMVeFDP features, and providing a more convenient method for optimizing WAF.

[0104] Step S23: Process the current operation request using the custom file system to determine the type of the target operation and the target handle corresponding to the current subdirectory.

[0105] The current operation request is processed according to the logic of the mounted file system. It is either forwarded to the underlying file system or passed to other kernel subsystems for further processing to determine the type of the target operation and the target handle corresponding to the current subdirectory. Specifically, because there is a first mapping relationship between each subdirectory in the mount point directory and each handle identifier of the NVMe storage device, and a second mapping relationship between the reclaimed unit handle and the handle identifier, once the current subdirectory is determined, the target handle identifier corresponding to the current subdirectory and the target reclaimed unit handle corresponding to the target handle identifier can be determined sequentially based on the first and second mapping relationships.

[0106] Step S24: If the type of the target operation is write, the NVMe storage device will place the target data carried by the current operation request in the target recycling unit corresponding to the target handle.

[0107] The current FDP software stack is built on the io_uring interface with the NVMe generic character device interface. io_uring supports efficient and scalable asynchronous I / O operations for storage and network tasks. It uses a ring buffer for communication between the application and the kernel, reducing system calls and improving performance. io_uring relies on two shared ring buffers: a commit queue (SQ) for sending requests and a completion queue (CQ) for receiving results. These queues are used to handle communication between user programs and the kernel. It processes I / O by extracting an entry named SQE from the SQ, populating the SQE, and calling the io_uring_enter system call.

[0108] In this embodiment, before the NVMe storage device places the target data carried by the current operation request in the target recycling unit corresponding to the target handle, it also includes: calling the io_uring interface to submit the current operation request to the NVMe storage device.

[0109] For example Figure 8 The diagram shows a specific data writing process. If the target operation is a write operation, the io_uring interface is called to submit the current operation request to the NVMe storage device. The NVMe storage device places the target data carried by the current operation request into the target recycling unit corresponding to the target recycling unit handle.

[0110] In this embodiment, the step of calling the io_uring interface to submit the current operation request to the NVMe storage device includes: creating an io_uring circular buffer; wherein, the io_uring circular buffer includes a submission queue; obtaining an idle request entry from the submission queue; filling the idle request entry with the opcode of the current operation request and the target handle identifier corresponding to the current subdirectory to obtain a submission request entry; and controlling the submission queue to submit the submission request entry to the NVMe storage device.

[0111] An io_uring circular buffer is created. Specifically, the io_uring circular buffer includes a commit queue, which contains multiple request entries, including currently occupied request entries (i.e., commit request entries) and free request entries. A free request entry is selected, and then the opcode of the current operation request and the target handle identifier corresponding to the current subdirectory are filled into the free request entry to obtain a commit request entry. The commit queue is controlled to submit the commit request entry to the NVMe storage device, so that the NVMe storage device receives the current operation request and the target handle identifier. The NVMe storage device determines the target reclamation group pointed to by the target reclamation unit handle, and determines the target reclamation unit in the target reclamation group. The target data carried by the current operation request is placed in the target reclamation unit.

[0112] In this embodiment, after the NVMe storage device places the target data carried by the current operation request in the target recycling unit corresponding to the target handle, it further includes: controlling the user-space daemon to write the response result of the current operation request into the device file. After the NVMe storage device places the target data carried by the current operation request in the target recycling unit corresponding to the target handle, the host controls the user-space daemon to write the response result of the current operation request into the device file.

[0113] In this embodiment, controlling the user-space daemon to write the response result of the current operation request to the device file includes: controlling the user-space daemon to write the response result of the current operation request returned by the NVMe storage device to the device file. After the NVMe storage device places the target data carried in the current operation request in the target recycling unit corresponding to the target handle, the NVMe storage device returns the response result of the current operation request to the host. The host's user-space daemon receives the response result of the current operation request, and the host controls the user-space daemon to write the response result of the current operation request returned by the NVMe storage device to the device file.

[0114] In this embodiment, after controlling the user-mode daemon to write the response result of the current operation request to the device file, the method further includes: controlling the FUSE kernel driver to obtain the response result written by the user-mode daemon from the device file; and returning the response result to the application through the standard file interface to complete the input / output operation of the current operation request.

[0115] Because the user-space daemon writes the response to the current operation request to the device file, the FUSE kernel driver can be controlled to retrieve the response written by the user-space daemon from the device file. Then, the response is returned to the application via the standard file interface to complete the input / output operation of the current operation request. Essentially, the operation request is sent from the application to the host via the standard file interface. The host receives the request, processes it accordingly, sends it to the NVMe storage device, receives the response from the NVMe storage device, and then returns the response to the application via the standard file interface. This completes the input / output operation of the current operation request.

[0116] In this embodiment, the io_uring circular buffer includes a completion queue; controlling the user-space daemon to write the response result of the current operation request returned by the NVMe storage device to the device file includes: receiving the response result of the current operation request returned by the NVMe storage device using the completion queue; wherein, the response result of the current operation request is a completion request entry encapsulated by the NVMe storage device using the completion status, status code, and error information of the current operation request; and controlling the user-space daemon to write the response result of the current operation request to the device file.

[0117] Furthermore, the io_uring circular buffer includes a completion queue, which is used to write the response result to the device file. Specifically, after the NVMe storage device places the target data carried by the current operation request into the target reclaim unit corresponding to the target handle, it returns the response result of the current operation request to the host. The response result of the current operation request is a completion request entry encapsulated by the NVMe storage device using the completion status, status code, and error information of the current operation request. The completion status of the current operation request is either a success status or a failure status. A success status indicates that the NVMe storage device has successfully placed the target data into the target reclaim unit, while a failure status indicates that the NVMe storage device has not successfully placed the target data into the target reclaim unit. Next, the user-space daemon process controls the writing of the response result of the current operation request to the device file. In this way, the FUSE kernel driver controls the acquisition of the response result written by the user-space daemon process from the device file.

[0118] Therefore, the user-defined file system based on the FUSE framework abstracts the complex underlying interfaces of NVMe FDP (such as Placement Handles and RUH) into a directory structure of the file system. This allows users to optimize data layout simply by performing file read and write operations like operating a regular directory, without directly manipulating the underlying technical details of FDP. Specifically, during the SDFS initialization phase, the FDP capabilities of the NVMe device are automatically detected, and subdirectories corresponding one-to-one with Placement Handles are dynamically generated. User read and write operations on directories (such as copying files to a specific directory) are captured by the SDFS daemon and converted into underlying FDP I / O requests. Data is then directed to the corresponding RUH storage area through liburing and io_uring mechanisms. SDFS is fully compatible with standard file operation interfaces (such as open, write, and read). Users can use regular commands (such as cp and mv) or programming interfaces (such as fopen) to operate on directories, and the system automatically handles complex processes such as FDP Placement Handle binding and NVMe command submission. This design hides the technical details of FDP through directory mapping and transparent transformation, while using asynchronous I / O mechanisms to ensure performance. Ultimately, it enables users to complete data grouping and storage in a simple way by operating local directories, thereby reducing WAF and optimizing SSD lifespan, without needing to understand the underlying NVMe FDP protocol.

[0119] The invention will now be described. RAID (Redundant Arrays of Independent Disks) distributes data across multiple hard drives, allowing input / output operations to overlap in a balanced manner, thereby improving performance and increasing the overall system's data fault tolerance. NVMe (Non-Volatile Memory Express) is a high-performance interface protocol for accessing non-volatile storage media (such as solid-state drives, SSDs). Designed specifically for SSDs, it features high performance (supporting multi-queue and high-concurrency operations, significantly improving data transfer speeds) and low latency (NVMe has lower latency compared to the traditional SATA protocol). FDP is an emerging interface in the NVMe storage standard, designed to reduce write WAF in SSDs by allowing users to explicitly control data placement. It is a technology that optimizes storage performance and lifespan by flexibly allocating data storage locations. WAF (Web Application Firewall) refers to the ratio of the actual amount of data written to the NAND flash memory in a solid-state drive (SSD) to the amount of data written by the host. It is an important indicator of SSD performance and durability. A high WAF can lead to decreased write performance and a shorter SSD lifespan; therefore, an ideal WAF is 1. FS (File System) is a mechanism in an operating system used to manage and organize files. It defines how files and directories are named, stored, and accessed. NVMe Namespace is a logical storage area that divides the non-volatile storage space of an NVMe device into multiple independent logical blocks. Each logical block can be formatted and presented as a standard block device. In Linux systems, each namespace typically appears as an independent block device. Garbage collection (GC) is a mechanism in SSDs used to optimize storage space and improve write performance. In SSDs, if you want to write new data to a block storing useless data, you need to erase the entire block first. In other words, SSDs do not have the ability to directly overwrite old data. For SSDs, GC refers to the process of moving the valid data in the old data block to another location and erasing the old data block. An RU is a collection of NAND blocks in which the host can write logical blocks, similar to the concept of a super block in an SSD's flash translation layer (FTL). A typical size is several GB (e.g., 6 GB). A reclaim group is a collection of reclaim units, usually used to partition data across different NAND chips according to host policies.Reclaim Unit Handles (RUHs) are internal SSD resources used to manage and buffer logical blocks written to reclaim units. A namespace can access one or more RUHs. If a namespace can access multiple RUHs, the host can write data to multiple RUs simultaneously. Many FDP devices have 8, 16, 256, etc., RUH counts. Endurance Groups (UGCs) represent an FDP configuration that contains multiple RGs and RUs, with one or more RUHs provided for each RG. User-space file systems are mechanisms used in Unix-like operating systems that allow users to create custom file systems without editing kernel code by running file system code in user space.

[0120] Furthermore, during the data placement process of NVMe storage devices, the system first performs device capability detection and initialization: When mounting the SDFS file system, the system first obtains the FDP capability set of the underlying storage device through the NVMe Identify command, parses the number of Placement Handles supported by the device and related parameters (such as RUH, EG, and other configuration information), and dynamically creates subdirectories corresponding one-to-one with the Placement Handles in the mount point directory. Then, file operation interception and routing are performed: When a user performs file operations on a specific subdirectory (such as / mnt / sdfs / handle_2), the FUSE kernel module intercepts system calls (open / write, etc.) and passes the request to the user-space SDFS daemon process through the / dev / fuse character device. The daemon process parses the target path and extracts the corresponding Placement Handle identifier. Next, request encapsulation: When preparing the io_uring SQE (commit queue entry) via liburing: Set the target Placement Handle in the FDP extension field of the NVMe command; Attach an FDP Directive (such as NVMe_IO_FDP_DIRECTIVE_WITH_PH) to the write request; Associate the user buffer data with the LBA address. Further, asynchronous commit and execution: Batch commit SQE via the io_uring_enter system call - the NVMe controller according to the FDP configuration; Write the data to the NAND physical location corresponding to the specified RUH; Maintain the binding relationship between data and Placement Handle when updating the FTL mapping table; Return the operation status via CQE (complete queue entry). Next, metadata management: Maintain the mapping table records in memory: the correspondence between file inodes and Placement Handles; the remaining capacity and wear status of each RUH; Periodically synchronize the FDP status information on the device side via the NVMe Get Log Page command. Further garbage collection optimizations: GC is performed based on Placement Handle grouping: priority is given to collecting RUH groups with higher WAF; invalid data within the same RUH is centrally cleaned up; valid data blocks maintain their PH when migrating within a group.

[0121] The system transforms the physical storage management of FDP into intuitive directory operations through a file system abstraction layer. While ensuring data locality, it reduces the write amplification factor (WAF) compared to traditional SSDs. Users can achieve the following through standard file APIs: hot data is written to low-latency RUH, cold data is stored centrally in high-endurance EG, and data with similar lifecycles is automatically grouped and stored.

[0122] Figure 9This invention provides a schematic diagram of a data placement device for an NVMe storage device, applied to a host connected to an NVMe storage device, comprising:

[0123] The system mount module 11 is used to mount a custom file system to a target directory to obtain a mount point directory; wherein, the custom file system is built based on the FUSE system framework, and the mount point directory includes subdirectories, each of which corresponds to a handle of the NVMe storage device.

[0124] The request acquisition module 12 is used to acquire the current operation request; wherein, the current operation request represents a target operation to be performed on the current subdirectory in the mount point directory.

[0125] The handle determination module 13 is used to process the current operation request using the custom file system to determine the type of the target operation and the target handle corresponding to the current subdirectory.

[0126] The data placement module 14 is used to place the target data carried by the current operation request into the target recycling unit corresponding to the target handle if the type of the target operation is write.

[0127] Therefore, this invention is applied to a host connected to an NVMe storage device, comprising: mounting a custom file system to a target directory to obtain a mount point directory; wherein the custom file system is built based on the FUSE system framework, the mount point directory includes subdirectories, each subdirectory corresponding to a handle of the NVMe storage device; obtaining a current operation request; wherein the current operation request represents a target operation on the current subdirectory in the mount point directory; processing the current operation request using the custom file system to determine the type of the target operation and the target handle corresponding to the current subdirectory; if the type of the target operation is a write type, the NVMe storage device places the target data carried by the current operation request in the target recycling unit corresponding to the target handle.

[0128] The beneficial effects are as follows: This invention mounts a custom file system based on the FUSE framework to the target directory, forming a subdirectory corresponding to the NVMe storage device handle. This allows the host to determine the target handle by parsing operation requests to the subdirectory of the mount point directory. During write operations, the target handle guides the NVMe device to accurately place data into the corresponding recycling unit, thereby reducing the write amplification factor, improving storage performance, and extending the lifespan of the solid-state drive. Furthermore, it eliminates the need for users to directly operate the complex FDP underlying interface. In other words, users only need to perform write operations on the file system directory, which is no different from operating a local hard drive directory, i.e., performing standard file operations. The host automatically converts standard file operation requests into control commands for the NVMe storage device handle through the custom file system, hiding the details of FDP technology and improving the convenience for users to place data.

[0129] Furthermore, embodiments of this application also disclose an electronic device, Figure 10 This is a structural diagram of an electronic device according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. Specifically, the electronic device may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the NVMe storage device data placement method disclosed in any of the foregoing embodiments. Furthermore, the electronic device in this embodiment may specifically be a computer.

[0130] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0131] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0132] The operating system 221 is used to manage and control the various hardware devices on the electronic device and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the NVMe storage device data placement method executed by the electronic device as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0133] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned NVMe storage device data placement method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0134] Furthermore, embodiments of this application also disclose a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the NVMe storage device data placement method disclosed in any of the foregoing embodiments.

[0135] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0136] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0137] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0138] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0139] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for placing data on an NVMe storage device, characterized in that, For use with hosts connected to NVMe storage devices, including: A custom file system is mounted to the target directory to obtain a mount point directory; wherein, the custom file system is built based on the FUSE system framework, and the mount point directory includes subdirectories, each of which corresponds to a handle of the NVMe storage device; Obtain the current operation request; wherein, the current operation request represents a target operation to be performed on the current subdirectory in the mount point directory; The current operation request is processed using the custom file system to determine the type of the target operation and the target handle corresponding to the current subdirectory; If the type of the target operation is write, the NVMe storage device will place the target data carried by the current operation request in the target recycling unit corresponding to the target handle; The NVMe storage device data placement method further includes: Establish a first mapping relationship between each subdirectory in the mount point directory and each handle identifier of the NVMe storage device; wherein each handle identifier corresponds to each handle of the NVMe storage device; Accordingly, the current operation request is processed using the custom file system to determine the target handle corresponding to the current subdirectory, including: Based on the first mapping relationship, and using the custom file system to process the current operation request, a target handle identifier corresponding to the current subdirectory is determined; and the target handle is locked from each of the handles according to the target handle identifier. The NVMe storage device data placement method further includes: The FDP function parameters supported by the NVMe storage device are detected; wherein, the FDP function parameters include the structural information of each durability group and the second mapping relationship between the reclaim unit handle and the handle identifier in the durability group, and the structural information of the durability group represents each reclaim unit contained in each reclaim group under the durability group. The current operation request is processed using the custom file system to determine the target handle corresponding to the current subdirectory, including: The current operation request is processed using the custom file system to determine the target handle identifier corresponding to the current subdirectory; the target recycling unit handle corresponding to the target handle identifier is locked in each of the handles according to the second mapping relationship; The step of obtaining the current operation request includes: The original operation request is structured and encapsulated using the FUSE kernel driver to obtain each structured encapsulated operation request; each structured encapsulated operation request is then stored in the kernel queue. Select the current operation request from the structured and encapsulated operation requests in the kernel queue; The step of selecting the current operation request from each of the structured, encapsulated operation requests in the kernel queue includes: The user-mode daemon selects the current operation request from the structured, encapsulated operation requests in the kernel queue through the device file; Accordingly, after the NVMe storage device places the target data carried by the current operation request into the target recycling unit corresponding to the target handle, it further includes: The user-mode daemon is controlled to write the response result of the current operation request to the device file.

2. The data placement method for NVMe storage devices according to claim 1, characterized in that, The NVMe storage device places the target data carried by the current operation request into the target recycling unit corresponding to the target handle, including: The NVMe storage device determines the target reclamation group pointed to by the target reclamation unit handle, determines the target reclamation unit in the target reclamation group, and places the target data carried by the current operation request in the target reclamation unit.

3. The data placement method for NVMe storage devices according to claim 1, characterized in that, The original operation request indicates that the application requests to write target data to a subdirectory under the mount point directory through a standard file interface.

4. The data placement method for NVMe storage devices according to claim 3, characterized in that, After the step of controlling the user-mode daemon to write the response result of the current operation request to the device file, the method further includes: The FUSE kernel driver is controlled to obtain the response result written by the user-mode daemon from the device file; The response result is returned to the application through the standard file interface to complete the input / output operation of the current operation request.

5. The data placement method for NVMe storage devices according to claim 1, characterized in that, Before placing the target data carried by the current operation request into the target reclamation unit corresponding to the target handle, the NVMe storage device further includes: Call the io_uring interface to submit the current operation request to the NVMe storage device; Accordingly, controlling the user-mode daemon to write the response result of the current operation request to the device file includes: The user-space daemon is controlled to write the response result of the current operation request returned by the NVMe storage device to the device file.

6. The data placement method for NVMe storage devices according to claim 5, characterized in that, The step of calling the io_uring interface to submit the current operation request to the NVMe storage device includes: Create an io_uring circular buffer; wherein, the io_uring circular buffer includes a commit queue; Retrieve idle request entries from the submission queue; The opcode of the current operation request and the target handle identifier corresponding to the current subdirectory are filled into the idle request entry to obtain the submit request entry; The submit queue is controlled to submit the submit request entry to the NVMe storage device.

7. The data placement method for an NVMe storage device according to claim 6, characterized in that, The io_uring circular buffer includes a completion queue; The step of controlling the user-space daemon to write the response result of the current operation request returned by the NVMe storage device to the device file includes: The completion queue is used to receive the response result of the current operation request returned by the NVMe storage device; wherein, the response result of the current operation request is a completion request entry encapsulated by the NVMe storage device using the completion status, status code and error information of the current operation request; The user-mode daemon is controlled to write the response result of the current operation request to the device file.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the NVMe storage device data placement method according to any one of claims 1 to 7.

9. An electronic device, characterized in that, include: memory for storing computer programs; A processor for executing the computer program to implement the steps of the NVMe storage device data placement method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the NVMe storage device data placement method as described in any one of claims 1 to 7.

Images