NVMe storage device data placement method, product, device and medium

Abstract

The invention discloses an NVMe storage device data placement method, a product, a device and a medium, relates to the technical field of solid state disk storage, is applied to a host connected with an NVMe storage device, and comprises the following steps: mounting a user-defined file system to a target directory to obtain a mounting point directory; the custom file system is constructed based on a FUSE system framework, and each subdirectory of the mounting point directory corresponds to each handle of the NVMe storage device; acquiring a current operation request; the current operation request represents that target operation is carried out on the current subdirectory; processing the current operation request by using the custom file system, and determining the type of a target operation and a target handle corresponding to the current subdirectory; and if the type of the target operation is a write type, the NVMe storage device places target data carried by the current operation request in a target recovery unit corresponding to the target handle. And the write amplification factor is reduced, so that the storage performance is improved, and the service life of the solid state disk is prolonged.

Description

Technical Field

[0001] The present invention relates to the technical field of solid - state drive storage, and particularly to a data placement method, product, device, and medium for NVMe storage devices. Background Art

[0002] In the technical field of solid - state drive (SSD) storage, with the growth of data storage requirements, the performance and service life of SSDs have become key issues. Inside an SSD, data is stored through flash chips, and its performance and life are significantly affected by the garbage collection (GC) mechanism. When a user places data, the SSD needs to migrate valid data to a new flash block through GC and erase the old block for reuse. This process increases the write amplification factor (WAF). Write amplification causes the actual amount of data written to the flash to far exceed the amount of data written by the user, not only reducing the write performance of the SSD but also accelerating the wear of flash chips and shortening the service life of the SSD.

[0003] Although there are technologies such as flexible data placement (FDP) that attempt to optimize the data storage location, it is difficult to achieve precise data placement through user - layer control, and there are obvious deficiencies in balancing user operation convenience and storage performance optimization.

[0004] It can be seen that how to improve the convenience of data placement and reduce the write amplification factor to enhance storage performance and extend the service life of solid - state drives is a problem that those skilled in the art need to solve. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a data placement method, device, device, and medium for NVMe storage devices, which improve the convenience of data placement and reduce the write amplification factor to enhance storage performance and extend the service life of solid - state drives. The specific solutions are as follows: In a first aspect, the present invention discloses a data placement method for an NVMe storage device, which is applied to a host connected to the NVMe storage device and includes: Mount a custom file system to a target directory to obtain a mount - point directory; wherein, the custom file system is constructed based on the FUSE system framework, and the mount - point directory includes each sub - directory, and each sub - directory corresponds to each handle of the NVMe storage device; Obtain a current operation request; wherein, the current operation request represents a target operation on the current sub - directory in the mount - point directory; Process the current operation request by using the custom file system to determine the type of the target operation and the target handle corresponding to the current subdirectory; If the type of the target operation is a write type, so that the NVMe storage device places the target data carried by the current operation request in the target recycling unit corresponding to the target handle.

[0006] Optionally, the method for placing NVMe storage device data further includes: Establish a first mapping relationship between each subdirectory in the mount point directory and each handle identifier of the NVMe storage device; wherein, each of the handle identifiers corresponds to each handle of the NVMe storage device; Correspondingly, processing the current operation request by using the custom file system to determine the target handle corresponding to the current subdirectory includes: Based on the first mapping relationship, and processing the current operation request by using the custom file system to determine the target handle identifier corresponding to the current subdirectory; Lock the target handle from each of the handles according to the target handle identifier.

[0007] Optionally, the method for placing NVMe storage device data further includes: Detect the FDP function parameters supported by the NVMe storage device; wherein, the FDP function parameters include the structure information of each durability group and a second mapping relationship between the recycling unit handle and the handle identifier in the durability group, and the structure information of the durability group characterizes each recycling unit included in each recycling group under the durability group.

[0008] Optionally, processing the current operation request by using the custom file system to determine the target handle corresponding to the current subdirectory includes: Process the current operation request by using the custom file system to determine the target handle identifier corresponding to the current subdirectory; Lock the target recycling unit handle corresponding to the target handle identifier from each of the handles according to the second mapping relationship.

[0009] Optionally, the NVMe storage device placing the target data carried by the current operation request in the target recycling unit corresponding to the target handle includes: The NVMe storage device determines the target recycling group pointed to by the target recycling unit handle, and determines the target recycling unit in the target recycling group, and places the target data carried by the current operation request in the target recycling unit.

[0010] Optionally, the obtaining of the current operation request includes: Structurally encapsulating the original operation request by using a FUSE kernel driver to obtain each structurally encapsulated operation request; Storing each of the structurally encapsulated operation requests into a kernel queue; Selecting the current operation request from each of the structurally encapsulated operation requests in the kernel queue.

[0011] Optionally, the selecting of the current operation request from each of the structurally encapsulated operation requests in the kernel queue includes: Controlling a user-space daemon process to select the current operation request from each of the structurally encapsulated operation requests in the kernel queue through a device file; Correspondingly, after the NVMe storage device places the target data carried by the current operation request in a target recovery unit corresponding to the target handle, it further includes: Controlling the user-space daemon process to write the response result of the current operation request into the device file.

[0012] Optionally, the original operation request represents that an application program requests to write target data to a subdirectory under the mount point directory through a standard file interface.

[0013] Optionally, after controlling the user-space daemon process to write the response result of the current operation request into the device file, it further includes: Controlling the FUSE kernel driver to obtain the response result written by the user-space daemon process from the device file; Returning the response result to the application program through the standard file interface to complete the input / output operation of the current operation request.

[0014] Optionally, before the NVMe storage device places the target data carried by the current operation request in a target recovery unit corresponding to the target handle, it further includes: Invoking an io_uring interface to submit the current operation request to the NVMe storage device; Correspondingly, the controlling the user-space daemon process to write the response result of the current operation request into the device file includes: Controlling the user-space daemon process to write the response result of the current operation request returned by the NVMe storage device into the device file.

[0015] Optionally, the invoking of the io_uring interface to submit the current operation request to the NVMe storage device includes: Create an io_uring ring buffer; wherein, the io_uring ring buffer includes a submission queue; Obtain an idle request entry from the submission queue; Fill the opcode of the current operation request and the target handle identifier corresponding to the current subdirectory into the idle request entry to obtain a submission request entry; Control the submission queue to submit the submission request entry to the NVMe storage device.

[0016] Optionally, the io_uring ring buffer includes a completion queue; The controlling the user-space daemon to write the response result of the current operation request returned by the NVMe storage device to the device file includes: Receive the response result of the current operation request returned by the NVMe storage device by using the completion queue; wherein, the response result of the current operation request is a completion request entry encapsulated by the NVMe storage device by using the completion status, status code, and error information of the current operation request; Control the user-space daemon to write the response result of the current operation request to the device file.

[0017] In a second aspect, the present invention discloses a computer program product, including a computer program / instructions, which when executed by a processor, implement the steps of the NVMe storage device data placement method disclosed above.

[0018] In a third aspect, the present invention discloses an electronic device, including: A memory for storing a computer program; A processor for executing the computer program to implement the steps of the NVMe storage device data placement method disclosed above.

[0019] In a fourth aspect, the present invention discloses a computer-readable storage medium for storing a computer program; wherein, the computer program when executed by a processor implements the steps of the NVMe storage device data placement method disclosed above.

[0020] As can be seen, the present invention is applied to a host connected to an NVMe storage device, including: mounting a custom file system to a target directory to obtain a mounted point directory; wherein, the custom file system is constructed based on the FUSE system framework, and the mounted point directory includes subdirectories, and each of the subdirectories corresponds to a handle of the NVMe storage device; obtaining a current operation request; wherein, the current operation request represents a target operation on the current subdirectory in the mounted point directory; using the custom file system to process the current operation request to determine the type of the target operation and the target handle corresponding to the current subdirectory; if the type of the target operation is a write type, so that the NVMe storage device places the target data carried by the current operation request in a target recycling unit corresponding to the target handle.

[0021] The beneficial effects are as follows: By mounting a custom file system based on the FUSE framework to a target directory to form subdirectories corresponding to the handles of the NVMe storage device, the host can determine the target handle by parsing the operation request for the subdirectory of the mounted point directory. During a write operation, the target handle guides the NVMe device to accurately place the data into the corresponding recycling unit, thereby achieving a reduction in the write amplification factor, an improvement in storage performance, and an extension of the service life of the solid-state drive. Moreover, users do not need to directly operate the complex FDP underlying interface. That is to say, users only need to perform a write operation on the file system directory, which is no different from operating a local hard disk directory, that is, performing a standard file operation. The host automatically converts the standard file operation request into a control instruction for the handle of the NVMe storage device through the custom file system, hiding the technical details of FDP and improving the convenience for users to complete data placement. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] To more clearly illustrate the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 It is a flowchart of a method for placing data in an NVMe storage device provided by an embodiment of the present invention; Figure 2 It is a comparison diagram of the amplification advantages of a specific flexible data placement provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of a specific FDP functional unit of an NVMe storage device provided by an embodiment of the present invention; Figure 4 It is a flowchart of a method for placing data in an NVMe storage device provided by an embodiment of the present invention; Figure 5 A specific initialization schematic diagram provided by an embodiment of the present invention; Figure 6 A schematic diagram of the mapping relationship between a specific handle identifier and a subdirectory provided by an embodiment of the present invention; Figure 7 A specific FUSE result schematic diagram provided by an embodiment of the present invention; Figure 8 A specific data writing flow chart provided by an embodiment of the present invention; Figure 9 A structural schematic diagram of an NVMe storage device data placement device provided by an embodiment of the present invention; Figure 10 A structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0024] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0025] In the field of solid-state drive storage technology, with the growth of data storage requirements, the performance and service life of SSDs have become key issues. Inside an SSD, data is stored through flash memory chips, and its performance and life are significantly affected by the garbage collection mechanism. When a user performs data placement, the SSD needs to migrate valid data to a new flash block through GC and erase the old block for reuse. This process will increase the write amplification factor (WAF). Write amplification causes the actual amount of data written to the flash memory to far exceed the amount of data written by the user, which not only reduces the write performance of the SSD but also accelerates the wear of the flash memory chips and shortens the service life of the SSD.

[0026] Although there are technologies such as flexible data placement trying to optimize the data storage location, it is difficult to achieve precise data placement through user-level control, and there are obvious deficiencies in balancing user operation convenience and storage performance optimization.

[0027] The terms "including" and "having" in the specification of the present invention and the above-mentioned accompanying drawings, as well as any variations related to "including" and "having", are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may include steps or units not listed.

[0028] To enable those skilled in the art to better understand the solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] Next, a data placement solution for an NVMe storage device provided by an embodiment of the present invention will be introduced in detail. Figure 1 A data placement method for an NVMe storage device provided by an embodiment of the present invention is applied to a host connected to the NVMe storage device and includes: Step S11: Mount a custom file system to a target directory to obtain a mount point directory; wherein, the custom file system is built based on the FUSE system framework, and the mount point directory includes each sub-directory, and each sub-directory corresponds to each handle of the NVMe storage device.

[0030] Due to its advantages such as small volume, fast access speed, and high data density, solid-state drives have become increasingly popular in various storage systems. However, the performance and lifespan of SSDs are significantly affected by the garbage collection mechanism. GC is a basic technology for SSDs to clean up invalid data to free up space, but this process will cause additional write operations, increase the write amplification factor, thereby reducing the write performance of SSDs and shortening their limited write lifespan. With the development of the NVMe (Non-Volatile Memory Express) standard, flexible data placement has been proposed as a new feature to reduce the overhead of GC, such as Figure 2 As shown in a schematic diagram comparing the amplification advantages of a specific flexible data placement, FDP allows the host system to more flexibly control the placement location of data in the SSD, thereby optimizing the data layout, reducing the generation of invalid data, and the frequency of GC.

[0031] The host is connected to the NVMe storage device. On the host side, a custom file system (Self-defined File System, i.e., SDFS) is built based on the FUSE system framework and mounted to the target directory to obtain a mount point directory. Among them, the mount point directory includes each sub-directory, and each sub-directory corresponds to each handle of the NVMe storage device. That is to say, there is a mapping relationship between each sub-directory and each handle, so as to map different handles to the user layer in the form of directories.

[0032] In this embodiment, it further includes: establishing a first mapping relationship between each sub-directory in the mount point directory and each handle identifier of the NVMe storage device; wherein, each handle identifier corresponds to each handle of the NVMe storage device.

[0033] Specifically, set the handle identifiers of each handle, that is, there is a corresponding relationship between the handle and the handle identifier. Further, establish a first mapping relationship between each sub-directory in the mount point directory and each handle identifier of the NVMe storage device. In this way, when a sub-directory is determined, the corresponding handle identifier can be determined according to the first mapping relationship, and then the corresponding handle can be determined.

[0034] In this embodiment, it further includes: detecting the FDP function parameters supported by the NVMe storage device; wherein, the FDP function parameters include the structure information of each endurance group and the second mapping relationship between the reclaim unit handle and the handle identifier in the endurance group, and the structure information of the endurance group characterizes each reclaim unit included in each reclaim group under the endurance group.

[0035] Detect the FDP function parameters supported by the NVMe storage device, specifically detect the structure information of each endurance group in the NVMe storage device, the second mapping relationship between the reclaim unit handle and the handle identifier in the endurance group, and the structure information of the endurance group characterizes each reclaim unit included in each reclaim group under the endurance group. For example Figure 3 As shown in a schematic diagram of a specific FDP functional unit of an NVMe storage device, the NVMe storage device is connected to the host. The NVMe storage device includes a controller, a namespace, and an endurance group (i.e., EG). The endurance group includes one or more reclaim groups (i.e., RG), and each reclaim group includes one or more reclaim units (i.e., RU). Among them, the reclaim unit handle (i.e., RUH) points to the reclaim unit in the corresponding reclaim group, and there is a second mapping relationship between the handle identifier (Placement Handle) and the reclaim unit handle, that is, there is a mapping relationship between the handle identifier and the Identity of the reclaim unit handle. That is to say, determining the handle identifier can determine the reclaim unit handle, and the reclaim unit in the reclaim group pointed to by the reclaim unit handle can be determined according to the reclaim unit handle.

[0036] Step S12: Obtain the current operation request; wherein, the current operation request represents a target operation on the current sub-directory in the mount point directory.

[0037] The host side obtains the current operation request, and the current operation request represents a target operation on the current sub-directory in the mount point directory, and the original operation request corresponding to the current operation request is an operation request initiated by the application program to the host.

[0038] It should be noted that when the original operation request after structured encapsulation is saved in the kernel queue, it is necessary to filter out the current operation request from the kernel queue. Specifically, the current operation request can be filtered out based on the storage time of each operation request in the kernel queue, or the current operation request can be filtered out according to the processing priority of each operation request.

[0039] Step S13: Process the current operation request by using the custom file system to determine the type of the target operation and the target handle corresponding to the current subdirectory.

[0040] Process the current operation request by using the custom file system to determine the type of the target operation. The type of the target operation specifically includes a data writing type, a deletion type, and a reading type.

[0041] In this embodiment, processing the current operation request by using the custom file system to determine the target handle corresponding to the current subdirectory includes: based on the first mapping relationship, and processing the current operation request by using the custom file system to determine the target handle identifier corresponding to the current subdirectory; locking the target handle from each of the handles according to the target handle identifier.

[0042] It can be understood that since there is a first mapping relationship between each subdirectory in the mount point directory and each handle identifier of the NVMe storage device, and the current operation request is to perform a target operation on the current subdirectory in the mount point directory, that is, the custom file system can determine the current subdirectory, and then determine the target handle identifier corresponding to the current subdirectory based on the first mapping relationship. In this way, the target handle can be locked from each of the handles according to the target handle identifier.

[0043] In this embodiment, processing the current operation request by using the custom file system to determine the target handle corresponding to the current subdirectory includes: processing the current operation request by using the custom file system to determine the target handle identifier corresponding to the current subdirectory; locking the target recycle unit handle corresponding to the target handle identifier from each of the handles according to the second mapping relationship.

[0044] Furthermore, since there is a second mapping relationship between the recycle unit handle and the handle identifier, when processing the current operation request by using the custom file system to lock the target handle identifier corresponding to the current subdirectory, the target recycle unit handle corresponding to the target handle identifier can be locked from each of the handles according to the second mapping relationship, that is, the target recycle unit handle is the target handle.

[0045] Step S14: If the type of the target operation is a write type, so that the NVMe storage device places the target data carried by the current operation request in a target recycling unit corresponding to the target handle.

[0046] In this embodiment, the NVMe storage device places the target data carried by the current operation request in a target recycling unit corresponding to the target handle, including: the NVMe storage device determines a target recycling group pointed to by the target recycling unit handle, determines a target recycling unit in the target recycling group, and places the target data carried by the current operation request in the target recycling unit.

[0047] If the type of the target operation is a write type, the NVMe storage device places the target data carried by the current operation request in a target recycling unit corresponding to the target handle. Specifically, after determining the target handle, the NVMe storage device determines a target recycling group pointed to by the target recycling unit handle, then determines a target recycling unit in the target recycling group, and then can place the target data carried by the current operation request in the target recycling unit.

[0048] After the NVMe storage device places the target data carried by the current operation request in a target recycling unit corresponding to the target handle, the NVMe storage device returns a response result of the current operation request. The response result of the current operation request indicates that the target data is successfully placed or the target data is not successfully placed. Next, the user-mode daemon process writes the response result of the current operation request into a device file, the FUSE kernel driver obtains the response result from the device file, and then returns the response result to the application program through a standard file interface to complete the input / output operation of the current operation request.

[0049] It can be seen that the present invention is applied to a host connected to an NVMe storage device, including: mounting a custom file system to a target directory to obtain a mount point directory; wherein, the custom file system is built based on the FUSE system framework, and the mount point directory includes each sub-directory, and each sub-directory corresponds to each handle of the NVMe storage device; obtaining a current operation request; wherein, the current operation request represents a target operation on the current sub-directory in the mount point directory; using the custom file system to process the current operation request to determine the type of the target operation and a target handle corresponding to the current sub-directory; if the type of the target operation is a write type, so that the NVMe storage device places the target data carried by the current operation request in a target recycling unit corresponding to the target handle.

[0050] The beneficial effects are as follows: By mounting the custom file system based on the FUSE framework to the target directory, the present invention forms subdirectories corresponding to the NVMe storage device handles, enabling the host to determine the target handle by parsing the operation requests for the subdirectories of the mount point directory. During the write operation, the target handle guides the NVMe device to accurately place the data into the corresponding recovery unit, thereby reducing the write amplification factor, improving the storage performance, and extending the service life of the solid-state drive. Moreover, the user does not need to directly operate the complex FDP underlying interface. That is to say, the user only needs to perform writes on the file system directory, which is no different from operating the local hard disk directory, that is, performing standard file operations. The host automatically converts the standard file operation requests into control instructions for the NVMe storage device handles through the custom file system, hiding the FDP technical details and improving the convenience for the user to complete data placement.

[0051] See Figure 4 The embodiment of the present invention discloses a specific method for placing data in an NVMe storage device. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Applied to a host connected to an NVMe storage device, it includes: Step S21: Mount the custom file system to the target directory to obtain a mount point directory; wherein, the custom file system is constructed based on the FUSE system framework, the mount point directory includes each subdirectory, and each subdirectory corresponds to each handle of the NVMe storage device.

[0052] The host side mounts the custom file system based on FUSE to the target directory to obtain a mount point directory. By mapping different handle identifiers to the user layer in the form of directories, an explicit data placement function is realized at the file system level, enabling the user to more easily use the flexible data placement function, thereby optimizing data access performance and storage management. Specifically, for example Figure 5 As shown in a specific initialization schematic diagram, the custom file system based on FUSE is mounted to the target directory to obtain a mount point directory, and the FDP function parameters supported by the NVMe storage device are collected and detected to obtain the FDP capability set. The FDP function parameters of the FDP capability set include the structure information of each durability group and the second mapping relationship between the recovery unit handles and the handle identifiers in the durability group. The structure information of the durability group represents each recovery unit included in each recovery group under the durability group, and a first mapping relationship is established between each subdirectory in the mount point directory and each handle identifier of the NVMe storage device. That is to say, a corresponding subdirectory is created for each handle identifier in the mount point directory.

[0053] For example Figure 6The diagram shows a specific mapping relationship between handle identifiers and subdirectories. This is a user-level custom file system based on the user space file system framework. Its main function is to identify the FPD function of the attached NVMe device and the handle identifier corresponding to the device RUH. These handle identifiers are presented to the user layer in the form of directories. When read, write, or delete operations occur on files in this directory, a series of operations are performed using the liburing interface to write the data to the unique handle identifier bound to this folder. The mount point directory of the user-level custom file system is mounted. This directory contains several subdirectories, each of which corresponds to a specific handle identifier. User programs can write data to these subdirectories through standard file operation interfaces.

[0054] Step S22: Use the FUSE kernel driver to perform structured encapsulation on the original operation request to obtain each structured encapsulated operation request; store each of the structured encapsulated operation requests in the kernel queue; select a current operation request from each of the structured encapsulated operation requests in the kernel queue; wherein the current operation request represents a target operation on the current subdirectory in the mount point directory.

[0055] For example Figure 7 As shown in the figure, when the application layer program interacts with the mounted custom file system, the operating system forwards these original operation requests to the FUSE kernel driver. When the FUSE kernel driver receives the original operation request, it encapsulates the original operation request into a structured manner to obtain each structured encapsulated operation request, and then stores each structured encapsulated operation request into the kernel queue. At this time, the application that initiates the request will enter a waiting state until a response is received.

[0056] The current operation request is selected from the structured and encapsulated operation requests in the kernel queue. It can be understood that the current operation request represents a target operation on the current subdirectory in the mount point directory. Furthermore, when selecting the current operation request from the structured encapsulated operation requests in the kernel queue, the operation requests may be first screened in descending order of their processing priorities, and then screened in the order of the time the operation requests were stored in the kernel queue. Specifically, the processing priorities of the structured encapsulated operation requests in the kernel queue are determined. If the processing priorities of the structured encapsulated operation requests are different, the current operation request is selected from the kernel queue based on the processing priorities from high to low. If the processing priorities of the structured encapsulated operation requests are the same, the time when the structured encapsulated operation requests were stored in the kernel queue is determined, and the current operation request is selected from the kernel queue in the order of the time they were stored in the kernel queue. In this way, it is ensured that operation requests with higher processing priorities are processed first, and it is also ensured that operations generated earlier can be processed earlier. In other words, this scheduling strategy uses a dual-layer screening mechanism of priority + time order to ensure that high-priority tasks are executed first while taking into account the fairness of tasks with the same priority, thereby meeting real-time requirements and avoiding task starvation. By dynamically balancing the response speed of critical tasks and the overall system throughput, resource utilization is effectively improved and system performance is optimized.

[0057] In this embodiment, selecting the current operation request from the structured encapsulated operation requests in the kernel queue includes: controlling the user-mode daemon process to select the current operation request from the structured encapsulated operation requests in the kernel queue through a device file.

[0058] like Figure 7 As shown, when a structured encapsulated operation request is detected in the kernel queue, the user-state daemon, i.e., the user-state SDFS daemon, is awakened, and then the user-state daemon is controlled to select the current operation request from the structured encapsulated operation requests in the kernel queue through the device file (i.e., / dev / fuse).

[0059] In this embodiment, the original operation request represents an application request to write target data to a subdirectory under the mount point directory through a standard file interface. The application initiates the original operation request to write target data to a subdirectory under the mount point directory through a standard file interface. In other words, SDFS exposes the FDP SSD as a file directory to user mode, allowing users to easily group similar data, thereby more efficiently managing data storage, reducing the difficulty of using the NMVeFDP feature, and providing a more convenient method for optimizing WAF.

[0060] Step S23: Process the current operation request by using the custom file system to determine the type of the target operation and the target handle corresponding to the current subdirectory.

[0061] Process the current operation request according to the logic of the mounted file system, forward the current operation request to the underlying file system, or pass the current operation request to other kernel subsystems for further processing to determine the type of the target operation and the target handle corresponding to the current subdirectory. Among them, because there is a first mapping relationship between each subdirectory in the mount point directory and each handle identifier of the NVMe storage device, and a second mapping relationship between the reclaim unit handle and the handle identifier, when the current subdirectory is determined, the target handle identifier corresponding to the current subdirectory and the target reclaim unit handle corresponding to the target handle identifier can be determined in sequence according to the first mapping relationship and the second mapping relationship.

[0062] Step S24: If the type of the target operation is the write type, so that the NVMe storage device places the target data carried by the current operation request in the target reclaim unit corresponding to the target handle.

[0063] The current FDP software stack is built based on io_uring and the NVMe generic character device interface. io_uring supports efficient and scalable asynchronous I / O operations for storage and network tasks. It uses a ring buffer to communicate between the application and the kernel, reducing system calls and improving performance. io_uring relies on two shared ring buffers: a submission queue (SQ) for sending requests and a completion queue (CQ) for receiving results, and uses these two queues to handle the communication between the user program and the kernel. It processes I / O by extracting an entry called SQE from the SQ, filling the SQE, and calling the io_uring_enter system call.

[0064] In this embodiment, before the NVMe storage device places the target data carried by the current operation request in the target reclaim unit corresponding to the target handle, it further includes: calling the io_uring interface to submit the current operation request to the NVMe storage device.

[0065] For example Figure 8 As shown in a specific data write flowchart, if the type of the target operation is the write type, call the io_uring interface to submit the current operation request to the NVMe storage device, and the NVMe storage device places the target data carried by the current operation request in the target reclaim unit corresponding to the target reclaim unit handle.

[0066] In this embodiment, submitting the current operation request to the NVMe storage device by invoking the io_uring interface includes: creating an io_uring ring buffer; where the io_uring ring buffer includes a submission queue; obtaining a free request entry from the submission queue; filling the operation code of the current operation request and the target handle identifier corresponding to the current subdirectory into the free request entry to obtain a submission request entry; and controlling the submission queue to submit the submission request entry to the NVMe storage device.

[0067] Create an io_uring ring buffer. Specifically, the io_uring ring buffer includes a submission queue, and the submission queue includes multiple request entries, including occupied request entries (i.e., submission request entries) and free request entries. Select a free request entry, then fill the operation code of the current operation request and the target handle identifier corresponding to the current subdirectory into the free request entry to obtain a submission request entry, and control the submission queue to submit the submission request entry to the NVMe storage device, so that the NVMe storage device receives the current operation request and the target handle identifier. The NVMe storage device determines the target recovery group pointed to by the target recovery unit handle and determines the target recovery unit in the target recovery group, and places the target data carried by the current operation request in the target recovery unit.

[0068] In this embodiment, after the NVMe storage device places the target data carried by the current operation request in the target recovery unit corresponding to the target handle, it further includes: controlling the user-space daemon process to write the response result of the current operation request into the device file. After the NVMe storage device places the target data carried by the current operation request in the target recovery unit corresponding to the target handle, the host controls the user-space daemon process to write the response result of the current operation request into the device file.

[0069] In this embodiment, controlling the user-space daemon process to write the response result of the current operation request into the device file includes: controlling the user-space daemon process to write the response result of the current operation request returned by the NVMe storage device into the device file. After the NVMe storage device places the target data carried by the current operation request in the target recovery unit corresponding to the target handle, the NVMe storage device returns the response result of the current operation request to the host. The user-space daemon process of the host receives the response result of the current operation request, and the host controls the user-space daemon process to write the response result of the current operation request returned by the NVMe storage device into the device file.

[0070] In this embodiment, after controlling the user-space daemon process to write the response result of the current operation request into the device file, the method further includes: controlling the FUSE kernel driver to obtain the response result written by the user-space daemon process from the device file; and returning the response result to the application program through the standard file interface to complete the input / output operation of the current operation request.

[0071] Since the user-space daemon process writes the response result of the current operation request into the device file, the FUSE kernel driver can be controlled to obtain the response result written by the user-space daemon process from the device file, and then the response result is returned to the application program through the standard file interface to complete the input / output operation of the current operation request. It can be understood that the operation request is sent by the application program to the host through the standard file interface. After receiving the operation request, the host processes it accordingly, sends it to the NVMe storage device, and receives the response result returned by the NVMe storage device. Then the host returns the response result to the application program through the standard file interface, thus completing the input / output operation of the current operation request.

[0072] In this embodiment, the io_uring ring buffer includes a completion queue; the step of controlling the user-space daemon process to write the response result of the current operation request returned by the NVMe storage device into the device file includes: using the completion queue to receive the response result of the current operation request returned by the NVMe storage device; where the response result of the current operation request is a completion request entry encapsulated by the NVMe storage device using the completion status, status code, and error information of the current operation request; and controlling the user-space daemon process to write the response result of the current operation request into the device file.

[0073] Further, the io_uring ring buffer includes a completion queue, which is used to write the response result into the device file. Specifically, after the NVMe storage device places the target data carried by the current operation request in the target recovery unit corresponding to the target handle, it returns the response result of the current operation request to the host. And the response result of the current operation request is a completion request entry encapsulated by the NVMe storage device using the completion status, status code, and error information of the current operation request. Among them, the completion status of the current operation request is a success status or a failure status. The success status indicates that the NVMe storage device has successfully placed the target data in the target recovery unit, and the failure status indicates that the NVMe storage device has not successfully placed the target data in the target recovery unit. Next, control the user-space daemon process to write the response result of the current operation request into the device file. In this way, control the FUSE kernel driver to obtain the response result written by the user-space daemon process from the device file.

[0074] It can be seen that the user-level custom file system based on the FUSE framework abstracts the complex underlying interfaces of NVMe FDP (such as Placement Handle, RUH, etc.) into the directory structure of the file system, enabling users to optimize data layout without directly operating on the technical details of the FDP at the bottom layer. Specifically, during the initialization phase of SDFS, it automatically detects the FDP capabilities of NVMe devices and dynamically generates subdirectories that correspond one-to-one with the Placement Handle. The read and write operations of the user on the directory (such as copying a file to a specific directory) will be captured by the SDFS daemon process and converted into I / O requests for the underlying FDP. Through the liburing and io_uring mechanisms, the data is directed to be written into the corresponding RUH storage area. SDFS is fully compatible with standard file operation interfaces (such as open, write, read). Users can operate on directories using conventional commands (such as cp, mv) or programming interfaces (such as fopen), while the system automatically handles complex processes such as the binding of the Placement Handle of FDP and the submission of NVMe commands. This design hides the technical details of FDP through directory mapping and transparent conversion, and at the same time uses the asynchronous I / O mechanism to ensure performance. Finally, users can complete data grouped storage in a simple way of operating on local directories, achieving the effects of reducing WAF and optimizing the SSD lifespan without having to understand the underlying NVMe FDP protocol.

[0075] The following is a corresponding description of the present invention. RAID (Redundant Arrays of Independent Disks, that is, an array with redundancy capabilities composed of several independent disks) achieves the purpose of improving performance and increasing the data fault tolerance of the entire system by overlapping input and output operations in a balanced manner by placing data on multiple hard disks. NVMe (Non-Volatile Memory Express, that is, the Non-Volatile Memory Host Controller Interface Specification) is a high-performance interface protocol for accessing non-volatile storage media (such as solid-state drives SSD). It is designed specifically for solid-state drives and has the characteristics of high performance (supporting multi-queue and high-concurrency operations, which can significantly improve data transfer speed) and low latency (compared with the traditional SATA protocol, NVMe has lower latency). FDP is an emerging interface in the NVMe storage standard, aiming to reduce the Write Amplification Factor (WAF) in SSDs through explicit user control of data placement. It is a technology that optimizes storage performance and lifespan by flexibly allocating data storage locations. WAF refers to the ratio of the actual amount of data written to the NAND flash in a solid-state drive (SSD) to the amount of data written by the host, and it is an important indicator for measuring the performance and durability of SSDs. A higher WAF will lead to a decline in write performance and a shortening of the lifespan of the SSD. Therefore, the ideal WAF is 1. FS (File System, that is, the file system) is a mechanism in the operating system for managing and organizing files. It defines the naming, storage, and access methods of files and directories. The NVMe Namespace is a logical storage area that divides the non-volatile storage space of an NVMe device into multiple independent logical blocks (Logical Blocks). Each logical block can be formatted and presented as a standard block device. In the Linux system, each namespace usually appears as an independent block device. Garbage collection is a mechanism in solid-state drives for optimizing storage space and improving write performance. In an SSD hard disk, if you want to write new data to a block storing useless data, you need to first erase the entire block before you can write new data. That is to say, solid-state drives do not have the ability to directly overwrite old data. For an SSD, GC refers to the process of transferring the valid data in an old data block to other locations and erasing the old data block. RU is a collection of NAND blocks in which the host can write logical blocks, similar to the concept of a super block in the Flash Translation Layer (FTL) of an SSD, with a typical size of several GB (for example, 6GB). A recovery group is a collection of recovery units, usually used to separate data into different NAND chips according to the host policy.The reclaim unit handle is a resource within the SSD that manages and buffers the logical blocks written to the reclaim units. A namespace can access one or more RUHs. If a namespace can access multiple RUHs, the host can write data to multiple RUs simultaneously. Many FDP devices have 8, 16, 256, etc. RUHs. A durability group represents an FDP configuration that contains multiple RGs and RUs and provides one or more RUHs for each RG. The user space file system is a mechanism for Unix-like operating systems that enables users to create custom file systems without editing the kernel code by running the file system code in user space.

[0076] Further, during the data placement process of the NVMe storage device, first, device capability detection and initialization: When mounting the SDFS file system, the system first obtains the FDP capability set of the underlying storage device through the NVMe Identify command, parses the number of Placement Handles supported by the device and related parameters (such as configuration information like RUH, EG, etc.), and dynamically creates subdirectories corresponding one-to-one with the Placement Handles under the mount point directory. Then, file operation interception and routing: When the user performs a file operation on a specific subdirectory (such as / mnt / sdfs / handle_2), the FUSE kernel module intercepts system calls (open / write, etc.) and passes the request to the user-space SDFS daemon process through the / dev / fuse character device. The daemon process parses the target path and extracts the corresponding Placement Handle identifier. Next, request encapsulation: When preparing the io_uring SQE (submission queue entry) through liburing: Set the target Placement Handle in the FDP extension field of the NVMe command; attach the FDP Directive (such as NVMe_IO_FDP_DIRECTIVE_WITH_PH) to the write request; associate the user buffer data with the LBA address. Further, asynchronous submission and execution: Batch submit SQEs through the io_uring_enter system call - the NVMe controller according to the FDP configuration; Write the data to the NAND physical location corresponding to the specified RUH; Maintain the binding relationship between the data and the Placement Handle when updating the FTL mapping table; Return the operation status through the CQE (completion queue entry). Next, metadata management: Maintain the mapping table records in memory: The correspondence between the file inode and the Placement Handle; The remaining capacity and wear status of each RUH; Regularly synchronize the FDP status information on the device side through the NVMe Get Log Page command. Further, garbage collection optimization: Execute GC based on Placement Handle grouping: Prioritize recycling the RUH group with a higher WAF; Clean up the invalid data concentrated within the same RUH; Keep the PH unchanged when migrating the valid data blocks within the group.

[0077] It realizes the transformation of the physical storage management of FDP into an intuitive directory operation through the file system abstraction layer. While ensuring data locality, the write amplification factor (WAF) is reduced compared to traditional SSDs. Users only need to use the standard file API to achieve: Directing hot data to be written to low-latency RUH, storing cold data centrally in high-durability EG, and automatically grouping and storing data with similar lifecycles.

[0078] Figure 9Schematic diagram of the structure of a data placement device for an NVMe storage device provided by an embodiment of the present invention, which is applied to a host connected to the NVMe storage device and includes: A system mounting module 11, configured to mount a custom file system to a target directory to obtain a mounted point directory; wherein, the custom file system is built based on the FUSE system framework, and the mounted point directory includes each sub-directory, and each of the sub-directories corresponds to each handle of the NVMe storage device.

[0079] A request acquisition module 12, configured to acquire a current operation request; wherein, the current operation request represents a target operation on the current sub-directory in the mounted point directory.

[0080] A handle determination module 13, configured to process the current operation request by using the custom file system to determine the type of the target operation and the target handle corresponding to the current sub-directory.

[0081] A data placement module 14, configured to, if the type of the target operation is a write type, so that the NVMe storage device places the target data carried by the current operation request in a target recovery unit corresponding to the target handle.

[0082] It can be seen that the present invention is applied to a host connected to an NVMe storage device and includes: mounting a custom file system to a target directory to obtain a mounted point directory; wherein, the custom file system is built based on the FUSE system framework, and the mounted point directory includes each sub-directory, and each of the sub-directories corresponds to each handle of the NVMe storage device; acquiring a current operation request; wherein, the current operation request represents a target operation on the current sub-directory in the mounted point directory; processing the current operation request by using the custom file system to determine the type of the target operation and the target handle corresponding to the current sub-directory; if the type of the target operation is a write type, so that the NVMe storage device places the target data carried by the current operation request in a target recovery unit corresponding to the target handle.

[0083] The beneficial effects are as follows: By mounting the custom file system based on the FUSE framework to the target directory, the present invention forms a sub-directory corresponding to the NVMe storage device handle, enabling the host to determine the target handle by parsing the operation requests for the sub-directories of the mounted point directory. During the write operation, the target handle guides the NVMe device to accurately place the data into the corresponding recycling unit, thereby reducing the write amplification factor, improving the storage performance, and extending the service life of the solid-state drive. Moreover, users do not need to directly operate the complex FDP underlying interface. That is to say, users only need to perform writes on the file system directory, which is no different from operating a local hard disk directory, namely, performing standard file operations. The host automatically converts the standard file operation requests into control instructions for the NVMe storage device handle through the custom file system, hiding the FDP technical details and improving the convenience for users to complete data placement.

[0084] Further, the embodiment of the present application also discloses an electronic device. Figure 10 It is a structural diagram of an electronic device shown according to an exemplary embodiment. The content in the figure should not be considered as any limitation to the scope of use of the present application. The electronic device may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the NVMe storage device data placement method disclosed in any of the foregoing embodiments. In addition, the electronic device in this embodiment may specifically be an electronic computer.

[0085] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and no specific limitation is imposed on it here; the input / output interface 25 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and no specific limitation is made here.

[0086] In addition, as a carrier for resource storage, the memory 22 may be a read-only memory, a random access memory, a disk, or an optical disc, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0087] Among them, the operating system 221 is used to manage and control each hardware device and computer program 222 on the electronic device, and it can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the NVMe storage device data placement method executed by the electronic device disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs that can be used to complete other specific tasks.

[0088] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the NVMe storage device data placement method disclosed above. For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0089] Furthermore, the embodiments of the present application also disclose a computer program product, including a computer program / instructions, and when the computer program / instructions are executed by a processor, they implement the steps of the NVMe storage device data placement method disclosed in any of the foregoing embodiments.

[0090] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and reference can be made to the description of the method part for the relevant parts.

[0091] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0092] The steps of the methods or algorithms described in combination with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of the two. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0093] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0094] The technical solutions provided in this application have been introduced in detail above. Specific examples are used in this text to elaborate on the principles and implementation manners of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A method for data placement in an NVMe storage device, characterized in that, Applied to a host connected to an NVMe storage device, including: Mount a custom file system to a target directory to obtain a mount point directory; wherein, the custom file system is built based on the FUSE system framework, and the mount point directory includes each sub-directory, and each sub-directory corresponds to each handle of the NVMe storage device; Obtain a current operation request; wherein, the current operation request represents a target operation on the current sub-directory in the mount point directory; Use the custom file system to process the current operation request to determine the type of the target operation and the target handle corresponding to the current sub-directory; If the type of the target operation is a write type, so that the NVMe storage device places the target data carried by the current operation request in a target recycle unit corresponding to the target handle.

2. The method for placing NVMe storage device data according to claim 1, wherein, Also includes: Establish a first mapping relationship between each sub-directory in the mount point directory and each handle identifier of the NVMe storage device; wherein, each handle identifier corresponds to each handle of the NVMe storage device; Correspondingly, using the custom file system to process the current operation request to determine the target handle corresponding to the current sub-directory includes: Based on the first mapping relationship, and using the custom file system to process the current operation request to determine the target handle identifier corresponding to the current sub-directory; Lock the target handle from each of the handles according to the target handle identifier.

3. The method for placing NVMe storage device data according to claim 1, wherein Also includes: Detect the FDP function parameters supported by the NVMe storage device; wherein, the FDP function parameters include the structure information of each durability group and the second mapping relationship between the recycle unit handle and the handle identifier in the durability group, and the structure information of the durability group represents each recycle unit included in each recycle group under the durability group.

4. The method for placing NVMe storage device data according to claim 3, wherein Using the custom file system to process the current operation request to determine the target handle corresponding to the current sub-directory includes: Using the custom file system to process the current operation request to determine the target handle identifier corresponding to the current sub-directory; Lock the target recycle unit handle corresponding to the target handle identifier from each of the handles according to the second mapping relationship.

5. The method for placing NVMe storage device data according to claim 4, wherein The NVMe storage device places the target data carried by the current operation request in a target recycle unit corresponding to the target handle, including: The NVMe storage device determines the target recycle group pointed to by the target recycle unit handle, and determines a target recycle unit in the target recycle group, and places the target data carried by the current operation request in the target recycle unit.

6. The method for placing NVMe storage device data according to any one of claims 1 to 5, characterized in that, The obtaining the current operation request includes: Use the FUSE kernel driver to structurally encapsulate the original operation request to obtain each structurally encapsulated operation request; Store each of the structurally encapsulated operation requests into a kernel queue; Select the current operation request from each of the structurally encapsulated operation requests in the kernel queue.

7. The method for placing NVMe storage device data according to claim 6, wherein Selecting the current operation request from each of the structured encapsulated operation requests in the kernel queue includes: Controlling the user-mode daemon process to select the current operation request from each of the structured encapsulated operation requests in the kernel queue through the device file; Correspondingly, after the NVMe storage device places the target data carried by the current operation request in the target recovery unit corresponding to the target handle, it further includes: Controlling the user-mode daemon process to write the response result of the current operation request to the device file.

8. The method for placing NVMe storage device data according to claim 7, wherein The original operation request represents that the application requests to write target data to a subdirectory under the mount point directory through the standard file interface.

9. The method for placing NVMe storage device data according to claim 8, wherein After controlling the user-mode daemon process to write the response result of the current operation request to the device file, it further includes: Controlling the FUSE kernel driver to obtain the response result written by the user-mode daemon process from the device file; Returning the response result to the application through the standard file interface to complete the input / output operation of the current operation request.

10. The method for placing NVMe storage device data according to claim 7, wherein Before the NVMe storage device places the target data carried by the current operation request in the target recovery unit corresponding to the target handle, it further includes: Invoking the io_uring interface to submit the current operation request to the NVMe storage device; Correspondingly, controlling the user-mode daemon process to write the response result of the current operation request to the device file includes: Controlling the user-mode daemon process to write the response result of the current operation request returned by the NVMe storage device to the device file.

11. The method for placing NVMe storage device data according to claim 10, characterized in that, The invoking the io_uring interface to submit the current operation request to the NVMe storage device includes: Creating an io_uring circular buffer; wherein, the io_uring circular buffer includes a submission queue; Obtaining an idle request entry from the submission queue; Filling the operation code of the current operation request and the target handle identifier corresponding to the current subdirectory into the idle request entry to obtain a submission request entry; Controlling the submission queue to submit the submission request entry to the NVMe storage device.

12. The method for placing NVMe storage device data according to claim 11, wherein The io_uring circular buffer includes a completion queue; Controlling the user-mode daemon process to write the response result of the current operation request returned by the NVMe storage device to the device file includes: Using the completion queue to receive the response result of the current operation request returned by the NVMe storage device; wherein, the response result of the current operation request is a completion request entry encapsulated by the NVMe storage device using the completion status, status code, and error information of the current operation request; Controlling the user-mode daemon process to write the response result of the current operation request to the device file.

13. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, the steps of the NVMe storage device data placement method according to any one of claims 1 to 12 are implemented.

14. An electronic device, characterized in that, Including: A memory for storing a computer program; A processor for executing the computer program to implement the steps of the NVMe storage device data placement method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the NVMe storage device data placement method according to any one of claims 1 to 12 are implemented.

Images