Nvme command transparent transmission method, generation method and cross test method

Abstract

The present disclosure relates to an NVMe command transparent transmission method, belonging to the technical field of storage. The method comprises: identifying the data transmission type of a preset field domain in an NVMe command; based on the data transmission type, applying memory according to the length of user data and the offset of a virtual address in the NVMe command; performing physical address translation on the applied memory to ensure that the offset of the virtual address of the applied memory corresponds to the offset of the physical address. The transparent transmission technology in the embodiment of the present disclosure enables the application program to more flexibly control the operation of SSD data transmission, allows the application program to directly configure PRP / SGL without the need for implementation at the driver layer, and improves flexibility and efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of storage technology, and in particular to an NVMe command pass-through method, generation method, and cross-testing method. Background Technology

[0002] Solid State Drive (SSD) is a type of storage device. NVM Express (NVMe), or Non-Volatile Memory Host Controller Interface Specification (NVMHCIS), is a logical device interface specification and a high-speed transmission protocol designed specifically for SSDs.

[0003] In SSD usage and testing, data is transferred between the host and the SSD using the SSD's controller queue. The host writes Physical RegionPages (PRPs) or Scatter Gather Lists (SGLs) pointing to user data into the SSD controller's commit queue. The PRP is used for data transfer between the controller and memory, while the SGL describes the data structure of the data buffer in the memory address space. The hardware engine then performs the data transfer using Direct Memory Access (DMA). Commands sent from the host to the SSD are placed in the Submission Queue (SQ), with each command being a 64-byte data structure. Commands sent from the SSD to the host are placed in the Completion Queue (CQ), with each command being a 16-byte data structure. The Submission Queue structure contains a field specifying the user data address and whether the user address is of PRP or SGL type.

[0004] In both kernel and user space, the Application Programming Interface (API) only provides basic command parameters. PRP / SGL configuration can only be implemented at the driver layer. The PRP or SGL type is configured by the kernel or user space driver, and the application cannot configure it. This mode cannot fully verify the field fields and related functions of NVMe's PRP / SGL. Summary of the Invention

[0005] Based on this, it is necessary to address the above-mentioned technical problems by providing an NVMe command pass-through method and apparatus, an NVMe command generation method, and a PRP / SGL cross-testing method in this embodiment of the invention. This solves the problem that PRP / SGL can only be configured at the driver layer, which has limitations and cannot fully verify the field fields and related functions of NVMe's PRP / SGL.

[0006] In a first aspect, embodiments of the present invention provide an NVMe command pass-through method, the method comprising:

[0007] Identify the data transfer type in the preset field of the NVMe command;

[0008] Based on the data transfer type, memory is allocated according to the length of the user data in the NVMe command and the offset of the virtual address;

[0009] The requested memory is physical address translated to ensure that the offset of the virtual address of the requested memory corresponds to the offset of the physical address.

[0010] In some embodiments of this disclosure, the data transmission type includes PRP type and SGL type;

[0011] The process of allocating memory based on the length of the user data and the offset of the virtual address in the NVMe command includes:

[0012] In response to the data transfer type being PRP, based on the user data length of the NVMe command and the offset of the PRP virtual address, the virtual address is divided into user data, PRP virtual address, and PRP manifest virtual address; and / or

[0013] In response to the data transmission type being SGL, the virtual address is divided into user data and SGL virtual address according to the SGL address and length alignment requirements of the NVMe command;

[0014] Memory is allocated based on the virtual address partitioning results.

[0015] In some embodiments of this disclosure, the user data and data transfer type in the NVMe command are populated with virtual addresses.

[0016] In some embodiments of this disclosure, the method further includes:

[0017] During the physical address translation process for the requested memory, virtual addresses that exceed the memory size of the storage device are identified and translated into invalid physical addresses.

[0018] Secondly, embodiments of this disclosure also provide an NVMe command generation method, the method comprising:

[0019] In response to the user's NVMe command generation instruction, generate an NVMe command set containing the corresponding data structure according to the preset PRP / SGL combination algorithm;

[0020] The preset PRP / SGL combination algorithm includes at least one of the following: a PRP / SGL combination algorithm with a random generation strategy, a PRP / SGL combination algorithm with a proportional generation strategy, and a PRP / SGL combination algorithm with a user-defined configuration strategy based on pass-through technology.

[0021] In some embodiments of this disclosure, the PRP / SGL combination algorithm for the random generation strategy includes:

[0022] Based on the random seed contained in the NVMe command generation instruction, determine the type and corresponding quantity of randomly generated PRPs, and / or the type and corresponding quantity of SGLs;

[0023] If the randomly generated type is PRP, then it is randomly determined whether to use a single-page PRP, a two-page PRP, or a PRP list to construct PRP entries;

[0024] If the randomly generated type is SGL, then the length and hierarchy of the generated SGL descriptor chain are randomly determined, and SGL entries and hierarchical nesting structures are constructed.

[0025] In some embodiments of this disclosure, the PRP / SGL combination algorithm for the scaling generation strategy includes:

[0026] Based on the preset ratio of PRP and SGL types in the NVMe command generation instructions, PRP and SGL type data, nested formats of PRP and SGL types, and N-level nesting of SGL types are generated proportionally.

[0027] In some embodiments of this disclosure, the PRP / SGL combination algorithm based on user-defined configuration strategies using pass-through technology includes:

[0028] Retrieve user-defined configurations for PRP or SGL fields in commands;

[0029] The custom configuration is passed through to the user-space driver for execution, generating NVMe commands of type PRP and type SGL.

[0030] Thirdly, embodiments of this disclosure also provide a PRP / SGL cross-testing method, including:

[0031] Obtain the NVMe command set generated by the above NVMe command generation method, wherein the NVMe command set includes NVMe commands of type PRP and NVMe commands of type SGL;

[0032] Mix and insert PRP-type NVMe commands and SGL-type NVMe commands in the submission queue;

[0033] Perform I / O tests on the submission queue.

[0034] Fourthly, embodiments of this disclosure also provide a storage device, including: a memory and a processor, the processor including multiple cores, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the NVMe command pass-through method, or the NVMe command generation method, or the PRP / SGL cross-testing method described above.

[0035] This disclosure provides an NVMe command pass-through method. The technical solution provided by the embodiments of this disclosure brings at least the following beneficial effects: the application uses this interface to request the virtual address of memory to assemble the PRP / SGL configuration in the NVMe command, and then performs physical address translation through the user-space NVMe driver to request the NVMe request, thereby realizing the function of directly passing through the NVMe command. This pass-through technology enables the application to control the operation of SSD data transfer more flexibly, allowing the application to directly configure PRP / SGL without implementation at the driver layer, thus improving flexibility and efficiency.

[0036] Furthermore, an NVMe command generation method is provided. The technical solution provided by the embodiments of this disclosure brings at least the following beneficial effects: NVMe commands generated by the PRP / SGL combination algorithm with a random generation strategy have a wide test coverage and can simulate a large number of boundary or abnormal scenarios; NVMe commands generated by the PRP / SGL combination algorithm with a proportional generation strategy can accurately define parameters such as the PRP / SGL structure ratio and nesting depth; they are easy to reproduce and verify specific design requirements, and are suitable for protocol consistency and stability verification; NVMe commands generated by the PRP / SGL combination algorithm with a user-defined configuration strategy based on pass-through technology are pass-through based on the complete NVMe command structure defined by the user in the APP. The user has complete control over the PRP / SGL structure, which is convenient for constructing illegal boundaries or extreme nesting, and has maximum flexibility. The application layer has complete control over the structure content and can construct arbitrarily complex or illegal structures for accurate testing. It is suitable for senior test engineers or developers and can also be used for scenarios such as controller crash testing and protocol consistency testing; Based on this, NVMe commands generated based on the preset PRP / SGL combination algorithm can verify the processing capability of SSD in mixed SGL and PRP scenarios.

[0037] Furthermore, a PRP / SGL cross-testing method is provided. The technical solution provided by the embodiments of this disclosure brings at least the following beneficial effects: the application can more flexibly control the operation of the SSD, combine various NVMe commands, verify PRP or SGL error injection, and complete functional verification tests.

[0038] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This diagram illustrates the kernel-mode and user-mode NVMe command delivery process and the PRP / SGL assembly location.

[0041] Figure 2 A flowchart illustrating an NVMe command pass-through method provided in an embodiment of this disclosure;

[0042] Figure 3 This is a schematic diagram illustrating the specific implementation process of an NVMe command pass-through method provided in an embodiment of this disclosure;

[0043] Figure 4 This is a schematic diagram of the module structure of NVMe command pass-through technology in a user-space NVMe driver provided in an embodiment of this disclosure;

[0044] Figure 5 A flowchart illustrating an NVMe command generation method provided in this embodiment of the disclosure;

[0045] Figure 6 A schematic diagram illustrating the filling of the submission queue for PRP-type NVMe commands in this embodiment of the disclosure;

[0046] Figure 7 A schematic diagram illustrating how an SGL-type NVMe command fills the submission queue in an embodiment of this disclosure;

[0047] Figure 8 A schematic diagram illustrating the filling of the submission queue for PRP and SGL type NVMe commands in this embodiment of the disclosure;

[0048] Figure 9 This is a schematic diagram illustrating the nested continuous physical pages and non-contiguous physical pages in the nested format of PRP in the embodiments of this disclosure;

[0049] Figure 10 This is a schematic diagram of the N-level nesting of SGL Segments in the nested format of SGL in the embodiments of this disclosure;

[0050] Figure 11 A schematic diagram of a PRP / SGL cross-test provided for an embodiment of this disclosure;

[0051] Figure 12 A schematic diagram of a PRP / SGL hybrid cross-test supporting multiple strategies is provided for embodiments of this disclosure;

[0052] Figure 13 This is a schematic diagram illustrating the "Invalid Address For PRP List Or Entry" invalid verification in an embodiment of this disclosure;

[0053] Figure 14 This is a schematic diagram illustrating the verification of illegal values ​​for SGL's address, offset, and Type in an embodiment of this disclosure;

[0054] Figure 15 This is a schematic diagram illustrating the simultaneous participation of different queues and commands in mixed crossover in an embodiment of this disclosure;

[0055] Figure 16 This is a schematic diagram of the structure of a storage device provided in an embodiment of this disclosure. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this disclosure.

[0057] In the relevant embodiments disclosed herein, the API only provides basic command parameters in kernel mode and user mode. PRP / SGL configuration can only be implemented at the driver layer. The PRP or SGL type is configured by the kernel or user mode driver; the application cannot configure it. This mode cannot fully verify the field domains and related functions of NVMe's PRP / SGL. Both PRP and SGL describe a data space in memory. PRP points to a physical memory page and serves as a distribution / collection mechanism for data transfer between the controller and memory. To achieve efficient out-of-order data transfer between the controller and the host, PRP entries are of fixed size. SGL is a data structure in the memory address space used to describe a data buffer, which can be a source buffer or a destination buffer. PRP can only map to individual physical pages, corresponding to a fixed-length physical space, while SGL can map to a contiguous physical space of arbitrary size.

[0058] SGLs are linked lists, with one SGL containing one or more SGL segments. An SGL segment is a Qword-aligned data structure within a contiguous region of physical memory, describing all, part, or all of the data buffer and the next SGL segment (if any). The SGL descriptor is the most basic unit of an SGL, used to describe a contiguous block of physical memory: starting address + size. The SGL segment descriptor describes the next SGL segment, not the last. A special SGL segment descriptor corresponds to the last SGL segment, namely the SGL end segment descriptor.

[0059] Based on the aforementioned issues, the method provided in this disclosure allows the application to use this interface to request virtual addresses of memory to assemble the PRP / SGL configuration in NVMe commands. Then, the application performs physical address translation through the user-space NVMe driver to request NVMe requests, thereby realizing the function of directly passing through NVMe commands. This pass-through technology enables the application to control SSD data transfer operations more flexibly, allowing the application to directly configure PRP / SGL without implementation at the driver layer, thus improving flexibility and efficiency.

[0060] In the relevant embodiments of this disclosure, Figure 1 This diagram illustrates the kernel-mode and user-mode NVMe command delivery process and the PRP / SGL assembly location, as shown below. Figure 1 As shown, the user-space NVMe driver provides more flexibility and convenience to the application layer. This disclosure implements a method in user space for user applications to customize and issue PRP / SGL type NVMe commands, thereby enabling PRP / SGL cross-testing, during SSD usage and testing.

[0061] Specifically, the APP is used to provide command opcodes and other parameters, memory alloc, and ioctrl (input / output control).

[0062] like Figure 1 As shown, the NVMe command issuance process in kernel mode includes:

[0063] The app provides the command opcode and other parameters to memory alloc, then to ioctrl, and finally outputs the NVMe command.

[0064] NVMe commands are sent to the block device in the file system via the system call interface (syscalls{});

[0065] In the kernel-mode NVMe driver, the NVM command is assembled, followed by an NVMe Request; the PRP / SGL assembly is performed within the NVMe Request.

[0066] The NVMe command assembled by the NVMe Request is sent to the PCIe bus and finally to the NVM SSD.

[0067] like Figure 1 As shown, the NVMe command issuance process in user space includes:

[0068] The app provides the command opcode and other parameters to the memory alloc, and finally outputs the NVMe command.

[0069] NVMe commands are sent to the user-space NVMe driver via the user-space API interface;

[0070] In the user-space NVMe Driver, the NVM command is assembled, followed by an NVMe Request; the PRP / SGL assembly location is within the NVMe Request.

[0071] The NVMe command assembled by the NVMe Request is sent to the PCIe bus and finally to the NVMe SSD.

[0072] Figure 2 This is a flowchart illustrating an NVMe command pass-through method provided in an embodiment of this disclosure, as shown below. Figure 2 As shown, the NVMe command pass-through technology applicable to user-space NVMe drivers includes the following steps:

[0073] In S210, the data transfer type of the preset field in the NVMe command is identified;

[0074] In S220, memory is allocated based on the data transfer type, the length of the user data in the NVMe command, and the offset of the virtual address.

[0075] In S230, the physical address of the requested memory is translated to ensure that the offset of the virtual address of the requested memory corresponds to the offset of the physical address.

[0076] Based on the above, this disclosure provides a specific implementation process for an NVMe command pass-through method, such as... Figure 3 As shown, the specific steps include:

[0077] In the S310, NVMe commands are obtained through the API interface; the API interface provides a dedicated DMA memory alloc interface.

[0078] In S320, it is determined whether the PSDT type of the NVMe command is of type PRP. If it is, the virtual address is divided into user data, PRP virtual address and PRPList virtual address according to the user data length and PRP virtual address offset of the NVMe command. If not, the virtual address is divided into user data and SGL virtual address according to the SGL address and length alignment requirements of the NVMe command.

[0079] In S330, while ensuring that the offset of the virtual address and the offset of the physical address are consistent, the virtual address of the data structure corresponding to PRP / SGL in the NVMe command is translated into the physical address;

[0080] In S340, an NVMe Request is made, NVMe commands are assembled into it, sent to the submission queue and written to the DoorBell, and data is read from the completion queue and written to the DoorBell.

[0081] As can be seen, the API interface provides a dedicated DMA memory alloc interface and obtains NVMe commands through the API interface. This allows applications to directly configure PRP / SGL, and the user-space NVMe driver performs physical address translation and sends requests. That is, the application uses this interface to request the virtual address of memory to assemble the PRP / SGL configuration in the NVMe command, and then the user-space NVMe driver performs physical address translation to request NVMe, realizing the function of directly passing through NVMe commands. This pass-through technology enables applications to control SSD data transfer operations more flexibly, allowing applications to directly configure PRP / SGL without implementation at the driver layer, improving flexibility and efficiency.

[0082] Figure 4 This is a schematic diagram of the module structure of an NVMe command pass-through technology in a user-space NVMe driver provided in an embodiment of this disclosure, as shown below. Figure 4 As shown, it includes the following modules:

[0083] The M10 module provides various API interfaces for apps to use.

[0084] The M11 module is a virtual address allocation and translation module. Its function is to allocate PRP / SGL virtual addresses and map them to physical addresses, providing valid physical addresses for DMA. The specific implementation is as follows: Based on the "PSDT" field of the NVMe command provided by the APP, it determines whether it is PRP or SGL. If it is PRP, it verifies which virtual addresses are user data, which are PRP / SGL virtual addresses, and which are PRP List virtual addresses by using the user data length and the PRP virtual address offset. Then, it performs address translation and conversion, ensuring that the virtual address offset and the physical address offset correspond consistently.

[0085] The M12 module handles Request requests and NVMe assembly commands. This module maintains the commit / completion queues and how the Head / Tail functions.

[0086] The M13 module puts the NVMe commands of the Request into the commit queue and writes them to the Doorbell, and reads data from the completion queue and writes it to the Doorbell as well.

[0087] In some specific embodiments of this disclosure, when the APP uses an API interface, such as requesting memory (user data memory request, PRP List memory request) through the M111 interface, the requested memory address is assembled into the NVMe command data structure according to the PRP / SGL format requirements in the NVMe protocol. In the NVMe command, both PRP data and user data are filled with virtual addresses (i.e., the user data and user data type (PRP) in the NVMe command data structure are allocated virtual addresses, and the application fills the virtual addresses into the command structure); and the page alignment of the virtual address must meet the NVMe protocol PRP offset requirements (e.g., Entry1 must be DWord aligned; Entry2 must be page aligned when it is user data, and QWord aligned when it is an Entry list); similarly, when requesting SGL type, it must meet the SGL address and length alignment requirements. Then, when translating the virtual address of the data structure corresponding to PRP / SGL in the NVMe command into a physical address through M112, the offset of the virtual address and the offset of the physical address must be consistent. An NVMe Request is initiated via M121, and M122 assembles the NVMe command data into it before sending it to the send queue. The M13 module handles the queue's transactions. The application (APP) fills the virtual address into the command structure, including the user data and its transmission type, and allocates the virtual address.

[0088] For verification of illegal PRP / SGL cases, the APP allocates memory according to page alignment in the M111 stage, constructs the desired PRP / SGL offset according to the offset test requirements, and ensures that the physical address is also the PRP / SGL offset required for the test by translating the offset from the address in the M112 stage. This ensures that the physical address is also the PRP / SGL offset required for the test, and allows for more flexible verification.

[0089] Figure 5 This is a flowchart illustrating an NVMe command generation method provided in an embodiment of this disclosure, as shown below. Figure 5 As shown, it includes the following steps:

[0090] In S510, in response to the user's NVMe command generation instruction, an NVMe command set containing corresponding data structures is generated according to a preset PRP / SGL combination algorithm. The preset PRP / SGL combination algorithm includes at least one of the following: a PRP / SGL combination algorithm with a random generation strategy, a PRP / SGL combination algorithm with a proportional generation strategy, and a PRP / SGL combination algorithm with a user-defined configuration strategy based on pass-through technology.

[0091] In some specific embodiments of this disclosure, the API interface is used to obtain whether the data transfer type customized by the APP is PRP or SGL, and NVMe command data structures are assembled based on NVMe command pass-through technology. NVMe commands are then generated, which are then accepted and processed by the SSD.

[0092] In some specific embodiments of this disclosure, the above-mentioned NVMe command generation instructions include the number of commands, the maximum / minimum number of segments, buffer alignment options, and a random seed.

[0093] The PRP / SGL combination algorithm for the above random generation strategy includes:

[0094] Based on the number of commands, the maximum / minimum number of segments, buffer alignment options, and a random seed, a PSDT type and corresponding data structure are randomly determined for each command to generate a first NVMe command set. The first NVMe command set includes PRP type data, or SGL type data, or both PRP type and SGL type data.

[0095] The PRP / SGL combination algorithm of the random generation strategy is the random generation mode of NVMe commands. In this mode, the algorithm takes a random seed as input and randomly generates the contents of the PRP or SGL fields in several NVMe commands.

[0096] The specific process is as follows:

[0097] Input parameters include: number of commands, random seed, maximum / minimum number of segments, and buffer alignment options;

[0098] The generation logic includes:

[0099] For each command, a PSDT type is randomly selected, such as PRP or SGL.

[0100] If you select a PRP type, it will be randomly determined whether to use a single-page PRP, a two-page PRP, or a PRP List.

[0101] If the SGL type is selected, the length and hierarchy of the descriptor chain (Segment + Data Block) will be randomly generated.

[0102] The output includes:

[0103] The complete PRP / SGL structure for each command, including buffer virtual address, length, alignment parameters, etc.

[0104] Automatically construct PRP Entry, SGL Entry, and nested structures;

[0105] Assembly includes: translating the data via the M11 module, writing it into an NVMe command, and submitting it to the submission queue.

[0106] By using a random seed, the number of PRP and SGL types for N commands is randomly generated and randomly sent to the I / O submission queue to form the first NVMe command set, ensuring that different I / O queues have different numbers and types of SGL and PRP.

[0107] Figure 6 A schematic diagram illustrating how to fill the submission queue with PRP-type NVMe commands in this embodiment of the present disclosure. Figure 7 A schematic diagram illustrating how to fill the submission queue for SGL-type NVMe commands in this embodiment of the present disclosure. Figure 8 This is a schematic diagram illustrating how PRP and SGL type NVMe commands fill the submission queue in this embodiment of the disclosure, as shown below. Figures 6-8 As shown, an NVMe queue can contain only PRP type commands, only SGL type commands, or both PRP and SGL type commands.

[0108] In summary, the PRP / SGL combined algorithm with a random generation strategy has significant advantages, including broad test coverage for NVMe commands generated by the algorithm, capable of simulating numerous boundary or abnormal scenarios; easy triggering of potential controller bugs; and suitability for large-scale automated testing. However, it should be noted that this random generation strategy is highly uncontrollable, making it difficult to reproduce problems and hard to target and verify specific structures or errors.

[0109] In some specific embodiments of this disclosure, the NVMe command generation instructions mentioned above include a random seed, nesting level, alignment parameters, and PRP / SGL percentage;

[0110] The PRP / SGL combined algorithm for the above-mentioned scaling generation strategy includes:

[0111] A second NVMe command set is generated proportionally based on the random seed, nesting level, alignment parameters, and PRP / SGL percentage. The second NVMe command set includes different PRP and SGL type data, nesting formats of PRP and SGL types, and nesting of SGL Segment N levels.

[0112] The PRP / SGL combination algorithm for the proportional generation strategy is the proportional generation mode of NVMe commands. This mode allows users to precisely define the proportional relationship between PRP and SGL in all commands and set advanced rules such as nesting levels and alignment parameters.

[0113] The specific process is as follows:

[0114] Input parameters include:

[0115] PRP / SGL command ratio, such as PRP: 60%, SGL: 40%; PRP List usage ratio; SGL Segment nesting level; Entry quantity range; Alignment requirements, such as page alignment, QWord alignment, 8B alignment, etc.

[0116] The generation logic includes:

[0117] Iterate through the command set, selecting types sequentially or randomly based on proportions;

[0118] For each type, use parameter-driven templates to populate the struct;

[0119] Supported nested descriptor generation logic includes:

[0120] SGL segment → Multiple SGL Data Blocks;

[0121] Segment can be nested within Segment or Last Segment;

[0122] The output includes:

[0123] A valid set of commands, i.e., the second NVMe command set, that satisfies the scaling and boundary conditions.

[0124] Figure 9 This is a schematic diagram illustrating the nested continuous and non-contiguous physical pages in the nested format of PRP in this embodiment of the present disclosure. Figure 10 This is a schematic diagram of the N-level nesting of SGL Segments in the nested format of SGL in the embodiments of this disclosure, as shown below. Figures 9-10 As shown, different PRP and SGL data types are generated proportionally based on various parameters, such as the percentage of PRP / SGL, the random seed value, the percentage of different types of PRP / SGL, and whether nesting is used. This includes nesting formats for PRP and SGL, as well as N-level nesting of SGLSegments, where N = thousands / ten thousand / hundred thousand / millions. In N-level nesting of SGL segments, this million-level nesting is a relatively extreme nesting level for the controller.

[0125] In summary, the proportional generation strategy of the PRP / SGL combination algorithm has significant advantages, including the ability to precisely define parameters such as the PRP / SGL structure ratio and nesting depth; ease of reproduction and verification of specific design requirements; and suitability for protocol consistency and stability verification. However, this proportional generation strategy has complex parameter settings, requires an understanding of underlying alignment and protocol rules, and is not suitable for testing abnormal structures.

[0126] In some specific embodiments of this disclosure, the above-mentioned NVMe command generation instructions include a complete NVMe command structure configured by the user;

[0127] The aforementioned user-defined PRP / SGL combination algorithm based on pass-through technology includes:

[0128] A third NVMe command set is generated based on the complete NVMe command structure configured by the user; the third NVMe command set is used to directly pass through to the commit queue.

[0129] The PRP / SGL combination algorithm, which allows users to customize configuration strategies based on pass-through technology, is the user pass-through mode for NVMe commands. In this mode, users can directly construct the PRP or SGL fields in the command and pass them through to the user-space driver for execution.

[0130] The specific process is as follows:

[0131] The input content includes:

[0132] The complete NVMe command structure defined in the user's APP; where the PRP entry or SGL entry is a virtual address applied for by the user.

[0133] The processing logic includes:

[0134] The system is only responsible for verifying the basic legality of the structure, such as whether it meets the alignment requirements;

[0135] The virtual address request and translation module M11 translates the user-provided virtual address into a physical address;

[0136] The mapping result is written to the NVMe command field, i.e., the third NVMe command set, such as prp1 / prp2 or sgl entry;

[0137] In summary, the PRP / SGL combination algorithm with a transparent custom configuration strategy has significant advantages. It is based on the complete NVMe command structure defined by the user in the application, allowing the user complete control over the PRP / SGL structure. This facilitates the construction of illegal boundaries or extreme nesting, thus offering the greatest flexibility compared to the aforementioned random generation and proportional generation strategies. The application layer has complete control over the structure content, enabling the construction of arbitrarily complex or illegal structures for precise testing, making it suitable for advanced test engineers or developers. It can also be used for scenarios such as controller crash testing and protocol consistency testing. However, this transparent custom configuration strategy requires a high level of user skill, may lead to misconfiguration, and is prone to system anomalies or crashes due to incorrect construction.

[0138] Based on the PRP / SGL combination algorithm that supports multiple strategies in the NVMe command generation process, the aim is to flexibly, efficiently, and systematically generate the PRP / SGL structure in NVMe commands to achieve protocol compliance testing, abnormal scenario simulation, and mixed cross-scenario verification.

[0139] Figure 11 This is a schematic diagram of a PRP / SGL cross-test provided in an embodiment of the present disclosure, as shown below. Figure 11 As shown, it includes the following steps:

[0140] In S1110, the NVMe command set generated by the above NVMe command generation method is obtained, wherein the NVMe command set contains NVMe commands of type PRP and NVMe commands of type SGL;

[0141] In S1120, PRP type NVMe commands and SGL type NVMe commands are mixed and inserted into the submission queue;

[0142] In S1130, I / O tests are performed on the submission queue.

[0143] In some specific embodiments of this disclosure, the test scenario of using a hybrid cross-injection mode by combining the three generation modes of the PRP / SGL combination algorithm to generate a hybrid cross-injection can be specifically set according to the user's actual needs. PRP type and SGL type commands are mixed and inserted into a submission queue; then different structures are selected according to the command type, for example, SGL type is used for read commands and PRP type is used for write commands; the command structure is periodically switched for the same queue to enhance the coverage and dynamism of the test.

[0144] It should be noted that the above three modes can be used in a complementary manner to maximize benefits. For example, using the PRP / SGL random generation mode, long-term verification based on a random seed can cover a wide range of situations. When a problem is reproduced, it can be determined which PRP / SGL model caused the bug. Then, using the PRP / SGL pass-through custom configuration mode, a PRP / SGL model that reproduces the bug can be directly built at the user level, directly hitting the reproduction. After the bug is fixed, its success can be verified. Next, using the PRP / SGL proportional generation mode, a certain proportion of PRP / SGL models can be generated using parameters and the random seed at the time of the problem's reproduction to verify the success of the fix over a long period. Finally, the PRP / SGL random generation mode and random seed can be used again for long-term verification. Based on this, bugs can be quickly verified and reproduced during the verification phase, the bug reproduction phase, and after bug fixing, greatly shortening the bug verification cycle and improving efficiency. It should be noted that this disclosure also has other specific use cases, which are not all listed here.

[0145] Figure 12 This disclosure provides a flowchart illustrating a PRP / SGL hybrid cross-testing approach supporting multiple strategies, as shown in the embodiments below. Figure 12 As shown, it includes the following steps:

[0146] In S1210, the data provided by the APP varies depending on the policy; in S1201, the APP provides the necessary NVMe Command data and random seed; in S1202, the APP provides the necessary NVMe Command data and PRP / SGL related parameters; in S1203, the APP provides the configured NVMe Command data, which includes PRP / SGL data.

[0147] In S1220, through the PRP / SGL API interface, different PRP / SGL combination algorithm modes are applied according to the different data provided by the APP, ultimately forming a PRP / SGL hybrid cross mode; in S1211, it is the PRP / SGL random generation mode; in S1212, it is the PRP / SGL proportional generation mode; in S1213, it is the PRP / SGL user pass-through mode; and in S1214, it is the final converged PRP / SGL hybrid cross mode (Crossed Injection).

[0148] In S1230, the test is completed in the user-space NVMe Driver based on the NVMe commands output in the PRP / SGL hybrid cross-connect mode.

[0149] It should be noted that, for the PRP / SGL random generation mode, in S1201, the APP provides the necessary data and random seed for the NVMe command; the PRP / SGL generator in S1211 generates different PRP or SGL type NVMe commands, and then S1214 performs PRP and SGL mixing and cross-processing; for example, read commands use SGL, write commands use PRP, Dataset Management commands use SGL, etc. At the same time, each commit queue is cross-stuffed with PRP type commands and SGL type commands, or the same commit queue is stuffed with commands of the same PRP or SGL type; finally, S1230 processes the commit queue / completion queue.

[0150] In the PRP / SGL random generation mode, SGL layout verification or PRP verification can also be performed. Since PRP verification is the standard procedure, the following explanation will focus on SGL layout verification.

[0151] By passing different parameters to the PRP / SGL generator through the S1211 interface, the generator can generate the SGL type required for the test according to the parameters, and thus complete the following functional verification:

[0152] (1) The field "DPTR" of the NVMe command is an SGL entry, and its entry type is an SGL Segment descriptor, which contains multiple SGL Data BLock descriptors; the scenario of verifying whether the last descriptor of the SGL Segment descriptor is an SGL Segment descriptor or an SGL Last Segment descriptor.

[0153] (2) Verify scenarios where the SGL Last Segment descriptor contains an SGL Segment descriptor or an SGL LastSegment descriptor;

[0154] (3) Verify the SGL Last Segment descriptor, which contains multiple SGL Data BLock descriptors and scenarios containing SGL Last Segment descriptors or SGL Segment descriptors.

[0155] For PRP / SGL proportional generation mode, PRP layout verification can also be performed. For example, when PRP lists are nested, some PRP list entries are non-contiguous physical pages, while others are contiguous physical pages. This verifies the controller's ability to handle the simultaneous existence of contiguous and non-contiguous physical pages.

[0156] In S1202, the APP provides the necessary data for NVMe commands and PRP / SGL related parameters, such as the ratio of PRP and SGL quantities, the number of Entries in the PRP List, the number of PRP Lists, and the offset alignment parameters of Entries; or, the type and number of SGL descriptors, the address alignment parameters of SGL descriptors, and the length alignment parameters, etc. S1212 generates the corresponding PRP or SGL type according to the ratio of PRP and SGL, including the nesting format of PRP and SGL, and N-level nesting, etc.; then S1214 performs PRP and SGL mixing and cross-validation to complete the relevant verification.

[0157] For example, PRP / SGL ratio generation is used to verify the controller's ability to handle PRP or SGL mixed processing. Using different parameters via the S1202 interface, the S1212 generates PRP and SGL in a ratio based on these parameters. Different commands are then selected to generate a specific ratio of IO commands and a specific ratio of PRP / SGL, thereby verifying the controller's ability to handle PRP or SGL mixed processing.

[0158] Specifically, a commit queue and a completion queue of size 1024 are created for I / O testing. For verification, an I / O depth of 256 is used as an example, ensuring that the I / O depth is at least 2. 256 commands are placed in the commit queue, each command having a PSDT type of PRP / SGL. Using the interface in S1212, PRP / SGL is generated proportionally based on parameters provided by the application. The command then selects the appropriate PRP / SGL and sends it to the commit queue.

[0159] For PRP / SGL user passthrough mode, in S1203, the APP uses NVMe command passthrough technology, and S1214 performs PRP and SGL hybrid cross-transmission, directly passing through user-defined PRP / SGL type commands and putting them into the submission queue. S1230 processes the submission queue / completion queue to complete the relevant verification.

[0160] For example, verifying an invalid PRP address: Using NVMe command pass-through technology, a user data buffer is requested in the APP, and the virtual address offset is aligned according to the parameters. A complete NVMe command, including the PRP part, is assembled in the APP and the command is issued using NVMe command pass-through technology. In the NVMe command pass-through technology, it is determined whether the PRP is a virtual address or a physical address. If it is a virtual address, it is translated, and the offset of the address page is aligned with the virtual address. This achieves the purpose of verifying the PRP offset field.

[0161] For example, Figure 13 This is a schematic diagram illustrating the invalid "Invalid Address For PRP List Or Entry" verification in an embodiment of this disclosure. Figure 14 This is a schematic diagram illustrating the verification of invalid values ​​for SGL's address, offset, and Type in an embodiment of this disclosure, as shown below. Figure 13 and Figure 14 As shown, the verification of an invalid PRP list is performed using NVMe command pass-through technology. The PRP or SGL data structure to be verified is assembled at the APP layer and then sent to the submission queue after address translation. The following verification can be completed:

[0162] (1) The PRP Entry2 of the DPTR field of the NVMe command is a PRP List that verifies whether the offset of its address is QWord aligned;

[0163] (2) The PRP Entry2 of the DPTR field of the NVMe command is a PRP List. The last entry in the PRP List is the address of the next PRP Entry List. Its PRP Entry address is memory page aligned. Verify whether it is page aligned.

[0164] (3) Invalid address, i.e., physical address exceeding the memory size, such as DMA Error. Based on the memory size, PRPEntry is translated into an invalid address. During the virtual address and physical address translation, the system will automatically determine which virtual addresses are translated into invalid physical addresses;

[0165] (4) Verification of SGL address, offset and length: Using NVMe command pass-through method, the APP selects SGL / PRP, and after translation, it is directly put into the submission queue to verify the invalidity of PRP or SGL field fields, etc.

[0166] For PRP / SGL hybrid crossover mode, it can be used to verify the controller's ability to handle PRP or SGL hybrid processing.

[0167] For example, through the S1201 interface, based on different parameters such as a random seed, S1211 randomly generates different PRP or SGL types, and then assembles them into the NVMe command according to whether the PSDT of the command is PRP or SGL. After S1214 performs PRP and SGL mixing and cross-processing, it is put into the submission queue, which can complete the following verification:

[0168] Create a commit queue and a completion queue with a size of 1024 to perform I / O testing. During verification, taking an I / O depth of 256 as an example, ensure that the I / O depth is at least greater than or equal to 2. Place 256 commands in the commit queue, with each command's PSDT being PRP / SGL. Use the S1214 interface to randomly mix PRP / SGL crosses (crosses for different commands) according to the parameters provided by the APP to generate the PSDT.

[0169] For example, Figure 15 This is a schematic diagram illustrating the simultaneous participation of different queues and commands in mixed crossover in an embodiment of this disclosure, such as... Figure 15 As shown, by using different parameters through the S1202 interface, S1212 generates PRP and SGL in proportion according to the parameters, and S1214 performs a cross-mixing of PRP and SGL, the following verification can be completed:

[0170] Multiple pairs of submit queue / complete queue IO tests are controlled by APP parameters. The S1212 interface generates a certain proportion of PRP / SGL for different commands. For example, read commands use SGL, write commands use PRP, Dataset Management commands use SGL, etc., and S1214 performs mixed cross-processing of PRP and SGL.

[0171] It should be noted that the above involves nested SGL segment descriptors and SGL segment descriptor verification. An SGL segment descriptor contains only one SGL DATA BLOCK, and the last SGL LastSegment descriptor contains one SGL DATA BLOCK.

[0172] In summary, the NVMe command pass-through method, NVMe command generation method, and PRP / SGL cross-testing method proposed in this disclosure can be seen as a process from pass-through technology to command generation and then to testing. Specifically, this disclosure proposes a method for cross-testing by controlling the issuance of PRP / SGL type NVMe commands by a user-space program. Its application can more flexibly control the operation of SSD, combine various NVMe commands, verify PRP or SGL error injection, and complete functional verification tests. Based on the PRP / SGL combination algorithm, the processing capability of SSD in mixed SGL and PRP scenarios can be verified.

[0173] It should be understood that, although Figure 2-4 and Figure 11 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 -and Figure 11 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0174] It is understood that the same / similar parts between the various embodiments of the methods described above in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the description of other method embodiments.

[0175] This disclosure also provides a storage device, which includes a processor and a memory. The processor contains multiple cores, and the memory stores a computer program. When the computer program is executed by the processor, it can implement the methods of any of the above embodiments. The execution method and beneficial effects are similar and will not be described again here.

[0176] Figure 16 This is a schematic diagram of the structure of a storage device provided in an embodiment of this disclosure, such as... Figure 16 As shown, the storage device may include a processor 1610 and a memory 1620. The memory 1620 stores a computer program. When the computer program is executed by the processor 1610, it can implement the method provided in any of the above embodiments. The execution method and beneficial effects are similar and will not be described again here.

[0177] Of course, for the sake of simplicity, Figure 16 Only some of the components of the storage device relevant to the present invention are shown, omitting components such as buses, input / output interfaces, input devices, and output devices. In addition, the storage device may include any other suitable components depending on the specific application.

[0178] This disclosure provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the methods of any of the above embodiments. The execution method and beneficial effects are similar, and will not be described again here.

[0179] The aforementioned computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0180] The computer program described above can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer device, partially on the user's device, as a standalone software package, partially on the user's computer device and partially on a remote computer device, or entirely on a remote computer device or server.

[0181] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0182] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An NVMe command pass-through method, characterized in that, The method includes: Identify the data transfer type in the preset field of the NVMe command; Based on the data transfer type, memory is allocated according to the length of the user data in the NVMe command and the offset of the virtual address; the data transfer type includes PRP type and SGL type; the allocation of memory according to the length of the user data in the NVMe command and the offset of the virtual address includes: in response to the data transfer type being PRP type, dividing the virtual address into user data, PRP virtual address, and PRP manifest virtual address according to the length of the user data in the NVMe command and the offset of the PRP virtual address; and / or in response to the data transfer type being SGL type, dividing the virtual address into user data and SGL virtual address according to the SGL address and length alignment requirements of the NVMe command; and allocating memory according to the virtual address allocation result; Perform physical address translation on the requested memory to ensure that the offset of the virtual address of the requested memory corresponds to the offset of the physical address. The NVMe command is sent to the user-space NVMe driver via the user-space API interface; in the user-space NVMe Driver, the NVM command is assembled, and then an NVMe Request is made; the PRP / SGL assembly location is in the NVMe Request.

2. The method according to claim 1, characterized in that, In the NVMe command, user data and data transfer type are filled with virtual addresses.

3. The method according to claim 1, characterized in that, The method further includes: During the physical address translation process for the requested memory, virtual addresses that exceed the memory size of the storage device are identified and translated into invalid physical addresses.

4. A method for generating NVMe commands, characterized in that, The method includes: In response to the user's NVMe command generation instruction, generate an NVMe command set containing the corresponding data structure according to the preset PRP / SGL combination algorithm; The preset PRP / SGL combination algorithm includes at least one of the following: a PRP / SGL combination algorithm with user-defined configuration strategy based on pass-through technology, a PRP / SGL combination algorithm with random generation strategy, and a PRP / SGL combination algorithm with proportional generation strategy. The PRP / SGL combination algorithm based on the user-defined configuration strategy using pass-through technology is the user pass-through mode of NVMe commands. In this mode, users can directly construct the PRP or SGL fields in the command and pass them through to the user-space driver for execution. The pass-through mode includes: inputting a complete NVMe command structure defined in the user's APP; wherein, the PRP entry or SGL entry is a virtual address applied for by the user; verifying the basic legality of the structure; translating the virtual address given by the user into a physical address; and writing the mapping result into the NVMe command field.

5. The method according to claim 4, characterized in that, The PRP / SGL combination algorithm for the random generation strategy includes: Based on the random seed contained in the NVMe command generation instruction, determine the type and corresponding quantity of randomly generated PRPs, and / or the type and corresponding quantity of SGLs; If the randomly generated type is PRP, then it is randomly determined whether to use a single-page PRP, a two-page PRP, or a PRP list to construct PRP entries; If the randomly generated type is SGL, then the length and hierarchy of the generated SGL descriptor chain are randomly determined, and SGL entries and hierarchical nesting structures are constructed.

6. The method according to claim 4, characterized in that, The PRP / SGL combination algorithm for the scaling generation strategy includes: Based on the preset ratio of PRP and SGL types in the NVMe command generation instructions, PRP and SGL type data, nested formats of PRP and SGL types, and N-level nesting of SGL types are generated proportionally.

7. The method according to claim 4, characterized in that, The PRP / SGL combination algorithm for user-customized configuration strategies based on pass-through technology includes: obtaining the user's custom configuration of the PRP or SGL fields in the command; The custom configuration is passed through to the user-space driver for execution, generating NVMe commands of type PRP and type SGL.

8. A PRP / SGL cross-testing method, characterized in that, include: Obtain the NVMe command set generated by the NVMe command generation method according to any one of claims 4-7, wherein the NVMe command set includes NVMe commands of type PRP and NVMe commands of type SGL; Mix and insert PRP-type NVMe commands and SGL-type NVMe commands in the submission queue; Perform I / O tests on the submission queue.

9. A storage device, characterized in that, include: The system includes a memory and a processor, wherein the processor contains multiple cores, and the memory stores a computer program that, when executed by the processor, implements the NVMe command pass-through method as described in any one of claims 1-3, or the NVMe command generation method as described in any one of claims 4-7, or the PRP / SGL cross-testing method as described in claim 8.

Images