NVMe queue resource dynamic allocation method and register based on SR-IOV protocol

Abstract

The present application provides a method and register for dynamically allocating NVMe queue resources based on the SR‑IOV protocol, the method comprising: receiving queue resource demand data of a target PCIe function sent by a host based on the NVMe protocol and the SR‑IOV protocol; if the target PCIe function meets the preset queue resource allocation conditions according to the function list corresponding to the current CPU core resources, then based on the function list and the queue resource demand data of the target PCIe function, dynamically allocate queue resources for the target PCIe function in the local target NVMe shared IO queue. The present application can realize the dynamic allocation of queue resources for PCIe functions through hardware, can effectively improve the utilization rate of IO queue resources, and can effectively solve the problem of insufficient or wasted IO queue resources in a certain PCIe function.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and register for dynamically allocating NVMe queue resources based on the SR-IOV protocol. Background Art

[0002] SR-IOV (Single Root I / O Virtualization) is an extension of the PCIe (peripheral component interconnect express) protocol, and is a method for implementing hardware virtualization. The SR-IOV protocol allows multiple high-speed serial computer expansion bus standard PCIe functions, including physical functions PF (Physical Function) and virtual functions VF (Virtual Function) to share hardware resources. Each PF and VF is assigned a unique PCIe BDF number, and the input / output IO (Input / Output) memory management unit IOMMU can distinguish data and interrupts between different PFs and VFs. NVMe is a storage interface protocol currently supported by high-speed solid-state drives, which supports high concurrent access. NVMe SSDs that implement the SR-IOV protocol can meet fast storage access requests in virtualization scenarios, and virtual machines accessing NVMe devices can also achieve the performance of non-virtualized environments.

[0003] Currently, in NVMe SSDs that support the SR-IOV protocol, the maximum number of queues supported by each PF or VF is fixed, which cannot satisfy the PF or VF's allocation of queues on demand. The number of NVMe IO queues is related to the number of CPU cores, but the number of CPU cores simulated in a virtual machine can be different from that of a physical machine. If the maximum number of queues specified in advance is followed, then when the number of CPU cores is less than the maximum number of queues, the excess IO queue resources will be wasted, and when the number of CPU cores is more than the maximum number of queues, the IO queue resources will be insufficient.

[0004] Therefore, it is urgent to design a method that can solve the problem that NVMe IO queues in the SR-IOV protocol cannot be allocated on demand. Summary of the invention

[0005] In view of this, an embodiment of the present application provides an NVMe queue resource dynamic allocation method and register based on the SR-IOV protocol to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of the present application provides a method for dynamically allocating NVMe queue resources based on the SR-IOV protocol, which is implemented by hardware, and the method includes:

[0007] Receiving a queue resource allocation request sent by a host based on the NVMe protocol and the SR-IOV protocol, wherein the queue resource allocation request includes queue resource demand data of a target PCIe function, and the target PCIe function includes: a physical function and / or a virtual function;

[0008] If it is found out that the target PCIe function meets the preset queue resource allocation conditions according to the function list corresponding to the current CPU core resources, queue resources are dynamically allocated to the target PCIe function in the local target NVMe shared IO queue based on the function list and the queue resource requirement data of the target PCIe function.

[0009] In some embodiments of the present application, the queue resource demand data includes: a unique identifier of the target PCIe function and a requested queue resource; the function list stores a unique identifier of each physical function and each virtual function and a corresponding status identifier, wherein the status identifier includes a valid status identifier or an invalid status identifier;

[0010] Correspondingly, after receiving the queue resource allocation request sent by the host based on the NVMe protocol and the SR-IOV protocol, it also includes:

[0011] The state identifier corresponding to the unique identifier of the target PCIe function is searched from the function list corresponding to the current CPU core resources to see whether it is a valid state identifier. If so, it is determined that the target PCIe function meets the preset first queue resource allocation condition.

[0012] In some embodiments of the present application, the queue resource requirement data further includes: an offset of the target PCIe function;

[0013] Correspondingly, after determining that the target PCIe function meets the preset first queue resource allocation condition, the method further includes:

[0014] It is determined whether the offset of the target PCIe function is out of bounds, and if not, it is determined that the target PCIe function meets a preset second queue resource allocation condition.

[0015] In some embodiments of the present application, the function list further stores queue base addresses corresponding to each physical function and each virtual function;

[0016] If the target PCIe function meets the preset queue resource allocation condition according to the query of the function list corresponding to the current CPU core resources, then based on the function list and the queue resource requirement data of the target PCIe function, the queue resources are dynamically allocated to the target PCIe function in the local target NVMe shared IO queue, including:

[0017] If it is obtained according to the function list that the target PCIe function satisfies the first queue resource allocation condition and the second queue resource allocation condition, the queue base address corresponding to the unique identifier of the target PCIe function in the function list is concatenated with the offset of the target PCIe function to obtain the queue address of the target NVMe shared IO queue corresponding to the target PCIe function, and the target NVMe shared IO queue is obtained based on the queue address of the target NVMe shared IO queue corresponding to the target PCIe function;

[0018] Dynamically allocate queue resources for the target PCIe function in the target NVMe shared IO queue.

[0019] In some embodiments of the present application, dynamically allocating queue resources for the target PCIe function in the target NVMe shared IO queue includes:

[0020] According to the request queue resources corresponding to the target PCIe function, the target NVMe shared IO queue is scanned from the beginning to find out whether the target NVMe shared IO queue currently contains continuous free queue space corresponding to the request queue resources. If so, the continuous free queue space corresponding to the request queue resources is allocated to the target PCIe function in the target NVMe shared IO queue.

[0021] In some embodiments of the present application, it also includes:

[0022] If it is found that the target NVMe shared IO queue does not contain continuous idle queue space corresponding to the request queue resource, fragment recovery processing is performed on each discontinuous idle queue space in the target NVMe shared IO queue;

[0023] According to the request queue resources corresponding to the target PCIe function, determine whether the target NVMe shared IO queue after fragment recovery processing contains continuous free queue space corresponding to the request queue resources. If so, allocate the continuous free queue space corresponding to the request queue resources to the target PCIe function in the target NVMe shared IO queue.

[0024] In some embodiments of the present application, it also includes:

[0025] If it is determined that the target NVMe shared IO queue does not contain continuous free queue space corresponding to the request queue resource, it is determined whether the target NVMe shared IO queue currently contains remaining free queue space. If so, the remaining free queue space is allocated to the target PCIe function in the target NVMe shared IO queue.

[0026] In some embodiments of the present application, it also includes:

[0027] After dynamically allocating queue resources for the target PCIe function, a corresponding queue resource allocation message is sent to the host, so that after the host receives the queue resource allocation message, the target PCIe function in the host calls the queue resources corresponding to the target PCIe function in the target NVMe shared IO queue based on the SR-IOV protocol and the NVMe protocol.

[0028] In some embodiments of the present application, it also includes:

[0029] If a queue resource cancellation request sent by the host is received based on the NVMe protocol and the SR-IOV protocol, wherein the queue resource cancellation request includes queue resource cancellation data of the target PCIe function;

[0030] If it is found out that the target PCIe function meets the preset queue resource allocation conditions according to the function list corresponding to the current CPU core resources, the queue resources corresponding to the target PCIe function are released in the local target NVMe shared IO queue based on the function list and the queue resource cancellation data of the target PCIe function.

[0031] Another aspect of the present application provides an NVMe register, wherein the NVMe register is used to execute the NVMe queue resource dynamic allocation method based on the SR-IOV protocol;

[0032] The NVMe register is set in an NVMe solid-state drive based on the SR-IOV protocol, and the NVMe register is respectively communicated with the PCIe interface in the NVMe solid-state drive and the CPU on the SOC chip to receive the queue resource allocation request sent by the host based on the PCIe interface and receive the current CPU core resources from the CPU.

[0033] The NVMe queue resource dynamic allocation method based on the SR-IOV protocol provided in the present application is implemented by hardware, and a queue resource allocation request sent by a host is received based on the NVMe protocol and the SR-IOV protocol, wherein the queue resource allocation request includes queue resource demand data of a target PCIe function, and the target PCIe function includes: a physical function and / or a virtual function; if the target PCIe function satisfies a preset queue resource allocation condition according to a query of a function list corresponding to the current CPU core resources, then based on the function list and the queue resource demand data of the target PCIe function, queue resources are dynamically allocated to the target PCIe function in a local target NVMe shared IO queue, so that the dynamic allocation of queue resources for the PCIe function can be implemented through hardware, the utilization rate of the IO queue resources can be effectively improved, and the problem of insufficient or wasted IO queue resources in a certain PCIe function can be effectively solved, thereby effectively improving the reliability and effectiveness of the PCIe function accessing the NVMe SSD supporting the SR-IOV protocol.

[0034] Additional advantages, purposes, and features of the present application will be partially described in the following description, and will become partially apparent to those skilled in the art after studying the following, or may be learned from the practice of the present application. The purposes and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0035] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present application are not limited to the above specific description, and the above and other purposes that can be achieved by the present application will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings described herein are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application. The components in the drawings are not drawn to scale, but are only for the purpose of illustrating the principles of the present application. In order to facilitate the illustration and description of some parts of the present application, the corresponding parts in the drawings may be enlarged, that is, they may become larger relative to other components in the exemplary device actually manufactured according to the present application. In the drawings:

[0037] Figure 1 The following is a schematic diagram of an example of an SR-IOV structure.

[0038] Figure 2 This is a diagram showing an example of a fixed IO queue for an SR-IOV NVMe SSD.

[0039] Figure 3 This is a first flow chart of a method for dynamically allocating NVMe queue resources based on the SR-IOV protocol in one embodiment of the present application.

[0040] Figure 4 This is a second flow chart of a method for dynamically allocating NVMe queue resources based on the SR-IOV protocol in one embodiment of the present application.

[0041] Figure 5 This is a third flow chart of a method for dynamically allocating NVMe queue resources based on the SR-IOV protocol in one embodiment of the present application.

[0042] Figure 6 This is a schematic diagram of the execution logic of the hardware allocation queue access process in an example of the present application.

[0043] Figure 7 This is a fourth flow chart of a method for dynamically allocating NVMe queue resources based on the SR-IOV protocol in an embodiment of the present application.

[0044] Figure 8 This is a schematic diagram of a logic example of shared IO queue allocation in an example of the present application.

[0045] Fig. 9 This is a schematic diagram of an example process of allocating a shared IO queue in an example of the present application.

[0046] Fig.10 Schematic diagram of the structure of the NVMe register in one embodiment of the present application.

[0047] Fig.11 This is a schematic diagram of an example structure of an NVMe solid-state drive based on the SR-IOV protocol in one embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the implementation modes and the accompanying drawings. Here, the illustrative implementation modes and descriptions of the present application are used to explain the present application, but are not intended to limit the present application.

[0049] It should also be noted here that in order to avoid obscuring the present application due to unnecessary details, only the structures and / or processing steps closely related to the scheme according to the present application are shown in the accompanying drawings, while other details that are not very relevant to the present application are omitted.

[0050] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0051] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0052] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0053] In one or more embodiments of the present application, the PCIe BDF number consists of three parts: Bus Number (ranging from 0 to 255), Device Number (ranging from 0 to 31), and Function Number (ranging from 0 to 7).

[0054] In one or more embodiments of the present application, “IO” may also be written as “I / O”, which refers to input / output.

[0055] like Figure 1 As shown, the software side assigns VF to a fixed virtual machine through the SR-IOV management unit, and the virtual machine can bypass the virtual machine management (VMM) layer and directly access the SR-IOV hardware, which can shorten the data access path and improve performance. Among them, VM1 represents the first virtual machine, and VM2 represents the second virtual machine. VF1 and VF2 are different virtual functions. PF0 represents a physical function.

[0056] It is understandable that SR-IOV hardware refers to hardware devices based on the SR-IOV protocol, such as solid-state drives (SSDs).

[0057] In NVMe SSDs that support the SR-IOV protocol (also written as SR-IOV NVMe SSD), the maximum number of queues supported by each PF or VF is fixed, which cannot satisfy the PF or VF to allocate queues according to demand. Figure 2 As shown, if the maximum number of queues specified in advance is followed, when the number of CPU cores is less than the maximum number of queues, the excess IO queue resources are wasted, and when the number of CPU cores is more than the maximum number of queues, insufficient IO queue resources occur. Among them, NVMe VF1 and NVMe VF2 are units where different virtual functions based on the NVMe protocol are located. PF0 represents a physical function. FTL represents the Flash Translation Layer in the solid-state drive SSD; NS1, NS2 and NS3 are different and independent logical spaces divided into by the flash memory space. VF1, VF2 and PF0 correspond to the VF queue, VF2 queue and PF0 queue respectively in the NVMe solid-state drive that supports the -IOV protocol.

[0058] In one or more embodiments of the present application, the NVMe solid-state drive based on the SR-IOV protocol can also be written as an NVMe SSD with an SR-IOV protocol or abbreviated as SR-IOV NVMe SSD.

[0059] Based on this, in order to solve the problem that the NVMe IO queue in the above-mentioned SR-IOV protocol cannot be allocated on demand, the embodiments of the present application respectively provide an NVMe queue resource dynamic allocation method based on the SR-IOV protocol and an NVMe register for executing the NVMe queue resource dynamic allocation method based on the SR-IOV protocol. By uniformly managing the NVMe IO queues in different PFs and VFs, when VFs need to be generated, they are allocated one by one according to the queue requests of different VFs, thereby solving the problem that IO queue resources are insufficient or wasted in a certain PF or VF.

[0060] The details are described in detail through the following examples.

[0061] Based on this, the embodiment of the present application provides a method for dynamically allocating NVMe queue resources based on the SR-IOV protocol that can be implemented by hardware devices such as NVMe registers, and is also a hardware implementation method for managing NVMe queues based on SR-IOV, which belongs to the field of computer technology, especially to the field of storage, see Figure 3 The NVMe queue resource dynamic allocation method based on the SR-IOV protocol specifically includes the following contents:

[0062] Step 100: Receive a queue resource allocation request sent by a host based on the NVMe protocol and the SR-IOV protocol, wherein the queue resource allocation request includes queue resource demand data of a target PCIe function, and the target PCIe function includes: a physical function and / or a virtual function.

[0063] In one or more embodiments of the present application, the target PCIe function refers to at least one PCIe function of the request for allocation of queue resources sent by the host, and the at least one PCIe function refers to at least one of a physical function PF and a virtual function VF.

[0064] It is understood that the host side may include physical functions set in the host, such as Figure 2 The host side may also include virtual functions based on the NVMe protocol that are respectively set in each virtual machine, such as Figure 2NVMe VF1 and NVMe VF2 in SR-IOV. SR-IOV allocates VF to the specified virtual machine VM, so that network traffic can flow directly between VF and sub-partitions, thereby enabling data traffic to bypass the software emulation layer of the virtualization stack, that is, the virtual machine can directly access the hardware device. Compared with software-implemented virtualization, it can reduce the IO overhead in the software emulation layer and achieve almost the same performance in accessing IO devices as in a non-virtualized environment.

[0065] Step 200: If it is found through querying the function list corresponding to the current CPU core resources that the target PCIe function meets the preset queue resource allocation conditions, then based on the function list and the queue resource requirement data of the target PCIe function, queue resources are dynamically allocated to the target PCIe function in the local target NVMe shared IO queue.

[0066] In step 200, a hardware device such as an NVMe register dynamically allocates queue resources to a target PCIe function on the premise that the target PCIe function satisfies a preset queue resource allocation condition, thereby improving the reliability of dynamic allocation of queue resources to the PCIe function and avoiding invalid queue resource allocation processes to reduce resource consumption in the process.

[0067] In one or more embodiments of the present application, the function list can be abbreviated as func_table and stored locally in a hardware device such as an NVMe register. The function list corresponds to the current CPU core resources. Specifically, the DPU on the SOC chip can send corresponding configuration information to a hardware device such as an NVMe register regularly or in real time according to the current resource status of the CPU core, so that the hardware device such as the NVMe register updates the function list according to the configuration information, and specifically updates the number of queues of the NVMe shared IO queues corresponding to each PF and VF in the function list.

[0068] Based on this, in step 200, hardware devices such as NVMe registers use the current latest version of the function list each time to further improve the reliability and effectiveness of dynamically allocating queue resources for the target PCIe function.

[0069] In one or more embodiments of the present application, the target NVMe shared IO queue refers to at least one NVMe shared IO queue corresponding to the target PCIe function in each NVMe shared IO queue. The target NVMe shared IO queue can be abbreviated as reg_list or shared IO queue.

[0070] From the above description, it can be seen that the NVMe queue resource dynamic allocation method based on the SR-IOV protocol provided in the embodiment of the present application can realize the dynamic allocation of queue resources for the PCIe function through hardware, can effectively improve the utilization rate of IO queue resources, can effectively solve the problem of insufficient or wasted IO queue resources in a certain PCIe function, and thus can effectively improve the reliability and effectiveness of the PCIe function accessing the NVMe SSD that supports the SR-IOV protocol.

[0071] In order to ensure that the current state of the target PCIe function to which queue resources are to be dynamically allocated is valid, so as to further improve the effectiveness and reliability of the dynamic allocation of NVMe queue resources based on the SR-IOV protocol, in a method for dynamic allocation of NVMe queue resources based on the SR-IOV protocol provided in an embodiment of the present application, the queue resource demand data includes: a unique identifier of the target PCIe function and requested queue resources; the function list stores the unique identifiers of each physical function and each virtual function and the corresponding status identifiers, wherein the status identifier includes a valid status identifier or an invalid status identifier.

[0072] For the corresponding Figure 4 , the steps between step 100 and step 200 in the NVMe queue resource dynamic allocation method based on the SR-IOV protocol further specifically include the following contents:

[0073] Step 010: Search the function list corresponding to the current CPU core resources to see whether the status identifier corresponding to the unique identifier of the target PCIe function is a valid status identifier. If so, execute step 020; if not, execute step 030.

[0074] Step 020: Determine whether the target PCIe function meets a preset first queue resource allocation condition.

[0075] Step 200 may be directly executed after step 020, that is, in the current case, the preset queue resource allocation condition mentioned in step 200 is the first queue resource allocation condition.

[0076] Step 030: Stop the subsequent processing of the dynamic allocation of queue resources, and send a notification message to the host that the target PCIe function has failed.

[0077] On this basis, in order to ensure that the current state of the target PCIe function to which queue resources are to be dynamically allocated is valid, it can also be ensured that the offset of the target PCIe function does not cross the boundary, so as to further improve the effectiveness and reliability of the dynamic allocation of NVMe queue resources based on the SR-IOV protocol, in a method for dynamic allocation of NVMe queue resources based on the SR-IOV protocol provided in an embodiment of the present application, the queue resource demand data also includes: the offset of the target PCIe function.

[0078] For the corresponding Figure 5 , the following contents are specifically included between step 020 and step 200 in the NVMe queue resource dynamic allocation method based on the SR-IOV protocol:

[0079] Step 040: Determine whether the offset of the target PCIe function is out of bounds, if not, execute step 050; if yes, execute step 060.

[0080] In one or more embodiments of the present application, the offset is written as offset.

[0081] Step 050: Determine whether the target PCIe function satisfies the preset second queue resource allocation condition. Then execute step 200. That is to say, in the current case, the preset queue resource allocation condition mentioned in step 200 is composed of the first queue resource allocation condition and the second queue resource allocation condition. It should be noted that in order to further improve the effectiveness and reliability of the dynamic allocation of NVMe queue resources based on the SR-IOV protocol, the preset queue resource allocation condition may also include other queue resource allocation conditions, which can be set according to the actual application scenario.

[0082] Step 060: Stop the subsequent processing of the dynamic allocation of queue resources, and send a notification message to the host that the offset of the target PCIe function has exceeded the boundary.

[0083] It can be understood that the specific method of determining whether the offset of the target PCIe function is out of bounds is: determining whether the actual queue address of the NVMe hardware formed by splicing the queue base address and the offset exceeds 2K. If not, the offset of the target PCIe function is not out of bounds.

[0084] In order to further improve the security and reliability of accessing the target NVMe shared IO queue corresponding to the target PCIe function, in a dynamic allocation method of NVMe queue resources based on the SR-IOV protocol provided in an embodiment of the present application, the function list also stores the queue base addresses corresponding to each physical function and each virtual function.

[0085] For the corresponding Figure 5 , step 200 in the NVMe queue resource dynamic allocation method based on the SR-IOV protocol specifically includes the following contents:

[0086] Step 210: If it is found through querying the function list that the target PCIe function satisfies the first queue resource allocation condition and the second queue resource allocation condition, the queue base address corresponding to the unique identifier of the target PCIe function in the function list is concatenated with the offset of the target PCIe function to obtain the queue address of the target NVMe shared IO queue corresponding to the target PCIe function, and the target NVMe shared IO queue is obtained based on the queue address of the target NVMe shared IO queue corresponding to the target PCIe function.

[0087] Step 220: Dynamically allocate queue resources for the target PCIe function in the target NVMe shared IO queue.

[0088] In one example, Figure 6 As shown, the function list is used to store the unique codes corresponding to each PF and VF, which can be written as Func_id. The function list is also used to store the status identifiers corresponding to the unique codes of each PF and VF, the number of queues of the NVMe shared IO queue, and the queue base address. The number of queues can be written as length, the queue base address can be written as base, and the status identifier can include a valid status identifier or an invalid status identifier, wherein the valid status identifier can be written as Valid.

[0089] exist Figure 6 When a hardware device such as an NVMe register receives a request, it determines whether the request comes from the host or the CPU in the SoC based on the type identifier in the request. The type identifier in the request may include a CPU request type indicating that the request comes from the host, which can be written as type(host); the type identifier in the request may also include a CPU request type indicating that the request comes from the SoC, which can be written as type(SoC).

[0090] Specifically, when a hardware device such as an NVMe register receives a queue resource allocation request sent by the host, it obtains the unique code Func_id of the target PCIe function and the offset offset of the target PCIe function from the queue resource allocation request sent by the host.

[0091] When a hardware device such as an NVMe register receives a CPU core configuration information reception request sent by the CPU in the SoC, it obtains the offset of the PF and / or VF corresponding to the current CPU core resource from the CPU core configuration information reception request.

[0092] Then, the queue base address base corresponding to the unique identifier of the target PCIe function in the function list func_table is spliced ​​with the offset offset of the target PCIe function to obtain the queue address of the target NVMe shared IO queue corresponding to the target PCIe function, and the target NVMe shared IO queue reg_list is obtained based on the queue address of the target NVMe shared IO queue corresponding to the target PCIe function.

[0093] Among them, the NVMe shared IO queue or the target NVMe shared IO queue includes submission queue head information SQ_head, submission queue tail information SQ_tail, completion queue head information CQ_head and completion queue tail information CQ_tail.

[0094] That is to say, when the hardware queries the PF and / or VF queues and updates the corresponding queue doorbell registers, when a request is received, the func_table entry is first queried based on Func_id. Each entry includes the queue base address, whether the PF and / or VF is valid, and the number of VF queues. First, check whether the PF and / or VF is valid, and then check whether the offset is out of bounds. After the check, the queue base address and offset are concatenated to form the real queue address of the NVMe hardware. In this way, the dynamic allocation of NVMe queues based on SR-IOV is realized.

[0095] In order to further improve the effectiveness and flexibility of the dynamic allocation of NVMe queue resources based on the SR-IOV protocol, in a method for dynamic allocation of NVMe queue resources based on the SR-IOV protocol provided in an embodiment of the present application, see Figure 7 , step 220 in the NVMe queue resource dynamic allocation method based on the SR-IOV protocol specifically includes the following contents:

[0096] Step 221: According to the request queue resource corresponding to the target PCIe function, scan the target NVMe shared IO queue from the beginning to find out whether the target NVMe shared IO queue currently contains continuous free queue space corresponding to the request queue resource. If so, execute step 222.

[0097] Step 222: Allocate continuous idle queue space corresponding to the request queue resource to the target PCIe function in the target NVMe shared IO queue.

[0098] In order to further improve the success rate of on-demand allocation of NVMe queue resources, in a method for dynamic allocation of NVMe queue resources based on the SR-IOV protocol provided in an embodiment of the present application, see Figure 7 , the step 220 in the NVMe queue resource dynamic allocation method based on the SR-IOV protocol further specifically includes the following contents:

[0099] If it is determined through step 221 that the target NVMe shared IO queue currently does not contain continuous idle queue space corresponding to the request queue resource, step 223 is executed.

[0100] Step 223: Perform fragment recovery processing on each discontinuous free queue space in the target NVMe shared IO queue.

[0101] Step 224: According to the request queue resources corresponding to the target PCIe function, determine whether the target NVMe shared IO queue after the fragment recovery process contains continuous free queue space corresponding to the request queue resources. If so, execute step 225.

[0102] Step 225: Allocate continuous idle queue space corresponding to the request queue resource to the target PCIe function in the target NVMe shared IO queue.

[0103] In order to further improve the flexibility of NVMe queue resource allocation, in a method for dynamically allocating NVMe queue resources based on the SR-IOV protocol provided in an embodiment of the present application, see Figure 7 , step 220 in the NVMe queue resource dynamic allocation method based on the SR-IOV protocol further specifically includes the following contents:

[0104] If it is determined through step 224 that the target NVMe shared IO queue after the fragment recovery process does not contain continuous free queue space corresponding to the request queue resource, step 226 is executed.

[0105] Step 226: Determine whether the target NVMe shared IO queue currently contains any remaining free queue space, if so, execute step 227, if not, execute step 228.

[0106] Step 227: Allocate the remaining free queue space to the target PCIe function in the target NVMe shared IO queue.

[0107] Step 228: Stop subsequent processing and send a notification message to the host indicating that the dynamic allocation of NVMe queue resources has failed.

[0108] In order to further improve the efficiency and reliability of the host side in learning the success of dynamic allocation of NVMe queue resources, in a method for dynamic allocation of NVMe queue resources based on the SR-IOV protocol provided in an embodiment of the present application, see Figure 7 , the NVMe queue resource dynamic allocation method based on the SR-IOV protocol further specifically includes the following contents after step 222, step 225 and step 227:

[0109] Step 300: After dynamically allocating queue resources for the target PCIe function, a corresponding queue resource allocation message is sent to the host, so that after the host receives the queue resource allocation message, the target PCIe function in the host calls the queue resources corresponding to the target PCIe function in the target NVMe shared IO queue based on the SR-IOV protocol and the NVMe protocol.

[0110] In one example, see Figure 8 and Fig. 9 , for 1 PF and 255 VFs, resource space is allocated in 2K shared IO queues. Before initializing the NVMe device, each PF or VF needs to apply for the maximum number of queues and allocate a continuous space (the size of the space is the maximum number of queues applied) from the maximum total number of queues (2K). When applying for a queue, the method of scanning from the beginning is adopted until a continuous free space that meets the demand is scanned, and then the allocation is made. If there is not enough continuous space but there are many fragments, the fragments are recycled. The recycling principle is to allocate the continuous queue space of the PF or VF to be allocated from the beginning according to the current PF or VF to be allocated.

[0111] In order to further improve the flexibility and reliability of dynamic allocation of NVMe queue resources, in an NVMe queue resource dynamic allocation method based on the SR-IOV protocol provided in an embodiment of the present application, the NVMe queue resource dynamic allocation method based on the SR-IOV protocol also includes steps 400 and 500 that can be performed before or after any step, see Figure 7 , taking step 400 and step 500 sequentially following step 300 as an example for demonstration, the step 400 and step 500 specifically include the following contents:

[0112] Step 400: If a queue resource cancellation request sent by the host is received based on the NVMe protocol and the SR-IOV protocol, wherein the queue resource cancellation request includes queue resource cancellation data of the target PCIe function;

[0113] Step 500: If it is found through querying the function list corresponding to the current CPU core resources that the target PCIe function meets the preset queue resource allocation conditions, then based on the function list and the queue resource cancellation data of the target PCIe function, the queue resources corresponding to the target PCIe function are released in the local target NVMe shared IO queue.

[0114] That is to say, when a VF is deregistered, the corresponding queue space is released and waits for other VFs to use it. Fragments are not actively recycled. Fragment recycling is performed only when there is no continuous free space to be allocated.

[0115] Based on the aforementioned embodiment of the NVMe queue resource dynamic allocation method based on the SR-IOV protocol, the present application also provides an NVMe register for executing all or part of the NVMe queue resource dynamic allocation method based on the SR-IOV protocol, and the NVMe register is used for all the latter parts of the NVMe queue resource dynamic allocation method based on the SR-IOV protocol provided in the aforementioned embodiment.

[0116] See also Fig.10 , the NVMe register may include the following contents:

[0117] The allocation request receiving module 10 is used to receive a queue resource allocation request sent by a host based on the NVMe protocol and the SR-IOV protocol, wherein the queue resource allocation request contains queue resource demand data of a target PCIe function, and the target PCIe function includes: a physical function and / or a virtual function.

[0118] The queue resource allocation module 20 is used to dynamically allocate queue resources for the target PCIe function in the local target NVMe shared IO queue based on the function list corresponding to the current CPU core resources and the queue resource requirement data of the target PCIe function if the target PCIe function meets the preset queue resource allocation conditions.

[0119] See also Fig.11 The NVMe register is set in an NVMe solid-state drive based on the SR-IOV protocol, and the NVMe register is respectively communicated with the PCIe interface in the NVMe solid-state drive and the CPU on the SOC chip to receive the queue resource allocation request sent by the host based on the PCIe interface and receive the current CPU core resources from the CPU.

[0120] Specifically, the dynamic allocation of queues is implemented in NVMe registers. NVMe registers receive access from host (also written as host) registers, and at the same time, NVMe registers receive the configuration of the CPU in the SoC. The host follows the standard NVMe protocol and SR-IOV protocol when accessing, that is, the dynamic allocation of queues is transparent to the host side.

[0121] In addition, the CPU in the SoC can also sense the needs of different VFs for queues and configure the queue information into the NVMe register module.

[0122] The NVMe register is also connected to the NVMe controller (NVMe Controller), and the NVMe controller is connected to the memory DDR and the flash translation layer FTL respectively. Fig.11 The Flash in it refers to the flash memory.

[0123] That is to say, the NVMe register provided in the embodiment of the present application dynamically manages the NVMe IO queues in different VFs and PFs; by sharing the NVMe IO queues, the queues in different PFs and VFs are allocated on demand, and the requirements of different PFs and VFs for NVMe queues are met, so as to solve the problem that the NVMe IO queues in the SR-IOV protocol cannot be allocated on demand.

[0124] The embodiment of the NVMe register provided in the present application can be specifically used to execute the processing flow of the embodiment of the method for dynamic allocation of NVMe queue resources based on the SR-IOV protocol in the above-mentioned embodiment. Its function will not be repeated here, and reference can be made to the detailed description of the embodiment of the method for dynamic allocation of NVMe queue resources based on the SR-IOV protocol.

[0125] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0126] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0127] In the present application, features described and / or illustrated for one embodiment may be used in the same manner or in a similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0128] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for dynamically allocating NVMe queue resources based on the SR-IOV protocol, characterized in that: Implemented by hardware, the method includes: Receiving a queue resource allocation request sent by a host based on the NVMe protocol and the SR-IOV protocol, wherein the queue resource allocation request includes queue resource demand data of a target PCIe function, and the target PCIe function includes: a physical function and / or a virtual function; If the target PCIe function meets the preset queue resource allocation condition according to the function list corresponding to the current CPU core resources, then based on the function list and the queue resource demand data of the target PCIe function, queue resources are dynamically allocated to the target PCIe function in the local target NVMe shared IO queue; The queue resource requirement data includes: the unique identifier of the target PCIe function and the requested queue resources; the function list stores the unique identifiers of each physical function and each virtual function and the corresponding status identifiers, wherein the status identifier includes a valid status identifier or an invalid status identifier; Correspondingly, after receiving the queue resource allocation request sent by the host based on the NVMe protocol and the SR-IOV protocol, it also includes: Searching the function list corresponding to the current CPU core resource to see whether the state identifier corresponding to the unique identifier of the target PCIe function is a valid state identifier, and if so, determining that the target PCIe function meets the preset first queue resource allocation condition; The queue resource requirement data also includes: an offset of the target PCIe function; Correspondingly, after determining that the target PCIe function meets the preset first queue resource allocation condition, the method further includes: Determine whether the offset of the target PCIe function is out of bounds, and if not, determine that the target PCIe function meets a preset second queue resource allocation condition; The function list also stores queue base addresses corresponding to each physical function and each virtual function; If the target PCIe function meets the preset queue resource allocation condition according to the query of the function list corresponding to the current CPU core resources, then based on the function list and the queue resource requirement data of the target PCIe function, the queue resources are dynamically allocated to the target PCIe function in the local target NVMe shared IO queue, including: If it is obtained according to the function list that the target PCIe function satisfies the first queue resource allocation condition and the second queue resource allocation condition, the queue base address corresponding to the unique identifier of the target PCIe function in the function list is concatenated with the offset of the target PCIe function to obtain the queue address of the target NVMe shared IO queue corresponding to the target PCIe function, and the target NVMe shared IO queue is obtained based on the queue address of the target NVMe shared IO queue corresponding to the target PCIe function; Dynamically allocate queue resources for the target PCIe function in the target NVMe shared IO queue.

2. The NVMe queue resource dynamic allocation method based on the SR-IOV protocol according to claim 1, characterized in that: The dynamically allocating queue resources for the target PCIe function in the target NVMe shared IO queue includes: According to the request queue resources corresponding to the target PCIe function, the target NVMe shared IO queue is scanned from the beginning to find out whether the target NVMe shared IO queue currently contains continuous free queue space corresponding to the request queue resources. If so, the continuous free queue space corresponding to the request queue resources is allocated to the target PCIe function in the target NVMe shared IO queue.

3. The NVMe queue resource dynamic allocation method based on the SR-IOV protocol according to claim 2, characterized in that: Also includes: If it is found that the target NVMe shared IO queue does not contain continuous idle queue space corresponding to the request queue resource, fragment recovery processing is performed on each discontinuous idle queue space in the target NVMe shared IO queue; According to the request queue resources corresponding to the target PCIe function, determine whether the target NVMe shared IO queue after fragment recovery processing contains continuous free queue space corresponding to the request queue resources. If so, allocate the continuous free queue space corresponding to the request queue resources to the target PCIe function in the target NVMe shared IO queue.

4. The NVMe queue resource dynamic allocation method based on the SR-IOV protocol according to claim 3, characterized in that: Also includes: If it is determined that the target NVMe shared IO queue does not contain continuous free queue space corresponding to the request queue resource, it is determined whether the target NVMe shared IO queue currently contains remaining free queue space. If so, the remaining free queue space is allocated to the target PCIe function in the target NVMe shared IO queue.

5. The NVMe queue resource dynamic allocation method based on the SR-IOV protocol according to any one of claims 1 to 4, characterized in that: Also includes: After dynamically allocating queue resources for the target PCIe function, a corresponding queue resource allocation message is sent to the host, so that after the host receives the queue resource allocation message, the target PCIe function in the host calls the queue resources corresponding to the target PCIe function in the target NVMe shared IO queue based on the SR-IOV protocol and the NVMe protocol.

6. The NVMe queue resource dynamic allocation method based on the SR-IOV protocol according to claim 1, characterized in that: Also includes: If a queue resource cancellation request sent by the host is received based on the NVMe protocol and the SR-IOV protocol, wherein the queue resource cancellation request includes queue resource cancellation data of the target PCIe function; If it is found out that the target PCIe function meets the preset queue resource allocation conditions according to the function list corresponding to the current CPU core resources, the queue resources corresponding to the target PCIe function are released in the local target NVMe shared IO queue based on the function list and the queue resource cancellation data of the target PCIe function.

7. An NVMe register, characterized in that: The NVMe register is used to execute the NVMe queue resource dynamic allocation method based on the SR-IOV protocol according to any one of claims 1 to 6; The NVMe register is set in an NVMe solid-state drive based on the SR-IOV protocol, and the NVMe register is respectively communicated with the PCIe interface in the NVMe solid-state drive and the CPU on the SOC chip to receive the queue resource allocation request sent by the host based on the PCIe interface and receive the current CPU core resources from the CPU.

Images