Managing the distribution of I / O queue pairs of a target across hosts

Abstract

Examples include managing the distribution of a total number of I / O queue pairs of an NVMe target across a plurality of NVM subsystems of the NVMe target and a plurality of hosts connected to the NVMe target. A target controller of the NVMe target defines a number of I / O queue pairs for a dedicated pool and a number of I / O queue pairs for a reserved pool based on the total number of I / O queue pairs. The target controller distributes a number of I / O queue pairs to each of the hosts from the number of I / O queue pairs of the dedicated pool and uses the number of I / O queue pairs of the reserved pool to balance the number of I / O queue pairs on each of the hosts by selectively changing the number of I / O queue pairs of the reserved pool.

Description

BACKGROUNDNon-volatile memory (NVM) is a type of computer memory that maintains its content over power cycles and can therefore be used as memory. In contrast to volatile memory that requires power to store data, the NVM may continue to store data even after the computer is powered down.NVM Express™ Basic Specification, revision 1.4b, September 21, 2020, refers to an NVM ExpressTM (NVMeTM) interface that enables host software to communicate with a non-volatile memory subsystem. This interface is optimized for enterprise and client solid state drives and is normally connected to the PCI Express interface at the register level.US 2020 / 0 004 441 A1 describes a method of access control in a memory in a memory system comprising a memory and a controller communicating with a plurality of hosts, each host comprising its own set of input-output queues but only one host comprising an admin queue. The method includes receiving an association that limits access to portions of the memory to particular hosts of the plurality of hosts, the association being generated by the host including the admin queue, and limiting access to the portions of the memory based on the association.BRIEF DESCRIPTIONA method according to claims 1 to 7, a target controller according to claims 8 to 14, and a non-transitory machine readable storage medium according to claims 15 to 18 are disclosed.BRIEF DESCRIPTION OF THE DRAWINGSThe following detailed description refers to the drawings in which: FIG. 1 is a block diagram of a system having an NVMe target and a plurality of hosts connected to the NVMe target, in accordance with one example; FIGS. 2A and 2B collectively show a flow diagram of a method for managing the distribution of a number of I / O queue pairs of an NVMe target to a plurality of hosts connected to the NVMe target, in accordance with one example; FIGS. 3A and 3B collectively show a flowchart of a method for managing the distribution of a number of I / O queue pairs of an NVMe target among a plurality of hosts connected to the NVMe target, in accordance with another example; and FIG. 4 is a block diagram of a computer system including a processing resource and a machine readable storage medium encoded with example instructions to manage the distribution of a number of I / O queue pairs of an NVMe target among a plurality of hosts connected to the NVMe target, in accordance with an example.DETAILED DESCRIPTIONThe following detailed description refers to the accompanying drawings. Wherever possible, the same reference numbers will be used in the drawings and the following description to refer to the same or like parts. It is expressly understood, however, that the drawings are for purposes of illustration and description only. Although several examples are described in this document, modifications, adaptations, and other implementations are possible. Accordingly, the following detailed description does not limit the disclosed examples. Rather, the true scope of the disclosed examples may be defined by the appended claims.The terminology used herein is for the purpose of describing particular examples only and is not to be taken as limiting. As used herein, the singular forms "a", "an" and "the" also include the plural forms, unless the context clearly indicates otherwise. The term "plurality" as used herein is defined as two or more than two. The term "another" as used herein means at least a second or more. The term "connected" as used herein is defined as coupled or associated, either directly without intermediate members or indirectly with at least one intermediate member, unless otherwise specified. Two elements may be mechanically, electrically, or communicatively connected via a communication channel, path, network, or system. As used herein, the term "and / or" refers to and encompasses all possible combinations of the listed elements. The term "based on" means based at least in part on. Although the terms "first", "second", "third", etc. are used herein to describe various elements, these elements should not be limited by these terms, as these terms are only used to distinguish one element from another unless otherwise stated or otherwise indicated by context.Non-volatile memory (NVM) Express (NVMe) refers to techniques by which a host may communicate with a storage device (e.g., a solid state drive (SSD)) via a peripheral component interconnect express (PCIe) bus. With NVMe, the memory device may process a plurality of I / O operations in parallel. To provide this benefit to enterprise class data centers, NVMe may be extended across fabrics to improve scalability and sharing. In this context, NVMe over Fabrics (NVMe-oF) refers to a flexible transport abstraction layer for a wide range of storage network fabrics such as Ethernet and Fibre Channel. A data storage system using NVMe or NVMe-oF may include an Ethernet switch connected to multiple storage devices within a chassis (referred to as "NVMe chassis" or "NVMe destination"). The data storage system may include an additional NVMe target without an Ethernet switch (referred to as an Ethernet switchless chassis) that may be connected to the Ethernet switch of the NVMe target with an Ethernet switch. Such an NVMe target without an Ethernet switch is referred to as a Just a Bunch of Flash (JBOF) chassis or target. In some examples, a JBOF target may have an Ethernet repeater or re-timer instead of an Ethernet switch to reduce the cost of a data storage system. Another example of an NVMe target may be a bridge target that connects Ethernet to an interface such as PCIe.A storage device that uses NVMe and can process commands (e.g., read commands, write commands, management commands, etc.) consistent with and / or provided in accordance with the NVMe specification is referred to as an "NVM subsystem.". An example of an "NVM subsystem" may be solid state drives (SSDs) compatible with the NVMe specification. A host may be a computing device that can access and write data to data stored in one or more NVM subsystems. In one example, the host may be a server that provides data services to client devices based on the data stored in the NVM subsystem(s).The NVMe specification defines both an interface (e.g., a register-level interface) and a command protocol for communication with the NVM subsystems. In a system that uses NVMe, one or more NVM subsystems (e.g., including port(s) of the NVM subsystem / subsystems) of an NVMe target may be configured to communicate with one or more hosts (e.g., by including port(s) of the host(s)). The host(s) may / may communicate with the NVMe target, more particularly, with the NVM subsystem(s) of the NVMe target based on a mapping defining which host(s) may access which NVM subsystem(s) of the NVMe target. In some examples, one of the hosts may be mapped to one or more NVM subsystems and one of the NVM subsystems may be mapped to one or more hosts. Communication between the host and the NVM subsystem or subsystems may be realized by a target controller. In the examples described herein, the target controller is a storage array controller or a front end I / O module that can manage one or more NVM subsystems such as SSDs at the back end. Each host may be connected to a host port on the target controller, thereby linking the host port to the host. In one example, the host port may be a physical port that serves as an interface between the host and the target controller.The interface between a host and an NVM subsystem of an NVMe target may be based on multiple queue pairs (i.e., paired submission and completion queues) shared between the host (e.g., including port(s) of the host) and the NVM subsystem (e.g., including port(s) of the NVM subsystem). The queue pairs may be located in either a host memory (e.g., memory space in the host) or a memory provided by the NVM subsystem. In one example, the NVMe specification allows up to 64K individual queue pairs, and each queue pair may have up to 64K entries. Once the queue pairs are configured, these queue pairs may be used for communication between the host and the NVM subsystem using the command protocol. Each new entry may be transmitted to the NVM subsystem via the transmission queue with a transmission command. When the submission command is processed, an entry (previously associated with the submission queue from which the command was retrieved) with a completion command may be placed in the completion queue and an interrupt generated. There may be separate queue pairs for management operations (e.g., creating and deleting queues or updating firmware in the NVM subsystem) and for I / O operations (e.g., reading and writing). Separate queue pairs for I / O operations may avoid delays in I / O operations due to lengthy administrative operations.Generally, there are a total number of I / O queue pairs that an NVMe target can support (e.g., between 128 and 512 I / O queue pairs) depending on the hardware resources and the implementation of the NVMe target. The total number of I / O queue pairs supported may be divided among a number of hosts associated with the NVMe target. Each host may be assigned an upper limit to the number of I / O queue pairs that may be assigned (or granted) to the host within the total number of I / O queue pairs supported. The upper limit for the number of I / O queue pairs that may be allocated or used by a particular host (which may be referred to herein as an I / O queue pair limit for the particular host) may be an upper limit for the number of I / O queue pairs that may be granted to the particular host for use (e.g., when requested by the particular host). If the subject host requests a certain number of I / O queue pairs (e.g., by sending connection commands), the subject host may be assigned a number of I / O queue pairs up to its assigned maximum number of I / O queue pairs. The number of I / O queue pairs allocated to the respective host may be used by the respective host to perform one or more I / O operations. For example, if four hosts access an NVMe target that may support 128 I / O queue pairs, each host may be assigned a boundary of 32 I / O queue pairs. In such cases, each host may be assigned a maximum of 32 I / O queue pairs.If the given host requests a certain number of I / O queue pairs that is equal to or greater than the maximum number of I / O queue pairs assigned to it, the given host may be granted a number of I / O queue pairs that is equal to the maximum number of I / O queue pairs assigned to it to be used with a first NVM subsystem of the NVMe target associated with the given host. Generally, when a new host is allocated to the first NVM subsystem, the new host may not be allocated an I / O queue pair for use with the first NVM subsystem. If a particular host is assigned multiple NVM subsystems, it may be assigned no or only a few I / O queue pairs for use with a second NVM subsystem. This disproportionate distribution of the I / O queue pairs may result in imbalanced load distribution between the hosts and the NVM subsystems of the NVMe target, which may cause network congestion and degrade overall system performance.The examples described herein provide techniques for managing the distribution of a total number of I / O queue pairs of an NVMe target to multiple NVM subsystems of the NVMe target and hosts mapped to the NVMe target to achieve a balanced load distribution to the hosts and the NVM subsystems. In particular, the examples may provide techniques for assigning a limit to the number of I / O queue pairs that may be used by each of the hosts with each of the NVM subsystems of the NVMe target associated with the host at a particular time.In some examples, a target controller of an NVMe target may determine a number of I / O queue pairs for a dedicated pool and a number of I / O queue pairs for a reserved pool based on the total number of I / O queue pairs of the NVMe target. The target controller may determine a limit on the number of I / O queue pairs that may be used by each of a plurality of NVM subsystems of the NVMe target (which may be referred to herein as an I / O queue pair limit for an NVM subsystem) based on the number of I / O queue pairs defined for the dedicated pool and a total number of NVM subsystems in the plurality of NVM subsystems of the NVMe target. At a first time, the target controller may determine a number of hosts associated with a given NVM subsystem and determine a first value of per-host I / O queue pairs for the given NVM subsystem based on the I / O queue pair boundary for the given NVM subsystem and the number of hosts associated with the given NVM subsystem. The target controller may determine an integer closest to the first value when the first value is not an integer. The target controller may then set the closest integer as the second value of the I / O queue pairs per host for the given NVM subsystem. In these cases, the target controller may selectively change the number of I / O queue pairs for the reserved pool by an amount based on the second value. The target controller may set a limit on the number of I / O queue pairs that may be used by each host (referred to herein as an I / O queue pair limit for a host) associated with the given NVM subsystem, where this limit is equal to the second value. In some examples, the target controller may determine whether to update the number of I / O queue pairs of the reserved pool at a second time based on the determination of a second number of hosts associated with the given NVM subsystem.Examples are described in more detail herein with reference to FIGS. 1-4. It should be understood that the description and drawings are merely illustrative of the principles of the present subject matter along with the examples described herein, and should not be construed as limiting the present subject matter. Although some examples are described herein with reference to a single NVM subsystem of an NVMe target, the examples for multiple NVM subsystems may be used in one or more NVMe targets. Moreover, any functionality described herein that is executed by a component (e.g., a target controller) of a system may be executed by at least one processing resource of the component executing instructions (stored on a machine readable storage medium) to execute the functionalities described herein. Various implementations of the present subject matter are described below using several examples.FIG. 1 shows an example system 100 in a network using the NVMe-oF specification. The system 100 may include an NVMe target 102 (hereinafter referred to as "target 102") and a plurality of hosts 104- 1, 104- 2,... 104-N (hereinafter collectively referred to as "hosts 104") to communicate with the target 102. The target 102 may include a plurality of NVM subsystems 106- 1, 106- 2,... 106-L (hereinafter collectively referred to as "NVM subsystems 106", or simply "subsystems 106") and a target controller 110. In an example, the target 102 may be an NVMe target with Ethernet switch, a JBOF target (i.e., an NVMe target without Ethernet switch), or a bridge target. In one example, the target 102 may include the subsystems 106 in a memory array and include the target controller 110 as a controller of the memory array. In other examples, the target 102 may include multiple memory arrays and a target controller for each of the memory arrays.The target controller 110 may facilitate the connection between the hosts 104 and the subsystems 106 in accordance with an assignment of the hosts 104 to the subsystems 106. The mapping may indicate which hosts 104 may access which subsystem(s) 106 of the target 102 to communicate with the subsystem(s) 106. Each of the subsystems 106 may be accessed by a corresponding subset of the hosts 104. For example, a first subset of the hosts 104 may be associated with one subsystem 106- 1, a second subset of the hosts 104 may be associated with another subsystem 106- 2, etc. In some examples, a particular host of the hosts 104 may be associated with two or more of the subsystems 106 (i.e., the particular host may belong to two or more subsets of the hosts 104).The target controller 110 may be a computing device, or instead a service or application executing on one or more computing devices. In an example, the target controller 110 may be located in a switch (e.g., embedded in a container), an external virtual machine (VM), or an NVM subsystem (e.g., subsystem 106- 1). As used herein, a "computing device" may be a server, a server cluster, a storage array, a computing device, a workstation, a desktop computer, a laptop, a switch, a router, or other processing device, or other equipment having a processing resource. In certain examples, the target controller 110 may be an I / O module of the target 102. In an example, the target controller 110 may be connected to, a portion of, connected to, and / or otherwise connected to the target 102. The target controller 110 may include a processing resource 112 communicatively coupled to a machine-readable storage medium 114 including instructions 116 that, when executed by the processing resource 112, cause the target controller 110 to perform certain actions and functionalities as described herein.In the examples described herein, for example, the processing resource 112 may include one or more processors included in a single computing device or distributed across multiple computing devices. As used herein, a "processor" may be at least a central processing unit (CPU), a semiconductor-based microprocessor, a graphics processing unit (GPU), a field programmable gate array (FPGA), or other hardware devices such as integrated circuits (IC), control logic, electronic circuits, or combinations thereof that include a number of electronic components. In the examples described herein, the processing resource 112 may fetch, decode, and execute the instructions 116 stored on the machine-readable storage medium 114 to perform the functions described with respect to the instructions 116 stored on the machine-readable storage medium 114. The machine-readable storage medium 114 may be located either on the computing device executing the instructions 116, or may be remote from the computing device but accessible for execution (e.g., via a computer network). In the examples depicted herein, the target controller 110 may be implemented by one or more machine readable storage media.A "machine-readable storage medium" may be any electronic, magnetic, optical, or other physical storage medium that contains or stores information such as executable instructions, data, and the like. For example, any machine-readable storage medium described herein may be a RAM, EEPROM, volatile memory, non-volatile memory, flash memory, a storage drive (e.g., a HDD, an SSD), any type of storage disk (e.g., a compact disc, a DVD, etc.), or the like, or a combination thereof. Moreover, any machine-readable storage medium described herein may be non-transitory.The target controller 110 may include a plurality of host ports (not shown in FIG. 1 ). Each of the hosts 104 may be connected to one of the host ports. Thus, each of the host ports may be connected to a particular host 104. A particular host 104 connected to one of the host ports may be enabled to communicate with one subsystem of the plurality of subsystems 106. In one example, the communication can take place via a transport protocol such as RDMA over converged Ethernet v1 (RoTMv1), RoTMv2, Internet wide area RDMA protocol (iWAR) or transmission control protocol / Internet protocol (TCP / IP) for the exchange of messages.The target 102 may support a total number of I / O queue pairs (e.g., between 128 and 512 I / O queue pairs) depending on its hardware resources and implementation. Accordingly, the target 102 may be able to assign the total number of I / O queue pairs to the hosts 104 that may be mapped to the target 102.In accordance with the examples described herein, the target controller 110 may perform multiple functions to manage the distribution of I / O queue pairs from the predetermined total number of I / O queue pairs in a balanced manner among the hosts 104 associated with the subsystems 106 of the target 102. In an example, the functions performed by the target controller 110 may be performed by the processing resource 112 executing the instructions 116 stored in the machine readable storage medium (e.g., a non-transitory machine readable storage medium) 114. The functions performed by the target controller 110 to manage the distribution of the total number of I / O queue pairs of the target 102 to the hosts 104 associated with the subsystems 106 of the target 102 will be described herein with reference to the flowcharts in FIGS. 2A-2B and 3A-3B. FIGS. 2A-2B collectively show a flow diagram of an example method for managing the distribution of the predetermined total number of I / O queue pairs of the target 102 among the hosts 104 associated with the subsystems 106 of the target 102 at a first time, and FIGS. 3A-3B collectively show a flow diagram of another example method for managing the distribution of the predetermined total number of I / O queue pairs of the target 102 among the hosts 104 associated with the subsystems 106 of the target 102 at a second time. In particular, FIGS. 2A-2B collectively show the flow diagram of the example method for assigning a limit to the number of I / O queue pairs that may be used by each of the hosts 104 associated with a given subsystem 106 of the target 102 at a first time, and FIGS. 3A-3B collectively show the flow diagram of the other example method for assigning a limit to the number of I / O queue pairs that may be used by each of the hosts 104 associated with the given subsystem 106 of the target 102 at a second time.Although the example methods of FIGS. 2A-2B and 3A-3B are described with reference to the target controller 110 of FIG. 1, other applications or devices suitable for executing the example methods of FIGS. 2A-2B and 3A-3B may also be used. Moreover, the implementation of the methods is not limited to such examples. Although the example methods of FIGS. 2A-2B and FIGS. 3A-3B individually show a particular order of execution of particular functions, the methods are not limited to this order. For example, the functions shown in the flowcharts sequentially may be performed in a different order, may be performed simultaneously or partially simultaneously, or combinations thereof.In FIG. 2A, a portion 200A of the flowchart includes a first set of method blocks 202- 212 of the example method for assigning a limit to the number of I / O queue pairs that may be used by each of the hosts 104 associated with the given subsystem 106 of the target 102 at the first time. In some examples, the first time may be representative of a first occurrence of the I / O queue pair boundary assignment for each host 104 associated with the given subsystem 106 at the first time. In an example, the method blocks 202- 212 may be performed by the target controller 110, in particular by the processing resource 112 of the target controller 110.In block 202, the target controller 110 may set a number of I / O queue pairs for a dedicated pool and a number of I / O queue pairs for a reserved pool based on the total number of I / O queue pairs. The dedicated pool may refer to a portion of the total number of target 102 I / O queue pairs used to establish a limit on the number of I / O queue pairs that may be used by a subset of hosts associated with each of the target 102 subsystems 106. The reserved pool may refer to another portion (e.g., the remaining portion) of the total number of I / O queue pairs that is used as a buffer to accommodate the I / O queue pair constraint for each host associated with a particular subsystem 106. In an example, the target controller 110 may distribute the total number of I / O queue pairs between the dedicated pool and the reserved pool based on instructions from an administrator (e.g., a user) and / or depending on the implementation of the target 102. In one example, the number of I / O queue pairs for the dedicated pool and the number of I / O queue pairs for the reserved pool may vary between a ratio of 1:5 and 5:1, for example. For example, the dedicated pool may be defined to include 70 percent and the reserved pool to include 30 percent of the total number of I / O queue pairs. In an example, the sum of the number of dedicated pool I / O queue pairs and the number of reserved pool I / O queue pairs is equal to the total number of target 102 I / O queue pairs. The number of I / O queue pairs defined for the dedicated pool may be referred to as dedicated I / O queue pairs, and the number of I / O queue pairs defined for a reserved pool may be referred to as reserved I / O queue pairs.In block 204, the target controller 110 may determine a threshold for the number of I / O queue pairs that may be used by each of the subsystems 106 (which may be referred to herein as "α1") based on the dedicated I / O queue pairs and the total number of subsystems 106. In some examples, the target controller 110 may distribute the dedicated I / O queue pairs equally among the subsystems 106 of the target 102 to determine the I / O queue pair boundary for each subsystem 106. In one example, the target controller 110 may calculate the I / O queue pair boundary for each subsystem using an example relationship represented by equation (1).In other examples, the target controller 110 may distribute the dedicated I / O queue pairs to the subsystems 106 depending on their respective resources. In such cases, the target controller 110 may set different limits on the number of I / O queue pairs that may be used by some of the subsystems 106.In block 206, the target controller 110 may determine a first number (e.g., "m") of hosts associated with the given subsystem 106 at the first time. The target controller 110 may determine the first number (m) of hosts in the subset of hosts 104 associated with the given subsystem 106, as defined by the association.In block 208, the target controller 110 may determine a first value of the per-host I / O queue pairs for the given subsystem 106 (which may be referred to herein as a first value and may be referred to as "β1") based on the I / O queue pair boundary for the given subsystem 106 (α1) and the first number (m) of hosts associated with the given subsystem 106. In one example, the target controller 110 may calculate the first value (β1) by dividing the total boundary for I / O queue pairs (α1) for the given subsystem 106 by the first number (m) of hosts associated with the given subsystem 106 using an example relationship represented by equation (2).The first value (β1) for the given subsystem 106 may or may not be an integer. In block 210, the target controller 110 may perform a check to determine whether the first value (β1) is an integer. If it is determined in block 210 that the first value (β1) is an integer ("YES" in block 210), the target controller 110 may assign a limit to number of I / O queue pairs that may be used by each host 104 (which may be referred to herein as an assigned I / O queue pair limit for a host and may be referred to as "z1") associated with the given subsystem 106, where the assigned I / O queue pair limit for each host (z1) is equal to the first value (β1).If it is determined in block 210 that the first value (β1) is not an integer ("NO" in block 210), the method may proceed to a place holder "A" that results in another portion of the flow chart shown in FIG. 2B. FIG. 2B shows a part 200B of the flow chart with a second set of method blocks 220- 230 of the example method. In one example, the operations in blocks 220- 230 may be performed by the target controller 110, particularly by the processing resource 112 of the target controller 110.Specifically, in block 210 (see FIG. 2A ), if it is determined that the first value (β1) is not an integer, the target controller 110 may determine an integer closest to the first value (β1) in block 220. An integer closest to the given value (i.e., a non-integer value) may refer to an integer closest to the given value. The integer closest to the given value may be an integer greater than or less than the given value. In some examples, an integer closest to the given value may be determined by rounding the given value up or down to the next integer. For example, the next integer to a value of 2.8 is 3. In some examples, the target controller 110 may be configured to determine the next integer to the first value (β1) that is greater or less than the first value (β1). For illustration, in the following description, the target controller 110 is described as determining an integer that is closest to the first value (β1) that is larger than the first value (β1), without limiting the scope of the present disclosure.In block 222, the target controller 110 may set the integer closest to the first value (β1) as the second value of the per-host I / O queue pairs for the given subsystem 106 (hereinafter referred to as "second value" and referred to as "β'1"). In one example, the second value (β'1) is an integer closest to the first value (β1) when the first value (β1) is not an integer. Further, in block 224, the target controller 110 may set the second value (β'1) as a threshold for I / O queue pairs for each host 104 associated with the given subsystem 106.Moreover, in some examples, at block 226, the target controller 110 may determine an updated limit on the number of I / O queue pairs that may be used for the given subsystem 106 (which may be referred to herein as an updated I / O queue pair limit for a subsystem and may be referred to as "α'1") based on the second value (β'1) and the number of hosts (m) associated with the given subsystem 106 at the first time. The target controller 110 may determine the updated I / O queue pair boundary for the given subsystem 106 by determining a boundary for the total number of I / O queue pairs that may be used by all hosts 104 associated with the given subsystem 106 based on the second value (β'1). In one example, the target controller 110 may calculate the updated threshold for I / O queue pairs (α'1) for the given subsystem 106 using an example relationship represented by equation (3).The updated I / O queue pair boundary (α'1) for the given subsystem 106 may be different (e.g., larger or smaller) from the previously determined I / O queue pair boundary (α1) for the given subsystem 106. In such cases, the target controller 110 may use the reserved I / O queue pairs of the reserved pool to assign the I / O queue pair boundary (z1) for each host 104 associated with the given subsystem 106 in accordance with the updated I / O queue pair boundary (α'1) for the given subsystem 106.Moreover, in block 228, the target controller 110 may selectively change the number of I / O queue pairs of the reserved pool by an amount (i.e., a first amount) based on the second value (β'1). As used herein, the term "selectively change" or "selectively change" may refer to an action in which the number of I / O queue pairs of the reserved pool is changed when the updated threshold for I / O queue pairs (α'1) for the given subsystem 106 differs from the previously determined I / O queue pairs (α1) for the given subsystem 106 and the number of I / O queue pairs is maintained (i.e., the number of I / O queue pairs of the reserved pool is maintained (i.e., no change) when the updated I / O queue pair boundary (α'1) for the given subsystem 106 is not different than the previously determined I / O queue pair boundary (α1) for the given subsystem 106.In some examples, the updated I / O queue pair boundary (α'1) for the given subsystem 106 is different than the previously determined I / O queue pair boundary (α1) for the given subsystem 106. In these examples, the target controller 110 may change the number of I / O queue pairs of the reserved pool by the first amount representing a difference between the updated I / O queue pair boundary (α'1) for the given subsystem 106 and the previously determined I / O queue pair boundary (α1) for the given subsystem 106. For example, the target controller 110 may calculate the first amount from an example relationship represented by equation (4).In other examples, the updated I / O queue pair boundary (α'1) for the given subsystem 106 is not different from the previously determined I / O queue pair boundary (α1) for the given subsystem 106. In these examples, the target controller 110 may not change the number of I / O queue pairs of the reserved pool.As described, the target controller 110 may change the number of I / O queue pairs of the reserved pool to accommodate the difference between the updated I / O queue pair boundary (α'1) for the given subsystem 106 and the previously determined I / O queue pair boundary (α1) for the given subsystem 106. In examples where the second value (β'1) is greater than the first value (β1) (i.e., (β'1>β1) at the first time, the updated I / O queue pair boundary (α'1) for the given subsystem 106 is greater than the previously determined I / O queue pair boundary (α1) for the given subsystem 106 (α'1>α1). In such examples, the target controller 110 may reduce the number of I / O queue pairs of the reserved pool by the first amount (α'1∼α1) to balance the difference between the updated I / O queue pair boundary (α'1) for the given subsystem 106 and the previously determined I / O queue pair boundary (α1) for the given subsystem 106. The target controller 110 may reduce the number of I / O queue pairs of the reserved pool by allowing a number of I / O queue pairs to be the first set (α'1∼α1) from the reserved pool to increase the I / O queue pair boundary to the second value (β'1) for each host 104 associated with the given subsystem 106. In this manner, the target controller 110 may equalize the I / O queue pair boundary for each host 104 associated with the given subsystem 106 by decreasing the number of I / O queue pairs in the reserved pool.In examples where the second value (β'1) is less than the first value (β1) (i.e., (β'1<β1), the updated I / O queue pair boundary (α'1) for the given subsystem 106 is less than the predetermined I / O queue pair boundary (α1) for the given subsystem 106 (α'1<α1). In such examples, the target controller 110 may increase the reserved pool I / O queue pairs by the first amount (α'1∼α1) to compensate for the difference between the updated I / O queue pair boundary (α'1) for the given subsystem 106 and the predetermined I / O queue pair boundary (α1) for the given subsystem 106. The target controller 110 may increase the number of I / O queue pairs of the reserved pool by adding to the reserved pool a number of I / O queue pairs corresponding to the first value (α'1∼α1) to lower the I / O queue pair boundary to the second value (β'1) for each host 104 associated with the given subsystem 106. In this manner, the target controller 110 may equalize the I / O queue pair boundary on each of the hosts 104 associated with the given subsystem 106, thereby increasing the number of I / O queue pairs in the reserved pool.In block 230, the target controller 110 may assign a limit to the number of I / O queue pairs that may be used by each of the hosts 104 associated with the given subsystem 106. In these cases, the assigned I / O queue pair boundary (z1) for each host 104 associated with the given subsystem 106 is equal to the second value (β'1) (which was set as the I / O queue pair boundary for each host 104 associated with the given subsystem 106 in block 224).In this way, in examples where the first value (β1) is an integer, each host 104 associated with the given subsystem 106 may be assigned the I / O queue pair boundary (z1) equal to the first value (β1). In examples where the first value (β1) is not an integer, each host 104 associated with the given subsystem 106 may be assigned the I / O queue pair boundary (z1) equal to the second value (β'1) by changing the number of I / O queue pairs of the reserved pool based on the second value (β'1).In some examples, the target controller 110 may determine an aggregated limit for the number of I / O queue pairs that may be used by all hosts 104 associated with the given subsystem 106 (which may be referred to herein as an aggregated I / O queue pair limit for the hosts and may be referred to as "μ1") at the first time. In examples where the assigned I / O queue pair boundary (z1) for each host 104 associated with the given subsystem 106 is equal to the first value (β1) (at block 212 of FIG. 2A ), the aggregated I / O queue pair boundary (μ1) for all hosts 104 associated with the given subsystem 106 is equal to the previously determined I / O queue pair boundary (α1) for the given subsystem 106. In examples where the updated I / O queue pair boundary (α'1) for the given subsystem 106 is determined (at block 226 of FIG. 2B ) based on the second value (β'1) and / or the assigned I / O queue pair boundary (z1) for each host 104 associated with the given subsystem 106 is equal to the second value (β'1) (at block 230 of FIG. 2B ), the aggregated I / O queue pair boundary (μ1) for all hosts 104 associated with the given subsystem 106 is equal to the updated I / O queue pair boundary (α'1) for the given subsystem 106.In some examples, the process blocks in FIGS. 2A-2B may be individually executed by the target controller 110 for the subsystems 106 of the target 102 to assign the respective I / O queue pair limits for the respective hosts 104 associated with the respective subsystems 106 at the first time. In some examples, the target controller 110 may determine the respective assigned I / O queue pair limits for the given host 104 with the respective subsystems 106 when a given host 104 associated with the given subsystem 106 is also associated with another subsystem 106 at the first time. The target controller 110 may then determine a total limit for the number of I / O queue pairs assigned to the given host 104 (which may be referred to herein as a total limit for assigned I / O queue pairs and may be referred to as "Z1") at the first time. The total assigned I / O queue pair boundary (Z1) for the given host 104 is the sum of the respective assigned I / O queue pair boundaries (z1) for the given host 104 with the respective subsystems 106 at the first time. For example, if the given host 104 is mapped to subsystems 106- 1 (e.g., a first subsystem S 1) and 106- 2 (e.g., a second subsystem S 2), the total assigned I / O queue pair boundary (Z 1) for the given host 104 may be calculated using an example relationship represented by equation (5).In some examples, the target controller 110 may maintain first information including an I / O queue pair boundary (α1) for each subsystem 106, a number of hosts (m) associated with each subsystem 106, a first value (β1), a second value (β'1), an updated I / O queue pair boundary (α'1) for each subsystem 106, a first amount (α'1 ~ α1), a change in the number of I / O queue pairs of the reserved pool (i.e., a change in the reserved pool), and an assigned I / O queue pair boundary (z1) for each host 104 associated with each subsystem 106 of the target 102 at the first time. As used herein, the term "a change in number" may represent a number. As an example, Table 1 shows the first information for the subsystem 106- 1 (e.g., the first subsystem S 1) and the subsystem 106- 2 (e.g., the second subsystem S 2) of the target 102 at the first time. Table 1: Example of First Information at First Time Table 1: Example of First Information at First TimeS1824----4S2832,66391-13In Table 1, the number of I / O queue pairs for each of subsystems S1 and S2 is limited to eight (i.e., α1=8). The number of hosts associated with subsystem S1 is two (i.e., m = 2). Accordingly, a first value for the subsystem S1 is four (β1=8 / 2=4), which is an integer. For subsystem S1, the assigned I / O queue pair boundary (z1) for each host associated with subsystem S1 is equal to the first value (β1=4).For subsystem S2, the number of hosts associated with subsystem S2 is three (i.e., m=3). Accordingly, a first value for subsystem S2 is 2.66 (β1=8 / 3=2.66) which is not an integer. Therefore, a second value for subsystem S2 is set to three (β'1 = 3). An updated I / O queue pair boundary for subsystem S2 is nine (α'1 = β'1 × m = 9) based on the second value (β'1 = 3). Since the updated I / O queue pair boundary (α'1) for subsystem S2 is greater than the I / O queue pair boundary (α1) for subsystem S2, the number of I / O queue pairs of the reserved pool is decreased by the first value (α'1 ~ α1 = 9-8 = 1). Accordingly, a change in the reserved pool is equal to one, which is indicated by "-1" in Table 1. For subsystem S2, the assigned boundary for I / O queue pairs (z1) for each host associated with subsystem S2 corresponds to the second value (β'1=3).In examples where a given host 104 associated with the given subsystem 106 requests a number of I / O queue pairs (e.g., "x" number of I / O queue pairs) that the given host 104 is to use at a first time or are to be granted to it, the target controller 110 may determine the assigned I / O queue pair boundary (z 1) for the given host 104 from the maintained first information (e.g., Table 1). Based on the assigned I / O queue pair boundary (z 1) for the given host 104, the target controller 110 may grant to the given host 104 a number of I / O queue pairs (which may be referred to herein as granted I / O queue pairs and may be referred to as "y 1") based on the assigned I / O queue pair boundary (z 1) for the given host 104. In some examples, if the requested number of I / O queue pairs (x) is greater than the assigned I / O queue pair boundary (z1) for the given host 104 (i.e., x>z1), the target controller 110 may grant (i.e., y1=z1) a number of I / O queue pairs to the given host with the given subsystem 106 equal to the assigned I / O queue pair boundary (z1). In some examples, the target controller 110 may grant the given host 104 a number of I / O queue pairs corresponding to the requested number of I / O queue pairs (x) (i.e., y1=x) when the requested number of I / O queue pairs (x) is less than the assigned I / O queue pair boundary (z1) for the given host 104 (i.e., x<z1).If the given host 104 is associated with two or more subsystems 106, the target controller 110 may assign a particular number of I / O queue pairs (y1) to the given host 104 with the respective subsystems 106 in a similar manner as described above. In such examples, the total number of I / O queue pairs granted to the given host 104 (referred to herein as total granted I / O queue pairs and referred to as "Y1") may correspond to the sum of the I / O queue pairs granted to the given host 104 with the respective subsystems 106. For example, if the given host 104 is mapped to subsystems 106- 1 (e.g., first subsystem S 1) and 106- 2 (e.g., second subsystem S 2), the sum of the I / O queue pairs (Y 1) granted to the given host 104 may be calculated using an example relationship represented by equation (6).In some examples, the target controller 110 may maintain second information including a total assigned I / O queue pair boundary (Z 1) and a total granted I / O queue pair (Y 1) for each of the hosts 104 associated with the subsystem(s) 106 of the target 102 at the first time. For example, Table 2 shows such second information, including an assigned total boundary for I / O queue pairs (Z 1) and a granted total boundary for I / O queue pairs (Y 1) for each of the hosts 104 (e.g., H 1, H 2,... HN) associated with the subsystem(s) 106 of the target 102 at the first time. Table 2: Example of second information at the first time Table 2: Example of second information at the first timeHostZ1Y1H 184H 21612......................................H N44In some examples, the target controller 110 may determine whether to update the number of I / O queue pairs of the reserved pool based on a second number of hosts associated with the given subsystem 106 at a second time. In FIG. 3A, a portion 300A of the flowchart includes a first set of method blocks 302- 326 of the other example method for assigning a limit to the number of I / O queue pairs for each of the hosts 104 associated with the given subsystem 106 of the target 102 at a second time. In one example, the second time may be later than the first time. At the second time, one or more new hosts 104 may be associated with the given subsystem 106, or one or more hosts 104 associated with the given subsystem 106 at the first time may be removed. In one example, the operations in blocks 302- 326 may be performed by the target controller 110, particularly by the processing resource 112 of the target controller 110. Method blocks 306- 326 may be performed after execution of method blocks 202- 212 of FIG. 2A and method blocks 220- 230 of FIG. 2B.FIG. 3A includes certain process blocks similar to one or more process blocks described in FIG. 2A or 2B, the details of which are not repeated here for brevity. For example, blocks 306, 308, 310, and 312 of FIG. 3A are similar to blocks 206, 208, 210, and 212 of FIG. 2A, and blocks 320, 322, 324, and 326 of FIG. 3A are similar to blocks 220, 222, 224, and 226 of FIG. 2B.In block 306, the target controller 110 may determine a second number (p) of hosts associated with the given subsystem 106 at the second time. The second number (p) of hosts may be different than the first number (m) of hosts associated with the given subsystem 106 at the first time. The second number (p) of hosts associated with the given subsystem 106 at the second time may be greater (when one or more new hosts are associated with the given subsystem 106) or less (when one or more hosts 104 are removed to access the given subsystem 106) than the first number (m) of hosts associated with the given subsystem 106 at the first time. Based on the second number (p) of hosts 104, the target controller 110 may perform similar operations as described with reference to one or more process blocks of FIGS. 2A-2B to assign a limit to the number of I / O queue pairs that may be used by each of the hosts 104 associated with the given subsystem 106 at the second time and to determine whether to update the number of I / O queue pairs of the reserved pool.For example, in block 308, the target controller 110 may determine a first value of per-host I / O queue pairs for the given subsystem 106 (which may be referred to herein as "β2") based on the I / O queue pair boundary (α1) for the given subsystem 106 and the second number of hosts (p) associated with the given subsystem 106 at the second time. In one example, the target controller 110 may calculate the first value (β1) by dividing the I / O queue pair boundary (α1) for the given subsystem 106 by the second number of hosts (p) associated with the given subsystem 106 using an example relationship represented by equation (7)Further, at block 310, the target controller 110 may perform a check to determine whether the first value (β2) is an integer. If it is determined that the first value (β2) is an integer ("YES" in block 310), the target controller 110 may assign a limit to the number of I / O queue pairs that may be used by each host 104 associated with the given subsystem 106 (which may be referred to herein as "z2" at the second time) in block 312. In these examples, the assigned I / O queue pair boundary (z2) for each host 104 associated with the given subsystem 106 may be equal to the first value (β2) at the second time.If it is determined in block 310 that the first value (β2) is not an integer ("NO" in block 310), the target controller 110 may determine an integer closest to the first value (β2) in block 320. In block 322, the target controller 110 may set the integer closest to the first value (β2) as a second value (which may be referred to herein as "β'2") at the second time. In block 324, the target controller 110 may set the second value (β'2) as a limit to the number of I / O queue pairs that may be used by each host 104 associated with the given subsystem 106 at the second time. In block 326, the target controller 110 may determine an updated threshold for I / O queue pairs for the given subsystem 106 (which may be referred to herein as "α'2" at the second time) based on the second value (β'2) and the number of hosts (p) associated with the given subsystem 106 at the second time. In one example, the target controller 110 may calculate the updated I / O queue pair boundary (α'2) for the given subsystem 106 using an example relationship represented by equation (8).The method may now proceed to a place holder "B" which leads to another part of the flow chart shown in FIG. 3B. FIG. 3B shows a portion 300B of the flowchart with a second set of method blocks 328- 338 of the other example method at the second time. In one example, the operations in blocks 328- 338 may be performed by the target controller 110, more specifically, by the processing resource 112 of the target controller 110.In block 328, the target controller 110 may determine an aggregated threshold for the number of I / O queue pairs (referred to herein as "μ2") that may be used by all hosts 104 associated with the given subsystem 106 at the second time. In examples where the target controller 110 assigns the I / O queue pair boundary (z2) for each host 104 associated with the given subsystem 106 equal to the first value (β2) (at block 314 of FIG. 3A ), the aggregated I / O queue pair boundary (μ2) for all hosts 104 associated with the given subsystem 106 is equal to the previously determined total I / O queue pair boundary (α1) for the given subsystem 106 (i.e., μ2= α1). In examples where the target controller 110 determines (at block 326 of FIG. 3A ) the updated I / O queue pair boundary (α'2) for the given subsystem 106 based on the second value (β'2), the aggregated I / O queue pair boundary (μ2) for all hosts 104 associated with the given subsystem 106 is equal to the updated I / O queue pair boundary (α'2) for the given subsystem 106 (i.e., μ2= α'2).In block 330, the target controller 110 may perform a check to determine whether the aggregated I / O queue pair boundary (μ2) for all hosts associated with the given subsystem 106 at the second time is different than the aggregated I / O queue pair boundary (μ1) for all hosts associated with the given subsystem 106 at the first time. In an example, the target controller 110 may compare the aggregated I / O queue pair boundary (μ2) at the second time with the aggregated I / O queue pair boundary (μ1) at the first time.If it is determined in block 332 that the aggregated I / O queue pair boundary (μ2) at the second time is different from the aggregated I / O queue pair boundary (μ1) at the first time ("YES" in block 332), the target controller 110 may change the number of I / O queue pairs of the reserved pool to balance the difference between the aggregated I / O queue pair boundary (μ2) for all hosts associated with the given subsystem 106 at the second time and the aggregated I / O queue pair boundary (μ1) for all hosts associated with the given subsystem 106 at the first time in block 334. For example, if the aggregated I / O queue pair boundary (μ2) at the second time is greater than the aggregated I / O queue pair boundary (μ1) at the first time, the target controller 110 may decrease the number of I / O queue pairs of the reserved pool by allowing a number of I / O queue pairs by a second amount (μ2-μ1) to increase the I / O queue pair boundary to the second value (β'2) for each host 104 associated with the given subsystem 106. In some examples, the aggregated number of allowed I / O queue pairs (μ2) at the second time is less than the aggregated number of allowed I / O queue pairs (μ1) at the first time, the target controller 110 may release a number of I / O queue pairs corresponding to the second amount (μ2-μ1) from one or more hosts 104 associated with the given subsystem 106 to reduce the I / O queue pair limit for each host 104 associated with the given subsystem 106 to the second value (β'2). In these cases, the target controller 110 may add or fill the enabled number of I / O queue pairs to the number of I / O queue pairs of the reserved pool. The method may then proceed to block 336.In block 336, the target controller 110 may set a limit on the number of I / O queue pairs that may be used by each host 104 associated with the given subsystem 106. In these examples, the assigned threshold for I / O queue pairs (z2) for each host 104 associated with the given subsystem 106 is equal to the second value (β'2).If it is determined in block 332 that the aggregated I / O queue pair boundary (μ2) for all hosts 104 associated with the given subsystem 106 at the second time is not different from the aggregated I / O queue pair boundary (μ1) for all hosts 104 associated with the given subsystem 106 at the first time ("NO" in block 332), the target controller 110 cannot change the number of I / O queue pairs of the reserved pool in block 338. In such examples, each host 104 associated with the given subsystem 106 may be permitted to use the number of I / O queue pairs corresponding to the first value (β2) assigned to each host 104 in block 314 of FIG. 3A.In some examples, the sets of method blocks in FIGS. 3A-3B may be individually executed by the target controller 110 for the subsystems 106 present in the target 102 to assign the respective I / O queue pair boundaries for the respective hosts 104 associated with the respective subsystems 106 at the second time. The target controller 110 may then determine an entire assigned I / O queue pair boundary (Z 2) for the given host 104 at the second time. The total assigned I / O queue pair boundary (Z2) for the given host 104 may be the sum of the respective assigned I / O queue pair boundary (z2) for the given host 104 with the respective subsystems 106 at the second time. For example, if the given host 104 is mapped to subsystems 106- 1 (e.g., first subsystem S 1) and 106- 2 (e.g., second subsystem S 2), the entire assigned I / O queue pair boundary (Z 2) for the given host 104 may be calculated using an example relationship represented by equation (9).The target controller 110 may maintain first information at the second time that is similar to the first information (e.g., Table 1) at the first time. In one example, the target controller 110 may update the first information at the second time with additional information about the second amount (μ2-μ1) instead of the first amount (α'1-α1). The updated first information may include a total boundary for I / O queue pairs (α1) for each subsystem 106, a number of hosts (p) associated with each subsystem 106, a first value (β2), a second value (β'2), an updated boundary for I / O queue pairs (α'2) for each subsystem, an amount (i.e., a second amount (μ2-μ1), an assigned I / O queue pair boundary (z2) for each host 104 associated with each subsystem 106, and a change in the number of I / O queue pairs of the reserved pool at the second time. For example, Table 1 is updated as Updated Table 1 that contains the updated first information at the second time for subsystems S 1 and S 2 of the target 102 when a new host is associated with each of subsystems S 1 and S 2 compared to the number of hosts associated with each of subsystems S 1 and S 2 at the first time. Updated Table 1: Example of First Information at Second Time Updated Table 1: Example of First Information at Second TimeS1832,66391-13S284281+12Referring to Table 1 and Updated Table 1, if S 1 at the second time is associated with three hosts (p=3 in Updated Table 1) as compared to two hosts (m=2 in Table 1) at the first time, the target controller 110 at the second time assigns each of the three hosts a boundary of three I / O queue pairs (z2=3 in Updated Table 1) as compared to a boundary of four I / O queue pairs (z1=4 in Table 1) associated with each of the two hosts at the first time. In this case, since the boundary for aggregated I / O queue pairs for three hosts (μ2) at the second time is nine (μ2=α'2=9 according to the updated Table 1), while the boundary for aggregated I / O queue pairs for two hosts (μ1) at the first time is eight (μ1α1=8 according to Table 1), an I / O queue pair from the reserved pool (change in reserved pool=-1) is reduced to assign an I / O queue pair (μ2∼μ1=9-8=1) to the new host.When S2 is mapped to four hosts (p=4 in the updated table 1) at the second time as compared to three hosts (m=3 in table 1) at the first time, the target controller 110 assigns a boundary of two I / O queue pairs (z2=2 in the updated table 1) to each of the four hosts at the second time as compared to a boundary of three I / O queue pairs (z1=3 in table 1) for each of the three hosts at the first time. In this case, since the boundary for aggregated I / O queue pairs for four hosts at the second time is eight (i.e., μ2= α1=8 according to Updated Table 1), while the boundary for aggregated I / O queue pairs for three hosts at the first time is nine (μ1= α'1=9 according to Table 1), an I / O queue pair (μ2∼ μ1= 9-8=1) is added to the reserved pool (change of reserved pool=11).In some examples, when a given host 104 is associated with two or more subsystems 106, the target controller 110 may individually grant a corresponding number of I / O queue pairs to the given host 104 with the respective subsystems 106 in a similar manner as described above with respect to a given host 104 at the first time. In such examples, at the second time, the given host 104 may have a total number of granted I / O queue pairs (referred to herein as "Y2") that corresponds to the sum of the respective number of granted I / O queue pairs (referred to herein as "y2") with the respective subsystems 106. For example, if the given host 104 is associated with subsystems 106- 1 (e.g., first subsystem S 1) and 106- 2 (e.g., second subsystem S 2) at the second time, the total number of I / O queue pairs (Y 2) granted to the given host 104 may be calculated from an example relationship represented by equation (10).In some examples, the target controller 110 may then update the second information (e.g., Table 2) at the second time. The updated second information at the second time may include a total assigned I / O queue pair boundary (Z 2) for each of the hosts 104 and a total number of granted I / O queue pairs (Y 2) for each of the hosts 104 associated with the subsystem(s) 106 of the target 102 at the second time. For example, Table 2 is updated as Table 2 to maintain the total assigned I / O queue pair boundary (Z 2) at the second time and the total number of granted I / O queue pairs (Y 2) at the second time for each of the hosts (H 1, H 2,... HN) mapped to the subsystem(s) 106 of the target 102 at the second time. Updated Table 2: Example of second information at the second time Updated Table 2: Example of second information at the second timeHostZ2Y2H 166H 21210......................................H N44In some examples, the target controller 110 may dynamically update the first information (e.g., Table 1) and the second information (e.g., Table 2). As used herein, the term "dynamically update" may refer to updating the first information and the second information based on a change (e.g., addition or removal) in the number of hosts associated with the subsystem(s) 106 at a particular time.FIG. 4 is a block diagram of a computer system 400 including a processing resource 402 and a machine readable storage medium 404 encoded with example instructions to manage the distribution of a number of I / O queue pairs of an NVMe target (e.g., the target 102 of FIG. 1 ) among a plurality of hosts connected to the NVMe target, in accordance with one example. The machine readable storage medium 404 may be non-transitory and is alternatively referred to as a non-transitory machine readable storage medium 404. As described in detail herein, machine-readable storage medium 404 may be encoded with executable instructions 406, 408, 410, 412, 414, 416, and 418 (hereinafter collectively referred to as instructions 406- 418) for performing one or more process blocks of flowcharts 200A and 200B of FIGS. 2A-2B. Although not shown, in some examples, the machine readable storage medium 404 may be encoded with certain additional executable instructions to perform one or more of the method blocks of flowcharts 300A and 300B of FIGS. 3A-3B and / or other operations performed by the target controller 110, without limiting the scope of the present disclosure.In some examples, the machine readable storage medium 404 may be accessed by the processing resource 402. In some examples, the computer system 400 may be included in (e.g., as part of) a target controller (e.g., the target controller 110 of FIG. 1 ). In some examples, processing resource 402 may represent an example of processing resource 112 of target controller 110. Further, the machine readable storage medium 404 may represent an example of the machine readable storage medium 114 of the target controller 110. In some examples, the processing resource 402 may fetch, decode, and execute the instructions 406- 418 stored in the machine readable storage medium 404 to assign a number of I / O queue pairs to each host 104 connected to the target 102.The instructions 406, when executed by the processing resource 402, may cause the processing resource 402 to define a number of I / O queue pairs for a dedicated pool and a number of I / O queue pairs for a reserved pool based on a total number of I / O queue pairs of the target 102. Further, the instructions 408, when executed by the processing resource 402, may cause the processing resource 402 to determine a limit on the number of I / O queue pairs that may be used by each of the subsystems (α1) based on the dedicated I / O queue pairs and a total number of the subsystems 106. The instructions 410, when executed by the processing resource 402, may cause the processing resource 402 to determine a first number (m) of hosts associated with a particular subsystem 106 at a first time. Further, the instructions 412, when executed by the processing resource 402, may cause the processing resource 402 to determine a first value (β1) of per-host I / O queue pairs for the given subsystem 106 based on the I / O queue pair boundary for the given subsystem 106 (α1) and the first number (m) of hosts associated with the given subsystem 106. Moreover, the instructions 414, when executed by the processing resource 402, may cause the processing resource 402 to determine an integer closest to the first value (β1) if the first value (β1) is not an integer. Moreover, instructions 416, when executed by processing resource 402, may cause processing resource 402 to set the integer closest to the first value (β1) as the second value (β'1). Moreover, instructions 418, when executed by processing resource 402, may cause processing resource 402 to selectively change the number of I / O queue pairs of the reserved pool by an amount based on the second value (β'1).In the examples described herein, functions described as being executed by "instructions" may be understood as functions executable by those instructions when executed by a processing resource. In other examples, functions described with respect to instructions may be implemented by one or more modules, which may be any combination of hardware and programming to implement the functions of the module / modules.The foregoing description of the various examples is provided for purposes of illustration and description. The foregoing description is not intended to be exhaustive or to limit the disclosed examples, and modifications and variations are possible in light of the above teachings or may be acquired from the practice of the various examples. The examples discussed herein have been chosen and described in order to explain the principles and type of the various examples of the present disclosure and their practical application, to enable one skilled in the art to utilize the present disclosure in various examples and with various modifications as are suited to the particular use contemplated. The features of the examples described herein may be combined in all possible combinations of methods, apparatuses, modules, systems, and computer program products, except combinations where at least some of these features are mutually exclusive.

Claims

A method (200A, 200B) comprising: in a network with a non-volatile memory (NVM) Express (NVMe) target (102) capable of allocating a total number of I / O queue pairs and using NVMe over Fabrics (NVMe-oF), defining (204), by a target controller (110; 400) of the NVMe target (102), a number of I / O queue pairs for a dedicated pool, and a number of I / O queue pairs for a reserved pool based on the total number of I / O queue pairs of the NVMe target (102), wherein the dedicated pool refers to a portion of the total number of I / O queue pairs of the NVMe target (102), and the reserved pool refers to a remaining portion of the total number of I / O queue pairs; determining (206), by the target controller (110; 400) a first boundary for the number of I / O queue pairs that may be used by each of a plurality of NVM subsystems (106) of the NVMe target (102) based on the number of I / O queue pairs defined for the dedicated pool and a total number of NVM subsystems (106) in the plurality of NVM subsystems (106) of the NVMe target (102), wherein the target controller (110; 400) distributes the dedicated I / O queue pairs equally among the NVM subsystems of the NVMe target (102) to determine the first boundary; At a first time, determining (206), by the target controller (110; 400), a first number (m) of hosts (104) associated with a given NVM subsystem (106); determining (208), by the target controller (110; 400), a first value of per-host I / O queue pairs for the given NVM subsystem (106) based on the first limit for the number of I / O queue pairs allowed to be used by the given NVM subsystem (106) and the number of hosts (104) associated with the given NVM subsystem (106) at the first time; determining (220), by the target controller (110; 400), a closest integer to the first value, if the first value is not an integer; setting (222), by the target controller (110; 400), the closest integer as a second value of per-host I / O queue pairs for the given NVM subsystem (106); determining (226), by the target controller (110; 400), an updated limit for the number of I / O queue pairs that may be used by the given NVM subsystem as a limit for the total number of I / O queue pairs that may be used by all hosts associated with the given NVM subsystem based on the second value; and selectively changing (228), by the target controller (110; 400); 400) the number of I / O queue pairs for the reserved pool by an amount based on the second value, the amount representing a difference between the updated limit for the number of I / O queue pairs for the given NVM subsystem and the first limit for the number of I / O queue pairs.The method (200A, 200B) of claim 1, further comprising: assigning a limit to the number of I / O queue pairs that may be used by each of the hosts (104) associated with the given NVM subsystem (106), wherein the limit equals the second value.The method (200A, 200B) of claim 2, further comprising: responsive to receiving a request from a given host of the hosts (104) associated with the given NVM subsystem (106) for a number of I / O queue pairs to be granted to the given host, granting a number of I / O queue pairs for the given host based on the assigned limit for the number of I / O queue pairs for the given host.The method (200A, 200B) of claim 1, further comprising: determining (226) the updated limit for the number of I / O queue pairs that may be used by the given NVM subsystem (106) based on the second value and the number of hosts (104) associated with the given NVM subsystem (106) at the first time.The method (200A, 200B) of claim 1, wherein selectively changing the number of I / O queue pairs for the reserved pool comprises decreasing the number of I / O queue pairs for the reserved pool if the second value is greater than the first value.The method (200A, 200B) of claim 1, wherein selectively changing the number of I / O queue pairs for the reserved pool comprises increasing the number of I / O queue pairs for the reserved pool if the second value is less than the first value.The method (200A, 200B) of claim 1, further comprising: determining, by the target controller (110; 400), whether to update the number of I / O queue pairs for the reserved pools based on determining that a second number (p) of hosts (104) is associated with the given NVM subsystem (106) at a second time.A target controller (110; 400) of a non-transitory storage (NVM) Express (NVMe) target (102) capable of allocating a total number of I / O queue pairs, comprising: at least one processing resource (112; 402); and a non-transitory machine readable storage medium (114; 404) comprising instructions derived from the at least one processing resource (112; 402) to: define (406) a number of I / O queue pairs for a dedicated pool and a number of I / O queue pairs for a reserved pool based on the total number of I / O queue pairs of the NVMe target (102), wherein the dedicated pool refers to a portion of the total number of I / O queue pairs of the NVMe target (102), and the reserved pool refers to a remaining portion of the total number of I / O queue pairs; determining (408) a first limit for the number of I / O queue pairs that may be used by each of a plurality of NVM subsystems (106) of the NVMe target (102) based on the number of I / O queue pairs defined for the dedicated pool and a total number of NVM subsystems (106) in the plurality of NVM subsystems (106) of the NVMe target (102), wherein the dedicated I / O queue pairs are distributed evenly among the NVM subsystems of the NVMe target to determine the first limit; determining (410) a first number (m) of hosts (104) associated with a given NVM subsystem (106); determining (412) a first value of per-host I / O queue pairs for the given NVM subsystem (106) based on the first limit for the number of I / O queue pairs permitted to be used by the given NVM subsystem (106) and the number of hosts (104) associated with the given NVM subsystem (106) at the first time; determining (414) an integer closest to the first value if the first value is not an integer; setting (416) the closest integer as a second value of per host I / O queue pairs for the given NVM subsystem (106); determining an updated limit on the number of I / O queue pairs that may be used by the given NVM subsystem as a limit on the total number of I / O queue pairs that may be used by all hosts associated with the given NVM subsystem based on the second value; and selectively changing (418) the number of I / O queue pairs for the reserved pool by an amount based on the second value, the amount representing a difference between the updated limit for the number of I / O queue pairs for the given NVM subsystem and the first limit for the number of I / O queue pairs.The target controller (110; 400) of claim 8, wherein the instructions comprise instructions executable by the at least one processing resource (112; 402) to: assign a limit to the number of I / O queue pairs that may be used by each of the hosts (104) associated with the given NVM subsystem (106), the limit equal to the second value.The target controller (110; 400) of claim 9, wherein the instructions comprise instructions executable by the at least one processing resource (112; 402) to: responsive to receiving a request from a given host from the hosts (104) associated with the given NVM subsystem (106) for a number of I / O queue pairs to be granted to the given host, grant to the given host a number of I / O queue pairs based on the assigned limit for the number of I / O queue pairs for the given host.The target controller (110; 400) of claim 8, wherein the instructions comprise instructions executable by the at least one processing resource (112; 402) to: determine the updated limit for the number of I / O queue pairs permitted to be used by the given NVM subsystem (106) based on the second value and the number of hosts (104) associated with the given NVM subsystem (106) at the first time.The target controller (110; 400) of claim 8, wherein the instructions for selectively changing the number of I / O queue pairs of the reserved pool comprise instructions for decreasing the number of I / O queue pairs of the reserved pool when the second value is greater than the first value.The target controller of claim 8, wherein the instructions to selectively change the number of I / O queue pairs of the reserved pool comprise instructions to increase the number of I / O queue pairs of the reserved pool when the second value is less than the first value.The target controller (110; 400) of claim 8, wherein the instructions comprise instructions executable by the at least one processing resource (112; 402) to: determine whether to update the number of I / O queue pairs of the reserved pool based on determining that a second number (p) of hosts (104) is associated with the given NVM subsystem (106) at a second time.A non-transitory machine readable storage medium (114; 404) including instructions executable by at least one processing resource (112; 402) to: define (406) a number of I / O queue pairs for a dedicated pool and a number of I / O queue pairs for a reserved pool based on a total number of I / O queue pairs of an NVMe (102) non-volatile memory (NVM) express (406), wherein the dedicated pool refers to a portion of the total number of I / O queue pairs of the NVMe (102), and the reserved pool refers to a remaining portion of the total number of I / O queue pairs; determining (408) a first limit for the number of I / O queue pairs that may be used by each of a plurality of NVM subsystems (106) of the NVMe target (102) based on a number of I / O queue pairs defined for the dedicated pool and a total number of NVM subsystems in the plurality of NVM subsystems (106) of the NVMe target (102), wherein the dedicated I / O queue pairs are distributed evenly among the NVM subsystems of the NVMe target (102) to determine the first limit; determining (410) a first number (m) of hosts (104) associated with a given NVM subsystem (106); determining (412) a first value of per-host I / O queue pairs for the given NVM subsystem (106) based on the first limit for the number of I / O queue pairs permitted to be used by the given NVM subsystem (106) and the number of hosts (104) associated with the given NVM subsystem (106) at the first time; determining (414) an integer closest to the first value if the first value is not an integer; setting (416) the closest integer as a second value of per host I / O queue pairs for the given NVM subsystem (106); determining an updated limit on the number of I / O queue pairs that may be used by the given NVM subsystem as a limit on the total number of I / O queue pairs that may be used by all hosts associated with the given NVM subsystem based on the second value; and selectively changing (418) the number of I / O queue pairs for the reserved pool by an amount based on the second value, the amount representing a difference between the updated limit for the number of I / O queue pairs for the given NVM subsystem and the first limit for the number of I / O queue pairs.The non-transitory machine readable storage medium (114; 404) of claim 15, wherein the instructions comprise instructions to assign a limit to the number of I / O queue pairs that may be used by each of the hosts (104) associated with the given NVM subsystem (106), the limit equal to the second value.The non-transitory machine readable storage medium (114; 404) of claim 15, wherein the instructions comprise instructions to determine whether to update the number of I / O queue pairs of the reserved pool based on determining that a second number (p) of hosts (104) at a second time is associated with the given NVM subsystem (106).The non-transitory machine-readable storage medium (114; 404) of claim 15, wherein the instructions for selectively changing comprise instructions for changing to: decrease the number of I / O queue pairs of the reserved pool if the second value is greater than the first value; or increase the number of I / O queue pairs of the reserved pool if the second value is less than the first value.

Images