Method and system for managing input / output queue pairs - Patents.com

Abstract

To provide systems and methods for managing input-output queue pairs that adjust queue pair numbers and queue pair depths.SOLUTION: A method for managing input-output queue pairs includes calculating a system utilization ratio; determining that the system utilization ratio exceeds a first threshold utilization during a time period exceeding a first threshold length so that adding a new queue pair is expected to improve system performance; and adding the new queue pair; where the system utilization ratio is a ratio of a service rate to an arrival rate of input-output requests.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a data storage system using NVMeoF (Non-Volatile Memory express over Fabrics), and more particularly to a method and system for managing input / output queue pairs to improve the performance of such a data storage system. [Background technology]

[0002] Conventional NVMeoF systems exhibit degraded performance in some situations. This is caused by one or both of the following problems: First, some NVMeoF implementations (i) Maintain an extra idle queue pair, or (ii) Wasting system resources by not maintaining enough queue pairs and underutilizing the system. Second, some NVMeoF implementations do not have an adequate flow control scheme to prevent delays in queue pair service rates due to high I / O pressure (e.g., a lot of queuing I / O work in a queue pair).

[0003] Therefore, there is a need to develop a system and method for improving the performance of NVMeoF data storage systems. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-18514 A Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention has been made in consideration of the problems in the conventional NVMeoF systems described above, and an object of the present invention is to provide a method and system for managing input / output queue pairs that adjusts the number of queue pairs and the depth of the queue pairs. [Means for solving the problem]

[0006] A method for managing an input / output queue pair according to the present invention, which has been made to achieve the above object, comprises the steps of: A system having a processing circuit and one or more permanent storage devices. 1. A method for managing an input / output queue pair, comprising: wherein the one or more I / O queue pairs are for storing I / O requests to be sent to the one or more persistent storage devices and command completions received from the one or more persistent storage devices, and wherein the processing circuitry: calculating system utilization; Here, the system utilization rate is the ratio of the arrival rate of input / output requests to the service rate. , the input / output request arrival rate is an input / output request arrival rate to a centralized queue, the service rate is a service rate of the centralized queue, and by the processing circuitry: During a first time period that exceeds a first long threshold, the system utilization Overuse threshold The first usage threshold has been exceeded and a new Input / Output Queue Pair Adding , the new Input / Output The change in the service rate according to the queue pair is estimated, so that the performance of the system is improved. but improvement death , the new Input / Output Adding queue pairs increases the service rate. 、 In response to the presumption that 、 The new Input / Output Steps to determine if a queue pair should be added and The service rate is Per I / O request The first length threshold corresponds to an average processing delay and is greater than zero.

[0007] It is preferable that the method further includes a step of determining, by the processing circuitry, that the system utilization remains below a second utilization threshold, which is an under-utilization threshold, during a second time period that exceeds the first long threshold, and that removing the input / output queue pair is expected to improve system performance, and a step of removing the input / output queue pair by the processing circuitry. Preferably, the method further comprises the steps of: determining, by the processing circuitry, that the system utilization is less than the first utilization threshold and greater than a third utilization threshold; and adjusting, by the processing circuitry, a lower queue height threshold such that the service rate is equal to a target third utilization threshold. It is preferable that the method further includes a step of determining, by the processing circuitry, that a submission queue of a first input / output queue pair of the input / output queue pair is unlocked so that a centralized queue stores the input / output request and the height of the submission queue is less than an upper queue height threshold, and a step of enqueuing, by the processing circuitry, the input / output request. Preferably, the method further comprises the steps of: determining, by the processing circuitry, that a current height of the transmit queue is equal to the upper limit queue height threshold; and locking, by the processing circuitry, the transmit queue. It is preferable that the method further includes the steps of: determining, by the processing circuitry, that a transmit queue of a first input / output queue pair of the input / output queue pair is locked; determining, by the processing circuitry, that a current height of the transmit queue does not exceed the lower queue height threshold; and unlocking, by the processing circuitry, the transmit queue. Said service rate The above Lower Queue Height Threshold By a function of , a model of the average processing delay per I / O request. The numerical value that minimizes the objective function is It is preferable that it be required. The model is the service rate for Preferably, it is a polynomial fit of the historical average processing delay per I / O request as a function of said lower queue height threshold. Preferably, the method further comprises the steps of: determining, by the processing circuitry, that a time interval having a length exceeding a second length threshold has elapsed since the most recent use of numerical minimization to determine and set an initial number of input / output queue pairs; performing, by the processing circuitry, a numerical minimization of a model of average processing delay per input / output request as a function of the initial number of input / output queue pairs to calculate a second number of input / output queue pairs to be used during operation; and adding, by the processing circuitry, zero or more new input / output queue pairs or removing zero or more input / output queue pairs such that the initial number of input / output queue pairs is the same as the calculated second number of input / output queue pairs to be used during operation.

[0008] In order to achieve the above object, the present invention provides a system comprising a processing circuit and one or more persistent storage devices, the processing circuit instantiating one or more I / O queue pairs for storing I / O requests sent to the one or more persistent storage devices and command completions received from the one or more persistent storage devices; The ratio of I / O request arrival rate to service rate Calculate the system utilization rate, , where: the arrival rate of the I / O requests is an arrival rate of the I / O requests to a centralized queue, the service rate is a service rate of the centralized queue, during a first time period exceeding a first length threshold, the system utilization exceeds a first utilization threshold that is an overutilization threshold, and adding a new I / O queue pair is configured to improve system performance because a change in the service rate is estimated according to the new I / O queue pair, and to determine the addition of the new I / O queue pair in response to an estimation that adding the new I / O queue pair will increase the service rate, the service rate corresponds to an average processing delay per I / O request, and the first length threshold is greater than 0.

[0009] Preferably, the processing circuitry is further configured to determine that during a second time period exceeding the first long threshold, the system utilization remains below a second utilization threshold, which is an underutilization threshold, and removal of the input / output queue pair is expected to improve system performance, and to remove the input / output queue pair. Preferably, the processing circuitry is further configured to determine that the system utilization is less than the first utilization threshold and greater than a third utilization threshold, and to adjust a lower queue height threshold such that the service rate is equal to a target third utilization threshold. Preferably, the processing circuitry is further configured to determine that a transmit queue of a first one of the input / output queue pairs is unlocked and a centralized queue is storing the input / output request and the transmit queue height is less than an upper queue height threshold, and to enqueue the input / output request. Preferably, the processing circuitry is further configured to determine that the current height of the transmit queue is equal to the upper queue height threshold and to lock the transmit queue. Preferably, the processing circuitry is further configured to determine that a transmit queue of a first one of the input / output queue pairs is locked, determine that the current height of the transmit queue does not exceed the lower queue height threshold, and unlock the transmit queue. Said service rate The above Lower Queue Height Threshold By a function of , a model of the average processing delay per I / O request. The numerical value that minimizes the objective function is It is preferable that it be required. The model is the service rate for Preferably it is a polynomial fit of the historical average processing delay per I / O request as a function of said lower queue height threshold. Preferably, the processing circuitry is further configured to determine that a time interval having a length exceeding a second length threshold has elapsed since the most recent use of numerical minimization to determine and set the initial number of input / output queue pairs, perform a numerical minimization of a model of average processing delay per input / output request as a function of the initial number of input / output queue pairs to calculate a second number of input / output queue pairs to be used during operation, and add zero or more new input / output queue pairs or remove zero or more input / output queue pairs such that the initial number of input / output queue pairs is identical to the calculated second number of input / output queue pairs to be used during operation.

[0010] The present invention also provides a system for achieving the above object, comprising a processing means and one or more persistent storage devices, the processing means instantiating one or more I / O queue pairs for storing I / O requests sent to the one or more persistent storage devices and command completions received from the one or more persistent storage devices; The ratio of I / O request arrival rate to service rate Calculate the system utilization rate, , where: The arrival rate of input / output requests is the arrival rate of input / output requests to a centralized queue, the service rate is the service rate of the centralized queue, and during a first time period exceeding a first length threshold, the system utilization exceeds a first utilization threshold which is an overutilization threshold, and adding a new input / output queue pair improves system performance because a change in the service rate is estimated in accordance with the new input / output queue pair, and the addition of the new input / output queue pair is determined in response to an estimation that the service rate will increase by adding the new input / output queue pair, the service rate corresponds to an average processing delay per input / output request, and the first length threshold is greater than 0.

[0011] The processing means detects that during a second time period that the system utilization exceeds the first long threshold, The second utilization threshold, which is an underutilization threshold kept below Input / Outputdetermining that removing the queue pair is expected to improve system performance; Input / Output Preferably, it is further configured to delete the queue pair. Effect of the Invention

[0012] The method and system for managing I / O queue pairs according to the present invention has the advantage of optimizing the delay time of I / O requests by adjusting the number of queue pairs and the queue pair depth. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating the component architecture of a performance optimizer for a system-based NVMeoF system employing NUMA (Non-Uniform Memory Access) according to an embodiment of the present invention. [Figure 2A] 1 is a table showing a summary of terms used in the embodiments of the present invention. [Figure 2B] A diagram for explaining the architecture of the NVMeoF protocol according to one embodiment of the present invention. [Figure 2C] FIG. 2 is a diagram for explaining a queuing model of a queue pair in a NIC (Network Interface Card) port according to an embodiment of the present invention. [Figure 2D] 1 is a table summarizing symbols and descriptions used in bottleneck-based service rate analysis and modeling according to an embodiment of the present invention. [Figure 2E] 1 is a graph illustrating an exemplary 3D surface used to find the maximum value of the service rate as a function of a watermark variable and a number of queue pair variables in accordance with one embodiment of the present invention. [Figure 3A] A diagram to explain a queue pair model of a client stack and a target stack according to one embodiment of the present invention. [Figure 3B]1 is a table illustrating a comparison of two methods (first method and second method) for adjusting the number of queue pairs and the queue pair depth according to an embodiment of the present invention. [Figure 3C] 1 is a table showing a comparison of two approaches (a first approach and a second approach) for adjusting the number of queue pairs in a first method according to an embodiment of the present invention. [Figure 3D] FIG. 2 is an exemplary diagram for explaining operational aspects of a switching optimization approach based on different system utilization cases according to an embodiment of the present invention. [Figure 3E] 1 is a table illustrating thresholds and target utilizations used to adjust the number of queue pairs and queue pair depth, and calculation of system utilization, according to one embodiment of the present invention. [Figure 3F] 1 is a table illustrating cases and corresponding actions used to adjust the number of queue pairs and queue pair depth according to one embodiment of the present invention. [Figure 3G] FIG. 2 is a workflow diagram of two components used to adjust the number of queue pairs according to one embodiment of the present invention. [Figure 3H] 1 is a table illustrating sub-components and functions of a first component used to adjust the number of queue pairs according to one embodiment of the present invention. [Figure 3I] A diagram illustrating the relationships between sub-components of a first component used to adjust the number of queue pairs according to one embodiment of the present invention. [Figure 3J] 1 is a graph illustrating an example objective function used in a regression-based offensive approach according to an embodiment of the present invention. [Figure 4A] FIG. 2 is an exemplary diagram illustrating a queuing model used to adjust the depth of a queue pair according to an embodiment of the present invention. [Figure 4B] 1 is a table illustrating terms and corresponding descriptions associated with adjusting queue pair depth according to an embodiment of the present invention. [Figure 4C]FIG. 2 illustrates an example of the relationship between queued input / output (I / O) height and service rate used in the context of adjusting queue pair depth in accordance with an embodiment of the present invention. [Figure 4D] FIG. 2 is a diagram to illustrate further illustration of the relationship between queued input / output (I / O) height and service rate used in the context of adjusting queue pair depth in accordance with an embodiment of the present invention. [Figure 4E] FIG. 2 is a diagram illustrating conditions and corresponding operations in watermark-based optimization for adjusting queue pair depth according to an embodiment of the present invention. [Figure 4F] FIG. 2 illustrates some example procedures for watermark-based optimization for adjusting queue pair depth according to one embodiment of the present invention. [Figure 4G] 4 is a flowchart illustrating an exemplary workflow for watermark-based optimization for adjusting queue pair depth in accordance with an embodiment of the present invention. [Figure 4H] 1 is a table illustrating terms and corresponding descriptions for queue pair depth adjustment according to an embodiment of the present invention. [Figure 4I] FIG. 2 is a diagram illustrating exemplary average queuing I / O request heights during different time periods for queue pair depth adjustment according to one embodiment of the present invention. [Figure 4J] FIG. 4 is an exemplary diagram of a three-dimensional graph used by the system of the present invention to select a low watermark in accordance with one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Next, a specific example of an embodiment for implementing a method and system for managing input / output queue pairs according to the present invention will be described with reference to the drawings.

[0015] The present invention is intended as a description of exemplary embodiments of systems and methods for improving performance of NVMeoF data storage systems, and is not intended to represent the only manner in which the invention may be constructed or utilized. The present description explains features of the present invention with reference to illustrated embodiments. It is to be understood, however, that the same or equivalent functions and structures may be accomplished by different embodiments that are also intended to be encompassed within the scope of the invention. As in other parts of the present invention, like component numbers are intended to indicate like components or features.

[0016] <<1. Introduction>> The present invention generally relates to a performance optimizer called "NVMeQPO (NVMe Queue Pair Performance Optimizer)" for improving I / O sending performance of queue pair based NVMeoF initiator target systems by adaptively adjusting the number of queue pairs and controlling the I / O flow to obtain improved performance in terms of service rate compared to conventional systems that do not adaptively adjust the number of queue pairs and do not control the I / O flow.

[0017] In one embodiment, we are motivated by certain characteristics of the NVMeoF system, such as the overall service rate of the client stack, which is determined by the number of queue pairs and the average queuing I / O height in each queue pair (reflected by the distribution pattern of the arriving workload).

[0018] NVMeoF is used to improve the parallelism and scalability of computation and storage, and to decouple computation from storage. In one embodiment, the overall service rate of some queue pairs on each Network Interface Controller (NIC) port and the corresponding queue pairs on the target side is independent of the arrival of I / O in the NIC port's centralized queue (NIC_CQ). Therefore, to improve performance by minimizing the latency of each I / O request in the system, the system of the present invention maximizes the service rate by dynamically adjusting the parameters of arriving I / O and queue pair settings during runtime.

[0019] As mentioned above, some NVMeoF systems have the following problems: (1) Without adaptive adjustment of the number of NIC queue pairs based on workload patterns and system performance changes, some NVMeoF implementations waste system resources. Such waste is (i) Maintain an extra pair of idling queues, (ii) This is caused by not maintaining enough queue pairs, resulting in under-utilization of the system. (2) Some NVMeoF implementations do not have a flow control scheme to prevent delays in queue pair service rates in the event of high I / O pressure (i.e., when there are many I / O operations queued in a queue pair).

[0020] FIG. 1 is a diagram illustrating a component architecture of a performance optimizer for a system-based NVMeoF system employing NUMA (Non-Uniform Memory Access) according to an embodiment of the present invention. In order to solve the above-mentioned problems, one embodiment of the present invention has two components, "first component: queue pair number manager" and "second component: queue pair depth based flow controller", as shown in Figure 1. The system of the present invention uses a first component to vary the number of queue pairs in an attempt to improve the service rate. Additionally, the system of the present invention may further implement a second component that uses two watermarks to control the arrival flow and adjust the distribution of the arriving I / O request stream. Further details are provided in the next section.

[0021] <<2. NVMeQPO System Overview>> In this section, we introduce the background technology and limitations of the data path in some NVMeoF implementations, and then introduce the problem formulation and related algorithms. FIG. 2A is a table summarizing the terminology used in this section.

[0022] <2.1. NVMeoF Background Technology> Due to their faster access speeds and lower latency, SSDs are gradually replacing traditional HDDs in certain markets (e.g., the data center market) as SSD capacities increase and SSD prices fall. Conventionally, SSDs are connected to server motherboards via Serial Advanced Technology Attachment (SATA) or Serial Attached Small Computer System (SAS) interface specifications by an Advance Host Controller Interface (AHCI) mechanism. However, there are two bottlenecks that limit the implementation performance mentioned above.

[0023] (1) The first bottleneck: lack of parallelization The high I / O speed of SSDs causes a bottleneck due to the SATA interface. SATA was originally designed for traditional HDDs and only offers a maximum bandwidth of 600MB / s, much lower than the bandwidth of SSDs. The interface specification, called NVMe (Non-Volatile Memory Express), describes parallelization and direct access to overcome bottleneck problems.

[0024] NVMe accounts for increased SSD bandwidth usage by using Peripheral Component Interconnect Express (PCIe) bus channels and supporting 64k command queues and up to 64 commands per queue. In comparison, SATA only supports a maximum of 32 command queues and one command per queue. NVMe improves overall computing performance by processing many I / O operations in parallel across a PCIe bus channel which helps utilize multi-core processor resources. Also, unlike SATA and SAS, NVMe devices communicate with the system CPU directly using a relatively fast PCIe connection, bypassing intermediate stack layers and reducing I / O latency.

[0025] (2) The second bottleneck: Scalability issues due to relatively severe compute-storage coupling One SSD placement method involves placing SSDs on servers where client instances (e.g., virtual machines, docker containers, or other applications) run on the same server and use the associated SSD resources. However, due to the coupling of storage and computational resources, such architectures do not scale. For example, storage is limited by the number of disk slots in a server, leading to over- or under-utilization of computation and storage. Such over-utilization or under-utilization leads to poor performance efficiency (eg, poor resource management) and poor financial efficiency (eg, high total cost of ownership).

[0026] To overcome this limitation, NVMeoF separates (decouples) storage and computation by connecting NVMe SSDs (which are part of dedicated storage and stored on dedicated hosts / servers) directly to the cluster rather than fixing them to general servers. Such decoupling makes it easier to combine many (e.g., thousands or more) NVMe SSDs into a cluster while offloading computational tasks to other servers. As a result, various scalability features (eg, data replication and migration) are achieved.

[0027] FIG. 2B is a diagram illustrating the architecture of the NVMeoF protocol according to one embodiment of the present invention, showing examples of different hierarchies in an NVMeoF system, such as different transport modes supported by NVMeoF. NVMeoF is based on the NVMe 1.2 protocol and includes changes to the NVMe transport hierarchy. NVMeoF supports three transmission modes: "memory", "message" and "memory and message". One difference between NVMe (over PCIe) and NVMeoF is that NVMePCIe uses shared memory, while NVMeoF runs over Fabrics, a message-based transport.

[0028] As shown in Figure 2B, NVMeoF also uses and relies on Remote Direct Memory Access (RDMA), including InfiniBand (which has a bandwidth of about 40 Gbps), RoCE (RDMA over Converged Ethernet), and iWARP (Internet Wide Area RDMA Protocol) (RoCE and iWARP are over Ethernet, which has a bandwidth of about 25 Gbps or 100 Gbps). Compared to iSCSI (Internet Small Computer System Interface), NVMeoF has even lower latency. This adds a small amount of delay (e.g., a few microseconds) to traffic passing through the network. NVMeoF enables faster access between hosts and storage systems.

[0029] NVMe and NVMeoF are used to overcome the limitations mentioned above, but the service rate used for NVMeoF is not independent of various factors such as the stream of arriving I / O requests and the number of queue pairs in the client stack. The following subsections provide implementation details.

[0030] 2.2. Problem formulation FIG. 2C is a diagram for explaining a queuing model of a queue pair in a network interface card (NIC) port according to one embodiment of the present invention, showing an example of an I / O path in an exemplary NVMeoF environment. As mentioned above, NVMeoF is used to separate compute and storage. Thus, FIG. 2C shows two stacks: a client stack (a data processing server) and a target stack (a storage pool).

[0031] I / O requests are generated by the "user application" in the client stack (left side of Figure 2C). The operating system dispatcher (which distributes I / O to queue pairs separately from the dispatcher shown on the right side of Figure 2C) selects a NUMA (e.g., one of multiple NUMA-enabled processing circuits) for each I / O request based on the NUMA preferred target list. Once a path is selected (e.g., NIC3 on NUMA1 is selected to communicate with the first target in the second target group), such I / O request and the corresponding computational task are assigned to this NUMA (e.g., CPU and memory resources are used).

[0032] The I / O access operations are communicated to the target stack via the network switch. In addition, the memory inside each NUMA is divided into two parts: "user buffer" and "DMA (Direct Memory Access)". A user buffer is a memory area that stores the results of I / O returned from the target stack. DMA is dedicated to sending and receiving I / O to and from NVMeoF targets. This DMA is used in part because the associated RDMA transmission protocols may require pinning a region of memory to obtain contiguous physical memory to perform the DMA. In some embodiments, performance is improved by adjusting parameters in the client stack.

[0033] Moreover, the I / O path is explained by the server queue model shown on the right side of FIG. 2C. Each NIC port has its own centralized queue (ie, NIC_CQ), and I / O assigned to a given NIC port is queued in the NIC_CQ first. The dispatcher then distributes the queued I / O one by one to the corresponding queue pair (NIC_QP). Each queue pair is connected to a corresponding queue pair (TAR_QP) in the target stack. The dispatcher balances the load between queue pairs in a client stack.

[0034] (NIC_QP) and (TAR_QP) are modeled as single servers (eg, black boxes) and the following is observed: (1) The number of (NIC_QP) affects the service rate, and the relationship between them may not be linear. (2) Also, the number of I / O requests queued in (NIC_QP) affects the service rate, and the relationship between them need not be linear. (3) The system guarantees that the number of I / Os queued in (NIC_CQ) is finite, i.e., the arrival rate is less than or equal to the server's service rate.

[0035] To minimize the latency of each I / O request, the optimization framework shown in Equation 2-1 below is used.

number

[0036] The objective function in Equation 2-1 represents the average delay of an I / O request according to the M / M / 1 queuing model, which generally represents a model that describes the behavior of a queue. The second equation represents the constraint that ensures that the system is stable. The third formula reflects the expression (or "regression", i.e., modeling using regression techniques) of the service rate as a function of two input parameters: the number of queue pairs in the client stack and the height of the average queued I / O requests. The last line of Equation 2-1 indicates that the system of the present invention uses a low watermark (described in more detail below) to control the height of the average queued I / O requests.

[0037] The detailed explanation and proof of how to adjust the queue pair depth is provided in Section 4. In other words, the system of the present invention applies a mechanism to ensure that the arrival rate is less than or equal to the service rate. If λ≦μ, then the goal is how to maximize the service rate μ.

[0038]

number

[0039] FIG. 2D is a table summarizing symbols and descriptions used in bottleneck-based service rate analysis and modeling according to an embodiment of the present invention; FIG. 2E is a graph illustrating an exemplary 3D surface used to find the maximum service rate as a function of a watermark variable and a number of queue pair variables in accordance with an embodiment of the present invention. L and the number of queue pairs N q 1 shows an exemplary diagram illustrating maximizing μ by adjusting The system of the present invention has an average queuing I / O height I don't have direct control over TIFF0007641711000003.tif19168, but I want to use low watermark W L By adjusting the Section 4 below contains more details on this.

[0040] For example, finding the maximum point on the 3D surface shown in FIG. 2E is computationally intensive during runtime because there is no strict guarantee that the surface is concave. Also, the tasks involved in changing the number of queue pairs and adjusting the low watermark have different associated overheads. Generally, adjusting the number of queue pairs is computationally more expensive since a queue pair involves a thread that consumes CPU and memory resources. Thus, in one embodiment, the system of the present invention holds one variable fixed and adjusts the other under different scenarios based on which variable has the most impact on the service rate. In other words, the system of the present invention applies two methods.

[0041] <Method 1> If the system of the present invention fixes the watermark, the system of the present invention optimizes (maximizes) the service rate μ by q Adjust.

number

[0042] <Second method> If the system of the present invention fixes the queue pairs, the system of the present invention optimizes (maximizes) the service rate μ by L Adjust.

number

[0043] The first method is performed by a first component and the second method is performed by a second component. The next section explains which approach to use.

[0044] <<3. First component: Queue pair number manager>> In this section, we first discuss bounds on the number of queue pairs in a client stack. Then, two different approaches for adjusting the number of queue pairs based on different objectives with different related scenarios are described. We also describe an NVMeQPO solution for switching between the first and second approaches, as mentioned above.

[0045] <3.1. Explanation of boundary on number of queue pairs> FIG. 3A is a diagram illustrating a model of a client stack and a target stack queue pair in accordance with an embodiment of the present invention, showing an example of a centralized queue and multiple queue pairs, with the entire portions of each shown in cross hatching. Both the client stack and the target stack have many queue pairs. For each queue pair in each stack, the system of the present invention creates a separate thread for each queue pair and configures a software FIFO (First In, First Out) queue serviced by each queue pair.

[0046] Since the configuration of the target stack may not be easily adjusted by the host system, an embodiment of the present invention describes adjusting the number of queue pairs in the client stack. In various implementations of cloud computing infrastructure, further details about the target stack may not be available.

[0047] Regarding the configuration of different queue pairs, several points need to be considered. (1) The number of queue pairs in the two stacks may be the same or different. (2) Each queue pair in the client stack is connected to one queue pair in the target stack. (3) Different numbers of queue pairs QP may result in different overall service rates μ, and the relationship between them may not be linear. (4) The maximum number of queue pairs that can be connected to an NVMe target (N Target ) to notify the user. (5) The number of QPs is limited. The system of the present invention also matches the upper limit of queue pairs in the client stack with the number of queue pairs in the target stack. (6) If the target has a single SSD to write to and the queue depth per queue pair is reported by the target (W max ), a single queue pair is sufficient to achieve a given line rate.

[0048] Therefore, the following formula 3-1 is obtained.

number

[0049] Why not use the maximum number of queue pairs? Empirically, the system of the present invention determines the maximum number of queue pairs that can be used to transmit the maximum workload and improve the service rate. For example, the host system may be configured to use the maximum number of queue pairs in the same client stack as the target stack advertised by the target vendor. However, using the maximum number of queue pairs advertised by a target is not optimal. For example, maintaining an additional number of queue pairs can serve as overhead on the client side, and using a large number of queue pairs for smaller workloads can adversely affect performance (e.g., idling queue pairs waste resources).

[0050] In other words, having more queue pairs in the client stack improves performance by allowing increased use of network bandwidth and CPU resources in the client and target stacks. However, having a queue pair comes at a cost. The cost is incurred because setting up a queue pair creates a thread that consumes host NUMA CPU and memory resources (in the client stack). In practice, NUMA CPU and memory resources are limited and expensive, and it is undesirable to have a large number of idling queue pairs beyond a certain threshold.

[0051] The present invention first introduces a "conservative approach" in subsection 3.2., which adaptively increases and decreases the number of queue pairs iteratively (once per unit time) based on system utilization. In subsection 3.3, we introduce a second approach, called the “aggressive approach”, to periodically achieve more precise performance optimization by directly varying the number of queue pairs at a given operating point using runtime monitoring and regression.

[0052] <3.2. Conservative approach> System utilization is used to determine when to switch strategies by indicating how "fully" utilized the system is. FIG. 3B is a table comparing two methods (method 1 and method 2) for adjusting the number of queue pairs and queue pair depth according to one embodiment of the present invention, and summarizing the differences between the two methods. FIG. 3C is also a table showing a comparison of two approaches (first approach and second approach) for adjusting the number of queue pairs in the first method according to one embodiment of the present invention, summarizing the two approaches developed for the first method. The present invention introduces a "conservative approach" in this subsection and describes an "aggressive approach" in subsection 3.3.

[0053] The first method causes the service rate to change immediately and dramatically, but triggers significant overhead, whereas the second method slowly converges the service rate to a given operating point with low overhead. Thus, in an embodiment of the present invention, the system removes (deactivates) queue pairs to conserve resources and improve the service rates of the remaining queue pairs, for example, when system utilization is below an underutilization threshold.

[0054] Similarly, if the system utilization is higher than the overutilization threshold, the system will create (activate) new queue pairs to avoid creating an unstable system. These two cases represent relatively extreme operating environments that may not need to be remedied in a relatively short period of time by modifying the watermark. On the other hand, within the non-underutilized and non-overutilized ranges, the system of the present invention uses watermarks to improve the service rate.

[0055] FIG. 3D is an exemplary diagram illustrating the operational aspects of a switching optimization approach based on different system utilization cases in accordance with an embodiment of the present invention, illustrating an example of how the two approaches switch during runtime based on system utilization. FIG. 3E is also a table showing thresholds and target utilizations used to adjust the number of queue pairs and queue pair depth, as well as system utilization calculations, in accordance with one embodiment of the present invention, and explains the symbols used in FIG. 3D. In some instances, due to computational costs, it is advantageous to limit how often the number of queue pairs is changed. To achieve such a limit, the system of the present invention introduces two additional conditions to decide whether to change the number of queue pairs once two thresholds are reached.

[0056] <First Condition> The period of over-utilization or under-utilization falls within a preset threshold time window T utl (or a "first length threshold") to determine whether it is sufficiently long. <Second Condition> Determine whether adding or removing queue pairs would improve the service rate (as explained in subsection 3.3. and the table in Figure 3H).

[0057] FIG. 3F is a table illustrating cases and corresponding actions used to adjust the number of queue pairs and queue pair depth according to one embodiment of the present invention. Based on the above conditions, the following actions are applied for seven different scenarios, as shown in the table in FIG. 3F.

[0058] Specifically, Case 1 and Case 2 relate to the situation where the system utilization is greater than the "overutilization threshold" (or the "first utilization threshold"), and the system of the present invention immediately increases the service rate. Case 1 represents a situation where the two conditions are not completely met. The system of the present invention classifies this as a temporary burst or N q If modifying the metric is not helpful, the system of the present invention does nothing. Case 2 represents a situation where both conditions are fully met, and the system of the present invention triggers the first component to take the first approach to immediately improve the subsequent service rate.

[0059] If the system utilization is below the "underutilization threshold" (or "second utilization threshold"), similar designs are used for Cases 3 and 4. When system utilization is neither over-utilized nor under-utilized (i.e., the "safe zone"), the system of the present invention maximizes the service rate by adjusting the watermark (further details are provided in Section 4 below).

[0060] The system of the present invention uses a large window T reg A regression-based optimization (e.g., numerical minimization) with a metric (or “second length threshold”) (also accomplished by the first component) is further performed to further improve performance by finding the best service rate over the number of data points for all queue pairs. This is called the "aggressive approach" and is shown in the table of FIG. 3E as case 7, where the system of the present invention directly adjusts the number of queue pairs to a given operating point, instead of slowly adjusting its value by 1, without considering the first and second conditions. Further details are provided in subsection 3.3. Such numerical minimization occurs if a time interval having a length exceeding a second length threshold has elapsed since the most recent use of numerical minimization to determine and set the number of queue pairs.

[0061] To further reduce the overhead of frequently changing configuration, the system of the present invention is configured to converge to a target system utilization rate instead of maximizing the service rate. Specifically, the system of the present invention increases the service rate μ but decreases the target ρ T (or "third usage threshold") is reached ( Notice that by the definition of ,an increase in the service rate decreases ρ under the same arrival rate λ), and because frequent adjustments are computationally expensive, our system does not need to make additional adjustments to further approach the maximum service rate.

[0062] Therefore, the system of the present invention configures system utilization as a metric (along with other conditions shown in FIG. 3H) to distinguish between situations that should be treated differently, and uses different approaches and corresponding actions to improve performance. In this manner, embodiments of the present invention adapt to changes in operating conditions and take into account system overhead costs incurred by certain parameter changes.

[0063] Specifically, case 5 represents a situation where utilization is higher than the target, but utilization is lower than the overutilization threshold. In this region, the system converges on the target by using a second component. When the target is reached, the utilization rate is (ρ T , ρ T ), the system of the present invention does not need to invoke the first and second components, as shown in case 6. Additionally, FIG. 3G is a workflow diagram of two components used to adjust the number of queue pairs according to one embodiment of the present invention, illustrating the main steps of such an approach switching operation.

[0064] <3.3. Offensive Approach> While conservative adjustment of the number of queue pairs addresses the performance improvement task in a relatively incremental manner, the system of the present invention also aggressively adjusts the number of queue pairs to a relatively computationally efficient number in cases where the conservative approach is too slow.

[0065] FIG. 3H is a table showing the sub-components and functions of the first component used to adjust the number of queue pairs in accordance with one embodiment of the present invention. The table in FIG. 3H shows three sub-components used in the first component for use in a conservative approach (to verify the estimated performance change after the second condition, i.e., increasing / decreasing one queue pair) and an aggressive approach (to find a predetermined operating point for the maximum service rate).

[0066] One difference between the two approaches using such subcomponents is that the conservative approach invokes the second subcomponent's "First Mode: General Prediction" which communicates with the third subcomponent's "First Mode: General Regression", whereas the aggressive approach invokes the second subcomponent's "Second Mode: Accurate Prediction" which communicates with the third subcomponent's "Second Mode: Accurate Regression". Therefore, the aggressive approach is computationally more expensive than the conservative approach, and for this reason, the system of the present invention does not call it frequently (i.e., in the table of FIG. 3E, T reg should be a relatively long interval, for example, between 1.0 and 100.0 seconds (e.g., 10 seconds).

[0067] FIG. 3I is a diagram illustrating the relationships between subcomponents of a first component used to adjust the number of queue pairs according to one embodiment of the present invention, illustrating the relationships between the subcomponents in the first component.

[0068] Once the system of the present invention has a performance prediction function, the system of the present invention performs the optimization framework of Equation 3-1 shown below. TIFF0007641711000008.tif12128 and WL When is fixed, the system of the present invention can provide a service rate of By maximizing TIFF0007641711000009.tif13128, sys Attempt to minimize

number

[0069] Here, the first formula indicates that the objective of the framework is to find the number of queue pairs that achieves a relatively low delay (eg, the delay is calculated based on M / M / 1 queuing theory). The second formula is W L We show that, given,,,, our system iterates over the number of queue pairs. The third formula is N q This is the boundary between The fourth formula indicates that the upper bound must match the maximum number of queue pairs advertised in the target stack. The last formula also ensures that the system is stable.

[0070] The system of the present invention then determines the minimum T sys Explore the points. FIG. 3J is a graph illustrating an exemplary objective function used in a regression-based offensive approach according to one embodiment of the present invention. The curve of the function μ and the number of queue pairs is not a linear curve, ie, it has one predetermined operating point. The system of the present invention periodically monitors and regresses the curve and uses algorithms (e.g., greedy algorithms, gradient descent, A* search algorithms and / or other machine learning methods, etc.) to find a predetermined operating point.

[0071] <<4. Second Component: Queue Pair Depth Based Flow Controller>> In addition to the number of queue pairs, the queue pair depth also affects the system performance. In other words, the service rate varies depending on the height of the average queued I / O requests (ie, queue depth) in a client stack's queue pair. Queue depth controls the I / O flow and the delay (or gap) between I / O requests; by having an appropriate delay (or gap), the target stack resources have more opportunity to schedule background processes that run in parallel. However, directly controlling the queue depth is computationally intractable. As a result, the second component of NVMeQPO improves service rate by adjusting two watermarks on queue depth (i.e., by controlling arrival patterns rather than arrival rates).

[0072] <4.1. Workload-based service rate queuing model> FIG. 4A is an example diagram illustrating a queuing model used to adjust the depth of a queue pair according to an embodiment of the present invention. A model is used to describe a storage controller queue pair, as shown in FIG. 4A and described in further detail below. FIG. 4B is a table illustrating terminology and corresponding descriptions associated with adjusting queue pair depth according to one embodiment of the present invention, summarizing some of the terms used in this section.

[0073] An I / O request (SCSi command) from the host's initiator HBA (Host Bus Adapter) to the storage controller's target adapter is first queued in a centralized queue (NIC_CQ) on the NIC port. I / O requests have an arrival rate λ, which is I / O units per second (IOPS). At that time, (NIC_CQ) forwards the I / O to the Nq Send it to the dispatcher that aims to distribute it evenly across the queue pair (NIC_QP) (N q is the number of queue pairs on the NIC). Such queue pairs associated with the same (NIC_CQ) have the same configuration.

[0074] Also, over a relatively long time interval, the queue pairs are q We get the arrival rate, and the queue pair has a μ service rate in IOPS. NIC_QP is not an infinite queue, it is associated with one CQ and has a fixed maximum depth Q max (Client), which is the maximum number of I / Os that may be queued and processed by the target of that QP.

[0075] In an embodiment of the present invention, the service rate is not fixed but is related to the number of queued I / Os. When the (NIC_QP) becomes full, any new I / O assigned to this (NIC_QP) (i.e. I / O that has not yet been processed arrives at the port limit) will be rejected with a "QFULL" (i.e. queue is full) error message sent from the storage port. When a QFULL message is sent to the initiator, the HBA pauses I / O for a period of time and then lazily tries to retry the storage port. Sending I / O to a full QP will trigger a "QFULL" error and degrade performance.

[0076] Besides QFULL errors, service rate is also related to queued I / O levels. FIG. 4D is a diagram illustrating further examples of the relationship between queued input / output (I / O) height and service rate used in the context of adjusting queue pair depth in accordance with one embodiment of the present invention. As shown in Figure 4C, with “no push” and “push”, the service rates for the three lightly cross-hatched I / O requests are different, with lower heights having larger service rates. The latter is more likely to result in a "QFULL" error from the queue pair between the client and target stacks, resulting in a significantly slower service rate due to the "QFULL" errors and corresponding retries.

[0077] However, the relationship between queue height and service rate is not linear, ie, "the lower the height, the better the service rate." For example, if the NIC_QP queued height is very low, the TAR_QP resources will not be fully utilized and therefore the service rate will not be maximized. Thus, there is a trade-off between "queue pair overutilization: too much QFULL" or "queue pair underutilization: less QFULL but idling resources". There is a predetermined operating point of average queued I / O requests (ie, an appropriate pressure value at which the system of the present invention achieves the best service rate). An example of this is shown in FIG. 4D, where h=5 has the highest service rate of 10.

[0078] Such a predetermined operating point exists for two reasons. (1) A given operating point is (i) Lower height to reduce "QFULL"; (ii) Balancing the above mentioned trade-offs by aiming to fully utilize resources by not having too low queue heights. (2) A given operating point also ensures that the target stack schedules background operations (e.g., system interrupts like "memory copy") to occur in parallel, and allows some idling time to synchronize with the I / O stack's heartbeat. Thus, the system of the present invention attempts to match the average queue height to a given operating point in order to obtain the best service rate.

[0079] <4.2. Service rate control using two watermarks> Although the system of the present invention does not directly control the average queue height to improve performance, the system of the present invention does control two watermarks: the high watermark W H (or "upper queue height threshold") and the low watermark W L (or "Lower Queue Height Threshold"). Specifically, W H is used to match the upper limit to prevent QFULL, and W L controls the height and is used to find the best service rate.

[0080] FIG. 4E is a diagram for explaining the conditions and corresponding operations in the watermark-based optimization for adjusting queue pair depth according to one embodiment of the present invention, summarizing the conditions and corresponding operations of the watermark-based optimization for each queue pair. FIG. 4F is a diagram showing some exemplary steps for watermark-based optimization for adjusting queue pair depth according to one embodiment of the present invention, FIG. 4G is a flowchart showing an exemplary workflow for watermark-based optimization for adjusting queue pair depth according to one embodiment of the present invention, and FIG. 4H is a table showing terms and corresponding descriptions for queue pair depth adjustment according to one embodiment of the present invention. 4F and 4G further illustrate the basic scheme of the watermark-based optimization. In an embodiment of the present invention, NVMeQPO uses an I / O flow control method rather than a load balancer or dispatcher. In practice, a load balancing algorithm is also implemented, even a very simple round robin algorithm.

[0081] Specifically, for each queue pair, there is a lock flag called “lock”, as shown in the first row of Figure 4F. This flag indicates whether the dispatcher is allowed to allocate new I / O to one single (NIC_QP). I / Os are assigned to each (NIC_QP) in turn, one at a time, and the system of the present invention classifies all possible scenarios into the following four cases:

[0082] Case 1: (NIC_QP) is unlocked and not full ("full" is W H When the I / O arrives at , the system of the present invention enqueues the I / O to (NIC_QP). The "enqueue()" procedure of (NIC_QP) is called to enqueue new I / O to this (NIC_QP), as shown in FIG. 4E(1) and lines 11 to 18 of FIG. 4F.

[0083] Case 2: While new I / O continues to queue on this (NIC_QP), NVMeQPO also checks if (NIC_QP) is full. As shown in FIG. 4E(2) and line 17 of FIG. 4F, when it becomes full, this (NIC_QP) is locked and no more new I / O requests are allowed to be queued in this (NIC_QP) (i.e., the “wait()” procedure is called). This NIC_QP is (W H -W L ) queued I / Os to complete.

[0084] Case 3: (NIC_QP) is locked, but (NIC_QP) is not (W H -W L ) queued I / Os are completed (i.e., the “WLTriggered” flag is true), (NIC_QP) is released from the lock (line 8 of FIG. 4F) and allows new I / Os to be enqueued (line 9 of FIG. 4F), which is also shown at (3) in FIG. 4E.

[0085] Case 4: Similar to case 2, (NIC_QP) is unlocked, but if it becomes full, NVMeQPO will lock and the “wait()” function will be called. This (NIC_QP) is (W H -W L ) queued I / Os to complete.

[0086] In summary, NVMeQP's watermark-based flow control keeps the data in the NIC_QP until it is full (i.e., W H ), allowing additional I / O to the host. When it becomes full, (W H -W L ) I / Os are locked until they are completed. Through this, NVMeQP indirectly reduces the number of "QFULL" errors and controls pressure (i.e., the average height of queued I / O requests over a long interval).

[0087] In the next subsection, we will show how to tune these two watermark values ​​and how to control the I / O flow (arrival pattern) to obtain a high service rate. L A method for dynamically adjusting the

[0088] <4.2.1. High Watermark: Upper Limit Matching for Maximum Depth of Target Stack Queue Pair> In an embodiment of the present invention, the queue depth of a queue pair in a client stack is configured to have the same value as the queue depth of a queue pair in a target stack. The reason is that if the queues in the client and target stacks are not the same, it is likely to trigger many "QFULL" rejections in the target stack, which can result in significant performance degradation. For example, Q max (Client) is Q max We will assume that it is greater than (Target). Then, over a long period of time, as the maximum queue depth of the target controller is reached, the target storage controller must reject more and more incoming commands by returning an expensive "QFULL" response to the client stack. Such "QFULL" rejections can severely degrade system performance and cause errors on some systems.

[0089] As a result, the service rate of each QP in the client stack is reduced from the dispatcher's perspective. To avoid such "QFULL" rejections, the system of the present invention provides a controllable upper limit W H to match the QP in the target stack. This "matching" process can be handled by a simple configuration, (i) Client Stack Q max (Client) (Client stack also has pre-defined limits) (ii) Target Stack Q max This is because the maximum number of queue pairs in (Target) and (Target) are advertised during connection setup (i.e., each QP in the client stack connects with the corresponding QP in the target stack).

[0090] Therefore,

number

[0091] <4.2.2. Low Watermark: Controlling Queue Height and Finding the Optimal Service Rate> As mentioned above, since it is difficult to directly control the height of the average queued I / O requests in (NIC_QP), the system of the present invention uses a low watermark to indirectly achieve this control of the height of the average queued I / O requests in (NIC_QP).

[0092] In this subsection, we first explain the feasibility and introduce an optimization framework that helps us find the best service rate. Explaining the positive correlation between TIFF0007641711000012.tif12128 and WL. In an embodiment of the present invention, the system of the present invention is a monotonically increasing function TIFF0007641711000013.tif15128, or directly Regress the function as TIFF0007641711000014.tif15128.

[0093] <4.2.2.1. Indirect control based on positive correlation> Fixed N q and W H Given, The TIFF0007641711000015.tif12128 function is fixed, and the system of the present invention uses W L Estimate the average height as a function. The average queued I / O height for a particular QP increases with decreasing number of queued I / Os. H From W L It is calculated by increasing by 1 until

[0094] Intuitively, this is expressed as the weighted value of the time length of each height and the sum of each height, W H From W L This is the value divided by the total time

number

[0095] W L2 The process for this case is separated into two parts. The first part is W H From W L1 The second part is W L1 From W L2 That is until now. Therefore, it is defined as shown in the following formula 4-2.

number

[0096] Therefore,

number

number

number

[0097]

number

[0098] NVMeQPO is L To guide how to control the average height using Regress a monotonically increasing function as TIFF0007641711000025.tif15128.

[0099] <4.2.2.2. Optimization Framework> The system of the present invention is sys Optimize (minimize) That is, the optimization goal is to minimize the average latency for I / O in the system (ie, (NIC_CQ) and (NIC_QP)).

number

[0100] FIG. 4J is an exemplary diagram of a three-dimensional graph used by the system of the present invention to select a low watermark in accordance with one embodiment of the present invention, illustrating an exemplary objective function. hyperboloid TIFF0007641711000027.tif16128 and watermark W for service rate L The function μ=f(g(W L The intersection of ) is related as shown by the curve that forms the shaded upper boundary of the curved surface in Figure 4J. To ensure a stable system, must be larger than , so we use the hyperbola of .

[0101] The function μ=f(g(W L ) is concave, so the example curve shown in FIG. 4J is also concave. Indeed, since in one embodiment we do not focus on one single curve shape but aim to find the minimum of the curve, there is no need to limit the way the curve is represented. Similar to subsection 3.3, the system of the present invention periodically monitors and regresses the curve to find the desired operating point, using a greedy algorithm, gradient descent, A* search algorithm, or other machine learning methods. For example, the system of the present invention uses a polynomial fitting model of the historical average processing delay per I / O request as a function of the service rate and the lower queue height threshold. The system of the present invention does not limit the use of technology to handle this task.

[0102] In one embodiment, the client includes processing circuitry configured to perform some or all of the methods described herein. The term "processing circuitry" is used herein to mean any combination of hardware, firmware, and software adapted to process data or digital signals. The hardware of the processing circuit includes, for example, an application specific integrated circuit (ASIC), a CPU, a DSP, a GPU, and a programmable logic device such as a field programmable gate array (FPGA). As used herein, each function of the processing circuitry is performed by either hardware configured to perform the function (i.e., built into the hardware), or by more general purpose hardware, such as a CPU configured to execute commands stored on a non-transitory storage medium. The processing circuitry may be fabricated on a single Printed Circuit Board (PCB) or may be distributed across several interconnected PCBs. The processing circuitry includes other processing circuits, for example the processing circuitry includes two processing circuits, an FPGA and a CPU, interconnected on a PCB.

[0103] Terms such as "first," "second," "third," etc. are used herein to describe various elements, components, regions, hierarchies, and / or sections, but these elements, components, regions, hierarchies, and / or sections are not limited by these terms. These terms are used to distinguish one element, component, region, tier, or section from another element, component, region, tier, or section, and thus a first element, component, region, tier, or section described herein could be termed a second element, component, region, tier, or section without departing from the spirit and scope of the present invention.

[0104] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the terms "substantially," "about," and similar terms are used as terms of approximations rather than terms of degree and are intended to account for inherent deviations in measured or calculated values ​​that are recognized by one of ordinary skill in the art.

[0105] In Japanese, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the terms "comprise" and / or "comprising" embody the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the addition or presence of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. An expression such as "at least one" preceding a list of elements modifies the entire list of elements, and not any individual element of the list. It should be noted that the use of "can" when describing embodiments of the present invention means "one or more embodiments of the present invention." It is to be noted that the term "exemplary" is intended to indicate an example or illustration. As used herein, the terms "use", "utilize", and "used" are considered synonymous with the terms "utilize", "utilize", and "utilized", respectively.

[0106] When a component or layer is referred to as "on," "connected," "coupled," or "adjacent" another component or layer, it is either directly connected, coupled, or adjacent to the other component or layer, or there are one or more intermediate components or layers. In contrast, when an element or layer is referred to as being "directly," "directly connected," "directly coupled," or "adjacent" to another element or layer, there are no intermediate elements or layers present. As used herein, the word "or" is inclusive, e.g., "A or B" means either one of (i) A, (ii) B, and (iii) A and B.

[0107] Any numerical range cited herein is intended to include all subranges of the same numerical precision contained within the cited range. For example, a range "from 1.0 to 10.0" or "between 1.0 and 10.0" is intended to include all subranges between (and including) the cited minimum value of 1.0 and the cited maximum value of 10.0. That is, it has a minimum value greater than or equal to 1.0 and a maximum value less than or equal to 10.0, such as, for example, from 2.4 to 7.6. Any maximum numerical limitation recited herein is intended to include every sub-numerical limitation subsumed therein, and any minimum numerical limitation recited herein is intended to include every higher numerical limitation subsumed therein.

[0108] Exemplary embodiments of methods and systems for improving performance of an NVMeoF system are specifically described and illustrated herein. The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the technical scope of the present invention.

Claims

1. 1. A method for managing input and output queue pairs in a system having a processing circuit and one or more permanent storage devices, comprising: wherein the one or more I / O queue pairs are for storing I / O requests sent to the one or more persistent storage devices and command completions received from the one or more persistent storage devices; calculating, by said processing circuitry, a system utilization factor; where the system utilization is the ratio of the arrival rate of input / output requests to the service rate; the arrival rate of I / O requests is an arrival rate of I / O requests to a centralized queue; the service rate is a service rate of a centralized queue, determining, by the processing circuitry, in response to an estimation that the system utilization exceeds a first utilization threshold, which is an overutilization threshold, during a first time period that exceeds a first long threshold, and adding a new input / output queue pair increases system performance because an estimated change in the service rate is in accordance with the new input / output queue pair, and adding the new input / output queue pair increases the service rate; The service rate corresponds to an average processing delay per I / O request; The method of managing an input / output queue pair, wherein the first length threshold is greater than zero.

2. determining, by the processing circuitry, that the system utilization remains below a second utilization threshold, which is an underutilization threshold, during a second time period that exceeds the first long threshold, and that removing an input / output queue pair is expected to improve system performance; 2. The method of claim 1, further comprising the step of: deleting, by said processing circuitry, said input / output queue pair.

3. determining, with the processing circuitry, that the system utilization is less than the first utilization threshold and greater than a third utilization threshold; 3. The method of claim 2, further comprising the step of: adjusting, by the processing circuitry, a lower queue height threshold such that the service rate is equal to the target third utilization threshold.

4. determining, by the processing circuitry, that a submission queue of a first one of the input / output queue pairs is unlocked and a centralized queue is storing the input / output request, and that the submission queue height is less than an upper queue height threshold; 4. The method of claim 3, further comprising the step of: enqueuing, by said processing circuitry, said I / O request.

5. determining, by the processing circuitry, that the current height of the transmission queue is equal to the upper queue height threshold; 5. The method of claim 4, further comprising the step of: locking, by said processing circuitry, said transmit queue.

6. determining, by the processing circuitry, that a transmit queue of a first one of the input / output queue pairs is locked; determining, by the processing circuitry, that the current transmit queue height does not exceed the lower queue height threshold; 4. The method of claim 3, further comprising the step of: unlocking, by said processing circuitry, said transmit queue.

7. 4. The method for managing an I / O queue pair according to claim 3, wherein a value that minimizes a model of average processing delay per I / O request as an objective function is found as a function of the lower queue height threshold against the service rate.

8. 8. The method of claim 7, wherein the model is a polynomial fit of historical average processing delay per I / O request as a function of the lower queue height threshold for the service rate.

9. determining, by the processing circuitry, that a time interval has elapsed having a length that exceeds a second length threshold after a most recent use of numerical minimization to determine and set an initial number of input and output queue pairs; performing, by the processing circuitry, a numerical minimization of a model of average processing delay per I / O request as a function of the initial number of I / O queue pairs to calculate a second number of I / O queue pairs to be used during operation; 3. The method of managing input / output queue pairs of claim 2, further comprising the step of: adding, by the processing circuit, zero or more new input / output queue pairs or deleting zero or more input / output queue pairs so that the initial number of input / output queue pairs is identical to the calculated second number of input / output queue pairs used during operation.

10. A processing circuit; one or more permanent storage devices; The processing circuitry includes: instantiating one or more I / O queue pairs for storing I / O requests sent to and command completions received from the one or more persistent storage devices; Calculate the system utilization, which is the ratio of the arrival rate of input / output requests to the service rate; wherein the input / output request arrival rate is the input / output request arrival rate to a centralized queue; the service rate is a service rate of a centralized queue, During a first time period exceeding a first long threshold, the system utilization exceeds a first utilization threshold, which is an overutilization threshold, and adding a new input / output queue pair improves system performance because the service rate is estimated to change according to the new input / output queue pair; configured to determine the addition of the new input / output queue pair in response to an estimation that adding the new input / output queue pair will increase the service rate; The service rate corresponds to an average processing delay per I / O request; The system, wherein the first length threshold is greater than zero.

11. the processing circuitry determines that the system utilization remains below a second utilization threshold, the second utilization threshold being an underutilization threshold, during a second time period that exceeds the first long threshold, and that removing an input / output queue pair is expected to improve system performance; The system of claim 10 , further configured to delete the input / output queue pair.

12. the processing circuitry determines that the system utilization is less than the first utilization threshold and greater than a third utilization threshold; The system of claim 11 , further configured to adjust a lower queue height threshold so that the service rate is equal to a target third utilization threshold.

13. the processing circuit determines that a transmit queue of a first one of the input / output queue pairs is unlocked and a centralized queue is storing the input / output request, and the transmit queue height is less than an upper queue height threshold; 13. The system of claim 12, further configured to enqueue the I / O request.

14. the processing circuitry determines that the current height of the transmission queue is equal to the upper queue height threshold; The system of claim 13 , further configured to lock the transmission queue.

15. the processing circuitry determines that a transmit queue of a first one of the input / output queue pairs is locked; determining that the current transmission queue height does not exceed the lower queue height threshold; The system of claim 12 , further configured to unlock the transmission queue.

16. 13. The system of claim 12, wherein the function of the lower queue height threshold against the service rate is evaluated to minimize a model of average processing delay per I / O request as an objective function.

17. 17. The system of claim 16, wherein the model is a polynomial fit of historical average processing delay per I / O request as a function of the lower queue height threshold for the service rate.

18. The processing circuitry determines that a time interval having a length exceeding a second length threshold has elapsed since a most recent use of numerical minimization to determine and set an initial number of input / output queue pairs; numerically minimizing a model of average processing delay per I / O request as a function of the initial number of I / O queue pairs to calculate a second number of I / O queue pairs for use during operation; 12. The system of claim 11, further configured to add zero or more new input / output queue pairs or remove zero or more input / output queue pairs so that the initial number of input / output queue pairs is identical to the calculated second number of input / output queue pairs used during operation.

19. A processing means; one or more permanent storage devices; the processing means instantiates one or more I / O queue pairs for storing I / O requests sent to the one or more permanent storage devices and command completions received from the one or more permanent storage devices; Calculate the system utilization, which is the ratio of the arrival rate of input / output requests to the service rate; wherein the input / output request arrival rate is the input / output request arrival rate to a centralized queue; the service rate is a service rate of a centralized queue, During a first time period exceeding a first long threshold, the system utilization exceeds a first utilization threshold, which is an overutilization threshold, and adding a new input / output queue pair improves system performance because the service rate is estimated to change according to the new input / output queue pair; configured to determine the addition of the new input / output queue pair in response to an estimation that adding the new input / output queue pair will increase the service rate; The service rate corresponds to an average processing delay per I / O request; The system, wherein the first length threshold is greater than zero.

20. the processing means determines that during a second time period that exceeds the first long threshold, the system utilization remains below a second utilization threshold, which is an under-utilization threshold, and removal of an input / output queue pair is expected to improve system performance; 20. The system of claim 19, further configured to delete the input / output queue pair.

Images