Exchangeable queue types for network connections

Abstract

Processing circuitry, methods, and code instructions are provided for selecting the type of queue pair (QP) for data transmission across a network, for example, based on a remote direct memory access (RDMA) protocol. A request to establish a connection for data transmission across the network is received. Resource-related parameters are analyzed. Each resource-related parameter indicates the corresponding resource-related state for the connection to be established. The type of QP is selected from multiple candidate types based on the analysis. The candidate QP types may be defined by a transport protocol. The connection is established using the selected type of QP for data transmission across the network.

Description

Background Technology

[0001] In some embodiments of the present invention, the present invention relates to network connectivity, and more specifically, but not exclusively, to resource management systems and methods for establishing network connectivity.

[0002] Network nodes, such as servers, can establish and simultaneously support thousands of network connections with other network nodes, such as storage servers, endpoint devices, and other servers, to facilitate data exchange across the network. A large number of simultaneous network connections consumes significant resources. Summary of the Invention

[0003] The object of this invention is to provide processing circuitry, computing devices, methods, and computer-readable storage media for cross-network data transmission.

[0004] The above and other objectives are achieved through the features of the independent claim. Other implementations will be apparent from the dependent claims, the description, and the drawings.

[0005] According to a first aspect, a processing circuit for selecting a type of queue pair (QP) for data transmission across a network, the processing circuit being configured to: receive a request to establish a network connection for data transmission across the network; analyze a plurality of resource-related parameters, each resource-related parameter indicating a corresponding resource-related state for the network connection to be established; select the type of the QP from a plurality of candidate types based on the analysis; and establish the network connection for data transmission across the network using the selected type of the QP.

[0006] According to a second aspect, a method for selecting a type of queue pair (QP) for data transmission across a network includes: receiving a request to establish a network connection for data transmission across the network; analyzing a plurality of resource-related parameters, each resource-related parameter indicating a corresponding resource-related state for the network connection to be established; selecting the type of the QP from a plurality of candidate types based on the analysis; and establishing the network connection using the selected type of the QP for data transmission across the network.

[0007] According to a third aspect, a computer program includes program instructions that, when executed by a processor, cause the processor to: receive a request to establish a network connection for data transmission across the network; analyze a plurality of resource-related parameters, each resource-related parameter indicating a corresponding resource-related state for the network connection to be established; select the type of the QP from a plurality of candidate types based on the analysis; and establish the network connection for data transmission across the network using the selected type of the QP.

[0008] The requested QP type used for a new connection does not necessarily represent the optimal use of existing resources on the host and / or device and / or network, such as memory, hardware resources, processing resources, cache, and network resources. For example, RC QP may be optimal in terms of transport, but it is very expensive in terms of available resources. It is possible to choose a QP type that provides the most comprehensive (e.g., globally for device, network, and / or host) utilization of available resources, rather than the QP that is best for the requesting application.

[0009] For high-end servers that establish thousands of QPs for thousands of network connections, such as 10,000, 100,000, or 1,000,000, overall resource optimization is especially important.

[0010] Compared to existing methods, dynamically selecting the QP type used to establish connections offers better resource utilization, where each application is assigned the QP type it requests. For example, for N network nodes, each node runs M processes that establish network connections. For all M processes wanting to communicate with all processes on all nodes, the number of RC QPs required for this "many-to-many" communication is (M^2)*(N–1) per node. RD has a lower occupancy: each node requires M QPs + N "end-to-end" (EE) connections to achieve the same many-to-many communication pattern. However, RD is limited in terms of transport. Its most significant limitation is the single incomplete message supported per EE context. As a proprietary solution, DC has a lower occupancy compared to RC, but it still consumes significant resources if it uses many DCI / DCTs and is not very efficient in caching. Another drawback of DC is that it must use frequent connect-disconnect operations.

[0011] In another implementation of the first, second, and third aspects, the method further includes: re-analyzing the plurality of resource-related parameters during data transmission via the established network connection; re-selecting another type of the QP from the plurality of candidate types based on the re-analysis; establishing a second network connection using the re-selected other type of the QP; and performing at least one of the following operations: dynamically transferring the network connection to the second network connection, and re-initiating the transmission of data of the network connection to be transmitted via the second network connection.

[0012] When the resource-related parameters change, the QP type of the existing connection can be dynamically changed during the use of the established network connection to transmit data over the network, in order to provide different QP types that are expected to improve resource utilization.

[0013] In another implementation of the first, second, and third aspects, the method further includes: the network connection including a first network connection; receiving a second request to establish a second network connection for data transmission across the network; performing a second analysis on the plurality of resource-related parameters, each resource-related parameter indicating a corresponding resource-related state of the established first network connection and used for the establishment of the second network connection; selecting a second type of the QP from the plurality of candidate types based on the second analysis; establishing the second network connection using the second type of the QP; and dynamically migrating the first network connection to the second network connection, wherein the second network connection transmits data across the network for both the first network connection and the second network connection.

[0014] Two network connections transmitting two datasets over the network can be merged into a single network connection. This merged second network connection can use fewer resources than two separate network connections.

[0015] In another implementation of the first, second, and third aspects, the request is provided by a first application, the second request is provided by a second application, and the second network connection transmits data for the first and second applications through the network.

[0016] Two separate network connections requested by two different applications for transferring two datasets over the network can be merged into a single network connection used by both applications. This merged second network connection can use fewer resources than two separate network connections.

[0017] In another implementation of the first, second, and third aspects, dynamic migration includes: pausing network traffic on the first network connection; receiving an acknowledgment message that packets sent via the first network connection prior to the pausing have been received by a device at the other end of the first network connection; and using the second type of the QP of the second network connection for additional network traffic destined for the first network connection.

[0018] The migration can be performed transparently without significant disruption by pausing current network traffic on previously established network connections.

[0019] In another implementation of the first, second, and third aspects, it is further used to and / or include: terminating the first network connection in response to an indication that at least one first packet of the additional network traffic has been delivered through the second network connection.

[0020] Resources bound to a previously established network connection will be released by terminating the previously established network connection.

[0021] In another implementation of the first, second, and third aspects, the method further includes: receiving a third request to establish a third network connection for data transmission across the network; performing a third analysis on the plurality of resource-related parameters, each resource-related parameter indicating a corresponding resource-related state of the established network connection and used for the establishment of the third network connection; selecting a third type of the QP from the plurality of candidate types based on the third analysis; prior to receiving the third request, a fourth network connection using the third type of the QP has been previously established by the processing circuitry for the fourth request; using the fourth network connection to transmit data across the network associated with the third request and the fourth request, wherein the third network connection is not established independently.

[0022] When a network connection with the selected second QP type already exists, network traffic destined for the newly established network connection can be added to the existing network connection. Resources can be saved by using the existing network connection instead of adding another.

[0023] In another implementation of the first, second, and third aspects, the request to establish the network connection is for a first type of QP, and wherein a second type of QP, different from the first type, is selected.

[0024] The selection process can be performed in implicit mode, where the request is ignored and the QP type that best optimizes the available resources is selected, without the requesting application and / or process being aware that the requested QP type has changed.

[0025] In another implementation of the first, second, and third aspects, the request to establish the network connection is for a QP reliable type or a QP unreliable type of a first subtype of QP, and wherein the selected type of QP is a QP reliable type or a QP unreliable type of a second subtype of QP defined by the request, the second subtype of QP defined by the request being different from the first subtype of QP defined by the request.

[0026] Maintaining the reliability or unreliability of the QP type based on the network connection request ensures compatibility with applications and / or processes using the data transmitted over the network. For example, applications expecting reliable data transmission receive data transmitted reliably and do not need to handle unexpectedly unreliable data transmissions.

[0027] In another implementation of the first, second, and third aspects, the network connection is for the QP reliable type, and the selected type of the QP is the QP unreliable type, wherein reliability is provided by a reliability layer implemented by at least one or a combination of software, firmware, and hardware.

[0028] The requested reliability can be provided using an unreliable QP type with the aforementioned reliability layer, instead of using a resource-intensive reliable QP type. This can improve resource utilization by using an unreliable QP type that uses fewer resources than a reliable QP type.

[0029] In another implementation of the first, second, and third aspects, the request to establish the network connection is for an interchangeable QP type (IQT), wherein the type of the QP is selected from the plurality of candidate types excluding dynamic types.

[0030] The highest and / or best resource utilization efficiency can be attributed to any candidate type of process and / or application that indicates the use of dynamic QP type selection.

[0031] In another implementation of the first, second, and third aspects, the plurality of resource-related parameters are selected from the group consisting of: the number of network nodes communicating with the application providing the request, the number of existing network connections of the application providing the request, the topology of the existing network connections of the application, the total number of active QPs of the network connections established by the processing circuitry, whether the QP of the type of the request is available, the transmission reliability of the network, the current memory utilization, and the current utilization of the processing circuitry.

[0032] These parameters provide a description of the existing resource usage, which allows selection of the QP type for the network connection that will provide the best utilization of the existing resources.

[0033] In another implementation of the first, second, and third aspects, the plurality of candidate types of the QP pair are selected from the group consisting of: reliable connection (RC), reliable datagram (RD), extended reliable connection (XRC), unreliable datagram (UD), unreliable connection (UC), scalable reliable datagram (SRD), and dynamic connection (DC).

[0034] In another implementation of the first, second, and third aspects, the analysis is performed using at least one of the following: a rule set for generating the plurality of resource-related parameters of the selected type of the QP; a classifier that receives the plurality of resource-related parameters as input and generates a result of the selected type of the QP, wherein the classifier is trained on a training dataset of the plurality of sample resource-related parameters and labels of the QP type; and optimization code that uses a mathematical model and / or a system of equations to compute an optimization of the type of the QP for the plurality of resource-related parameters.

[0035] In another implementation of the first, second, and third aspects, the data transmission across the network is performed according to the remote direct memory access (RDMA) protocol, and the plurality of candidate types of QP are defined by the network transport protocol for RDMA.

[0036] The feasibility of RDMA largely depends on its reliability, high bandwidth, and low latency characteristics. Therefore, the choice of QP type plays a crucial role in determining the balance between RDMA reliability, high bandwidth, and low latency, especially for a large number of network connections. At least some of the embodiments described herein select the optimal QP type for RDMA to maximize low latency while meeting the required reliability and / or lower memory footprint, and / or to optimally utilize device caching (e.g., to prevent large cache misses), particularly when establishing a large number of connections. In another implementation of the first, second, and third aspects, the network transport protocol for RDMA defining the plurality of candidate QP types is selected from the group consisting of: InfiniBand, IB, remote direct memory access over converged ethernet (RoCE), RoCEv2, iWARP, and derivatives thereof.

[0037] Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as those known to one of ordinary skill in the art. While similar or equivalent methods and materials to those described herein may be used in the practice or testing of embodiments of the invention, exemplary methods and / or materials are described below. In case of any conflict, this specification (including definitions) shall prevail. Furthermore, these materials, methods, and examples are illustrative only and are not necessarily restrictive. Attached Figure Description

[0038] This document describes some embodiments of the invention by way of example only, in conjunction with the accompanying drawings. It is now emphasized, in particular, that the details shown are merely illustrative and for the purpose of explaining embodiments of the invention. In this regard, how to practice the embodiments of the invention will be apparent to those skilled in the art based on the accompanying drawings.

[0039] In the attached diagram:

[0040] Figure 1 This is a block diagram of a computing device for selecting a QP type for data transmission across a network, according to some embodiments.

[0041] Figure 2 This is a block diagram illustrating how multiple computing devices use a selected QP type to transmit data across a network, according to some embodiments.

[0042] Figure 3 This is a flowchart of a method for selecting a QP type for data transmission across a network, according to some embodiments;

[0043] Figures 4A to 4FThis is a schematic diagram of an exemplary QP type for selection according to some embodiments;

[0044] Figure 5 This is a schematic diagram comparing the dynamic selection of QP type according to some embodiments with the standard method;

[0045] Figure 6 This is another schematic diagram comparing the dynamic selection of QP type according to some embodiments with the standard method;

[0046] Figure 7 This is yet another schematic diagram comparing the dynamic selection of QP types according to some embodiments with the standard method. Detailed Implementation

[0047] In some embodiments of the present invention, the present invention relates to network connectivity, and more specifically, but not exclusively, to resource management systems and methods for establishing network connectivity.

[0048] One aspect of some embodiments relates to processing circuitry, systems, methods, apparatus, and / or code instructions (i.e., stored on a computer-readable medium for execution by one or more hardware processors) for automatically selecting the type of queuepair (QP), also known as QP type, for transferring data between two devices across a network, for example, based on a remote direct memory access (RDMA) protocol, wherein the QP type is defined by a transport protocol, such as Infiniband. TM This includes Remote Direct Memory Access over Converged Ethernet (RoCE), RoCEv2, iWARP, proprietary QP types (e.g., from different vendors), and derivatives thereof. The selection of a QP type responds to a request to establish a connection for data transfer across the network, such as uploading and / or downloading data to a remote data storage device via an application. One or more resource-related parameters are analyzed. Each resource parameter indicates the corresponding resource-related status for establishing the connection, such as available resources for establishing the network connection, such as available memory, available QPs of the requested type, utilization status of one or more processors, and network status (e.g., presence of noise). Based on the analysis, a QP type is selected from multiple candidate QP types. The selected QP type is used to optimize the available resources for establishing the connection, for example, compared to the requested QP type and / or candidate QP types. The initially requested QP type can be changed to the selected QP type. A connection is established using the selected QP type for data transfer across the network.

[0049] Optionally, the QP type used for a connection can be dynamically reselected while the connection is active. Based on a re-analysis of the current state of resource-related parameters, another QP type can be selected from the candidate QP types. Another connection can be newly established using the dynamically reselected QP type. Existing connections can be migrated to connections newly established using the reselected QP type.

[0050] Optionally, the QP type is used to establish another connection where one or more other connections have already been established. Based on the analysis of resource-related parameters, the requested connection can join an existing connection using the selected QP type, for example, by multiplexing two (or more) data streams for transmission across an existing connection. In another implementation, a new connection is established using a QP type selected based on the analysis of resource-related parameters. One or more previously established connections will be migrated to the new connection.

[0051] The requested QP type used for a new connection does not necessarily represent the optimal use of existing resources on the host and / or device and / or network, such as memory, hardware resources, processing resources, cache, and network resources. For example, RC QP may be optimal in terms of transport, but it is very expensive in terms of available resources. It is possible to choose a QP type that provides the most comprehensive (e.g., globally for device, network, and / or host) utilization of available resources, rather than the QP that is best for the requesting application.

[0052] For high-end servers that establish thousands of QPs for thousands of network connections, such as 10,000, 100,000, or 1,000,000, overall resource optimization is especially important.

[0053] Compared to existing methods, dynamically selecting the QP type used to establish connections offers better resource utilization, where each application is assigned the QP type it requests. For example, for N network nodes, each node runs M processes that establish network connections. For all M processes wanting to communicate with all processes on all nodes, the number of RC QPs required for this "many-to-many" communication is (M^2)*(N–1) per node. RD has a lower occupancy: each node requires M QPs + N "end-to-end" (EE) connections to achieve the same many-to-many communication pattern. However, RD is limited in terms of transport. Its most significant limitation is the single incomplete message supported per EE context. As a proprietary solution, DC has a lower occupancy compared to RC, but it still consumes significant resources if it uses many DCI / DCTs and is not very efficient in caching. Another drawback of DC is that it must use frequent connect-disconnect operations.

[0054] Optionally, the data transfer between different computing devices described herein is based on the Remote Direct Memory Access (RDMA) protocol, which is the primary transfer method for remote memory operations. The feasibility of RDMA largely depends on its reliability, high bandwidth, and low latency characteristics. Therefore, the choice of QP type plays a crucial role in determining the balance between RDMA reliability, high bandwidth, and low latency, especially for a large number of network connections. At least some embodiments described herein select the optimal QP type for RDMA to maximize low latency while meeting the required reliability and / or lower memory footprint, and / or to optimally utilize device cache (e.g., to prevent large cache misses), particularly when establishing a large number of connections.

[0055] For RDMA, different QP types are available. Different QP types can be represented as a combination of a first subtype and a second subtype. The first subtype is reliable or unreliable. A reliable first subtype guarantees that a message will be delivered at most once, primarily in order, without corruption. An unreliable first subtype provides no guarantee that a message will be delivered or regarding packet order. In RDMA, each packet undergoes a cyclic redundancy check (CRC), and corrupted packets are discarded (for any transport type). The reliability of a QP transport type refers to the reliability of the entire message. The second subtype is connected or disconnected. A connected first subtype means that a send / receive QP is associated with exactly one other QP. A disconnected first subtype means that a send / receive QP is associated with multiple other QPs. Exemplary QP types are provided by Infiniband. TM The specification defines reliable connections (RC), reliable datagrams (RD), extended reliable connections (XRC), unreliable datagrams (UD), and unreliable connections (UC). Additional QP types were also developed, such as scalable reliable datagrams (SRD) and dynamic connections (DC), also known as dynamically connected transport (DCT).

[0056] Before explaining at least one embodiment of the present invention in detail, it should be understood that the invention is not necessarily limited to the details of the construction and setup of the components and / or methods illustrated in the following description and / or drawings and / or examples. The invention can have other embodiments, or can be practiced or performed in various ways.

[0057] This invention can be a system, method, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions that cause a processor to perform aspects of the invention.

[0058] The computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device. The computer-readable storage medium may be, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing.

[0059] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or downloaded to an external computer or external storage device via a network such as the Internet, local area network, wide area network, and / or wireless network.

[0060] The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, or as a separate software package partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including local area networks (LANs) or wide area networks (WANs), and may also be connected to an external computer (e.g., via the Internet provided by an Internet service provider). In some embodiments, electronic circuits including programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), etc., can be personalized using state information from the computer-readable program instructions to execute the computer-readable program instructions and perform various aspects of the invention.

[0061] This document describes various aspects of the invention in conjunction with flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products provided by embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0062] The flowcharts and block diagrams in the figures illustrate the structure, function, and operation of possible implementations of the systems, methods, and computer program products provided in various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing one or more specified logical functions. In some alternative implementations, the functions described in the blocks may not be implemented in the order shown in the figures. For example, in fact, two blocks shown consecutively may be executed almost simultaneously, or sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by systems based on dedicated hardware that perform specific functions or actions, or by combinations of dedicated hardware and computer instructions.

[0063] Now for reference Figure 1 , Figure 1 This is a block diagram of a computing device 104 for selecting a QP type to transmit data across network 120, according to some embodiments. Also referenced is... Figure 2 , Figure 2 This is a block diagram illustrating how multiple computing devices 120, according to some embodiments, use a selected QP type to transmit data across network 104. Also referenced is... Figure 3 , Figure 3 This is a flowchart of a method for selecting a QP type for data transmission across a network, according to some embodiments. The computing device 104 may implement the reference, for example, by one or more or a combination of the following: Figure 3 The described method involves the following actions: one or more processors 102A of computing device 104 execute code instructions (e.g., code 150) stored in memory 106A; one or more processors 102A of computing device 104 are implemented in hardware to execute the instructions defined by code 150; one or more processors 102B of network interface device (e.g., network interface card) 114 execute code instructions (e.g., code 150) stored in memory 106B; and / or one or more processors 102B are implemented in hardware to execute the instructions defined by code 150.

[0064] Queue 106B stores QPs of a selected type, as described herein. QPs 106B may be stored, for example, by memory 106A of computing device 104 and / or memory 106B of network interface device 114.

[0065] The computing device 104 can act as a network node and may sometimes be referred to as a network node in this document.

[0066] As shown in the figure, by establishing a network connection using a dynamically selected type of QP, computing device 104 communicates with one or more other instances of computing device 104 (e.g., another network node) via network 120, as described herein. Computing device 104 can be implemented as, for example, a server, a client, an initiating network node establishing an initiating network connection, and / or a target network node receiving requests from an initiating network connection.

[0067] The computing device 104 can be implemented as one or more of the following: a computing cloud, a single computing device (e.g., a client terminal), a group of computing devices arranged in parallel, a web server, a local server, a remote server, a client terminal, a mobile device, a fixed device, a kiosk, a smartphone, a laptop, a tablet computer, a wearable computing device, a glasses computing device, a watch computing device, and a desktop computer.

[0068] One or more processors 102 may be implemented as, for example, one or more central processing units (CPUs), one or more graphics processing units (GPUs), one or more field-programmable gate arrays (FPGAs), one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more custom circuits, processors for interfacing with other units, and / or dedicated hardware accelerators. One or more processors 102 may be implemented as a single processor, a multi-core processor, and / or a cluster of processors arranged for parallel processing (which may include homogeneous and / or heterogeneous processor architectures).

[0069] Memory 106A stores code instructions that can be implemented by one or more processors 102A, and / or memory 106B stores code instructions that can be implemented by processor 102B of network interface device 114. Memory 106A to 106B are implemented as, for example, random access memory (RAM), read-only memory (ROM), and / or storage devices such as non-volatile memory, magnetic media, semiconductor memory devices, hard disk drives, removable memory, and optical media (e.g., DVD, CD-ROM).

[0070] The memory 106A can store a virtual machine manager (VMM) 108 that manages and / or runs one or more virtual machines (VMs) 110. The VMM 108 can be implemented as a hypervisor. The VMM 108 can be implemented in hardware, software, firmware, and / or a combination of the above.

[0071] Each VM 110 executes one or more virtual function (VF) drivers 112.

[0072] The computing device 104 includes and / or communicates with one or more network interface devices 114, optionally a network interface card and / or a network adapter.

[0073] Network interface device 114 may include one or more processors 102B and memory 106B. Features of the methods described herein may be implemented by computing device 104 (e.g., one or more processors 102A executing code 105 stored in memory 106A) and / or by network interface device 114 (e.g., one or more processors 102B executing code 105 stored in memory 106B).

[0074] Computing device 104 may include and / or communicate with one or more data storage devices 118. Data storage device 118 may store, for example, selectable candidate QP types. It should be noted that code instructions may be selectively loaded from data storage device 118 into memory 106 for execution by processor 102. One or more data storage devices 118 may be implemented as, for example, memory, local hard disk, removable storage unit, optical disk, storage device, and / or remote server and / or computing cloud (e.g., accessed via network connection).

[0075] The computing device 104 can communicate with the network 120 via the network interface device 114, such as the Internet, local area network, virtual network, wireless network, cellular network, local bus, point-to-point link (e.g., wired) and / or a combination of the above.

[0076] Network interface device 114 may be associated with one or more physical function (PF) drives 116. Network interface device 114 may be virtualized so that it can be used by multiple VMs 110 through corresponding execution VF drives 112. For example, different VMs 110 may access network 120 through one or more VF drives 112 used to access the network interface via one or more PF drives 116.

[0077] The computing device 104 may include and / or communicate with one or more physical user interfaces 122, including mechanisms for user interaction, such as inputting data (e.g., defining a set of rules for selecting a QP type) and / or viewing data (e.g., viewing available resources and / or established connections).

[0078] The exemplary physical user interface 122 includes one or more of the following: a touch screen, a display, a gesture-activated device, a keyboard, a mouse, voice-activated software using a speaker and a microphone, and a coordinator that sends data via a network interface.

[0079] Now back Figure 2 Multiple instances of computing device 104 (including at least processors 102A and / or 102B) transmit data to each other via network 120 using a network connection of selected QP type 106B. Various implementations of multiple instances of computing device 104 are described below, such as data transmission between two instances of computing device 104 (e.g., one computing device is an initiating device / local device, and the other is a target device / remote device), many-to-many communication where each computing device 104 communicates with all other computing devices 104, and one-to-many communication where one computing device 104 communicates with multiple other computing devices 104.

[0080] Now back Figure 3 At position 302, a request to establish a network connection for data transfer across the network is received. The request may be initiated, for example, by an application running on the local computing device. In another example, the request may be initiated from a remote device (e.g., a client, such as the initiating device) to establish a network connection for communication with a local device (e.g., a server, such as the target device).

[0081] Requests may be made in response to and / or include devices notifying each other that they support QP-type dynamic selection capabilities, as described herein. Optionally, a handshake may be performed between the remote device and the local device (e.g., between the target device and the initiating device) to ensure that both devices support QP-type dynamic selection, as described herein. For example, capability discovery procedures and / or negotiation protocols may be used to enable and / or manage QP-type dynamic selection capabilities.

[0082] For example, a request may be received by one or more of the following: a processing circuit running an application, a network interface card that establishes a network connection for connecting to the network, and / or a hypervisor that hosts the application.

[0083] The request can establish a network connection using an application-defined QP type. Applications can choose the QP type that best suits their needs, regardless of available resources. When the selected QP differs from the requested QP type, the requesting application may not be aware of the actual selected QP.

[0084] Alternatively, a request can be made to define a dynamic QP type for establishing network connections. A dynamic QP type indicates that the application knows it can choose any candidate QP type, as described in this article.

[0085] At position 304, analyze one or more resource-related parameters. Each resource-related parameter indicates the corresponding resource-related status used to establish the network connection. Each resource-related parameter can indicate the real-time status of one or more resources used to establish the network connection and / or to process data transmitted through the established network connection. Resource-related parameters can indicate the current resource availability status of one or more components, such as one or more processors, memory, available QPs for request types, remotely connected nodes, and networks.

[0086] These parameters provide a description of the existing resource usage, which allows selection of the QP type for the network connection that will provide the best utilization of the existing resources.

[0087] Exemplary resource-related parameters include: the number of network nodes communicating with the application providing the request, the number of existing network connections of the application providing the request, the topology of the application's existing network connections, the total number of active QPs for previously established network connections, whether the remaining QPs of the request type are available for allocation, network transmission reliability, current memory utilization, and current utilization of processing circuitry.

[0088] Analysis can be performed using one or more methods. For example, a rule set is used for resource-related parameters of a selected type that generate a QP. The rule set can be defined manually and / or automatically. The rule set can be based on the prediction that following the rule set can improve the utilization of available resources and / or optimize available resources. In another example, a trained resource classifier is used to perform the analysis, which takes resource-related parameters as input and generates results of a selected type for the QP. The resource classifier is trained on a training dataset of sample resource-related parameters and labels for QP types. In yet another example, optimization code uses mathematical models and / or systems of equations to compute the type of QP that optimizes resource-related parameters and / or simulates resource results for different QP types.

[0089] At point 306, a QP type is selected from multiple candidate QP types based on the analysis. The QP type can be defined for RDMA and / or transport protocols. The QP type can be based on published definitions and / or on custom-created QP types.

[0090] The QP will be created based on the selected type. The type of the QP is specified when the QP is created.

[0091] Exemplary QP types include: reliable connection (RC), reliable datagram (RD), extended reliable connection (XRC), unreliable datagram (UD), unreliable connection (UC), scalable reliable datagram (SRD), and dynamic connection (DC), which will be discussed in detail below.

[0092] Optionally, the selected QP type is in the same group as the requested QP type. This group can be a reliable group for reliable QP types or an unreliable group for unreliable QP types. Within each group, the selected QP type may belong to a different subtype than the requested QP type. Maintaining the reliability or unreliability of the QP type based on the network connection request maintains compatibility with applications and / or processes using the data transmitted over the network. For example, applications that expect reliable data transmission receive data transmitted reliably and do not need to handle unexpectedly unreliable data transmissions.

[0093] Optionally, the network connection is for a reliable QP type, and the selected QP type is an unreliable type of QP, wherein reliability is provided by a reliability layer implemented by one or a combination of software, firmware, and hardware. The requested reliability can be provided using an unreliable QP type with said reliability layer instead of a resource-intensive reliable QP type, which can improve resource utilization by using an unreliable QP type that uses fewer resources than a reliable QP type.

[0094] A request to establish a network connection can be for one type of QP, and another QP type different from the requested QP type can be selected. The selection process can be performed in implicit mode, where the request is ignored and the QP type that best optimizes for available resources is selected, and the requesting application and / or process is unaware that the requested QP type has changed.

[0095] A request to establish a network connection can be made for a dynamically selected interchangeable QP type (IQT). An IQT is not an actual QP type, but rather an indication of the dynamic selection of a QP type, without specifying which particular QP type is requested to establish the network connection. An IQT indicates that the application will handle any QP type actually selected from the candidate types (excluding the IQT). The highest and / or best resource utilization efficiency can be attributed to the process and / or application that instructs the use of an IQT to select any candidate QP type.

[0096] Each QP type can include queue pairs: a send queue (SQ) and a receive queue (RQ). Message transmission requests can be published to the SQ, for example, when sending data across the network via an application. When each message is executed, the SQ logic transmits the outbound message transmission request to the RQ logic of the remote QP, i.e., only unmarked operations—send opcodes, since reading and writing messages do not go through the RQ. Work requests (WR) can be published to the RQ to handle certain types of inbound message transmission requests transmitted from the SQ logic of the remote QP to the RQ logic. Each computing device, i.e., a network node, can implement multiple QPs of one or more types, for example, up to one million possible QPs, each of which can be able to send messages to and receive messages from one or more QPs in the remote network nodes. The choice of QP type can reduce the total number of QPs and / or the total number of network connections and / or improve the utilization of processing resources, as described herein.

[0097] At 308, a network connection is established using the selected type of QP for data transfer across the network.

[0098] A connection is established between the QPs of two network nodes before any messages can be transmitted. The QP contexts of the two QPs can each be programmed using the identifier of the remote QP and the address of the port where the remote QP resides. For RoCE, the port is constant, and other fields exist to distinguish the application, such as:

[0099] 1. QP ID in the RoCE RC-BTH header

[0100] 2. RoCE RD-RDETH and DETH have EEC and source QP.

[0101] The following is an exemplary procedure for establishing a network connection between a client requesting to establish a network connection with a service on a remote network node and a server hosting the service provided to the client, using a selected QP type. The client sends a REQ message to the server, containing an indication of the service with which it intends to establish a network connection (e.g., ServiceID). The server verifies the service's existence, then creates a local QP and / or EEC and sends information from a message about the created QP and / or EEC back to the requesting client. At this point, the new local QP and / or EEC is in a ready-to-receive (RTR) state, ready to receive messages, but cannot send messages until the client's QP setup is complete. The client receives the message and uses the information in the message to complete the establishment of its local QP and / or EEC, transitioning it to a ready-to-send (RTS) state. The client sends a message to the server to transition its QP and / or EEC to the RTS state. Upon receiving the message, the server transitions its local QP and / or EEC to the RTS state. It should be noted that when the first packet sent by the remote QP and / or EEC is received, the server's local QP and / or EEC will automatically switch from RTR to RTS state (even if no message has been received from the client yet).

[0102] At 310, the iteration refers to one or more features described in 304 through 308. Iteration can be performed to monitor resource-related parameters to detect significant changes that trigger a reanalysis of these parameters, such as increased noise in the network, the formation of new connections to new client terminals, a decrease in available memory, and / or a decrease in processor capacity. Iteration can be performed without being triggered by monitoring. For example, resource-related parameters can be reanalyzed at defined time intervals (e.g., every minute, every 5 minutes, every 30 minutes, every hour, and other time intervals), and / or when data is transmitted over the network and / or when a network connection is active but not transmitting data, and / or each time a new network connection is initiated.

[0103] When the resource-related parameters change, the QP type of the existing connection can be dynamically changed during the use of the established network connection to transmit data over the network, in order to provide different QP types that are expected to improve resource utilization.

[0104] As in 304, when an established network connection is active, resource-related parameters can optionally be re-analyzed during data transmission through the established network connection and / or when no data transmission occurs. As in 306, based on the re-analysis, another QP type is reselected from the candidate types. As in 308, another new network connection can be established using the reselected QP type. Previously established network connections can be dynamically migrated to the new network connection, and / or data transmission through the previously established network connection can be resumed for transmission through the new network connection.

[0105] At 312, the iteration refers to one or more features described in 302 through 308. Iteration can be performed to handle new requests to establish new network connections, for example, by the same application and / or by different applications.

[0106] Following iterations 302 to 308, in 314, data traffic on a previously established network connection can be migrated to a new network connection and multiplexed with other data traffic specified for the new network connection. In this case, once the data traffic has been migrated to the new network connection, the previously established network connection can be terminated.

[0107] Alternatively, following iterations 302 through 308, in 316, the data specified for the new network connection is migrated to the previously established network connection and multiplexed with other data traffic transmitted through the previously established network connection. In this case, a new network connection is not necessarily established (as in 308).

[0108] At point 314, in response to iterations 302 through 308, data traffic on the previously established network connection can be migrated to the new network connection. Two network connections transmitting two datasets over the network can be merged into a single network connection. The merged second network connection can use fewer resources than two separate network connections.

[0109] As in 302, an additional request to establish another network connection for data transfer across the network is received from the same application that issued the previous request and / or from a different application. As in 304, additional analysis of resource-related parameters is performed. This additional analysis of resource-related parameters can be performed while the previously established network connection is active, optionally transmitting data. The additional analysis can represent the resource impact of the previously established network connection and / or the resource impact of establishing the additional network connection, optionally considering the resource impact of a combination of the previously established network connection and the requested network connection. As in 306, a different QP type is selected based on the additional analysis. As in 308, the additional network connection is established using the newly selected QP type. Now, in 314, the previously established network connection is migrated to the newly established (i.e., additional) network connection. The newly established (i.e., additional) network connection transmits data across the network for both the previous network connection and the newly established network connection, for example, by multiplexing the two data streams.

[0110] Two separate network connections, requested by two different applications for transferring two datasets over the network, can be merged into a single network connection used by both applications. The merged second network connection can use fewer resources than the two separate network connections. The previously established network connection may have been established in response to a request from a first application. The new request may be provided by a second application, different from the first application. The newly established network connection transfers data between the first and second applications over the network.

[0111] Optionally, the dynamic migration from a previously established network connection to a newly established network connection is performed by an exemplary process that suspends current network traffic on the previously established network connection to perform the migration transparently without significant disruption, and / or the application associated with the current network traffic is unaware of the migration.

[0112] Dynamic migration can be performed using the following exemplary procedure: Pause network traffic on a previously established network connection. Receive an acknowledgment message. The acknowledgment message indicates that packets transmitted over the previously established network connection before the pause have been received by a device at the other end of the previously established network connection. The acknowledgment message indicates that the previously established network connection is "empty," meaning that no packets are currently passing through the previously established network connection. Additional network traffic destined for the previously established network connection is redirected to an additional (i.e., newly established) network connection using the selected QP type.

[0113] A previously established network connection can be terminated when data traffic designed for it has been redirected to an additional (i.e., newly established) network connection. Alternatively, a previously established network connection can be terminated in response to an indication that at least one first packet of the rerouted network traffic has been delivered to a device at the other end of the additional network connection. Resources bound to the previously established network connection will be released by terminating the previously established network connection.

[0114] At point 316, data traffic destined for a new network connection can be initialized on a previously established network connection in response to iterations from 302 to 308. When a network connection with the selected second QP type already exists, network traffic destined for the newly established network connection can be added to the existing network connection. Resources can be saved by using an existing network connection instead of adding another.

[0115] As in 302, an additional request to establish another network connection for data transfer across the network is received from the same application that issued the previous request and / or from a different application. As in 304, additional analysis of resource-related parameters is performed. This additional analysis of resource-related parameters can be performed while the previously established network connection is active, optionally transmitting data. The additional analysis can represent the resource impact of the previously established network connection and / or the resource impact of establishing the additional network connection, optionally considering the resource impact of a combination of the previously established network connection and the requested network connection. As in 306, another QP type is selected based on the additional analysis. It should be noted that in 308, another new network connection is not established using the newly selected QP type. Instead, a previously established network connection using the same QP type as the newly selected QP type is identified. The previously established network connection was already established in response to the previous request before the new request to establish the additional network connection was received. Now, in 316, the previously established network connection is used to transmit data across the network traffic initially sent to the previously established network connection and data sent to the additional network connection (not independently established), for example, by multiplexing the two data streams. New data streams can be initialized on previously established network connections.

[0116] An example based on the method described in Reference 3 is now provided.

[0117] For example, in 302, during connection establishment (e.g., RDMA CM), devices (e.g., target and initiating devices, server and client) can handshake and / or notify whether they support dynamic QP type selection capability.

[0118] For example, in a 304 error, analyze resource-related parameters.

[0119] For example, in 306, select the first RC QP.

[0120] For example, in 308, a selected first RC QP and a first network connection are created on both devices. RC QP traffic begins.

[0121] For example, in 312, another iteration from 302 to 308 is performed.

[0122] For example, in a 302 response, additional requests are received. Requests may include establishing a network connection using an RC QP, or not specifying a particular QP (e.g., requesting a dynamic QP type).

[0123] For example, in a 304 error, perform additional analysis on the current state of resource-related parameters.

[0124] For example, in 306, the RD QP type is selected based on an analysis that determines that the RD QP type saves more resources than another QP type (e.g., the RC QP type). Even if the RC QP type is requested, the RD QP type can be selected.

[0125] For example, in 308, RD QPs and end-to-end contexts (EECs) are created, which are connected by a second network for use in transmitting traffic across the network.

[0126] It should be noted that there are now two QPs on each host: one is RC and the other is RD.

[0127] RD QP traffic starts from the first end-to-end (EE).

[0128] For example, in 314, data traffic on the first established network connection using the RC QP type is migrated to an additional network connection established using the RDQP type. After the EEC is created on the remote device, the additional data begins to be sent via the newly selected RD QP type.

[0129] Network interface devices and / or hypervisors use RC QPs to temporarily suspend traffic on previously established network connections to "drain the pipes," ensuring no packets are sent through the previously established network connections. In response to receiving acknowledgment messages (ACKs) for all packets sent through the previously established network connections, WQE is published on the RD QP of the attached (i.e., newly established) network connection, rather than on the RC QP of the previously established network connection.

[0130] There are now four QPs in total: one RC QP and one RD type on each side. Once the first packet is delivered via the newly established network connection using the RD QP, the previously established network connection using the RC QP is destroyed. There are now two QPs in total: one RD QP on the server and one RD QP on the client. Choosing the RD QP type improves resource efficiency compared to using only RC QPs, where there are a total of four QPs, and results in a higher memory footprint compared to choosing RD QPs and migrating from the previous RC QP to the newly established RD QP.

[0131] Now for reference Figures 4A to 4F , Figures 4A to 4F This is a schematic diagram of an exemplary QP type for selection according to some embodiments.

[0132] Figure 4A This describes the Reliable Connection (RC) QP type. RC is typically chosen for applications that require the highest quality of service when transmitting data over a network, such as mission-critical applications. RC provides the highest level of reliability and predictability. However, the RC protocol consumes significant bandwidth due to the generation of ACK (a positive acknowledgment packet returned to indicate successful reception and processing of transmit or RDMA write request packets) and NAK (a negative acknowledgment packet returned to indicate one of the following: temporary receiver not read condition, PSN sequence error NAK, and fatal NAK error).

[0133] In the RC implementation, the local QP 402 on local host 404 and the remote host 408... one A remote QP406 is associated, thus forming a dedicated channel between QPs for transmitting data 410. A hardware protocol between QP 402 and QP 406 provides reliable transmission. Reliable transmission is achieved by detecting lost, corrupted, or invalid packets and automatically pausing further activity between the two QPs, retransmitting the lost / failed packet, and then resuming operation. Each queued operation is acknowledged 412, each operation is completed exactly once, and sent in the same order. The maximum message size is not limited by packet size or the maximum transfer unit (MTU) defined for the channel. Segmentation and reassembly of long messages occur in hardware and are transparent to the application. RC supports all InfiniBands. TM Services: Send / Receive, RDMA Read, RDMA Write, and Atom.

[0134] Figure 4BThis describes the unreliable connection (UC) QP type. Because the target QP's RQ logic does not generate ACK and NAK, the bandwidth consumed by the UC protocol is significantly lower than that of the RC protocol.

[0135] The setup for unreliable connections is similar to the reference. Figure 4A The described reliable connection includes a dedicated channel between local QP 402 on local host 404 and remote QP 406 on remote host 408 for transmitting data 410. However, no acknowledgment is provided. Send queue operations are marked as complete immediately after being transmitted by the QP. Lost or erroneous messages are not automatically retried. They are discarded, and the QP does not provide the sender with any indication of whether the message was successfully delivered. Similar to the RC service, operations are performed sequentially, and the maximum message size is not limited by packet size or path MTU. Send / receive and RDMA writes are supported, but RDMA reads and atomic operations are not supported. UC provides efficient communication for certain applications (such as streaming media) where lost data is not critical.

[0136] Figure 4C This describes the unreliable datagram (UD) QP type. UD QPs can send and receive messages to and from any number of other UD QPs located in one or more other network nodes. For each received request packet, no ACK or NAK is returned. Because the target QP's logic does not generate ACKs and NAKs, the UD protocol consumes significantly less bandwidth than the RC and RD protocols.

[0137] Unreliable datagrams are a connectionless service. UD QP can send data to the system. any Other Unreliable Datagram (UD) QPs. QP 402 on initiating host 404 sends data 410A through 410C to any of QPs 406A through 406C on destination hosts 408A through 408C. The sending QP must send a key named Q_Key, which must match the receiver's Q_Key; otherwise, the message is discarded. This prevents writing to unexpected locations. The only valid operation is send / receive. QPs used for unreliable datagram services cannot detect lost data, out-of-order data, or data received multiple times. In all these cases, the message to which this data belongs is considered incorrectly received and is silently discarded. The maximum message size is limited by the maximum packet size (MTU) supported by the path (256 bytes to 4KB). If reliability is required, it is provided by the upper-layer software protocol.

[0138] Figure 4DThis describes the Reliable Datagram (RD) QP type. An RD QP can send and receive messages from any number of RD QPs residing in one or more other network nodes. It does this through one or more "pipelines" established between the local network node and one or more remote network nodes. Each "pipeline," called a Reliable Datagram Channel (RDC), acts as the conduit through which multiple local client RD QPs send and receive messages from RD QPs residing in remote network nodes. The RD protocol consumes significant bandwidth due to the generation of ACKs and NAKs.

[0139] The Reliable Datagram Service combines features of RC and UD services. Specifically, it allows the same QP to interact with multiple remote QPs simultaneously, while providing reliability. QP 402 on initiating host 404 sends data 410A to 410C to any of QPs 406A to 406C on target hosts 408A to 408C and receives acknowledgments (ACKs) 412A to 412C. Essentially, RD provides a multiplexed reliable connection channel. QP 402 is logically associated with a set of remote RD QPs 406A to 406C. This service is most useful for applications running many different processes on each node that need to communicate with each other reliably. Operations are acknowledged, completed only once, sequentially, and automatically retried in case of errors. All InfiniBands are supported. TM Services include send / receive, RDMA write, RDMA read, and atomic operations. The endpoints of a reliable datagram channel are called end-to-end contexts (EECs), EEC_1 to EEC_6, as shown in the figure. A reliable datagram domain (RDD) determines which set of RD QPs can access which set of EECs. Each EEC is shared by all reliable datagram QPs of that RDD.

[0140] Figure 4EThe Extended Reliable Connection (XRC) QP type is described. XRC is described in the context of data 410 and acknowledgments (ACKs) 412, which are transmitted between the local QP 402 of the corresponding localhost 404 and the remote QP 406 of remote hosts 404 and 408. XRC allows for significant savings in the number of QPs required to establish full-to-full-process connections in large clusters. XRC differs from RD in several ways, but first, it eliminates the most important limitation of RD transport services: each EE context has a single wait message. The savings in the total number of required QPs are achieved due to how XRC operates on the responder side. The responder connection context (denoted as XRC TGT QP) allows the requesting process to send messages for multiple destination XRCSRQ 414s belonging to multiple processes on the responder node. Therefore, using a single (XRC INI) QP, a process in one node can communicate with all processes on remote nodes, thus reducing the total number of QPs required for a full connection by a factor of p (processes per node) compared to using RC QP. XRC SRQ 414 are receive queues for each process on the responding node, which can be targeted to multiple remote end nodes via XRC TGT QP. They are somewhat equivalent to receive queues in RD QP, so each process only needs one to allow it to receive messages from any process on any node in the cluster. Similar to how RD QP is restricted to use within the same reliable datagram domain (RDD) and RD EE context, the XRC transport service implements an equivalent XRC domain mechanism serving the same purpose. XRC TGT QP can only be used as a pipe to access XRC SRQs set up on the same XRC domain.

[0141] Figure 4FThis describes the dynamically connected (DC) QP type. QP402 on initiating host 404 sends data 410A to 410C to any of QPs 406A to 406C on target hosts 408A to 408C and receives acknowledgments (ACKs) 412A to 412C. DC is a scalable transport service that reduces the number of QPs per node compared to RC. DC has RC-like reliability semantics. DC has a symmetric API. On the responder side, there is either a DC target device or a DCT 416, either one is sufficient. On the requester side, there is either a DC initiator device or a DCI 418, either one is sufficient. DC forms a “temporary connection.” The first sending WR on the DCI connects this DCI to a remote DCT. The second sending WR uses this open connection. The DCI disconnects after a period of idle time without sending. The DCI can “switch destinations” if the next sending WR specifies a different destination. The DCT has a “responder” (DCR) pool. Each incoming DC connection is assigned a DCR. DCI recycling trade-offs: Too few DCIs—the same DCIs are switched back and forth between destinations. Redundant connection / disconnection streams (worst-case: every send). Impaired latency. Too many DCIs—still not as bad as N^2RC QP. Consumes resources, detrimental to caching. Best practice is to maintain... <dcidest>Hash tables are used to reduce connection rebuilding. An LRU (Least Recently Used) eviction policy is used to increase the probability of selecting disconnected DCI transmissions.

[0142] Now for reference Figures 5 to 7 , Figures 5 to 7 This is a schematic diagram illustrating a comparison between the dynamic selection of QP types according to some embodiments and the standard method.

[0143] Figure 5 The process of establishing an RC connection according to standard methods is described. Schematic diagram 502A illustrates the use of RC QP type 510A to establish a first RC network connection 504A between server host 506 and client host 508 (for reliable data transmission by providing ACK).

[0144] Diagram 502B illustrates the use of RC QP type 510B to establish a second RC network connection 504B between server 506 and client 508. The first RC connection 504A connects process A 512 running on server 506 and process C 514 running on client 508. The second RC connection 504B connects process A 512 running on server 506 and process D 516 running on client 508.

[0145] Figure 6 An example of a process for establishing a connection using an embodiment of dynamically selected QP type described herein is presented. Schematic diagram 602A illustrates the establishment of a first RC network connection 604A (for reliable data transmission by providing ACK) between server host 606 and client host 608 using RC QP type 610A. It should be noted that the process described in schematic diagram 602A can be compared with... Figure 5 The method is the same as the standard method described in schematic diagram 502A.

[0146] Diagram 602B illustrates a response to a request from client 608 to add a second RC connection to server 606. The first RC connection 604A lies between process A 612 running on server 606 and process C 614 running on client 608. The request for the second RC connection lies between process A 612 running on server 606 (i.e., the same as the first RC connection) and process D 616 running on client 608 (i.e., different from the first RC connection). In response to receiving the request to establish the second RC connection, the QP type on both server 606 and client 608 changes from RC type 610A to RD type 610B. For example, the RD QP type can be selected to save hardware resources and / or when no more RC QPs are available.

[0147] Figure 7 Another example of a connection establishment process using the embodiment of dynamically selected QP type described herein is presented. Schematic diagram 702A illustrates the establishment of a first RC network connection 704A (for reliable data transmission by providing ACK) between server host 706 and client host 708 using RC QP type 710A. It should be noted that the process described in schematic diagram 702A can be compared with... Figure 5 Schematic diagram 502A and / or Figure 6 The method is the same as that described in schematic diagram 602A.

[0148] Schematic diagram 702B illustrates a response to a request from client 708 to add a second RC connection to server 706. Using standard methods, a second RC network connection would be established between server 706 and client 708. However, according to at least one embodiment described herein, DC QP type 710B is dynamically selected and used to establish network connection 704B between server 706 and client 708. The previous RC QP type 710A used to establish network connection 704A is migrated to network connection 704B established using DC QP 710B. For example, the DC QP type can be selected to save hardware resources and / or when no more RC QPs are available.

[0149] Other systems, methods, features, and advantages of the present invention will be apparent to those skilled in the art upon review of the accompanying drawings and detailed description below. It is intended that all such other systems, methods, features, and advantages be included in this specification, within the scope of the invention, and protected by the appended claims.

[0150] The description of various embodiments of the present invention is for illustrative purposes only and is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical advancement of the embodiments, or to enable others skilled in the art to understand the embodiments disclosed herein, compared to technologies available on the market.

[0151] It is anticipated that many related QP types will be developed during the term of the patent expiring in this application, and the scope of the term QP type is intended to a priori include all such new technologies.

[0152] The term "about" as used in this article refers to ±10%.

[0153] The terms "including," "having," and their variations mean "including but not limited to." This term includes the terms "consisting of" and "substantially consisting of."

[0154] The phrase "consistently of..." indicates that a composition or method may include additional ingredients and / or steps, provided that the additional ingredients and / or steps do not substantially alter the fundamental and novel characteristics of the claimed composition or method.

[0155] Unless the context clearly indicates otherwise, the singular forms "a" and "the" used herein include the plural meaning. For example, the terms "a complex" or "at least one complex" can include multiple complexes, including mixtures thereof.

[0156] As used herein, the term "exemplary" means "as an example, instance, or illustration." Any embodiment described as "exemplary" is not necessarily to be construed as being more preferred or advantageous than other embodiments, and / or as excluding combinations of features of other embodiments.

[0157] As used herein, the term "optionally" means "provided in some embodiments and not in others." Unless multiple "optional" features conflict, any particular embodiment of the invention may include these features.

[0158] In this application, various embodiments of the invention are presented in a range format. It should be understood that the range format description is for convenience and brevity only and should not be construed as a fixed limitation on the scope of the invention. Therefore, the description of a range should be considered as specifically disclosing all possible subranges and individual numerical values ​​within said ranges. For example, a description of a range, such as from 1 to 6, should be considered as specifically disclosing subranges from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., and individual numbers within those ranges, such as 1, 2, 3, 4, 5, and 6. This applies regardless of how broad the range may be.

[0159] When this document indicates a range of numbers, it means that any of the listed numbers (fractions or integers) within the indicated range are included. The phrases "the range between the first and second indicated numbers" and "the range from the first to the second indicated number" are used interchangeably in this document to mean that the first and second indicated numbers, as well as all fractions and integers in between, are included.

[0160] It should be understood that, for the sake of brevity, certain features of the invention described in the context of a single embodiment may also be provided in combination in a single embodiment. Conversely, for the sake of brevity, various features of the invention described in the context of a single embodiment may also be provided individually or in any suitable sub-combination or as any other suitable embodiment of the invention. Certain features described in the context of various embodiments are not considered essential features of these embodiments unless the embodiment is inoperable without these elements.

[0161] All publications, patents, and patent applications mentioned herein are incorporated herein by reference in their entirety, and each individual publication, patent, or patent application is also specifically and individually incorporated herein by reference. Furthermore, any reference or designation to any application shall not be construed as allowing such reference to take precedence over the invention in the prior art. Section headings should not be construed as necessary limitations with respect to their use.< / dcidest>

Claims

1. A processing circuit (102A, 102B) for selecting the type of QP for data transmission across a network (120), characterized in that, The processing circuit is used for: Receive a request to establish a network connection for data transmission across the network; Analyze multiple resource-related parameters, each indicating the corresponding resource-related status used to establish the network connection; Based on the analysis, the type of QP is dynamically selected from multiple candidate types; The network connection is established using the selected type of the QP to transmit data across the network.

2. The processing circuit according to claim 1, characterized in that, Also used for: The multiple resource-related parameters are re-analyzed during data transmission via the established network connection; Based on the reanalysis, another type of QP is selected from the plurality of candidate types; Establish a second network connection using the reselected type of the QP; Perform at least one of the following operations: dynamically transfer the network connection to the second network connection, and re-initiate the transmission of data of the network connection to be transmitted through the second network connection.

3. The processing circuit according to claim 1, characterized in that, Also used for: The network connection includes a first network connection. Receive a second request to establish a second network connection for data transmission across the network; A second analysis is performed on the plurality of resource-related parameters, each resource-related parameter indicating the corresponding resource-related status of the established first network connection and used for the establishment of the second network connection; Based on the second analysis, a second type of QP is selected from the plurality of candidate types; The second network connection is established using the second type of the QP; The first network connection is dynamically migrated to the second network connection, wherein the second network connection transmits data across the network for both the first network connection and the second network connection.

4. The processing circuit according to claim 3, characterized in that, The request is provided by the first application, the second request is provided by the second application, and the second network connection transmits data for the first application and the second application through the network.

5. The processing circuit according to claim 3, characterized in that, Dynamic migration includes: Suspend network traffic on the first network connection; Receive an acknowledgment message that a packet sent via the first network connection before the pause has been received by a device at the other end of the first network connection; The second type of the QP of the second network connection is used for additional network traffic sent to the first network connection.

6. The processing circuit according to claim 5, characterized in that, It is also used to: terminate the first network connection in response to an indication that at least one first packet of the additional network traffic has been delivered through the second network connection.

7. The processing circuit according to claim 1, characterized in that, Also used for: Receive a third request to establish a third network connection for data transmission across the network; A third analysis is performed on the multiple resource-related parameters, where each resource-related parameter indicates the corresponding resource-related status of the established network connection and is used for the establishment of the third network connection; The third type of QP is selected from the plurality of candidate types based on the third analysis; Prior to receiving the third request, a fourth network connection of the third type using the QP had previously been established by the processing circuitry for the fourth request; The fourth network connection is used to transmit data across the network associated with the third request and the fourth request, wherein the third network connection is not established independently.

8. The processing circuit according to claim 1, characterized in that, The request to establish the network connection is for a first type of QP, and a second type of QP, different from the first type, is selected.

9. The processing circuit according to claim 1, characterized in that, The request to establish the network connection is for a first subtype of QP, either a reliable or unreliable QP, and wherein the selected type of QP is a second subtype of QP defined by the request, which is different from the first subtype of QP defined by the request.

10. The processing circuit according to claim 9, characterized in that, The network connection is for the QP reliable type, and the selected type of the QP is the QP unreliable type, wherein reliability is provided by a reliability layer implemented by at least one or a combination of software, firmware and hardware.

11. The processing circuit according to claim 1, characterized in that, The request to establish the network connection is for an interchangeable QP type (IQT), wherein the type of the QP is selected from a plurality of candidate types excluding dynamic types.

12. The processing circuit according to any one of claims 1-11, characterized in that, The resource-related parameters are selected from the group consisting of: the number of network nodes communicating with the application providing the request, the number of existing network connections of the application providing the request, the topology of the application's existing network connections, the total number of active QPs for network connections established by the processing circuitry, whether the QP of the requested type is available, the transmission reliability of the network, the current memory utilization, and the current utilization of the processing circuitry.

13. The processing circuit according to any one of claims 1-11, characterized in that, The multiple candidate types of the QP are selected from the group consisting of the following: reliable connection (RC), reliable datagram (RD), extended reliable connection (XRC), unreliable datagram (UD), unreliable connection (UC), scalable reliable datagram (SRD), and dynamic connection (DC).

14. The processing circuit according to any one of claims 1-11, characterized in that, The analysis is performed using at least one of the following: a rule set, which is used to generate the plurality of resource-related parameters of the selected type of the QP; A classifier that receives the plurality of resource-related parameters as input and generates a result of the selected type of the QP, wherein the classifier is trained on a training dataset of the plurality of sample resource-related parameters and labels of the QP type; The optimization code uses mathematical models and / or systems of equations to calculate the type of QP that optimizes the plurality of resource-related parameters.

15. The processing circuit according to any one of claims 1-11, characterized in that, The data transmissions across the network are performed according to the remote direct memory access (RDMA) protocol, and the multiple candidate types of QP are defined by the network transport protocol used for RDMA.

16. The processing circuit according to claim 15, characterized in that, The network transport protocols for RDMA defined by the multiple candidate types of QP are selected from the group consisting of: InfiniBand, IB, remote direct memory access over converged ethernet (RoCE), RoCEv2, iWARP, and derivatives thereof.

17. A method for selecting the type of QP for data transmission across a network, characterized in that, include: Receive a request to establish a network connection for data transmission across the network (302); Analyze multiple resource-related parameters, each indicating the corresponding resource-related status of the network connection used to establish it (304); Based on the analysis, the type of QP is dynamically selected from multiple candidate types (306); The network connection is established using the selected type of the QP to transmit data across the network (308).

18. A computer program product comprising program instructions (150), characterized in that, When the program instructions (150) are executed by the processors (102A, 102B), the processors: Receive a request to establish a network connection for data transmission across the network; Analyze multiple resource-related parameters, each indicating the corresponding resource-related status used to establish the network connection; Based on the analysis, the type of QP is dynamically selected from multiple candidate types; The network connection is established using the selected type of the QP to transmit data across the network.

Images