Decoupling congestion management state from connection management state in high performance computing

Abstract

The invention relates to decoupling a congestion management state from a connection management state in high performance computing. A first network endpoint establishes a connection with a second network endpoint by sending a control packet including a first identifier associated with the connection and the first network endpoint. The first network endpoint stores a first connection state associated with the connection in a first data structure based on the first identifier, and stores a first congestion state associated with the connection in a second data structure based on the first connection state. The first network endpoint identifies, for a data stream associated with a first identifier, a congestion state corresponding to the data stream by obtaining a first connection state by searching a first data structure based on the first identifier; and identifying a first congestion state by searching a second data structure based on the obtained first connection state.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 584,676 (Agent Docket No. P173311USPRV), filed on September 22, 2023, by inventors Keith D. Underwood and Robert L. Alverson, entitled “DECOUPLING CONGESTION MANAGEMENT STATE AND CONNECTION STATE IN A HIGH PERFORMANCE NIC.” Background Art Technical Field

[0003] Congestion management can be a fundamental process in modern high-performance data centers. High-bandwidth networks in these data centers may experience congestion, for example, at the midplane or at the endpoints. In order to manage congestion, a network endpoint (e.g., a network interface controller (NIC)) can maintain a congestion management state, which can include information about how much data can be allowed to enter the network and the quality of the path through the network. In most current solutions, the congestion management state is associated with the connection state, wherein each connection can maintain its connection state independently. In some cases, a network may include two or more connections between a single pair of endpoints, wherein each connection is associated with its own congestion management state. Therefore, duplicating the congestion management state may incur space costs. In addition, when two flows from one NIC compete with each other for the bandwidth of the NIC, congestion management may not be performed efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 An exemplary network architecture according to an aspect of the present application is illustrated.

[0005] Figure 2A Pseudo code according to one aspect of the present application is illustrated for creating and accessing two data structures for connection status and congestion status based on indirect indexing.

[0006] Figure 2B The diagram shows a schematic diagram of a device according to one aspect of the present application. Figure 2A A diagram of the information stored and accessed in the connected array and the congested array shown in FIG.

[0007] Figure 3A Pseudo-code for creating and accessing two data structures for connection status and congestion status based on encoding of an identifier is illustrated in accordance with an aspect of the present application.

[0008] Figure 3B The diagram shows a schematic diagram of a device according to one aspect of the present application. Figure 3A A diagram of the information stored and accessed in the connected array and the congested array shown in FIG.

[0009] Figure 4A Pseudo code according to one aspect of the present application is illustrated for creating and accessing two data structures for connection status and congestion status based on encoding of identifiers and including multiple connection status embedded in congestion status.

[0010] Figure 4B The diagram shows a schematic diagram of a device according to one aspect of the present application. Figure 4A A diagram of the information stored and accessed in the connected array and the congested array shown in FIG.

[0011] Figure 5 A flow chart illustrating a method of facilitating decoupling congestion management from connection state in a NIC according to an aspect of the present application is presented.

[0012] Figure 6 A computer system that facilitates decoupling congestion management state from connection state in a NIC according to an aspect of the present application is illustrated.

[0013] Figure 7 A computer-readable medium that facilitates decoupling congestion management state from connection state in a NIC according to an aspect of the present application is illustrated.

[0014] In the drawings, like reference numerals refer to the same drawing elements. DETAILED DESCRIPTION

[0015] The following description is presented to enable any person skilled in the art to make and use the various aspects and examples, and is provided in the context of a specific application and its requirements. Various modifications to the disclosed aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects and applications without departing from the spirit and scope of the present disclosure. Therefore, the various aspects described herein are not limited to the aspects shown, but are intended to conform to the maximum scope consistent with the principles and features disclosed herein.

[0016] The described aspects address the efficiency and performance losses that occur when each connection between endpoints in a network must maintain its own congestion management state by decoupling congestion management state from connection state and maintaining the connection state and congestion state in different data structures.

[0017] As described above, congestion management can be a fundamental process in modern high-performance data centers. High-bandwidth networks in these data centers may experience congestion at network endpoints (e.g., NICs). To manage congestion, network endpoints can maintain congestion management states that can include information about how much data can be allowed into the network (e.g., "window size") and the quality of the path through the network. In most current solutions, the congestion management state is associated with the connection state, where each connection can maintain its connection state independently.

[0018] In some cases, a network may include two or more connections between a single pair of endpoints, where each connection is associated with its own congestion management state. One reason for including multiple connections may be for performance considerations, for example, multiple data packets may be required to maintain a full packet rate. Another reason for including multiple connections may be due to the need for concurrency in the data flow. Two or more different applications may want to send data between the same pair of endpoints without interference, because, for example, there are different types of traffic that may not require a connection at all. For example, a target node may send "a lot of data" in response to a read request from an initiator node. Congestion control may be expected for a large amount of data transmitted in response (i.e., from the target node to the initiator node), but there may be no connection flowing in the reverse direction (e.g., from the initiator node to the target node).

[0019] When two or more connections, each with their own congestion management state, share a single data path between a pair of endpoints (e.g., between a pair of NICs), this can result in efficiency and performance losses. Similarly, when one or more connections share a single data path with data flows that do not require a connection between a pair of endpoints, this can also result in efficiency and performance losses. In one example, an HPE Cray Slingshot network may not use a connection for data sent in response to a request for data. In this example, an initiator node (a first NIC) may issue a "get" or "read" operation to a target node (a second NIC). The target node may provide the requested data in response to the "get" or "read" operation, but the transmission of the response may not use a connection.

[0020] Some current solutions may lack the ability to tie together the congestion management state of response data (i.e., flowing from a target node to an initiator node) with the congestion management state of requests flowing in the same direction (i.e., requests from a target node to an initiator node). Similarly, some current solutions may be limited to a single connection between a pair of NICs because having independent congestion management states for separate connections results in efficiency and performance losses.

[0021] In one example of efficiency loss, duplicating congestion management state may incur a space cost. The congestion management state may constitute 32 bytes or more, and duplicating this information may incur a cost in silicon area. In one example of performance loss, when two flows from one NIC compete with each other for the bandwidth of the NIC, congestion management may not be performed efficiently.

[0022] Furthermore, "connections" in a network can be dynamic and transient. Connections in a network can be established to transfer data and can be torn down after a certain period of time without any data transfer. Technologies that may be suitable for static, long-term, persistent connections may not be as effective in the realm of dynamic, transient connections.

[0023] The described aspects decouple congestion management state from connection state by using three main solutions that can maintain connection state and congestion state in different data structures to address the challenges described above. In a first solution, the connection state table can include an "indirect index", that is, an index into a corresponding element of the congestion state table. In a second solution, an encoding of a connection identifier can be used to directly index into the congestion state table and identify the congestion state associated with the connection state. In a third solution based on the second solution, the encoding can be used to directly index into the congestion state, including the corresponding connection state of the congestion state as a sub-element in the congestion state table.

[0024] Thus, the described aspects can eliminate inefficient duplication of congestion management state for each connection and provide a solution for determining how a connection or data flow can identify its associated congestion management state and how a scheduler (which schedules data packets based on the congestion management state) can identify all connections associated with that connection management state.

[0025] Example Network Architecture

[0026] Figure 1A diagram 100 of an exemplary network architecture according to an aspect of the present application is illustrated. Diagram 100 may include a network 110 of switches (which may be referred to as a "switch fabric") and may include switches 112, 114, 116, 118, and 120. Each switch may have a unique address or identifier within the switch fabric 110. Various types of devices and networks may be coupled to the switch fabric. For example, a storage array 130 may be coupled to the switch fabric 110 via switch 112; a high performance computing (HPC) network (e.g., Infiniband, Slingshot, or any other high performance network) 132 may be coupled to the switch fabric 110 via switch 114; multiple end hosts (such as host 136 and host 138) may be coupled to the switch fabric 110 via switch 118; and an Internet Protocol (IP) / Ethernet network 134 may be coupled to the switch fabric 110 via switch 120. An HPC network may include multiple networked computers and storage devices that simultaneously run programs to complete different complex and performance-intensive tasks. An IP / Ethernet network can include application layer protocols between physical Ethernet cabling and IP-based network devices, including communications via Transport Communication Protocol (TCP) / IP and User Datagram Protocol (UDP) packets. Typically, a switch can have edge ports and fabric ports. Edge ports can be coupled to devices outside the fabric. Fabric ports can be coupled to another switch within the fabric via fabric links. Typically, traffic can be injected into the switch fabric 110 via an ingress port of an edge switch and can leave the switch fabric 110 via an egress port of another (or the same) edge switch. An ingress link can couple a NIC of an edge device (e.g., an HPC terminal host) to an ingress edge port of an edge switch. The switch fabric 100 can then send traffic to an egress edge switch, which in turn can pass traffic to a destination edge device via another NIC.

[0027] First solution: Indirect index from connection state table to congestion state table

[0028] When establishing a connection between two network endpoints or NICs (e.g., a first NIC and a second NIC), the first NIC may be an "initiator node" or "sender" and the second NIC may be a "target node" or "receiver." The first NIC may establish a connection with the second NIC by sending a control packet that includes a connection identifier (also referred to as a "connection ID" or "connection_ID") for the first NIC and a connection ID for the second NIC. Each of a pair of NICs may maintain its own connection state (referred to as a "connection array" or "connection_array") and a corresponding congestion management state (referred to as a "congestion array" or "congestion_array"). The network endpoints or NICs described herein may refer to, for example, Figure 1 network 110 or a switch in switch structure 110 .

[0029] In a first solution, the connection state table may include an "indirect index" as used herein, as described below with respect to Figure 2A Describes the index into the corresponding element of the congestion state table.

[0030] Figure 2A Pseudocode 200 is illustrated according to one aspect of the present application, which is used to create and access two data structures for connection status and congestion status based on indirect indexing. The pseudocode 200 may include a section 202 defining a data structure or table of connection_array, and a section 204 defining a data structure or table of congestion_array. The pseudocode 200 may also include: a line 206 that creates a data structure; a line 208 that obtains a connection status for a given connection ID; and a line 210 that identifies a corresponding congestion status for the obtained connection status.

[0031] Section 202 may represent the definition of a structure of a connection_array using a first solution (referred to as “solution1_connection_state”), wherein: “next_sequence_number” indicates the next sequence number to be sent (if the structure is on the sending end) or the expected next sequence number (if the structure is on the receiving end); “*pending packets” indicate packets to be processed by a given node; “congestion_index” (203) indicates the index of the corresponding entry in the congestion_array; and “active” indicates the state of the connection, for example, whether the entry is active or inactive.

[0032] Section 204 may represent the definition of the structure of a congestion_array using a first solution (referred to as “solution1_congestion_state”), where: “congestion_window_size” indicates the maximum number of packets or bytes that can be sent at one time or before receiving an acknowledgment; “total_outstanding_data” indicates the number of packets or bytes of data that have been sent but not yet acknowledged; “congestion_rate” indicates the maximum rate at which data can move along a path or the total capacity of a path; “path_quality

[64] ” indicates the ability of a path to transmit data; “connection_indices[8]” (205) indicates an array of connection IDs corresponding to a given congestion_state array; and “active” indicates a state, such as whether an entry is active or inactive.

[0033] Line 206 may indicate the following: connection_array may be defined based on the solution1_connection_state data structure defined in 202 to have, for example, 32768 elements; and congestion_array may be defined based on the solution1_congestion_state data structure defined in 204 to have, for example, 4096 elements.

[0034] The appropriate entry representing the connection state may be obtained by using the connection_ID as an index into the connection_array (as indicated by row 208). Subsequently, the corresponding congestion state may be identified by using the congestion index in the obtained connection state entry as an index into the congestion_array (as indicated by row 210). Rows 208 and 210 demonstrate how the first solution may use the congestion index (203) in the connection_array as an indirect index into the congestion_array.

[0035] Figure 2B The diagram shows a schematic diagram of a device according to one aspect of the present application. Figure 2A220 shows information stored and accessed in the connection array and congestion array shown in FIG. 220 depicts connection_array 230 having entries at indices 0 to M (indicated by dashed circles), where M may be, for example, 32768. The entries in connection_array 230 may be, for example, Figure 2A For example, the entry 231 at index "1" (i.e., when connection_ID is "1") may include the following elements: "next_sequence_number" 232; "*pending packets" 233; "congestion_index" 234 with a value of "3" (as indicated by label 236); and a state 235 of "active" whose value is set to "1", indicating that the connection state indicated by this entry 231 is active.

[0036] Multiple connections may share the same congestion state. For example, an entry 241 at index "M" (i.e., when connection_ID is "M") may include the following elements: "next_sequence_number" 242; "*pending packets" 243; "congestion_index" 244 with a value of "3" (as indicated by label 246); and a state 245 of "active" whose value is set to "1", indicating that the connection state indicated by this entry 241 is active.

[0037] Diagram 220 also depicts congestion_array 250 having entries at indices 0 to N (indicated by dashed circles), where N may be, for example, 4096. The entries in congestion_array 250 may be as follows: Figure 2A 204 of . For example, the entry 251 at index "3" may include the following elements: "congestion_window_size" 252; "total_outstanding_data" 253; "congestion_rate" 254; "path_quality

[64] " 255; "connection_indices[8]" 256 array value "[1,4,...M]" (as indicated by label 258); and "active" state 257, the value of which is set to "1", indicating that the congestion management state indicated by this entry 251 is active.

[0038] As indicated by congestion_index 234, entry 251 in congestion_array 250 may correspond to entry 231 in connection_array 230 (indicated by arrow 260). Thus, for a given data flow between an initiator node and a target node, a connection_ID (e.g., the connection_ID of the initiator node) may be used to obtain a connection state (e.g., entry 231 in connection_array 230) for the given data flow. After obtaining the connection state (entry 231), a congestion index (e.g., congestion_index 234 having a value of "3" as indicated by label 236) in the connection state (entry 231) may be used as an index for identifying a corresponding congestion state (e.g., entry 251 having an index of "3" in congestion_array 250).

[0039] The obtained connection state and the identified congestion management state can be used by a scheduler (which can schedule data packets based on congestion management) to identify all connections associated with a given congestion management state. Scheduling can be performed on an active congestion array. When an element in the congestion array indicates that it is ready to be scheduled, the scheduler can obtain an available connection from the congestion array. An acknowledgment sent in response to the sent or received data can be used to access the connection array to complete one or more pending data packets (233). After accessing the connection array, the scheduler can use the congestion index to update the congestion array, for example, to total outstanding data (253).

[0040] For example, the scheduler may determine that a congested element is ready for scheduling. After identifying the connection state (entry 231) and the corresponding congestion state (entry 241), the scheduler may check the state of the connection entry at the index indicated in "connection_indices[8]" (e.g., element 256 having a value of "[1,4,...,M]" as indicated by label 258). For each of the indices listed in element 256 (i.e., 1, 4,...,M), the scheduler may check the corresponding element in connection_array230 to determine whether the state is active (e.g., whether the value of the element "active" is "0" indicating inactivity or "1" indicating activity). If the element is active (or its Boolean "active" element is set to a value of "1"), the scheduler may schedule the pending data packet indicated in the given connection state entry.

[0041] In the example of FIG. 200 , the connection_indices for congestion_array entry 251 are listed as “[1, 4, …M],” which instructs the scheduler to look up the entries at those indices in connection_array 230, check the status, and schedule the pending packet if the status is active. Entry 231 corresponds to index 1 and indicates an active status (235), so the scheduler can schedule the pending packet (233) indicated in entry 231 to be processed. Entry 241 corresponds to index M and indicates an active status (245), so the scheduler can schedule the pending packet (243) indicated in entry 241 to be processed. Note that although only entries at index 1 and index M are depicted in connection_array 230 (and not at index 4), the scheduler can similarly look at the active status in each listed index to determine whether to schedule the pending packet indicated in a given entry.

[0042] In this first solution, elements of the congested array can be dynamically allocated. As part of establishing a connection between an initiator node and a target node, the system can determine whether a congested array element exists for the target node. If the congested array element does exist, the system can use the congested array element. If the congested array element does not exist, the system can allocate a new element in the congested array.

[0043] The connection can be expected to exist for at least a few microseconds. In general, the solution may not need to establish a connection every cycle. Therefore, various data structures can be used to track the allocated congestion state elements and identify the congestion state element after the connection is established. After the connection is established, accessing the congestion state can be a simple static random access memory (SRAM) access, which can be performed at one element per cycle.

[0044] In a first solution, a first network endpoint establishes a connection with a second network endpoint by sending a control packet, the control packet including a first identifier associated with the connection and the first network endpoint. The first network endpoint stores a first connection state associated with the connection in a first data structure based on the first identifier. The first network endpoint stores a first congestion state associated with the connection in a second data structure based on the first connection state. The system identifies a congestion state corresponding to a data flow associated with the first identifier. Identifying the corresponding congestion state includes: obtaining the first connection state by searching the first data structure based on the first identifier; and identifying the first congestion state by searching the second data structure based on the obtained first connection state. The first network endpoint stores the first connection state in a first entry in a first data structure, the first data structure including the connection state associated with the connection between the first network endpoint and at least the second network endpoint; and the first network endpoint stores the first congestion state in a second entry in a second data structure, the second data structure including the congestion state associated with the connection between the first network endpoint and at least the second network endpoint. The first entry indicates at least a first identifier associated with the connection, one or more data packets to be sent, a first state associated with the connection, and a congestion index associated with the first congestion state stored in the second data structure. The system identifies the first congestion state by searching the second data structure further based on the congestion index indicated in the first entry.

[0045] Second solution: Encode the connection ID to directly index the congestion state table

[0046] In the second solution, the encoding of the connection identifier (ID) can be used to directly index into the congestion state table and to identify the congestion state associated with the connection state. As described above, the connection ID can be used in the network when establishing a connection. In general, a data packet can include an initiator node (or source) connection ID and a target node (or destination) connection ID. The second solution can use these connection IDs to directly index the connection array and the congestion array, thereby removing the link index used in the first solution.

[0047] Figure 3APseudocode 300 is illustrated according to one aspect of the present application, which is used to create and access two data structures for connection status and congestion status based on the encoding of the identifier. Pseudocode 300 may include sections 302 and 304, which define the data structures or tables of connection_array and congestion_array, respectively. Pseudocode 300 may also include: row 306 that creates the data structure; rows 308 to 309 that obtain the connection status based on the first encoding and identify the corresponding congestion status; and rows 310 to 311 that obtain the connection status based on the second encoding and identify the corresponding congestion status.

[0048] Section 302 may represent the definition of a structure of a connection_array using a second solution (referred to as “solution2_connection_state”), where: “next_sequence_number” indicates the next sequence number to be sent (if the structure is on the sending end) or the expected next sequence number (if the structure is on the receiving end); “*pending packets” indicates data packets to be processed by a given node; and “active” indicates the state of the connection, for example, whether the entry is active or inactive.

[0049] Section 304 may represent the definition of the structure of a congestion_array using a second solution (referred to as “solution2_congestion_state”), where: “congestion_window_size” indicates the maximum number of packets or bytes that can be sent at one time or before receiving an acknowledgment; “total_outstanding_data” indicates the number of packets or data bytes that have been sent but not yet acknowledged; “congestion_rate” indicates the maximum rate at which data can move along a path or the total capacity of a path; “path_quality

[64] ” indicates the ability of a path to transmit data; and “active” indicates a state, such as whether an entry is active or inactive.

[0050] Line 306 may indicate the following: connection_array may be defined based on the solution2_connection_state data structure defined in 302 to have, for example, 32768 elements; and congestion_array may be defined based on the solution2_congestion_state data structure defined in 304 to have, for example, 4096 elements.

[0051] The system can directly index both the connection array and the congestion array using the appropriate connection ID (included in the data packet as the initiator connection ID and the target connection ID). As an example, rows 308 to 309 indicate how to access the connection state and the corresponding congestion state based on the connection_ID and the first encoding of the connection_ID. The appropriate entry representing the connection state can be obtained by using the connection_ID as a direct index to the connection_array (as indicated by row 308). Subsequently, the corresponding congestion state can be identified by using the encoding (i.e., the first encoding) of the connection_ID as an index to the congestion_array (rather than the congestion index as in the first solution). The encoding can be, for example, to shift the connection_ID to the right by three bits. In the case where the connection_ID contains 15 bits, the encoded index (in 309) can contain 12 bits, which can result in the maximum overlap between the two indexes (i.e., a total of 12 bits).

[0052] The first encoding of the connection ID indicated in rows 308 to 309 can represent a continuous linear space. Other variations including the use of data structures of the same size may also be possible. As another example, rows 310 to 311 indicate how to access the connection state and the corresponding congestion state based on connection_ID or the second encoding of connection_ID. The appropriate entry representing the connection state can be obtained by using some bits of connection_ID as an index to connection_array (as indicated by row 310). Subsequently, the corresponding congestion state can be identified by using (rather than the congestion index as in the first solution) the encoding of connection_ID (i.e., the second encoding) as an index to congestion_array (as indicated by row 311). The encoding can be, for example, isolating or masking the bit outside the connection_ID and using the bottom 15 bits as an index for connection_array (as indicated by row 310) and using the upper 12 bits as an index for congestion_array (as indicated by row 311). In the case where the connection_ID contains 27 or more bits, the encoded index (in 311) may contain 12 bits, which may result in zero overlap between the two indices (ie, 0 common bits).

[0053] The examples of rows 308 to 309 and rows 310 to 311 are provided for illustrative purposes only. Other variations may be used. That is, any combination of encodings or encoding sequences of connection_IDs may be used to directly index one or more relevant portions of connection_array and congestion_array. By using such encodings, searches performed by the system (e.g., a scheduler) do not need to be serialized and may be performed simultaneously.

[0054] Figure 3B The diagram shows a schematic diagram of a device according to one aspect of the present application. Figure 3A 320 shows the information stored and accessed in the connection array and congestion array shown in FIG. 320 depicts connection_array 330 having entries at indexes 0 to M (indicated by dashed circles), where M may be, for example, 32768. The entries in connection_array 330 may be as follows: Figure 3A For example, entry 331 may include the following elements: "next_sequence_number" 332; "*pending packets" 333; and a state 335 of "active" whose value is set to "1", indicating that the connection state indicated by this entry 331 is active.

[0055] Diagram 320 also depicts congestion_array 350 having entries at indices 0 to N (indicated by dashed circles), where N may be, for example, 4096. The entries in congestion_array 350 may be, for example, Figure 3A 304. For example, entry 351 may include the following elements: "congestion_window_size" 352; "total_outstanding_data" 353; "congestion_rate" 354; "path_quality

[64] " 355; and a state 357 of "active", whose value is set to "1", indicating that the congestion management state indicated by this entry 351 is active.

[0056] The index for entry 331 in connection_array 330 may be, for example, "conn_ID_1" 341 (as indicated by Figure 3A The system may apply the first encoding to index 341 to obtain a corresponding index "enc1(conn_ID_1)" 361 in congestion_array 350 (as indicated by Figure 3AArrow 370 may be used to indicate searching and obtaining the corresponding congestion state from the connection state.

[0057] The system may also apply different encodings to the connection ID. The index for entry 331 in connection_array 330 may be, for example, "conn_ID_2" 342 and the index 342 itself may be encoded (e.g., by Figure 3A The system may apply the second encoding to index 342 to obtain a corresponding index "enc2(conn_ID_2)" 362 in congestion_array 350 (as indicated by row 310 of FIG. 1 ). Figure 3A The arrow 370 may be used to indicate the search and acquisition of the corresponding congestion state from the connection state.

[0058] In some aspects, only two types of data streams may be multiplexed onto a single congestion state. The first type of data stream may include requests (including requests with data) and the second type of data stream may include responses with data. For example, a first network endpoint ("A") may transmit data to a second network endpoint ("B") (as a first type of data stream), and the first network endpoint may also transmit data to the second network endpoint in response to a request for data from the second network endpoint. In such a case, the system may use a portion of a connection identifier (e.g., one bit) to distinguish the first type of data stream (e.g., request) from the second type of data stream (e.g., response). That is, the first network endpoint may use a portion of a connection identifier (e.g., a connection identifier shifted right by one bit or an upper or lower portion of a connection identifier) ​​as an index to store a connection state (for a request from A to B) in a data structure that includes a connection state associated with the first network endpoint as a transmission entity (e.g., an array of "sending" connections). The first network endpoint may also use the same index portion to store the connection state (for the response from A to B) in another data structure that includes the connection state associated with the first network endpoint as a receiving entity (e.g., a "receive" connection array). The same index portion may be used to index into the congestion array. The remaining portion (e.g., one bit) of the connection identifier may be used to identify whether the data flow is of the first type (request) or the second type (response), and the corresponding congestion state may be stored and accessed in the congestion array based on the remaining portion (e.g., one bit).

[0059] In a second solution, a first network endpoint establishes a connection with a second network endpoint by sending a control packet, the control packet including a first identifier associated with the connection and the first network endpoint. The first network endpoint stores a first connection state associated with the connection in a first data structure based on the first identifier. The first network endpoint stores a first congestion state associated with the connection in a second data structure based on the first connection state. The system identifies a congestion state corresponding to a data flow associated with the first identifier. Identifying the corresponding congestion state includes: obtaining the first connection state by searching the first data structure based on the first identifier; and identifying the first congestion state by searching the second data structure based on the obtained first connection state. The first network endpoint stores the first connection state in a first entry in a first data structure, the first data structure including a connection state associated with a connection between the first network endpoint and at least the second network endpoint; and the first network endpoint stores the first congestion state in a second entry in a second data structure, the second data structure including a congestion state associated with the connection between the first network endpoint and at least the second network endpoint. A first entry in the first data structure indicates at least a first identifier associated with the connection, one or more packets to be sent, and a first state associated with the connection, and a second entry in the second data structure indicates at least a second state associated with the first congestion state and one or more of a congestion window size and a congestion rate. The system identifies the first congestion state by searching the second data structure further based on the code associated with the first identifier.

[0060] Third solution: Decode the connection ID to directly index the congestion status table and embed the connection status into the congestion state table. In a blocked state

[0061] In a third solution based on the second solution, the encoding may be used to directly index into the congestion state, where the corresponding connection state of the congestion state may be included as a sub-element in the congestion state table. The size of the congestion management state may dominate the size of the storage device, for example, because the measure of path quality (64 bytes) may be large compared to other elements of the congestion array. Therefore, a single congestion state entry may embed or include multiple states of the connection state, i.e., by including the corresponding connection state as a sub-element in the congestion array.

[0062] Figure 4APseudocode 400 is illustrated according to one aspect of the present application, which is used to create and access two data structures for connection state and congestion state based on the encoding of the identifier, and includes multiple connection states embedded in the congestion state. Pseudocode 400 may include sections 402 and 404, which define the data structures or tables of connection_array and congestion_array, respectively. Pseudocode 400 may also include: row 406 that creates a single congestion_state data structure; and rows 408 to 409 that identify the congestion state based on the encoding and obtain the corresponding connection state.

[0063] Section 402 may represent the definition of a structure of a connection_array using a third solution (referred to as “solution3_connection_state”), where: “next_sequence_number” indicates the next sequence number to be sent (if the structure is on the sending end) or the expected next sequence number (if the structure is on the receiving end); “*pending packets” indicates data packets to be processed by a given node; and “active” indicates the state of the connection, for example, whether the entry is active or inactive.

[0064] Section 404 may use a third solution (referred to as “solution3_congestion_state”) to represent the definition of a structure of a congestion_array, where: “congestion_window_size” indicates the maximum number of packets or bytes that can be sent at one time or before receiving an acknowledgment; “total_outstanding_data” indicates the number of packets or data bytes that have been sent but not yet acknowledged; “congestion_rate” indicates the maximum rate at which data can move along a path or the total capacity of a path; “path_quality

[64] ” indicates the ability of a path to transmit data; “conn_state[8]” indicates an array of structures of “solution2_connection_state”; and “active” indicates a state, such as whether an entry is active or inactive.

[0065] Line 406 may indicate that congestion_array may be defined to have, for example, 4096 elements based on the "solution3_congestion_state" data structure defined in 404. As described above, the "solution3_congestion_state" data structure contains a plurality of connection state entries (depicted as up to 8 connection state entries for illustrative purposes only). That is, connection state information may be embedded in a congestion state entry, and the connection state may be a sub-element of the congestion state.

[0066] Thus, rows 408 to 409 indicate how to identify and obtain the congestion state and the connection state. Because the third solution is based on using only one data structure (the congestion array), the appropriate entry representing the congestion state can be identified by using an encoding of the connection_ID (e.g., shifting the connection_ID right by three bits) (as indicated by row 408). Subsequently, the corresponding connection state can be obtained by masking and selecting the lower three bits of the connection_ID (as indicated by row 409). In this example, the index into the connection array sub-element can include three bits to cover up to eight entries in the connection array (as defined in section 404).

[0067] Thus, although the third solution performs lookups in the reverse order of the second solution to obtain and identify connection and congestion information, as described with respect to Figure 3B and Figure 4B As described, both the second solution and the third solution can achieve access in a single lookup by parsing the data in the lookup.

[0068] Figure 4B The diagram shows a schematic diagram of a device according to one aspect of the present application. Figure 4A 420 shows an embedded connection_array 430 having eight entries at indices 0 to 7 (indicated by dashed circles). The entries in the embedded connection_array 430 may be as follows: Figure 3A Section 302 and Figure 4A For example, entry 431 may include the following elements: "next_sequence_number" 432; "*pending packets" 433; and a state 435 of "active" whose value is set to "1", indicating that the connection state indicated by this entry 431 is active.

[0069] Diagram 420 also depicts a congestion_array 450 that may be embedded in an array of connection_arrays (such as embedded connection_array 430). Congestion_array 450 may include entries at indices 0 to N (indicated by dashed circles), where N may be, for example, 4096. The entries in congestion_array 450 may be, for example, Figure 4A 404 of . For example, entry 451 may include the following elements: "congestion_window_size" 452; "total_outstanding_data" 453; "congestion_rate" 454; "path_quality

[64] " 455; an array of "solution2_connection_state" arrays 456, each of which is defined as "conn_state[8]" (e.g., embedded connection_array 430); and a state 457 of "active", whose value is set to "1", indicating that the congestion management state indicated by this entry 451 is active.

[0070] The system may apply a first encoding to the connection_ID to determine an index of an entry 451 in the congestion_array 450 for identifying a congestion state, for example, "enc1(conn_ID)" 461 (as indicated by Figure 4A After identifying entry 451, the system can apply the second encoding to connection_ID to obtain a corresponding index "enc2(conn_ID)" 462 in embedded congestion_array 450 (child element 456 of entry 451) (as indicated by Figure 4A The arrow 470 may be used to indicate searching and obtaining the corresponding connection state based on the identified congestion state.

[0071] Elements in the structures of sections 202, 204, 302, 304, 402, and 404 are depicted for illustrative purposes only. Sections 202, 204, 302, 304, 402, and 404 may include fewer or more elements than depicted and may include one or more of the depicted elements or any combination of the depicted elements.

[0072] Similar to the second solution, in the third solution, the first network endpoint establishes a connection with the second network endpoint by sending a control packet, the control packet including a first identifier associated with the connection and the first network endpoint. The first network endpoint stores a first connection state associated with the connection in a first data structure based on the first identifier. The first network endpoint stores a first congestion state associated with the connection in a second data structure based on the first connection state. The system identifies a congestion state corresponding to the data flow associated with the first identifier. Identifying the corresponding congestion state includes: obtaining the first connection state by searching the first data structure based on the first identifier; and identifying the first congestion state by searching the second data structure based on the obtained first connection state. The first network endpoint stores the first connection state in a first entry in a first data structure, the first data structure including a connection state associated with the connection between the first network endpoint and at least the second network endpoint; and the first network endpoint stores the first congestion state in a second entry in a second data structure, the second data structure including a congestion state associated with the connection between the first network endpoint and at least the second network endpoint. A first entry in the first data structure indicates at least a first identifier associated with the connection, one or more packets to be sent, and a first state associated with the connection, and a second entry in the second data structure indicates at least a second state associated with the first congestion state and one or more of a congestion window size and a congestion rate. The system identifies the first congestion state by searching the second data structure further based on the code associated with the first identifier.

[0073] Additionally, in a third solution, the second entry in the second data structure further indicates an array including connection state entries from the first data structure, the second entry including a first congestion state. The first network endpoint includes a connection state associated with the connection as a sub-element of the congestion state. The system identifies the first congestion state by further searching the second data structure based on a first code associated with the first identifier and the system obtains the first connection state by searching the array based on a second code associated with the first identifier.

[0074] Methods for facilitating decoupling of congestion management state from connection state in NIC

[0075] Figure 5A flowchart 500 illustrating a method for facilitating decoupling congestion management state from connection state in a NIC according to one aspect of the present application is presented. During operation, the system establishes a connection with a second network endpoint by sending a control packet through a first network endpoint (operation 502), the control packet including a first identifier associated with the connection and the first network endpoint. The first identifier can be, for example, a connection ID used in the network when establishing the connection. As described above, the data packet can include an initiator node (or source) connection ID and a target node (or destination) connection ID. In the first solution, the connection ID can be used to directly index a connection array, while an indirect index can be used to access a congestion array. In the second solution and the third solution, the connection ID can be utilized (e.g., based on one or more encodings) to directly index into both the connection array and the congestion array.

[0076] The system stores a first connection state associated with the connection based on the first identifier in a first data structure by the first network endpoint (operation 504). The first connection state can be stored in a first entry in the first data structure, and the first data structure can include a connection state associated with the connection between the first network endpoint and at least the second network endpoint, as described above with respect to Figure 2A The data structure 202 ("solution1_connection_state") and Figure 2B The system stores a first congestion state associated with the connection in a second data structure based on the first connection state by the first network endpoint (operation 506). The first congestion state can be stored in a second entry in the second data structure, and the second data structure can include the congestion state associated with the connection between the first network endpoint and at least the second network endpoint, as described above with respect to Figure 2A The data structure 204 ("solution1_congestion_state") and Figure 2B As described in entry 251.

[0077] The system identifies a congestion state corresponding to the data flow associated with the first identifier (operation 508). Operation 508 may include the steps of operation 510 and operation 512. That is, the system identifies the corresponding congestion state by the following operations: obtaining a first connection state by searching a first data structure based on the first identifier (operation 510); and identifying the first congestion state by searching a second data structure based on the obtained first connection state (operation 512). For example, as described above in Figure 2BAs described in , the system may use the connection ID to search connection_array 230, which enables obtaining the connection state entry 231 including congestion_index 234. The system may then use the obtained congestion_index 234 to search congestion_array 250, which enables identification of the congestion state entry 251. In addition, the system determines whether to schedule the data packet associated with the first identifier based on the identified state of the first congestion state or the obtained state of the first connection state (operation 514). For example, the scheduler or scheduling component may use Figure 2B The value of the state 257 element in congestion_array 250 or Figure 2B The value of the state 235 element in the connection_array 230 of the connection_array 230 determines whether to schedule the corresponding data packet. The value of the element 235 / 257 can indicate "inactive" set to the value "0", or indicate "active" set to the value "0", as described above with respect to Figure 2B As described above for Figure 3B Elements 335 / 357 and Figure 4B Elements 435 / 457 of can be similarly described. A scheduler or scheduling component can use these values ​​as part of a scheduling algorithm to determine the order in which to schedule corresponding packets.

[0078] Computer system facilitating decoupling of congestion management state from connection state in a NIC

[0079] Figure 6 A computer system 600 is illustrated that facilitates decoupling of congestion management state from connection state in a NIC according to one aspect of the present application. The computer system 600 includes a processor 602, a memory 604, and a storage device 606. The memory 604 may include a volatile memory (e.g., a random access memory (RAM)) that is used as a management memory and may be used to store one or more memory pools. In addition, the computer system 600 may be coupled to a peripheral I / O user device 610 (e.g., a display device 611, a keyboard 612, and a pointing device 613). The storage device 606 includes a non-transitory computer-readable storage medium and stores an operating system 616, a content processing system 618, and data 628. The computer system 600 may include, for example, a processor 602, a memory 604, and a storage device 606. Figure 6 Fewer or more entities or instructions than those shown.

[0080] The content processing system 618 of the storage device 606 may include instructions that, when executed by the computer system 600 (e.g., by a processing resource of the computer system 600, such as the processor 602), may cause the computer system 600 to perform the methods and / or processes described in the present disclosure. Specifically, the content processing system 618 may include instructions 620 for sending a control packet by a first network endpoint, the control packet including a first identifier (ID) associated with the first network endpoint (N / E) and a connection to be established with a second network endpoint (N / E). The content processing system 618 may include instructions 622 for storing, by the first network endpoint, a first connection state associated with the connection based on the first identifier. The content processing system 618 may also include instructions 624 for storing, by the first network endpoint, a first congestion state associated with the connection based on the first connection state. The content processing system 618 may include instructions 626 for identifying, for a data stream associated with a first identifier, a congestion state corresponding to the data stream, for example by performing the following operations: using the first identifier to perform a first search in a first structure storing connection states associated with connections, the first search returning the first connection state; and using the returned first connection state to perform a second search in a second data structure storing congestion states associated with the connection states, the second search identifying the first congestion state.

[0081] Content processing system 618 may include Figure 6 For example, the content processing system 618 may also store instructions for performing the operations described above with respect to the following operations: storing and accessing Figure 2B , Figure 3B and Figure 4B Array of; Figure 5 The operations depicted in flowchart 500 of ; and Figure 7 The content processing system 618 may also store instructions for executing and using the above-described respective FIG. 2A to FIG. 2B (For the first solution), FIG. 3A to FIG. 3B (for the second solution) and FIG. 4A to FIG. 4B (For the third solution) Instructions for operations associated with any of the three solutions.

[0082] Data 628 may include any data required as input or generated as output by the methods, operations, communications, and / or processes described in the present disclosure. Specifically, data 628 may store at least: requests; identifiers; connection identifiers; identifiers of network endpoints; arrays; tables; data structures; entries in data structures; packets; data flows; connection states; congestion management states or congestion states; connection arrays; congestion arrays; states; determining whether to schedule a packet; indexes; congestion indexes; indirect indexes; arrays of connection indexes; embedded arrays or data structures; encodings; number of bits; encodings of number of bits.

[0083] Figure 7 A computer-readable medium / device 700 is illustrated that facilitates decoupling congestion management state from connection state in a NIC according to an aspect of the present application. The device 700 may be a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform a method. The device 700 may store instructions 710 to establish a connection between a first network endpoint (N / E) and a second network endpoint (N / E) by sending a control packet including a first identifier associated with the connection and the first network endpoint. The device 700 may store instructions 712 to store a first connection state associated with the connection in a first entry in a first data structure based on the first identifier. The device 700 may store instructions 714 to store a first congestion state associated with the connection in a second entry in a second data structure based on the first connection state.

[0084] The device 700 may further store instructions 720 for identifying a congestion state corresponding to a data flow associated with the first identifier. The instructions 720 may include instructions 722 for obtaining a first connection state by searching a first data structure based on the first identifier; and instructions 724 for identifying the first congestion state by searching a second data structure based on the obtained first connection state. The device 700 may also store instructions 730 for determining whether to schedule a data packet associated with the first identifier based on the identified state of the first congestion state or the returned state of the first connection state.

[0085] The device 700 may include Figure 7 For example, the device 700 may also store instructions for performing the operations described above with respect to the following operations: storing and accessing Figure 2B , Figure 3B and Figure 4B Array of; Figure 5 The operations depicted in flowchart 500 of ; and Figure 6 The device 700 may also store instructions for executing and using the above-described instructions for the content processing system 618. FIG. 2A to FIG. 2B (For the first solution), FIG. 3A to FIG. 3B(for the second solution) and FIG. 4A to FIG. 4B (For the third solution) Instructions for operations associated with any of the three solutions.

[0086] Various aspects

[0087] In general, the disclosed aspects provide a method, a computer system, and a non-transitory computer-readable storage medium for decoupling congestion management state from connection state in an HPC environment (e.g., in a network endpoint or NIC). In one aspect, the system establishes a connection with a second network endpoint by transmitting a control packet through a first network endpoint, the control packet including a first identifier associated with the connection and the first network endpoint. The system stores a first connection state associated with the connection in a first data structure based on the first identifier through the first network endpoint. The system stores a first congestion state associated with the connection in a second data structure based on the first connection state through the first network endpoint. The system identifies a congestion state corresponding to a data flow associated with the first identifier. Identifying the corresponding congestion state includes: obtaining a first connection state by searching a first data structure based on the first identifier; and identifying the first congestion state by searching a second data structure based on the obtained first connection state.

[0088] In a variation of this aspect, the system identifies a data packet to be scheduled. The data packet is associated with a first identifier. The system determines whether to schedule the data packet based on at least one of: a state of the identified first congestion state; a state of the obtained first connection state; or one or more states of a connection associated with the identified first congestion state.

[0089] In a further variation of this aspect, the system stores a first connection state in a first entry in a first data structure. The first data structure includes a connection state associated with a connection between a first network endpoint and at least a second network endpoint. The system stores a first congestion state in a second entry in a second data structure. The second data structure includes a congestion state associated with a connection between the first network endpoint and at least a second network endpoint.

[0090] In yet another variation of this aspect, the first entry indicates at least a first identifier associated with the connection, one or more packets to be sent, a first state associated with the connection, and a congestion index associated with the first congestion state stored in the second data structure. The system identifies the first congestion state by further searching the second data structure based on the congestion index indicated in the first entry.

[0091] In another variation, a second entry in the second data structure indicates at least a second state associated with the first congestion state, one or more connection indexes corresponding to elements in the first data structure, and one or more of a congestion window size and a congestion rate. The connection index includes at least a first identifier for the connection and corresponds to the connection associated with the first congestion state.

[0092] In another variation, a first entry in the first data structure indicates at least a first identifier associated with the connection, one or more packets to be sent, and a first state associated with the connection. A second entry in the second data structure indicates at least a second state associated with the first congestion state and one or more of a congestion window size and a congestion rate. The system identifies the first congestion state by searching the second data structure further based on a code associated with the first identifier.

[0093] In another variation, the second entry in the second data structure further indicates an array including connection state entries from the first data structure, the second entry including a first congestion state. The system includes the connection state associated with the connection as a sub-element of the congestion state. The system identifies the first congestion state by further searching the second data structure based on a first code associated with the first identifier. The system obtains the first connection state by searching the array based on a second code associated with the first identifier.

[0094] In another embodiment, the system encodes the first identifier based on at least one of: obtaining a first index for the second data structure by shifting one or more bits of the first identifier, the first identifier and the first index including completely overlapping bits; obtaining a second index for the second data structure by shifting one or more bits of the first identifier, the first identifier and the second index not including overlapping bits; or obtaining the first index or the second index by masking the first identifier prior to shifting one or more bits.

[0095] In a further variation, the system determines that the data flow includes at least: first data transmitted from the first network endpoint to the second network endpoint; and second data transmitted from the first network endpoint to the second network endpoint in response to a request from the second network endpoint. The system stores a first connection state in a first data structure using a portion of a first identifier by the first network endpoint in response to a data flow including the first data, the first data structure further including a connection state associated with the first network endpoint as a transmitting entity. The system stores the first connection state in a third data structure using the same portion of the first identifier by the first network endpoint in response to a data flow including the second data, the third data structure including a connection state associated with the first network endpoint as a receiving entity. The system stores a first congestion state in a second data structure using a remaining portion of the first identifier by the first network endpoint.

[0096] On the other hand, a computer system is provided that includes a processing resource and a non-transitory computer-readable storage device that stores instructions executable by the processing resource to: send, by a first network endpoint, a control packet including a first identifier associated with the first network endpoint and a connection to be established with a second network endpoint; store, by the first network endpoint, a first connection state associated with the connection based on the first identifier; store, by the first network endpoint, a first congestion state associated with the connection based on the first connection state; and for a data stream associated with the first identifier, identify a congestion state corresponding to the data stream by performing the following operations: perform a first search in a first structure storing connection states associated with the connection using the first identifier, the first search returning the first connection state; and perform a second search in a second data structure storing congestion states associated with the connection states using the returned first connection state, the second search identifying the first congestion state. The instructions executable by the processing resource may further include: storing and accessing Figure 2B , Figure 3B and Figure 4B Array instructions; Figure 5 The operations depicted in flowchart 500 of ; and Figure 7 Instructions of the computer-readable storage medium / device 700.

[0097] On the other hand, a non-transitory computer-readable storage medium is provided, which includes instructions that can be executed by a processing resource to perform the following operations: establish a connection between the first network endpoint and the second network endpoint by transmitting a control packet including a first identifier associated with the connection and the first network endpoint; store a first connection state associated with the connection in a first entry in a first data structure based on the first identifier; store a first congestion state associated with the connection in a second entry in a second data structure based on the first connection state; for a data flow associated with the first identifier, identify the congestion state corresponding to the data flow by the following operations: obtain the first connection state by searching the first data structure based on the first identifier; and identify the first congestion state by searching the second data structure based on the obtained first connection state; and determine whether to schedule a data packet associated with the first identifier based on the state of the identified first congestion state or the state of the returned first connection state. The instructions executable by the processing resource may further include: storing and accessing Figure 2B , Figure 3B and Figure 4B Array instructions; Figure 5 The operations depicted in flowchart 500 of ; and Figure 6 Instructions of the content processing system 618.

[0098] The foregoing descriptions of various aspects have been presented for purposes of illustration and description only. These descriptions are not intended to be exhaustive or to limit the various aspects described herein to the disclosed forms. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art. In addition, the above disclosure is not intended to limit the various aspects described herein. The scope of the various aspects described herein is defined by the appended claims.

Claims

1. A computer-implemented method comprising: establishing, by a first network endpoint, a connection with a second network endpoint by sending a control packet, the control packet comprising a first identifier associated with the connection and the first network endpoint; storing, by the first network endpoint, a first connection state associated with the connection in a first data structure based on the first identifier; storing, by the first network endpoint, a first congestion state associated with the connection in a second data structure based on the first connection state; as well as Identifying, for a data flow associated with the first identifier, a congestion state corresponding to the data flow, wherein identifying the corresponding congestion state includes: obtaining the first connection state by searching the first data structure based on the first identifier; as well as The first congestion state is identified by searching the second data structure based on the obtained first connection state.

2. The method of claim 1, further comprising: identifying a data packet to be scheduled, the data packet being associated with the first identifier; as well as Determining whether to schedule the data packet is based on at least one of the following: the status of the identified first congestion state; the status of the obtained first connection status; or One or more states of the connection associated with the identified first congestion state.

3. The method of claim 1, further comprising: storing the first connection state in a first entry in the first data structure, the first data structure comprising connection states associated with a connection between the first network endpoint and at least the second network endpoint; as well as The first congestion status is stored in a second entry in the second data structure, the second data structure comprising congestion status associated with the connection between the first network endpoint and at least the second network endpoint.

4. The method according to claim 3, the first entry indicating at least the first identifier associated with the connection, one or more packets to be sent, a first state associated with the connection, and a congestion index associated with the first congestion state stored in the second data structure, and The method further includes identifying the first congestion state by searching the second data structure further based on the congestion index indicated in the first entry.

5. The method according to claim 4, the second entry in the second data structure indicates at least a second state associated with the first congestion state, one or more connection indexes corresponding to elements in the first data structure, and one or more of a congestion window size and a congestion rate, and The connection index includes at least a first identifier for the connection and corresponds to a connection associated with the first congestion state.

6. The method according to claim 3, the first entry in the first data structure indicating at least the first identifier associated with the connection, one or more data packets to be sent, and a first state associated with the connection, The second entry in the second data structure indicates at least a second state associated with the first congestion state and one or more of a congestion window size and a congestion rate, and The method further includes identifying the first congestion state by searching the second data structure further based on a code associated with the first identifier.

7. The method according to claim 6, The second entry in the second data structure further indicates an array including connection status entries from the first data structure, the second entry including the first congestion status, and The method further comprises: including the connection state associated with the connection as a sub-element of the congestion state; identifying the first congestion state by searching the second data structure further based on a first code associated with the first identifier; and The first connection state is obtained by searching the array based on a second code associated with the first identifier.

8. The method of claim 7, further comprising encoding the first identifier based on at least one of: obtaining a first index for the second data structure by shifting one or more bits of the first identifier, the first identifier and the first index comprising completely overlapping bits; obtaining a second index for the second data structure by shifting one or more bits of the first identifier, the first identifier and the second index comprising no overlapping bits; or The first index or the second index is obtained by masking the first identifier before shifting the one or more bits.

9. The method of claim 7, further comprising: Determine that the data flow includes at least the following items: first data sent from the first network endpoint to the second network endpoint; as well as second data sent from the first network endpoint to the second network endpoint in response to a request from the second network endpoint; storing, by the first network endpoint in response to the data flow including the first data, the first connection state in the first data structure using a portion of the first identifier, the first data structure further comprising a connection state associated with the first network endpoint as a sending entity; storing, by the first network endpoint in response to the data flow including the second data, the first connection state in a third data structure using the same portion of the first identifier, the third data structure including a connection state associated with the first network endpoint as a receiving entity; as well as The first congestion status is stored in the second data structure by the first network endpoint using a remaining portion of the first identifier.

10. A computer system comprising: Processing resources; and a non-transitory machine-readable storage device storing instructions executable by the processing resource to: sending, by a first network endpoint, a control packet, the control packet comprising a first identifier associated with the first network endpoint and a connection to be established with a second network endpoint; storing, by the first network endpoint, a first connection state associated with the connection based on the first identifier; storing, by the first network endpoint, a first congestion state associated with the connection based on the first connection state; as well as For a data flow associated with the first identifier, identifying a congestion state corresponding to the data flow by performing the following operations: performing a first search in a first data structure storing connection states associated with connections using the first identifier, the first search returning the first connection state; as well as A second search is performed using the returned first connection state in a second data structure storing congestion states associated with connection states, the second search identifying the first congestion state.

11. The computer system of claim 10, wherein the instructions are executable by the processing resource to further perform the following operations: Determining whether to schedule a data packet associated with the first identifier is based on at least one of the following: the status of the identified first congestion state; the status of the first connection status returned; or One or more states of the connection associated with the identified first congestion state.

12. The computer system of claim 10, wherein the instructions are executable by the processing resource to further perform the following operations: storing the first connection state in a first entry in the first data structure, the first data structure comprising a connection state associated with a connection between the first network endpoint and at least the second network endpoint; and The first congestion status is stored in a second entry in the second data structure, the second data structure comprising congestion status associated with the connection between the first network endpoint and at least the second network endpoint.

13. The computer system of claim 12, wherein the instructions are executable by the processing resource to further perform the following operations: indicating, in the first entry in the first data structure, at least the first identifier associated with the connection, one or more packets to be sent, a first state associated with the connection, and a congestion index associated with the first congestion state stored in the second data structure; and The second search is performed based on the congestion index indicated in the first entry.

14. The computer system of claim 13, wherein the instructions are executable by the processing resource to further perform the following operations: indicating, in the second entry in the second data structure, at least a second state associated with the first congestion state, one or more connection indexes corresponding to elements in the first data structure, and one or more of a congestion window size and a congestion rate; and At least the first identifier for the connection is included in the connection index, and the connection index corresponds to the connection associated with the first congestion state.

15. The computer system of claim 12, wherein the instructions are executable by the processing resource to further perform the following operations: indicating in the first entry in the first data structure at least the first identifier associated with the connection, one or more data packets to be sent, and a first state associated with the connection; indicating in the second entry in the second data structure at least a second state associated with the first congestion state and one or more of a congestion window size and a congestion rate; and The second search is performed based on a code associated with the first identifier.

16. The computer system of claim 15, wherein the instructions are executable by the processing resource to further perform the following operations: indicating in the second entry in the second data structure an array comprising connection state entries from the first data structure, the second entry comprising the first congestion state; including the connection state associated with the connection as a sub-element of the congestion state; identifying the first congestion state by searching the second data structure based on a first code associated with the first identifier; as well as The first connection state is obtained by searching the array based on a second code associated with the first identifier.

17. The computer system of claim 16, the instructions executable by the processing resource to further encode the first identifier by at least one of the following operations: obtaining a first index for the second data structure by shifting one or more bits of the first identifier, the first identifier and the first index comprising completely overlapping bits; obtaining a second index for the second data structure by shifting one or more bits of the first identifier, the first identifier and the second index comprising no overlapping bits; or The first index or the second index is obtained by masking the first identifier before shifting the one or more bits.

18. A non-transitory computer-readable storage medium comprising instructions executable by a processing resource to: establishing a connection between the first network endpoint and a second network endpoint by sending a control packet including a first identifier associated with the connection and the first network endpoint; storing a first connection state associated with the connection in a first entry in a first data structure based on the first identifier; storing a first congestion state associated with the connection in a second entry in a second data structure based on the first connection state; For a data flow associated with the first identifier, identifying a congestion state corresponding to the data flow by: obtaining the first connection state by searching the first data structure based on the first identifier; as well as identifying the first congestion state by searching the second data structure based on the obtained first connection state; as well as Whether to schedule a data packet associated with the first identifier is determined based on the identified state of the first congestion state or the returned state of the first connection state.

19. The non-transitory computer readable storage medium of claim 18, the first entry indicating at least the first identifier associated with the connection, one or more packets to be sent, a first state associated with the connection, and a congestion index associated with the first congestion state stored in the second data structure, and The instructions are executable by the processing resource to further identify the first congestion state by searching the second data structure further based on the congestion index indicated in the first entry.

20. The non-transitory computer readable storage medium of claim 18, the first entry in the first data structure indicating at least the first identifier associated with the connection, one or more data packets to be sent, and a first state associated with the connection, The second entry in the second data structure indicates at least a second state associated with the first congestion state and one or more of a congestion window size and a congestion rate, and The instructions are executable by the processing resource to further perform the following operations: identifying the first congestion state by searching the second data structure further based on a code associated with the first identifier; and The data packet is further scheduled based on the first state and the second state.

Images