Message transmission through acceleration components configured as acceleration services

Abstract

Embodiments of the present disclosure relate to message transmission through an acceleration component configured to accelerate a service. Components, methods, and systems are provided that allow an acceleration component to transmit messages. An acceleration component is provided for use between a first plurality of acceleration components associated with a first top-of-rack (TOR) switch to transmit messages to other acceleration components in an acceleration plane that can be configured to provide service acceleration for a service. The acceleration component includes a transport component that is configured to transmit a first point-to-point message to a second acceleration component associated with a second TOR switch that is different from the first TOR switch, and a third acceleration component associated with a third TOR switch that is different from the first TOR switch and the second TOR switch. The transport component can also be configured to broadcast the second point-to-point message to all acceleration components of the second plurality of acceleration components associated with the second TOR switch and all acceleration components of the third plurality of acceleration components associated with the third TOR switch.

Description

[0001] This application is a divisional application of the Chinese invention patent application with international application number PCT / US2017 / 067150, international application date December 19, 2017, entering the Chinese national phase on July 1, 2019, Chinese national application number 201780081851.5, and invention name “Message transmission through an acceleration component configured as an acceleration service”. Technical Field

[0002] Embodiments of the present disclosure relate to systems, methods, and acceleration components for transmitting messages. Background Art

[0003] Increasingly, users access applications provided via computing, network and storage resources located in data centers. These applications run in a distributed computing environment, which is sometimes referred to as a cloud computing environment. Computer servers in a data center are interconnected via a network and therefore, applications running on the computer servers can communicate with each other via the network. In large data centers, passing messages between computer servers may include broadcasting or multicasting messages from a computer server to several other computer servers. Broadcasting or multicasting in such a data center may use a large portion of the bandwidth available for applications. That in turn may reduce the performance of these applications because they experience lower throughput and higher latency.

[0004] Therefore, there is a need for methods and systems that alleviate at least some of these problems. Summary of the invention

[0005] In one example, the present disclosure relates to a system, comprising: a software plane, comprising a plurality of host components configured to execute instructions corresponding to at least one service; and an acceleration plane, comprising a plurality of acceleration components configurable to accelerate at least one service. The system may also include a network configured to interconnect the software plane and the acceleration plane, the network may include: a first top of rack (TOR) switch associated with a first subset of the plurality of acceleration components, a second TOR switch associated with a second subset of the plurality of acceleration components, and a third TOR switch associated with a third subset of the plurality of acceleration components, wherein any acceleration component in the first subset of the plurality of acceleration components may be configured to transmit a point-to-point message to any acceleration component in the second subset of the plurality of acceleration components and transmit the point-to-point message to any acceleration component in the third subset of the plurality of acceleration components, and wherein any acceleration component in the second subset of the plurality of acceleration components may be configured to broadcast the point-to-point message to all acceleration components in the second subset of the plurality of acceleration components, and wherein any acceleration component in the third subset of the plurality of acceleration components may be configured to broadcast the point-to-point message to all acceleration components in the third subset of the plurality of acceleration components.

[0006] In another example, the present disclosure relates to a method for allowing a first acceleration component among a first plurality of acceleration components associated with a first top-of-rack (TOR) switch to transmit a message to other acceleration components in an acceleration plane that can be configured to provide service acceleration for at least one service. The method may include: the first acceleration component transmitting a point-to-point message to a second acceleration component associated with a second TOR switch different from the first TOR switch, and a third acceleration component associated with a third TOR switch different from the first TOR switch and the second TOR switch. The method may also include: the second acceleration component broadcasting the point-to-point message to all acceleration components of the second plurality of acceleration components associated with the second TOR switch. The method may also include the third acceleration component broadcasting the point-to-point message to all acceleration components of the third plurality of acceleration components associated with the third TOR switch.

[0007] In yet another example, the present disclosure relates to an acceleration component for use between a first plurality of acceleration components associated with a first top-of-rack (TOR) switch to transmit messages to other acceleration components in an acceleration plane that can be configured to provide service acceleration for at least one service. The acceleration component may include a transport component configured to transmit a first point-to-point message to a second acceleration component associated with a second TOR switch different from the first TOR switch, and a third acceleration component associated with a third TOR switch different from the first TOR switch and the second TOR switch. The transport component may also be configured to broadcast the second point-to-point message to all acceleration components of the second plurality of acceleration components associated with the second TOR switch and all acceleration components of the third plurality of acceleration components associated with the third TOR switch.

[0008] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to delineate the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present disclosure is illustrated by way of example and not limited by the accompanying drawings, in which like reference numerals indicate similar elements. The elements in the drawings are illustrated for simplicity and clarity and are not necessarily drawn to scale.

[0010] Figure 1 is a schematic diagram of an architecture that may include a software plane and an acceleration plane according to an example;

[0011] Figure 2 A schematic diagram of a system for transmitting messages through an acceleration component configured to accelerate a service according to an example is shown;

[0012] Figure 3A schematic diagram of a system for transmitting messages through an acceleration component configured to accelerate a service according to an example is shown;

[0013] Figure 4 A schematic diagram of an acceleration assembly according to an example is shown;

[0014] Figure 5 shows a schematic diagram of a 3-port switch according to an example;

[0015] Figure 6 A schematic diagram of a system for transmitting messages through an acceleration component configured to accelerate a service according to an example is shown;

[0016] Figure 7 A flowchart of a method for transmitting a message through an acceleration component configured to accelerate a service according to an example is shown; and

[0017] Figure 8 A flow chart of another method for transmitting a message through an acceleration component configured to accelerate a service according to an example is shown. DETAILED DESCRIPTION

[0018] The examples described in the present disclosure relate to providing methods and systems for transmitting messages between acceleration components that can be configured to accelerate services. Acceleration components include, but are not limited to, configurable (or configured) to perform functions corresponding to services being provided by, for example, a data center more efficiently than software running on a general-purpose central processing unit (CPU). Acceleration components may include field programmable gate arrays (FPGAs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), erasable and / or complex programmable logic devices (PLDs), programmable array logic (PAL) devices, general-purpose array logic (GAL) devices, and massively parallel processor arrays (MPPA) devices. Image files may be used to configure or reconfigure acceleration components, such as FPGAs. The information included in the image file may be used to program the hardware components of the acceleration components (e.g., logic blocks and configurable interconnects of the FPGA) to implement desired functionality. The desired functionality may be implemented to support any service that can be provided via a combination of computing, network, and storage resources (such as via a data center or other infrastructure for delivering services).

[0019] The described aspects can also be implemented in a cloud computing environment. Cloud computing can refer to a model for supporting on-demand network access to a shared pool of configurable computing resources. For example, cloud computing can be adopted in the market to provide ubiquitous and convenient on-demand access to a shared pool of configurable computing resources. The shared pool of configurable computing resources can be quickly provided via visualization and issued with low management effort or service provider interaction and then scaled accordingly. The cloud computing model can include various features, such as, for example, on-demand self-service, extensive network access, resource pooling, rapid elasticity, measured services, etc. The cloud computing model can also expose various service models, such as, for example, software as a service ("SasS"), platform as a service ("PaaS"), and infrastructure as a service ("IaaS"). The cloud computing model can also be deployed using different deployment models (such as private cloud, community cloud, public cloud, hybrid cloud, etc.).

[0020] Data center deployment may include a hardware acceleration plane and a software plane. The hardware acceleration plane may include multiple networked acceleration components (e.g., FPGA). The software plane may include multiple networked software-implemented host components (e.g., central processing unit (CPU)). Network infrastructure may be shared between the hardware acceleration plane and the software plane. In some environments, the software-implemented host components are locally linked to the corresponding acceleration components. The acceleration components may communicate with each other via network protocols. In order to provide reliable services to users of services provided via a data center, any communication mechanism may be required to meet certain performance requirements, including reliability. In some examples, the present disclosure provides a lightweight transport layer for meeting such requirements. In one example, the acceleration components may communicate with each other via a lightweight transport layer (LTL).

[0021] Figure 1 An architecture 100 is shown that may include a software plane 104 and an acceleration plane 106 according to one example. The software plane 104 may include a collection of software-driven host components (each represented by the symbol "S"), and the acceleration plane may include a collection of acceleration components (each represented by the symbol "A"). In this example, each host component may correspond to a server computer that executes machine-readable instructions using one or more central processing units (CPUs). In one example, these instructions may correspond to services, such as text / image / video search services, translation services, or any other services that may be configured to provide useful results to users of the device. Each CPU may execute instructions corresponding to various components (e.g., software modules or libraries) of the service. Each acceleration component may include hardware logic for implementing functionality, such as, for example, portions of a service provided by a data center.

[0022] The acceleration plane 106 can be constructed using a heterogeneous or homogeneous collection of acceleration components, including different types of acceleration components and / or the same type of acceleration components with different capabilities. For example, the acceleration plane 106 can include a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a system on a chip (SOC), a complex programmable logic device (CPLD), other types of programmable hardware logic devices, etc. The acceleration plane 106 can provide a reconfigurable structure of acceleration components.

[0023] The host component can generally be any computing component that can perform operations by executing machine-readable instructions using each of its CPU hardware threads. The acceleration component can perform operations using a number of parallel logic elements to perform computing tasks. As an example, an FPGA can include a number of gate arrays that can be configured to perform certain computing tasks in parallel. Therefore, the acceleration component can perform some operations in less time than a software-driven host component. In the context of architecture 100, "acceleration" reflects its potential for accelerating the functions performed by the host component.

[0024] In one example, architecture 100 may correspond to a data center environment including a large number of servers. The servers may correspond to host components in software plane 104. In another example, architecture 100 may correspond to an enterprise system. In another example, architecture 100 may correspond to a user device or appliance using at least one host component having access to two or more acceleration components. In fact, other implementations for architecture 100 are possible depending on the requirements of the service.

[0025] The network 120 may couple host components in the software plane 104 to other host components and acceleration components in the acceleration plane 106 to other acceleration components. In this example, host components may interact with each other using the network 120, and acceleration components may interact with each other using the network 120. Interactions between host components in the software plane 104 may be independent of interactions between acceleration components in the acceleration plane 106. In this example, two or more acceleration components may communicate in a transparent manner relative to the host components in the software plane 104, outside of the direction of the host components, and without the host components “knowing” that a particular interaction is even occurring in the acceleration plane 106.

[0026] The architecture 100 can use any of a variety of different protocols to facilitate communication between acceleration components through the network 120, and can use any of a variety of different protocols to facilitate communication between host components through the network 120. For example, the architecture 100 can use the Ethernet protocol to transmit Internet Protocol (IP) packets through the network 120. In one implementation, each local host component in the server is given a single physical IP address. The local acceleration components in the same server can take the same IP address. The server can determine in different ways whether the incoming packet is destined for a local host component or a local acceleration component. For example, a packet destined for a local acceleration component can be formulated as a UDP packet with a specific port; on the other hand, a host-defined packet may not be formulated in this manner. In another example, based on the value of the state flag of each packet in the packet, a packet belonging to the acceleration plane 106 can be distinguished from a packet belonging to the software plane 104. In one example, the architecture 100 can be viewed as two logical networks (software plane 104 and acceleration plane 106) that can share the same physical network communication link. The packets associated with the two logical networks can be distinguished from each other by their corresponding traffic categories.

[0027] In another aspect, each host component in the architecture 100 is coupled to at least one acceleration component in the acceleration plane 104 via a local link. For example, the host component and the acceleration component can be arranged together and maintained as a single serviceable unit (e.g., a server) within the architecture 100. In this arrangement, the server can be referred to as a "local" host component to distinguish it from other host components associated with other servers. Similarly, the acceleration component(s) of the server can be referred to as a "local" acceleration component(s) to distinguish them from other acceleration components associated with other servers.

[0028] As depicted in architecture 100, host component 108 may be coupled to acceleration component 110 via local link 112 (e.g., a peripheral component interconnect express (PCIe) link). Thus, host component 108 may be a local host component from the perspective of acceleration component 110 and acceleration component 110 may be a local acceleration component from the perspective of host component 108. The local links of host component 108 and acceleration component 110 may form part of a server. More generally, host components in software plane 104 may be collectively represented by local A -to-local SMany individual links of coupling 114 are locally coupled to acceleration components in acceleration plane 106. In this example, the host component can interact directly with any locally linked acceleration component. The host component can initiate communication to the locally linked acceleration component to cause further communication between multiple acceleration components. For example, the host component can issue a request for a service (or a portion thereof), wherein the functionality for the service (or a portion thereof) consists of a set of one or more acceleration components across the acceleration plane 106. The host component can also interact indirectly with other acceleration components in the acceleration plane 106 to which the host component is not locally linked. For example, the host component 108 can communicate indirectly with the acceleration component 116 via the acceleration component 110. In this example, the acceleration component 110 communicates with the acceleration component 116 via the link 118 of the network (e.g., network 120).

[0029] The acceleration components in the acceleration plane 106 can be advantageously used to robustly accelerate larger scale services in a data center. A large portion of a complex data center service can be mapped to an acceleration component (e.g., FPGA) using a low latency interconnect for computation across multiple acceleration components. The acceleration components can also be reconfigured to provide different service functionality at different times as appropriate. Although Figure 1 A particular number of components of the architecture 100 are shown arranged in a certain manner, but there may be a greater or lesser number of components arranged differently. Additionally, the various components of the architecture 100 may also be implemented using other technologies.

[0030] Figure 2 A schematic diagram of a system 200 for transmitting or retransmitting a message through an acceleration component configured to accelerate a service according to an example is shown. In one example, the system 200 can be implemented as a rack of servers in a data center. Servers 204, 206, and 208 can be included in a rack in a data center. Each of the servers 204, 206, and 208 can be coupled to a top-of-rack (TOR) switch 210. Although not shown, other racks can have similar configurations. Server 204 can also include a host component 212, which includes CPUs 214, 216, etc. The host component 212 can be included in the software plane 104 together with the host components from the servers 206 and 208. Server 204 can also include an acceleration component 218. The acceleration component 218 can be included in the acceleration plane 106 together with the acceleration components from the servers 206 and 208.

[0031] The acceleration component 218 can be directly coupled to the host component 212 via a local link 220 (e.g., a PCIe link). Thus, the acceleration component 218 can treat the host component 212 as a local host component. The acceleration component 218 and the host component 212 can also be indirectly coupled through a network interface controller 222 (e.g., used to communicate across the network infrastructure 120). In this example, the server 204 can load an image representing service functionality onto the acceleration component 218.

[0032] The acceleration component 218 may also be coupled to the TOR switch 210. Thus, in the system 200, the acceleration component 218 may represent a path through which the host component 212 interacts with other components in the data center (including other host components and other acceleration components). The system 200 allows the acceleration component 218 to perform processing on packets received from (and / or sent to) the TOR switch 210 (e.g., by performing encryption, compression, etc.) without burdening the CPU-based operations performed by the host component 212. Although Figure 2 A certain number of components of the system 200 are shown arranged in a certain manner, but there may be a greater or lesser number of components arranged differently. Additionally, the various components of the architecture 200 may also be implemented using other technologies.

[0033] Figure 3A schematic diagram of a system 300 for transmitting or retransmitting a message through an acceleration component configured as an acceleration service according to an example is shown. In this example, IP routing can be used to transmit or receive messages between TOR switches, including TOR switch 1 302, TOR switch 2 304, and TOR switch N 306. Each server or server group can have a single "physical" IP address that can be provided by a network administrator. Therefore, in this example, server group 1 320, server 2 322, and server group N 324 can each include servers, each of which can have a "physical" IP address. The acceleration component can use the physical IP of its server as its address. In order to distinguish between IP packets destined for the host and packets destined for the acceleration component, a UDP packet with a specific port specifying the acceleration component as the destination can be used. The acceleration component can use layer 3 functionality corresponding to the seven-layer open system interconnection (OSI) model to transmit messages to a selected set of acceleration components associated with different TOR switches. The layer 3 functionality can be similar to the layer 3 functionality provided by the network layer of the OSI model. In this example, the acceleration component can transmit a point-to-point message to each of the other related acceleration components associated with the corresponding TOR switch. Then, those acceleration components can use layer 2 Ethernet broadcast packets to send data to all acceleration components associated with the TOR switch. Layer 2 functionality can be similar to the layer 2 functionality provided by the data link layer of the OSI model. Layer 2 functionality may include media access control, flow control, and error checking. In one example, this step will not require any broadcast support from the network that interconnects the acceleration plane and the software plane. This can advantageously alleviate the need for multicast functionality provided by routers or other network infrastructure. In turn, this can reduce the complexity of deploying and managing acceleration components. In addition, in general, higher levels of a network (e.g., a network including routers and other TOR switches) may be overbooked, which in turn may reduce the bandwidth available to use the acceleration components of higher network communications. In contrast, in this example, the acceleration components of a shared TOR switch can advantageously have a higher bandwidth available to them for any transmission of messages from one acceleration component to another acceleration component.

[0034] Figure 4 A schematic diagram of an acceleration component 400 according to an example is shown. The acceleration component 400 can be included in the acceleration plane 106. The components included in the acceleration component 400 can be implemented on hardware resources (eg, logic blocks and programmable interconnects) of the acceleration component 400.

[0035] The acceleration component 400 may include an application logic 406, a soft shell 404 associated with a first set of resources, and a shell 402 associated with a second set of resources. The resources associated with the shell 402 may correspond to low-level interface-related components that may generally remain the same across many different application scenarios. The resources associated with the soft shell 404 may remain the same across at least some different application scenarios. The application logic 406 may also be conceptualized as including an application domain (e.g., a "role"). An application domain or role may represent a portion of the functionality included in a component service deployed on multiple acceleration components. The roles at each acceleration component in a set of acceleration components may be linked together to create a group that provides service acceleration for an application domain.

[0036] The application domain hosts application logic 406 that performs service-specific tasks (such as a portion of functionality for sorting documents, encrypting data, compressing data, facilitating computer vision, facilitating speech translation, machine learning, etc.) Resources associated with the soft shell 404 are generally less subject to change than application resources, and resources associated with the shell 402 are less subject to change than resources associated with the soft shell 404 (although changing (reconfiguring) any component of the acceleration component 400 is possible).

[0037] In this example, in operation, application logic 406 interacts with shell resources and soft shell resources in a similar manner to the interactive mode of the application and its underlying operating system resources implemented by software. From the application development point of view, the use of public shell resources and soft shell resources makes the developer need not recreate these public components for each service.

[0038] Referring first to the shell 402, the shell resources may include a bridge 408 for coupling the acceleration component 400 to a network interface controller (via a NIC interface 410) and a local top-of-rack switch (via a TOR interface 412). The bridge 408 also includes a data path that allows traffic to flow from the NIC or TOR into the acceleration component 400 and allows traffic to flow from the acceleration component 400 out to the NIC or TOR. Internally, the bridge 408 may include: various FIFOs (414, 416) that buffer received packets; and various selectors and arbitration logic that routes packets to their desired destinations. When activated, the bypass control component 418 can control the bridge 408 so that packets are transmitted between the NIC and the TOR without further processing by the acceleration component 400.

[0039] The memory controller 420 manages the interaction between the acceleration component 400 and a local memory 422, such as DRAM memory. The memory controller 420 may perform error correction as part of its services.

[0040] The host interface 424 may provide functionality that enables the acceleration component 400 to interact with a local host component (not shown). In one implementation, the host interface 424 may use a peripheral component interconnect express (PCIe) in conjunction with direct memory access (DMA) to exchange information with the local host component. The housing may also include various other features 426, such as a clock signal generator, status LEDs, error correction functionality, and the like.

[0041] The elastic router 428 can be used to route messages between various internal components of the acceleration component 400 and between the acceleration component and external entities (e.g., via the transport component 430). Each such endpoint can be associated with a corresponding port. For example, the elastic router 428 is coupled to the memory controller 420, the host interface 424, the application logic 406, and the transport component 430.

[0042] The transport component 430 can formulate packets for transmission to remote entities (such as other acceleration components) and receive components from remote entities (such as other acceleration components). In this example, when activated, the 3-port switch 432 takes over the function of the bridge 408 by connecting between the NIC and the TOR and between the NIC or TOR and the local port associated with the acceleration component 400.

[0043] The diagnostic logger 434 can store information about the operations performed by the router 428, the transport component 430, and the 3-port switch 432 in a circular buffer. For example, the information can include data about the origin and destination IP addresses of the packet, host-specific data, or timestamps. The logs can be stored as part of a telemetry system (not shown) so that a technician can study the logs to diagnose the cause of a failure or suboptimal performance in the acceleration component 400.

[0044] Multiple acceleration components similar to acceleration component 400 may be included in acceleration plane 106. Acceleration components may communicate with each other using different network topologies (instead of using network 120 for communication). In one aspect, acceleration components are directly connected to each other (such as, for example, in a two-dimensional torus). Although Figure 4 A specific number of components of the acceleration assembly 100 are shown arranged in a certain manner, but there may be a greater or lesser number of components arranged differently. Additionally, the various components of the acceleration assembly 400 may also be implemented using other techniques.

[0045] Figure 5A schematic diagram of a 3-port switch 500 according to an example is shown (solid lines represent data paths, and dashed lines represent control signals). The 3-port switch 500 can provide a feature that prevents packets for acceleration components from being sent to the host system. If the data network supports several lossless types of traffic, the 3-port switch 500 can be configured to provide sufficient support to buffer and pause incoming lossless streams to allow it to insert its own traffic into the network. To support that, the 3-port switch 500 can be configured to distinguish lossless traffic categories (e.g., remote direct memory access (RDMA)) from lossy (e.g., TCP / IP) traffic categories. A field in the packet header can be used to identify which traffic category the packet belongs to. The configuration memory 530 can be used to store any configuration files or data structures corresponding to the 3-port switch 500.

[0046] The 3-port switch 500 may have a first port 502 (host side) connected to a first MAC and a second port 504 (network side) connected to a second MAC. A third local port may provide internal services to a transport component (e.g., transport component 430). The 3-port switch 500 may generally operate as a network switch with some limitations. Specifically, the 3-port switch 500 may be configured to only pass packets (e.g., lightweight transport layer (LTL) packets) received on a local port to the second port 504 (not the first port 502). Similarly, the 3-port switch 500 may be designed not to deliver packets from the first port 502 to the local port.

[0047] The 3-port switch 500 may have two packet buffers: a packet buffer for receiving (Rx) the first port 502 and a packet buffer for receiving the second port 504. The packet buffer may be divided into several areas. Each area may correspond to a packet traffic category. When a packet arrives and is extracted from its frame (e.g., an Ethernet frame), it may be classified by a packet classifier (e.g., a packet classifier 520 and a packet classifier 522) as one of the available packet categories (lossy, lossless, etc.) and written into the corresponding packet buffer. If no buffer space is available for the inbound packet, the packet may be discarded. Once the packet is stored and ready to be transmitted, an arbitrator (e.g., an arbitrator 512 or an arbitrator 514) may select from the available packets and may transmit the packet. A priority flow control (PFC) insertion block (e.g., a PFC insertion 526 or a PFC insertion 528) may allow the 3-port switch 500 to insert a PFC frame between the flow packets at the transmission half of any one of the ports 502 and 504.

[0048] The 3-port switch 500 can handle the lossless traffic category as follows. All packets arriving at the receiving half of the first port 502 and the receiving half of the second port 504 should eventually be transmitted on the corresponding transmission (Tx) half of the port. The transmission packet can be routed in a store-and-forward manner. Priority flow control (PFC) can be implemented to avoid packet loss. For the lossless traffic category, the 3-port switch 500 can generate a PFC message and send it on the transmission part of the first port 502 and the second port 504. In one embodiment, when the packet buffer is full, the PFC message is sent. When the buffer is full or about to be full, the PFC message is sent to the link partner requesting that the traffic of this category be suspended. The PFC message can also be received and acted upon. If a PFC control frame is received for the lossless traffic category on the receiving part of the first port or the second port (502 or 504), the 3-port switch 500 can suspend sending packets on the transmission part of the port receiving the control frame. The packet can be internally buffered until the buffer is full, at which time a PFC frame will be generated to the link partner.

[0049] The 3-port switch 500 can handle lossy traffic classes as follows: Lossy traffic (all traffic not classified as lossless) can be forwarded on a best effort basis. The 3-port switch 500 is free to drop packets if congestion is encountered.

[0050] Signs of congestion may be detected in packets or flows before the packets traverse a host and its network stack. For example, if a congestion marker is detected in a packet on its way to a host, the transport component may quickly stop or start a flow, increase or decrease available bandwidth, throttle other flows / connections, etc. before the effects of congestion begin to manifest at the host.

[0051] Figure 6 is shown coupled to a 3-port switch 602 (corresponding to Figure 4 3 port 432 example) and the transport component 600 of the elastic router 604 (corresponding to Figure 4 Example of transport component 430 of FIG. 4 ). Transport component 600 can be configured to act as an autonomous node on a network. In one embodiment, transport component 600 can be configured within an environment or shell in which any process or execution unit can be instantiated. The use of transport component 600 can be advantageous due to the proximity between application logic and the network and the removal of host-based burdens (such as navigating complex network stacks, interrupt handling, and resource sharing). Therefore, applications or services using acceleration components and transport components (such as transport component 600) may be able to communicate with lower latency and higher throughput. Transport component 600 can itself be an agent that generates and consumes network traffic for its own purposes.

[0052] Transport component 600 may be used to implement functionality associated with a mechanism or protocol for exchanging data, including transmitting or retransmitting messages. In this example, transport component 600 may include transmit logic 610, receive logic 612, soft shell 614, connection management logic 616, configuration memory 618, transmit buffer 620, and receive buffer 622. These elements may operate to provide efficient and reliable communication between transport components that may be included as part of an acceleration component.

[0053] In one example, the transport component 600 can be used to implement functionality associated with a lightweight transport layer (LTL). Consistent with this example of LTL, the transport component 600 can expose two main interfaces for LTL: one main interface for communicating with a 3-port switch 602 (e.g., a local network interface that can then be connected to a network switch (such as a TOR switch)) and another main interface for communicating with an elastic router 604 (e.g., an elastic router interface). In this example, the local network interface (local_*) can contain a network stream (NetworkStream) that is ready and valid for both receiving (Rx) and sending (Tx) directions. In this example, the elastic router interface (router_*) can expose a FIFO-like interface that supports multiple virtual channels and a credit-based flow control scheme. The transport component 600 can be configured via a configuration data structure (structure) for runtime controllable parameters, and can output a state data structure (a structure for status monitoring by a host or other soft shell logic). Table 1 below shows an example of an LTL top-level module interface.

[0054]

[0055] Table 2 below shows example static parameters that may be set for an LTL instance at compile time. The values ​​for these parameters are examples only, and additional or fewer parameters may be specified.

[0056] Parameter name Configuration Values MAX_VIRTUAL CHANNELS 8 ER_PHITS_PER_FLIT 4 MAX_ER_CREDITS 256 EXTRA_SFQ_ENTRIES 32

[0057] Thus, as described above, in Table 3, this will configure a buffer of MAX_VIRTUAL_CHANNELS+EXTRA_SFQ_ENTRIES MTU size for the LTL instance. Elastic router credits (ER_CREDITS) can be issued with a guaranteed minimum of 1 credit for each virtual channel (VC), as well as a dynamically calculated number of additional credits. The transport component 600 can expose a configuration input port that sets several runtime values. The configuration port can be defined as part of the LTLConfiguration structure data structure. The fields for the example data structure are listed in the following table (Table 4):

[0058]

[0059]

[0060]

[0061] The functionality corresponding to the fields shown in Table 4 may be combined or further separated. Certain fields may also be in memory indexed by address or descriptor fields in the LTLConfiguration structure data structure. Similarly, special instructions may provide information related to any of the fields in Table 4 or they may combine information from such fields. Other changes may be made to the LTLConfiguration structure data structure and format without departing from the scope of the present disclosure.

[0062] In one example, as part of LTL, all messages can be encapsulated in IPv4 / UDP frames. Table 5 below shows an example packet format for encapsulating messages in such frames. The Group column shows the various groups of fields in the packet structure. The Description column shows the fields corresponding to each group in the packet structure. The Size column shows the size of each field in bits. The Value column provides values ​​for the fields and, if necessary, provides example descriptions of related fields.

[0063]

[0064]

[0065]

[0066] The functionality corresponding to the fields shown in Table 5 may be combined or further separated. Certain fields may also be in memory indexed by an address or descriptor field in the packet. Similarly, a special instruction may provide information related to any of the fields in Table 5 or it may combine information from such fields. Other changes may be made to the packet structure and format without departing from the scope of the present disclosure.

[0067] The connection management logic 616 can provide a registration interface for establishing connections between transport components. The connection management logic 616, together with software (e.g., a soft shell), can set up a connection before data can be transmitted or received. In one example, there are two connection tables that can control the state of the connection, a send connection table (SCT) and a receive connection table (RCT). Each of these tables can be stored as part of a configuration memory 618 or some other memory associated with the transport component 600. Each entry in the SCT (send connection table entry (SCTE)) can store the current sequence number of the packet and other connection states used to establish the packet, such as the destination MAC address. Requests arriving from the elastic router 604 can be matched with the SCTE by comparing the destination IP address with the virtual channel field provided by the elastic router 604. At most one connection can target the destination IP address and VC pair. Therefore, the tuple {IP, VC} can be a unique key (in database terms) in the table. It may be possible to have two entries in the table with the same VC - for example, {IP: 10.0.0.1, VC: 0} and {IP: 10.0.0.2, VC: 0}. It may also be possible to have two entries with the same IP address but different VCs: {IP: 10.0.0.1, VC: 0} and {IP: 10.0.0.2, VC: 1}. However, two entries with the same {IP, VC} pair may not be allowed. The number of entries supported by LTL can be configured at compile time.

[0068] The elastic router 604 can move data in micro-slices, which can be 128B in size (32B x 4 cycles). A message can include multiple micro-slices demarked by a start marker and a last marker. In one example, once the elastic router 604 selects a flow to send from an input port to an output port for a given virtual channel, the entire message must be delivered before another message will begin to arrive on the same virtual channel. The connection management logic 616 may need to packetize messages from the elastic router 604 into blocks of the maximum transport unit (MTU) size of the network. This can be done by buffering data on each virtual channel until one of the following conditions is met: (1) the last marker is seen in the micro-slice; and (2) the MTU's worth of data (or appropriately reduced in size to accommodate header and alignment requirements). In this implementation, the MTU for the LTL payload can be 1408 bytes. Once one of the requirements is met, the transport component 600 via the transmission logic 610 can attempt to send the packet. The packet destination can be determined by a combination of the virtual channel on which the message arrives at the input of the transport component 600 (from the elastic router 604) and the message header that may arrive during the first cycle of the message from the elastic router 604. These two values ​​can be used to index into the send connection table, which can provide the destination IP address and the sequence number for the connection. In this example, each packet transmitted on a given connection should have a sequence number one greater than the previous packet for that connection. The only exception can be for retransmissions, which can see discarded or unacknowledged packets retransmitted with the same sequence number as it was originally sent. The first packet sent on a connection can have a sequence number set to 1. Therefore, as an example, for a collection of micro slices on various virtual channels (VCs) arriving from the elastic router 604 into the transport component 600, a buffer (e.g., receive buffer 622) can be used to buffer the data until the end of the message or the MTU's worth of data has been received and the packet can then be output. Thus, as an example, if 1500B is sent from the elastic router 604 to at least one LTL instance associated with the transport component 600 as a single message (e.g., multiple microslices on the same VC with zero or one LAST flag), at least one packet may be generated. In this example, the LTL instance may send a message once it has buffered the data - i.e., it will not wait for an ACK for the first message before sending the next message. There may be no maximum message size. The LTL instance may simply chunk up the message into packets of the MTU size and transmit it once the MTU's worth of data is ready. Similarly, in this example, there is no "message length" field anywhere in the packet - only the payload size for each packet. The transport component 600 may have no incremental knowledge of how much data the message will contain.Preferably, the instance of LTL associated with transport component 600 can deliver arriving flits matching a given SCT entry in order (even before discarding and timing out). Flits matching different SCT entries may have no ordering guarantees.

[0069] In this example, the transport component 600 will output one credit for each virtual channel and then one credit for each shared buffer. Credits will be returned after each flits except when a flits completes the MTU buffer. This may occur if the last flag is received or when the flits contain the MTUth byte of the message. Credits consumed in this manner may be held by the transport component 600 until the packet is acknowledged.

[0070] In terms of receiving packets through an instance of LTL associated with the transport component 600, in one example, packets arriving from the network are matched to RCT entries (RCTEs) through fields in the packet header. Each RCTE stores the last sequence number and the virtual channel (VC) that outputs the packet from the transport component 600 to the elastic router 604. Multiple entries in the RCT can point to the same output virtual channel. The number of entries supported by the LTL can be configured at compile time. When a packet arrives on a local port from a network switch, the transport component 600 can determine which entry in the receive connection table (RCT) the packet is paired with. If no matching RCT table exists, the packet can be discarded. The transport component can check that the sequence number matches the expected value from the RCT entry. If the sequence number is greater than the RCT entry, the packet can be discarded. If the sequence number is less than what is expected by the RCT entry, an acknowledgment (ACK) can be generated and the packet can be discarded. If it matches, the transport component can grab the virtual channel field of the RCT entry. If the number of available elastic router (ER) credits for the virtual channel is sufficient to cover the packet size, the transport component 600 can accept the packet. If there are insufficient credits, the transport component 600 can drop the packet. Once the packet is accepted, an acknowledgment (ACK) can be generated and the RCT entry sequence number can be incremented. The elastic router 604 can use the packet header to determine the final endpoint to which the message is destined. The transport component may require sufficient credits to be able to transfer all packets' worth of data to the elastic router 604 to proceed. To help ensure that all VCs can proceed, the transport component 600 can require the elastic router 604 to provide dedicated credits for each VC to handle at least one MTU of data for each VC. In this example, no shared credits can be assumed.

[0071] SCT / RCT entries can be written by software. In one example, the software can maintain the mirror image established by the connection. In order to update the SCT or RCT entries, the user can write to the register_wrdata_in port, which can be hooked to the register in the soft shell or environment corresponding to the application logic. Table 6 below is an example of the format of the data structure that can be used to update the entries in the SCT or RCT.

[0072]

[0073] To write an SCT entry, scte_not_rcte may be set to 1, the sCTI value may be set to the value of the index to the SCT being written to, and then the other fields of the data structure in Table 6 may be appropriately set. With respect to timing, the value of register_write_in may be switched high for at least one cycle. rCTI may be set to the RCT entry of the remote acceleration component (in this example, rCTI is included in the UDP packet sent to the acceleration component and this is how to find the correct connection at the other end). IPAddr may be set to the IP address of the destination acceleration component. MacAddr may be set to the MAC address of the host on the same LAN segment as the acceleration component or the MAC address of the router for the remote host. VirtualChannel may be set by looking it up based on the micro slices arriving from the elastic router 604. To write an RCT entry, scte_not_rcte may be set to 0, the rCTI value may be set to the value of the index to the RCT being written to, and then the other fields of the data structure in Table 6 may be appropriately set. rCTI may be set to the RCT entry of the sending acceleration component. IPAddr can be set to the IP address of the sending acceleration component. MacAddr can be ignored for the purpose of writing RCT. VirtualChannel can be set to the channel to which the message will be sent to the elastic router 604.

[0074] As an example, to establish a unidirectional link from node A (e.g., transport component A (10.0.0.1)) to node B (e.g., transport component B (10.0.0.2)), one may: (1) create a SCTE {sCTI:1, rCTI:4, IP:10.0.0.2, VC:1, Mac:01-02-03-04-05-06} on transport component A; and (2) create a RCTE {rCTI:4, sCTI:1, IP:10.0.0.1, VC:2} on transport component B. In this example, this will take a message arriving from the elastic router on transport component A with DestIP == 10.0.0.2 and VC == 1 and send it to transport component B in a packet. The packet header will have the rCTI field set to 4 (the rCTI value read from the SCT). Transport component B will access its RCT entry 4 and learn that the message should be output on VC 2. It will also generate an ACK back to transport component A. In this packet, the sCTI field will have a value of 1 (filled from the sCTI value read from the RCT).

[0075] An instance of LTL associated with transport component 600 may buffer all packets being sent until it receives an acknowledgment (ACK) from a receive acceleration component. If an ACK for a connection does not arrive within a configurable timeout period, the packet may be retransmitted. In this example, all unacknowledged packets starting with the oldest packet will be retransmitted. The discarding of packets belonging to a given SCT may not change the behavior of any other connection—that is, packets for other connections may not be retransmitted. In one example, because an LTL instance may require a reliable communication channel and packets may sometimes be lost on the network, a timeout-based retry mechanism may be used.

[0076] If a packet does not receive an acknowledgment within a certain period of time, it may be retransmitted. The timeout period may be set via a configuration parameter. Once a timeout occurs, the transport component 600 may adjust the congestion inter-packet gap for the flow.

[0077] Transport component 600 can also provide congestion control. If the LTL instance transmits data to a receiver that cannot absorb traffic at full line rate, the congestion control functionality can allow it to gracefully reduce the frequency of packets being sent to the destination node. Each LTL connection can have an associated inter-packet gap state that controls the minimum number of cycles between the transmission of packets in the flow. When creating a new connection, the IPG can be set to 1, which effectively allows full use of any available bandwidth. If a timeout, ECN notification, or NACK occurs on the flow, the delay can be multiplied by the cfg.throttle_credit_multiple parameter (see Table 3) or increased by the cfg.throttle_credits_per_scrub parameter (see Table 3; depending on the selection of linear or exponential backoff). Each ACK received can be subtracted from the IPG by the cfg.throttle_credits_per_scrub parameter (see Table 3) or the IPG can be divided by the cfg.throttle_credit_multiple parameter (see Table 3; depending on the selection of linear or exponential recovery). The LTL instance may not increase the IPG of a flow more than once per a predetermined period of time; for example, no more than every 2 microseconds (in this example, this may be controlled by the cfg.throttle_scrub_delay parameter (see Table 3)).

[0078] Consistent with an example of LTL, the transport component 600 may attempt to retransmit 128 times. If after 128 retries the packet is still not acknowledged, the packet may be dropped and the buffer released. Unless the disable_timeouts configuration bit is set, the transport component 600 may also clean up the SCTE for the connection to prevent further messages and packets from being transmitted. In this example, at this point, no data can be exchanged between the link partners because their sequence numbers will be out of sync. The connection will need to be reestablished.

[0079] When the LTL instance associated with the transport component 600 successfully receives a packet, it will generate an acknowledgment (e.g., an empty payload packet with the ACK flag bit set). The acknowledgment (ACK) can include a sequence number that tells the sender the last packet successfully received and the SCTI to which the sender should attribute the ACK (this value can be stored in the RCT of the ACK generator). According to one example of LTL, the following rules can be used to generate the ack: (1) if the RX sequence number matches the expected sequence number (in the RCT), then generate an ACK using the received sequence number; (2) if the RX sequence number is less than the expected sequence number, then discard the packet, but generate an ACK with the highest received sequence number (this can cover the case where a packet was sent twice (perhaps due to a timeout) but then received correctly); and (3) if the RX sequence number is greater than the expected sequence number, then discard the packet and do not generate an ACK.

[0080] An instance of LTL associated with transport component 600 may generate NACKs under certain conditions. These may be packets marked with both the ACK and NACK flag bits set. A NACK may be a request for the sender to retransmit a particular packet and all subsequent packets.

[0081] In one example, transport component 600 may require generation of a NACK under two conditions: (1) if a packet is dropped due to insufficient elastic router credits to accept all packets, transport component 600 may send a NACK once sufficient credits exist to transport the packet; or (2) if a packet is dropped because another sender currently holds a lock on the destination virtual channel, transport component 600 may send a NACK once the VC lock is released.

[0082] When an LTL endpoint is receiving traffic from multiple senders, the receiver can maintain a side data structure per VC (VCLockQueue) that can track which senders had their packets dropped because another message was being received on a particular VC. This side data structure can be used to coordinate multiple senders through explicit retransmission requests (NACKs).

[0083] In one example, once an instance of LTL associated with transport component 600 begins receiving messages on a particular VC, the VC is locked to that single sender until all packets of the message have been received. If another sender attempts to send a packet on the same VC while it is locked or when there are not enough ER credits available, the packet will be dropped and the receiver will be placed on the VCLockQueue. Once the lock is released or there are enough ER credits, the LTL instance will pop the VCLockQueue and send a retransmission request (NACK) to the next sender placed in the VCLockQueue. After popping from the VCLockQueue, the sender can be given the highest priority for the next 200,000 cycles (~1.15ms). During these 200,000 cycles, packets from other senders on the same VC will be dropped. This ensures that all senders that have had packets dropped will eventually get a chance to send their messages.

[0084] When a receiver drops a sender's packet because another sender has a VC lock, the receiver can place the sender (which has its packets dropped) on the VCLockQueue and send a NACK that also includes the XOFF flag, which indicates that the sender should not attempt to retransmit for some time (indicated by the cfg.xoff_period parameter). If the receiver has no ER credits, the NACK may also include a congestion flag.

[0085] When a sender receives a NACK with the XOFF flag, it may delay the next packet per the backoff period (e.g., xoff_period). If the NACK does not include a congestion flag (i.e., the drop is not due to insufficient credit but to VC lock), then the sender may record that VC lock is in effect for the flow. When a flow has VC lock enabled, the sender may need to ensure that it slows down after completing each message because it knows that packets of subsequent messages will be dropped because it competes with other senders that will receive the VC lock next. However, in this example, the sender will need to ensure that the first packet of the subsequent message is sent before slowing down (even if the sender knows that the first packet will be dropped) to ensure that it is placed in the VCLockQueue.

[0086] Figure 7 A flowchart 700 of a method for providing a service using a transport component to process messages is shown according to one example. In one example, application logic (e.g., Figure 6The application logic 608 of the acceleration component can be segmented and mapped to the roles of multiple acceleration components. As previously described, the application logic can be conceptualized as including an application domain (e.g., a "role"). The application domain or role can represent a portion of the functionality included in the component service deployed on multiple acceleration components. The role at each acceleration component in a set of acceleration components can be linked together to create a group that provides service acceleration for the application domain. Each application domain can host application logic to perform service-specific tasks (such as a portion of functionality for sorting documents, encrypting data, compressing data, facilitating computer vision, facilitating speech translation, machine learning, etc.). Step 702 may include receiving a message from a host to perform a task corresponding to a service. Step 704 may include forwarding the message to an acceleration component at the head of a multi-stage pipeline of the acceleration component, where the acceleration component may be associated with a first switch. Therefore, when a request for a function associated with a service (e.g., a sorting request) arrives from the host in the form of a PCI Express message, it can be forwarded to the head of the multi-stage pipeline of the acceleration component. In step 706, the acceleration component receiving the message can determine whether the message is for the acceleration component at the head. If the message is for an acceleration component at the head of a pipeline stage, the acceleration component at the head of the pipeline stage may process the message at the acceleration component. As part of this step, an elastic router (e.g., Figure 6The elastic router 604) can forward the message directly to the role. In one example, the LTL packet format (e.g., as shown in Table 6) can include a broadcast flag (e.g., bit 3 under the flag header as shown in Table 6) and a retransmission flag (e.g., bit 2 under the flag header as shown in Table 6). The broadcast flag can signal the acceleration component that the message is intended for multiple acceleration components. The retransmission flag can indicate to the acceleration component that a retransmission of the message is requested. In both cases, the LTL packet format can include a header that lists the IP addresses for (multiple) specific destinations. Therefore, when the acceleration component receives a message with the broadcast flag set, in step 712, the acceleration component (e.g., using the transport component 600) can transmit the message as a point-to-point message to a selected set of acceleration components, where each acceleration component is associated with a different top-of-rack (TOR) switch. In one example, the point-to-point message is transmitted using layer 3 functionality. In this example, when the transport component 600 receives a packet with the broadcast flag set (e.g., part of a point-to-point message), it can process the list of destinations; if it contains more than just its own IP address, it adds the packet to its own transmit buffer, sets up a bit field by each destination in the destination to track reception and then begins transmitting to each receiver one by one (in the absence of a broadcast or retransmit field). The destination acceleration component can acknowledge (using ACK) each packet based on successful reception, and the sender will mark the appropriate bit in the bit field. Once the packet is acknowledged by all destinations, the transmit buffer can be released.

[0087] Next, in step 714, each of the set of acceleration components may broadcast the message to any other acceleration components associated with the corresponding TOR switch using data link layer functionality. An example of data link layer functionality may be a layer 2 Ethernet broadcast packet. In this example, when the transport component 600 receives a packet with a broadcast flag set (e.g., part of a point-to-point message), it may process the list of destinations; if it contains more than just its own IP address, it adds the packet to its own transmit buffer, sets up a bit field through each destination in the destination to track receipt and then sends a layer 2 Ethernet broadcast (along with the broadcast flag and destination list) to all acceleration components that share a common TOR switch. The destination acceleration component may acknowledge (using ACK) each packet based on successful receipt, and the sender will mark the appropriate bit in the bit field. Once the packet is acknowledged by all destinations, the send buffer may be released. Although Figure 7 A specific number of steps are shown listed in a specific order, but there may be fewer or more steps and such steps may be performed in a different order.

[0088] Acceleration components can be grouped together as part of a graph. Grouped acceleration components do not need to be physically close to each other; instead, they can be associated with different parts of a data center and still be grouped together by linking them as part of an acceleration plane. In one example, a graph can have a certain network topology, depending on which of the acceleration components associated with which of the TOR switches are coupled together to accelerate a service. The network topology can be dynamically created based on configuration information received from a service manager for a service. The service manager can be a higher-level software associated with a service. In one example, the network topology can be dynamically adjusted based on at least one performance metric associated with a network (e.g., network 120) interconnecting the acceleration plane and the software plane, the network including a host component configured to execute instructions corresponding to at least one service. The service manager can use a telemetry service to monitor network performance. The network performance metric can be selected in substantially real time based on at least one service requirement. The at least one network performance metric can include latency, bandwidth, or any other performance metric specified by a service manager or application logic corresponding to at least one service.

[0089] In one example, the acceleration components may broadcast messages using a tree-based transport process that includes point-to-point links for acceleration components connected via layer 3 and layer 2 Ethernet broadcasts to the acceleration components sharing a TOR switch. The tree may be two levels or may have more levels, depending on the bandwidth limitations imposed by the network interconnecting the acceleration components.

[0090] Figure 8 A flow chart of a method for transmitting a message according to an example is shown. In step 802, a first acceleration component associated with a first TOR switch receives a message from a host. As previously discussed, the acceleration component may include a transport component (e.g., Figure 4 The transport component of 430, which is relative to Figure 6The transport component 600 is further described). In step 804, using network layer functionality (e.g., layer 3 functionality), the first acceleration component can transmit the message to a second acceleration component associated with a second TOR switch different from the first TOR switch. In step 806, using network layer functionality (e.g., layer 3 functionality), the first acceleration component can transmit the message to a third acceleration component associated with a third TOR switch different from the first TOR switch and the second TOR switch. In this example, when the transport component 600 receives a packet with a broadcast flag set (e.g., a portion of a received message), it can process the destination list; if it contains more than just its own IP address, it adds the packet to its own transmit buffer, sets up a bit field with each destination in the destination to track reception and then begins transmitting to each receiver one by one (in the absence of a broadcast or retransmit field). The destination acceleration component can acknowledge (using ACK) each packet based on successful reception, and the sender will mark the appropriate bit in the bit field. Once the packet is acknowledged by all destinations, the send buffer can be released.

[0091] In step 810, using data link layer functionality (e.g., Layer 2 Ethernet broadcast functionality), the second acceleration component can broadcast the message to other acceleration components associated with the second TOR switch. In this example, when the transport component 600 receives a packet with a broadcast flag set (e.g., part of a point-to-point message), it can process the destination list; if it contains more than just its own IP address, it adds the packet to its own transmit buffer, sets up a bit field through each destination in the destination to track reception and then sends a Layer 2 Ethernet broadcast (along with the broadcast flag and destination list) to all acceleration components that share a common TOR switch. The destination acceleration component can acknowledge (using ACK) each packet based on successful reception, and the sender will mark the appropriate bit in the bit field. Once the packet is acknowledged by all destinations, the send buffer can be released.

[0092] In step 810, using a data link layer function line (e.g., Layer 2 Ethernet broadcast functionality), the third acceleration component can broadcast a message to other acceleration components associated with the third TOR switch. In this example, when the transport component 600 receives a packet with a broadcast flag set (e.g., part of a point-to-point message), it can process the destination list; if it contains more than just its own IP address, it adds the packet to its own transmit buffer, sets up a bit field through each destination in the destination to track reception and then sends a Layer 2 Ethernet broadcast (along with the broadcast flag and destination list) to all acceleration components that share a common TOR switch. The destination acceleration component can acknowledge (using ACK) each packet based on successful reception, and the sender will mark the appropriate bit in the bit field. Once the packet is acknowledged by all destinations, the send buffer can be released. Although Figure 8 A specific number of steps are shown listed in a specific order, but there may be fewer or more steps and such steps may be performed in a different order.

[0093] In summary, a system is provided, comprising: a software plane, comprising a plurality of host components configured to execute instructions corresponding to at least one service; and an acceleration plane, comprising a plurality of acceleration components configurable to accelerate at least one service. The system may also include a network configured to interconnect the software plane and the acceleration plane, the network may include: a first top of rack (TOR) switch associated with a first subset of the plurality of acceleration components, a second TOR switch associated with a second subset of the plurality of acceleration components, and a third TOR switch associated with a third subset of the plurality of acceleration components, wherein any acceleration component in the first subset of the plurality of acceleration components may be configured to transmit a point-to-point message to any acceleration component in the second subset of the plurality of acceleration components and transmit the point-to-point message to any acceleration component in the third subset of the plurality of acceleration components, and wherein any acceleration component in the second subset of the plurality of acceleration components may be configured to broadcast the point-to-point message to all acceleration components in the second subset of the plurality of acceleration components, and wherein any acceleration component in the third subset of the plurality of acceleration components may be configured to broadcast the point-to-point message to all acceleration components in the third subset of the plurality of acceleration components. The point-to-point message may be transmitted using layer 3 functionality. The point-to-point message may be broadcast using layer 2 Ethernet broadcast functionality. Point-to-point messages can be broadcast without relying on any broadcast support from higher layers of the network other than layer 2 of the network or any multicast support from higher layers of the network other than layer 2 of the network. Any acceleration component in the first subset of multiple acceleration components can be configured to dynamically create a network topology that includes a second subset of multiple acceleration components and any acceleration component in the third subset of multiple acceleration components. Any acceleration component in the first subset of multiple acceleration components can also be configured to dynamically adjust the network topology based on at least one performance metric associated with a network configured to interconnect the software plane and the acceleration plane. The at least one network performance metric is selected in substantially real time by any acceleration component in the first subset of multiple acceleration components based at least on the requirements of at least one service. The at least one network performance metric may include latency, bandwidth, or any other performance metric specified by application logic corresponding to at least one service.

[0094] In another example, the present disclosure relates to a method for allowing a first acceleration component among a first plurality of acceleration components associated with a first top-of-rack (TOR) switch to transmit a message to other acceleration components in an acceleration plane that can be configured to provide service acceleration for at least one service. The method may include the first acceleration component transmitting a point-to-point message to a second acceleration component associated with a second TOR switch different from the first TOR switch, and a third acceleration component associated with a third TOR switch different from the first TOR switch and the second TOR switch. The method may also include the second acceleration component broadcasting the point-to-point message to all acceleration components of the second plurality of acceleration components associated with the second TOR switch. The method may also include the third acceleration component broadcasting the point-to-point message to all acceleration components of the third plurality of acceleration components associated with the third TOR switch. The first acceleration component may use layer 3 functionality to transmit the point-to-point message. Each of the second acceleration component and the third acceleration component may use layer 2 Ethernet broadcast functionality to broadcast the point-to-point message. The method may also include dynamically adjusting a network topology based on at least one performance metric associated with a network interconnecting an acceleration plane and a software plane, the network including a plurality of host components configured to execute instructions corresponding to at least one service. At least one network performance metric is selected in substantially real time by any acceleration component in a first subset of the plurality of acceleration components based at least on a requirement of the at least one service. The at least one network performance metric may include latency, bandwidth, or any other performance metric specified by application logic corresponding to the at least one service.

[0095] In yet another example, the present disclosure relates to an acceleration component for use between a first plurality of acceleration components associated with a first top-of-rack (TOR) switch to transmit a message to other acceleration components in an acceleration plane that can be configured to provide service acceleration for at least one service. The acceleration component may include a transport component configured to transmit a first point-to-point message to a second acceleration component associated with a second TOR switch different from the first TOR switch, and a third acceleration component associated with a third TOR switch different from the first TOR switch and the second TOR switch. The transport component may also be configured to broadcast a second point-to-point message to all acceleration components of the second plurality of acceleration components associated with the second TOR switch and all acceleration components of the third plurality of acceleration components associated with the third TOR switch. The point-to-point message may be transmitted using layer 3 functionality. The point-to-point message may be broadcast using layer 2 Ethernet broadcast functionality. The acceleration component may also be configured to broadcast a second point-to-point message in a network interconnecting the acceleration components in the acceleration plane without relying on any support for broadcast or multicast from a higher layer of a network other than layer 2 of the network.

[0096] It will be understood that the systems, methods, modules and components described herein are exemplary only. Alternatively or in addition, the functionality described herein may be performed at least in part by one or more hardware logic components. For example and without limitation, the hardware logic components of illustrative types that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc. In an abstract but still clear sense, any arrangement of components that implement the same functionality is effectively "associated" so that the desired functionality is implemented. Therefore, any two components that are combined to implement a specific functionality herein may be considered to be "associated" with each other so that the desired functionality is implemented regardless of the architecture or intermediate components. Similarly, any two components that are so associated may also be considered to be "properly connected" or "coupled" to each other to implement the desired functionality.

[0097] Functionality associated with some examples described in this disclosure may also include instructions stored in non-transient media. As used herein, the term "non-transient media" refers to any medium that stores data and / or instructions that cause a machine to operate in a specific manner. Exemplary non-transient media include non-volatile media and / or volatile media. Non-volatile media include, for example, hard disks, solid-state drives, disks or tapes, optical disks or tapes, flash memories, EPROMs, NVRAMs, PRAMs or other such media or networked versions of such media. Volatile media include, for example, dynamic memories, such as DARMs, SRAMs, caches or other such media. Non-transient media are different from transmission media, but can be used in conjunction with transmission media. Transmission media are used to transfer data and / or instructions to or from a machine. Exemplary transmission media include coaxial cables, fiber optic cables, copper wires, and wireless media (such as radio waves).

[0098] In addition, those skilled in the art will recognize that the boundaries between the functionality of the operations described above are illustrative only. The functionality of multiple operations can be combined into a single operation, and / or the functionality of a single operation can be distributed in additional operations. Moreover, alternative embodiments can include multiple instances of a particular operation, and the order of the operations can be changed in various other embodiments.

[0099] Although the present disclosure provides specific examples, various modifications and changes may be made without departing from the scope of the present disclosure as set forth in the following claims. Accordingly, the specification and drawings are to be understood in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure. Any benefits, advantages, or solutions to problems described herein with respect to specific examples are not intended to be construed as key, necessary, or essential features or elements of any or all claims.

[0100] Furthermore, the terms "a" or "an" as used herein are defined as one or more than one. Moreover, the use of introductory phrases such as "at least one" and "one or more" in the claims should not be interpreted as implying that any particular claim containing such introduced claim element by the indefinite article "a" or "an" will be limited to inventions containing only one such element, even when the same claim includes the introductory phrases "one or more" or "at least one" and an indefinite article such as "a" or "an". The same applies to the use of definite articles.

[0101] Unless otherwise stated, terms such as "first" and "second" are used to arbitrarily distinguish between the elements such terms describe. Therefore, these terms are not necessarily intended to indicate temporal or other prioritization of such elements.

Claims

1. A system comprising: a software plane comprising a plurality of host components configured to execute instructions corresponding to at least one service; as well as an acceleration plane comprising a plurality of acceleration components configurable to accelerate the at least one service, wherein each of the plurality of acceleration components is configurable as a sending acceleration component for sending messages for the at least one service or a receiving acceleration component for receiving messages for the at least one service, and wherein each of the messages comprises a plurality of packets having an inter-packet gap, and wherein the sending acceleration component is configured to operate in a first mode for the at least one service or a second mode for the at least one service, wherein in the first mode, the transmit acceleration component is configured to transmit subsequent packets corresponding to the first message associated with the at least one service using an inter-packet gap that is larger than an inter-packet gap used for a previous packet corresponding to the first message associated with the at least one service in response to receiving a first indication from the receive acceleration component, wherein the first indication comprises at least one of: expiration of a timer associated with transmission of the first message, an explicit congestion notification from an elastic router associated with the receive acceleration component, or an explicit retransmission request for retransmission of at least one packet associated with the first message, and Wherein, in the second mode, the sending acceleration component is configured to delay the transmission of a next packet corresponding to the first message associated with the at least one service for a configurable amount of time in response to receiving a second indication from the receiving acceleration component, wherein the second indication includes a retransmission request including a backoff flag, and wherein in the second mode, the receiving acceleration component is also configured to track at least a subset of other sending acceleration components, and the at least a subset of the other sending acceleration components previously attempted to send at least one message to the receiving acceleration component and at least one packet corresponding to the at least one message was discarded. 2 . The system of claim 1 , wherein in the second mode, the receive acceleration component is further configured to maintain a queue, the queue comprising information indicative of each tracked send acceleration component.

3. The system of claim 2, wherein after receiving the complete first message, the receiving acceleration component is further configured to determine that one of the tracked sending acceleration components is popped out of the queue.

4. A system according to claim 1, wherein in the second mode, the sending acceleration component is configured to delay the transmission of the next packet corresponding to the message based on the state of any receiving acceleration component to which the sending acceleration component is transmitting the message or to which the sending acceleration component has previously attempted to transmit a message and the packet corresponding to the message was discarded.

5. A method in a system, the system comprising a plurality of endpoints coupled via a network, wherein each of the plurality of endpoints is configurable as a sending endpoint for sending a message or a receiving endpoint for receiving a message, wherein each of the messages comprises a plurality of packets, and wherein the method comprises: The first sending endpoint initiates transmission of a first packet corresponding to a first message to a first receiving endpoint; The first receiving endpoint stores the first packet in a first buffer associated with the first receiving endpoint, and the first receiving endpoint transmits a first indication to the first sending endpoint, wherein the first indication comprises at least one of: expiration of a timer associated with transmission of the first message, an explicit congestion notification from a resilient router associated with the receiving endpoint, or an explicit retransmission request for retransmission of at least one packet associated with the first message; The first transmitting endpoint increases an amount of an inter-packet gap between consecutive first packets in response to the first indication; The second sending endpoint initiates transmission of a second packet corresponding to a second message to the first receiving endpoint; If the first receiving endpoint is still receiving the first packet corresponding to the first message from the first sending endpoint, the first receiving endpoint discards the second packet and transmits a second indication to the second sending endpoint, wherein the second indication includes a retransmission request including a backoff flag; as well as The second transmitting endpoint processes the second indication, and if the second indication indicates a delay backoff period, the second transmitting endpoint delays transmission of a next second packet by the backoff period, and transmits the next second packet after expiration of the backoff period.

6. The method according to claim 5, further comprising: The first receiving endpoint places information indicative of the second sending endpoint in a queue for tracking any sending endpoints that previously attempted to send at least one message to the first receiving endpoint.

7. The method according to claim 6, further comprising: The first receiving endpoint determines that one of the tracked sending endpoints is popped from the queue.

8. The method according to claim 5, further comprising: The first receiving endpoint transmits a third indication to the first sending endpoint, wherein the third indication corresponds to a backoff method of a linear backoff method or an exponential backoff method and corresponding parameters determined to increase an amount of an inter-packet gap between the consecutive first packets.

9. A system comprising: a software plane comprising a plurality of host components configured to execute instructions corresponding to at least one service; as well as An acceleration plane comprising a plurality of acceleration components configurable to accelerate at least one service, wherein each of the plurality of acceleration components is configurable as a sending acceleration component for sending messages for the at least one service or a receiving acceleration component for receiving messages for the at least one service, and wherein each of the messages comprises a plurality of packets having an inter-packet gap, and wherein the sending acceleration component is configured to send subsequent packets corresponding to the first message associated with the at least one service using an inter-packet gap larger than an inter-packet gap used for a previous packet corresponding to the first message associated with the at least one service in response to receiving an indication from the receiving acceleration component, wherein the indication comprises at least one of the following: expiration of a timer associated with transmission of the first message, an explicit congestion notification from an elastic router associated with the receiving acceleration component, or an explicit retransmission request for retransmitting at least one packet associated with the first message, and wherein the receiving acceleration component is configured to track at least a subset of other sending acceleration components, and wherein the at least a subset of the other sending acceleration components previously attempted to send at least one message to the receiving acceleration component, and at least one packet corresponding to the at least one message was discarded. 10 . The system according to claim 9 , wherein in the receiving acceleration component, the receiving acceleration component is further configured to maintain a queue, the queue including information indicating each tracked sending acceleration component.

11. The system of claim 10, wherein after receiving a complete message, the receiving acceleration component is further configured to determine that one of the tracked sending acceleration components is popped out of the queue.

12. A system comprising: a software plane comprising a plurality of host components configured to execute instructions corresponding to at least one service; as well as an acceleration plane comprising a plurality of acceleration components configurable to accelerate the at least one service, wherein each of the plurality of acceleration components is configurable as a sending acceleration component for sending messages for the at least one service or a receiving acceleration component for receiving messages for the at least one service, and wherein each of the messages comprises a plurality of packets having an inter-packet gap, and wherein the sending acceleration component is configured to operate in a first mode for the at least one service or a second mode for the at least one service, wherein in the first mode, the sending acceleration component is configured to, in response to receiving a first indication from the receiving acceleration component, send subsequent packets corresponding to the first message associated with the at least one service using an inter-packet gap that is larger than an inter-packet gap used for a previous packet corresponding to the first message associated with the at least one service, and wherein the sending acceleration component is not permitted to increase the amount of the inter-packet gap more than once within a specified predetermined time period, wherein the first indication comprises at least one of: expiration of a timer associated with transmission of the first message, an explicit congestion notification from an elastic router associated with the receiving acceleration component, or an explicit retransmission request for retransmitting at least one packet associated with the first message, and Wherein in the second mode, the sending acceleration component is configured to delay the transmission of a next packet corresponding to the first message associated with the at least one service for a configurable amount of time in response to receiving a second indication from the receiving acceleration component, wherein the second indication includes a retransmission request including a backoff flag.

13. A system according to claim 12, wherein in the second mode, the receiving acceleration component is also configured to track at least a subset of other sending acceleration components, and the at least a subset of the other sending acceleration components previously attempted to send at least one message to the receiving acceleration component and at least one packet corresponding to the at least one message was discarded.

14. The system of claim 13, wherein in the second mode, the receive acceleration component is further configured to maintain a queue including information indicative of each tracked transmit acceleration component.

15. The system of claim 14, wherein after receiving the complete first message, the receiving acceleration component is further configured to determine that one of the tracked sending acceleration components is popped out of the queue.

16. A system according to claim 12, wherein in the second mode, the sending acceleration component is configured to delay the transmission of a next packet corresponding to the message based on the state of any receiving acceleration component to which the sending acceleration component is sending the message or to which the sending acceleration component has previously attempted to send a message and the packet corresponding to the message was discarded.

17. A method in a system, the system comprising a plurality of endpoints coupled via a network, wherein each of the plurality of endpoints is configurable as a sending endpoint for sending a message or a receiving endpoint for receiving a message, wherein each of the messages comprises a plurality of packets, and wherein the method comprises: The first sending endpoint initiates transmission of a first packet corresponding to a first message to a first receiving endpoint; The first receiving endpoint stores the first packet in a first buffer associated with the first receiving endpoint, and the first receiving endpoint sends a first indication to the first sending endpoint, wherein the first indication comprises at least one of: expiration of a timer associated with transmission of the first message, an explicit congestion notification from a resilient router associated with the receiving endpoint, or an explicit retransmission request for retransmission of at least one packet associated with the first message; the first transmitting endpoint increasing an amount of an inter-packet gap between consecutive first packets in response to the first indication, wherein the first transmitting endpoint is not permitted to increase the amount of the inter-packet gap between the consecutive first packets more than once within a first specified predetermined time period; The second sending endpoint initiates transmission of a second packet corresponding to a second message to the first receiving endpoint; If the first receiving endpoint is still receiving the first packet corresponding to the first message from the first sending endpoint, the first receiving endpoint discards the second packet and sends a second indication to the second sending endpoint, wherein the second indication includes a retransmission request including a backoff flag; as well as The second transmitting endpoint processes the second indication, and if the second indication indicates a delay backoff period, the second transmitting endpoint delays transmission of a next second packet by the backoff period and transmits the next second packet after expiration of the backoff period, wherein the second transmitting endpoint is not allowed to increase the amount of the inter-packet gap between the consecutive second packets more than once within a second specified predetermined time period.

18. The method according to claim 17, further comprising: The first receiving endpoint places an identifier corresponding to the second sending endpoint in a queue for tracking any sending endpoint that previously attempted to send at least one message to the first receiving endpoint.

19. The method according to claim 18, further comprising: The first receiving endpoint determines that one of the tracked sending endpoints is popped from the queue.

20. The method of claim 17, further comprising: The first receiving endpoint sends a third indication to the first sending endpoint, wherein the third indication corresponds to a backoff method of a linear backoff method or an exponential backoff method and corresponding parameters, which are determined to increase the amount of the inter-packet gap between the consecutive first packets.

21. The method of claim 17, wherein each of the first transmitting endpoint, the second transmitting endpoint, and the first receiving endpoint comprises one or more field programmable gate arrays (FPGAs), and wherein at least one or more of the field programmable gate arrays (FPGAs) are at least partially configurable using an image file.

22. A method in a system, the system comprising a plurality of endpoints coupled via a network, wherein each of the plurality of endpoints is configurable as a sending endpoint for sending a message or a receiving endpoint for receiving a message, wherein each of the messages comprises a plurality of packets, and wherein the method comprises: The first sending endpoint initiates transmission of a first packet corresponding to a first message to a first receiving endpoint; The first receiving endpoint stores the first packet in a first buffer associated with the first receiving endpoint, and the first receiving endpoint sends a first indication to the first sending endpoint, wherein the first indication comprises at least one of: expiration of a timer associated with transmission of the first message, an explicit congestion notification from a resilient router associated with the receiving endpoint, or an explicit retransmission request for retransmission of at least one packet associated with the first message; as well as The first receiving endpoint sends a third indication to the first sending endpoint, wherein the third indication corresponds to a backoff method and corresponding parameters of a linear backoff method or an exponential backoff method, which is determined to be used to increase the amount of the inter-packet gap between the consecutive first packets, and wherein the first sending endpoint is not allowed to increase the amount of the inter-packet gap more than once within a specified predetermined time period.

23. The method of claim 22, wherein the inter-packet gap corresponds to a delay value, and wherein the linear backoff comprises adding an additional delay to the delay value to increase the inter-packet gap.

24. The method of claim 22, wherein the inter-packet gap corresponds to a delay value, and wherein the exponential backoff comprises multiplying the delay value by a factor to increase the inter-packet gap.

Images