System and method for inter-device networking using intra-device protocols

Abstract

The present disclosure relates to systems and methods for inter-device networking using intra-device protocols. In some aspects, the present disclosure relates to methods and systems for encapsulating intra-device protocols such as CXL and / or PCIe packets into inter-device protocols such as IP packets, thereby allowing sharing of resources across one or more networks. For example, a CXL and / or PCIe address to an IP address lookup table can be used to determine a destination IP address or a source IP address so that the CXL and / or PCIe packets can be sent across various networks. Layer 3 forwarding enables communication between large distributed systems while leveraging existing network infrastructure.

Description

Technical Field

[0001] The present disclosure generally relates to networking and communications, including but not limited to using intra-device protocols across inter-device networks, such as the Compute Express Link (CXL) standard and the Peripheral Component Interconnect Express (PCIe) standard. Background Art

[0002] Communication protocols can be roughly described as intra-device or inter-device, where high-speed intra-device protocols are used between internal device components such as processors, memory, and input / output devices, such as CXL and PCIe protocols; and intra-device protocols are used for communication between devices, such as Internet Protocol (IP) and Transmission Control Protocol (TCP). As an intra-device protocol, CXL is a standard designed to facilitate high-speed and efficient communication between high-speed components using interfaces such as PCIe interfaces. For example, the CXL protocol can be used with PCIe to provide a low-latency path between a host processor (e.g., a central processing unit (CPU)) and devices that share memory resources (e.g., such as accelerators and memory expanders). Using CXL in conjunction with PCIe switching increases the number of lanes in the bus that can be used for communication. However, CXL and PCIe are limited to the scope of a single link of a CXL or PCIe switch. Summary of the invention

[0003] In one aspect, the present disclosure relates to a transmission method via an Internet Protocol (IP) network, comprising: receiving, by a processor, a service to be transmitted; mapping, by the processor, an in-device protocol address to a destination IP address; encapsulating, by the processor, the service to be transmitted into an IP packet addressed to the destination IP address; and transmitting, by the processor, the service using the IP packet addressed to the destination IP address.

[0004] In another aspect, the present disclosure relates to a method for receiving data via an Internet Protocol (IP) network, comprising: receiving transmitted traffic by a processor; mapping a source IP address to an intra-device protocol address by the processor; decapsulating at least a portion of the received traffic into one or more transport layer packets by the processor; and routing the one or more transport layer packets by the processor using the mapped intra-device protocol address.

[0005] In another aspect, the present disclosure relates to a system for transmitting data via an Internet Protocol (IP) network, comprising: a computing device including a processing circuit system and a network interface; wherein the network interface is configured to receive traffic to be transmitted; and wherein the processing circuit system is configured to: map an in-device protocol address to a destination IP address; encapsulate the traffic to be transmitted into an IP packet addressed to the destination IP address; and transmit the traffic to be transmitted using the IP packet addressed to the destination IP address. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Various objects, aspects, features and advantages of the present disclosure will become more apparent and better understood by reference to the detailed description taken in conjunction with the accompanying drawings, in which reference numerals identify various objects, aspects and features. In the accompanying drawings, like reference numerals (or similar reference numerals, duplicate reference numerals, etc.) generally indicate identical, functionally similar and / or structurally similar elements.

[0007] Figure 1A is a block diagram depicting a network environment including one or more access points in communication with one or more devices or stations in accordance with one or more embodiments of the solution.

[0008] Figure 1B and 1C is a block diagram depicting a computing device that may be used in connection with the methods and systems described herein in accordance with one or more embodiments of the solution.

[0009] 2 is a block diagram depicting an example system for connecting devices using a PCIe switch for connecting the devices in accordance with one or more embodiments of the solution.

[0010] Figure 3A and 3B is a block diagram depicting an example system for connecting devices across an IP network in accordance with one or more embodiments of the solution.

[0011] Figure 4 is a flowchart of a method for transmitting an intra-device protocol via an inter-device network protocol according to one or more embodiments of the solution.

[0012] Figure 5 is a flowchart of a method for receiving an intra-device protocol via an inter-device network protocol according to one or more embodiments of the solution.

[0013] Figure 6 is a block diagram depicting example packet formats given various transport protocols in accordance with one or more embodiments of the solution.

[0014] Figure 7is a block diagram depicting an example system connecting devices across an IP network using an intra-device protocol in accordance with one or more embodiments of the solution.

[0015] Figure 8 is a block diagram depicting an example system connecting devices across an IP network using another intra-device protocol in accordance with one or more embodiments of the solution.

[0016] Fig. 9 is a block diagram of an example TLP field according to one or more embodiments of the solution.

[0017] The details of various embodiments of the method and system are set forth in the accompanying drawings and the description below. DETAILED DESCRIPTION

[0018] For the purpose of reading the description of the various embodiments below, the following description of the sections of the specification and their corresponding contents may be helpful:

[0019] - Section A describes a network environment and a computing environment that may be useful for practicing the embodiments described herein; and

[0020] - Section B describes embodiments of systems and methods for inter-device networking using an intra-device protocol according to one or more embodiments of the solution.

[0021] A. Computing and Network Environment

[0022] Before discussing specific embodiments of the present solution, it may be helpful to describe aspects of the operating environment and associated system components (eg, hardware elements) in conjunction with the methods and systems described herein. Figure 1A , depicting an embodiment of a network environment. In a brief overview, the network environment includes a wireless communication system that includes one or more access points (APs) 106, one or more wireless communication devices 102, and network hardware components 192. The wireless communication device 102 may, for example, include a laptop computer 102, a tablet computer 102, a personal computer 102, an Internet of Things (IoT) device 102, and / or a cellular telephone device 102. For details of embodiments of each wireless communication device 102 and / or AP 106, refer to Figure 1B and 1CDescribed in more detail. In one embodiment, the network environment may be a self-organizing network environment, an infrastructure wireless network environment, a subnet environment, etc. AP106 may be operably coupled to the network hardware component 192 via a local area network connection. The network hardware component 192, which may include routers, gateways, switches, bridges, modems, system controllers, devices, etc., may provide a local area network connection for the communication system. Each of the APs 106 may have an associated antenna or antenna array to communicate with wireless communication devices in its area. The wireless communication device 102 may register with a specific AP 106 to receive services from the communication system (e.g., via SU-MIMO or MU-MIMO configuration). For direct connections (e.g., point-to-point communications), some wireless communication devices may communicate directly via allocated channels and communication protocols. Some wireless communication devices 102 may be mobile or relatively stationary relative to the AP 106.

[0023] In some embodiments, AP 106 includes a device or module (including a combination of hardware and software) that allows wireless communication devices 102 to connect to a wired network using wireless fidelity (WiFi) or other standards. AP 106 may sometimes be referred to as a wireless access point (WAP). AP 106 may be implemented (e.g., configured, designed and / or constructed) for operation in a wireless local area network (WLAN). In some embodiments, AP 106 may be connected to a router (e.g., via a wired network) as a standalone device. In other embodiments, AP 106 may be a component of a router. AP 106 may provide multiple device access to a network. AP 106 may, for example, be connected to a wired Ethernet connection and use a radio frequency link to provide a wireless connection for other devices 102 to utilize the wired connection. AP 106 may be implemented to support standards for sending and receiving data using one or more radio frequencies. For example, AP 106 may support bandwidths between 5 Gbps and 500 Gbps. Those standards and the frequencies they use may be defined by the IEEE (eg, IEEE 802.11 standards such as 802.11aa 60 GHz). AP 106 may be configured and / or used to support public Internet hotspots and / or to extend the Wi-Fi signal range of the network over a network.

[0024] In some embodiments, the AP 106 may be used for a wireless network (e.g., IEEE 802.11aa, any other type of radio frequency based network protocol, and / or variations thereof) (e.g., in a home or in a building). Each of the wireless communication devices 102 may include a built-in radio and / or be coupled to a radio. Such wireless communication devices 102 and / or APs 106 may operate in accordance with various aspects of the present disclosure presented herein to enhance performance, reduce cost and / or size, and / or enhance broadband applications. Each wireless communication device 102 may have the ability to act as a client node attempting to access resources (e.g., data, and connections to networked nodes such as servers) via one or more APs 106.

[0025] The network connection may include any type and / or form of network and may include any of the following: a point-to-point network, a broadcast network, a telecommunications network, a data communications network, a computer network. The topology of the network may be a bus, a star, or a ring topology. The network may have any such network topology known to those of ordinary skill in the art that is capable of supporting the operations described herein. In some embodiments, different types of data may be transmitted via different protocols. In other embodiments, the same type of data may be transmitted via different protocols.

[0026] The communication device 102 and the access point 106 may be deployed as and / or executed on any type and form of computing device, such as a computer, network device, or appliance capable of communicating over any type and form of network and performing the operations described herein. Figure 1B and 1C A block diagram of a computing device 100 is depicted that may be used to practice embodiments of a wireless communication device 102 or AP 106. Figure 1B and 1C As shown in FIG. 1 , each computing device 100 includes a central processing unit 121 and a main memory unit 122. Figure 1B As shown in FIG. 1 , computing device 100 may include storage device 128, installation device 116, network interface 118, I / O controller 123, display devices 124a to 124n, keyboard 126, and pointing device 127 such as a mouse. Storage device 128 may include an operating system and / or software. Figure 1C As shown in , each computing device 100 may also include additional optional elements, such as a memory port 103, a bridge 170, one or more input / output devices 130a to 130n, and a cache memory 140 that communicates with the central processing unit 121.

[0027] The central processing unit 121 is any logic circuitry that responds to instructions fetched from the main memory unit 122 and processes the instructions. In many embodiments, the central processing unit 121 is provided by a microprocessor unit such as one manufactured by Intel Corporation of Santa Clara, California; one manufactured by International Business Machines of White Plains, New York; or one manufactured by Advanced Micro Devices of Sunnyvale, California. The computing device 100 may be based on any of these processors or any other processor capable of operating as described herein, such as an integrated digital signal processor (DSP). For example, in many implementations, the processor may be an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or any other type and form of dedicated silicon logic or processing circuitry.

[0028] The main memory unit 122 may be one or more memory chips capable of storing data and allowing any storage location to be directly accessed by the microprocessor or central processing unit 121, such as any type or variation of static random access memory (SRAM), dynamic random access memory (DRAM), ferroelectric RAM (FRAM), NAND flash memory, NOR flash memory, and solid-state drive (SSD). The main memory unit 122 may be based on any of the memory chips described above or any other available memory chip capable of operating as described herein. Figure 1B In the embodiment shown in FIG. 1 , a processor or central processing unit 121 communicates with a main memory unit 122 via a system bus 150 (described in more detail below). Figure 1C An embodiment of the computing device 100 is depicted in which the processor communicates directly with the main memory unit 122 via the memory port 103. For example, in Figure 1C In the embodiment, the main memory unit 122 may be a DRDRAM.

[0029] Figure 1CAn embodiment is depicted in which the main processor or central processing unit 121 communicates directly with the cache memory 140 via a secondary bus, sometimes referred to as a backside bus. In other embodiments, the main processor or central processing unit 121 communicates with the cache memory 140 using a system bus 150. The cache memory 140 typically has a faster response time than the main memory unit 122 and is provided by, for example, SRAM, BSRAM, or EDRAM. Figure 1C , the processor or central processing unit 121 communicates with various I / O devices 130 via a local system bus 150. Various buses may be used to connect the central processing unit 121 to any of the I / O devices 130, such as a VESA VL bus, an ISA bus, an EISA bus, a Micro Channel Architecture (MCA) bus, a PCI bus, a PCI-X bus, a PCI-Express bus, or a NuBus. For embodiments in which the I / O device is a video display 124, the processor or central processing unit 121 may communicate with the display 124 using an Advanced Graphics Port (AGP). Figure 1C An embodiment of the computer device 100 is depicted in which the main processor or central processing unit 121 can communicate directly with the I / O device 130b, such as via HYPERTRANSPORT, RAPIDIO, or INFINIBAND communication technology. Figure 1C Also depicted are embodiments in which a local bus is mixed with direct communications: the processor or central processing unit 121 communicates with I / O device 130a using a local interconnect bus while communicating directly with I / O device 130b.

[0030] A variety of I / O devices 130a to 130n may be present in the computing device 100. Input devices include keyboards, mice, trackpads, trackballs, microphones, dials, touchpads, touchscreens, and drawing tablets. Output devices include video displays, speakers, inkjet printers, laser printers, projectors, and dye sublimation printers. I / O devices may be provided by Figure 1B 100. The I / O controller may control one or more I / O devices, such as a keyboard 126 and a pointing device 127 (such as a mouse or pen). In addition, the I / O devices may also provide storage and / or installation media or installation devices 116 for the computing device 100. In yet other embodiments, the computing device 100 may provide a USB connection (not shown) to receive a handheld USB storage device, such as the USB flash drive family of devices manufactured by Twintech Industry, Inc. of Los Alamitos, California.

[0031] Reference again Figure 1B , the computing device 100 may support any suitable installation device 116, such as a disk drive, a CD-ROM drive, a CD-R / RW drive, a DVD-ROM drive, a flash drive, a tape drive of various formats, a USB device, a hard drive, a network interface, or any other device suitable for installing software and programs. The computing device 100 may further include a storage device, such as one or more hard drives or redundant arrays of independent disks, for storing an operating system and other related software and for storing application software programs (e.g., any program or software 120 used to implement (e.g., configured and / or designed for) the systems and methods described herein). Optionally, any of the installation devices 116 may also be used as a storage device. In addition, the operating system and software may be run from a bootable media.

[0032] In addition, computing device 100 may include a network interface 118 to interface to network 104 through various connections, including but not limited to a standard telephone line, a LAN or WAN link (e.g., 802.11, 56kb, X.25, SNA, DECNET), a broadband connection (e.g., Gigabit Ethernet, Ethernet-over-SONET), a wireless connection, or some combination of any or all of the above. The connection may be established using various communication protocols (e.g., TCP / IP, Ethernet, SONET, SDH, RS232, IEEE 802.11, IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, IEEE802.11ac, IEEE 802.11ad, IEEE 802.11ax, CDMA, GSM, WiMax, and direct asynchronous connection). In one embodiment, the computing device 100 communicates with other computing devices 100 via any type and / or form of gateway or tunneling protocol, such as Secure Sockets Layer (SSL) or Transport Layer Security (TLS). The network interface 118 may include a built-in network adapter, a network interface card, a PCMCIA network card, a card bus network adapter, a wireless network adapter, a USB network adapter, a modem, or any other device suitable for interfacing the computing device 100 to any type of network capable of communicating and performing the operations described herein.

[0033] In some embodiments, the computing device 100 may include or be connected to one or more display devices 124a to 124n. Therefore, any one of the I / O devices 130a to 130n and / or the I / O controller 123 may include any type and / or form of appropriate hardware, software, or a combination of hardware and software to support, enable, or provide the computing device 100 with connection and use of the display devices 124a to 124n. For example, the computing device 100 may include any type and / or form of video adapter, video card, driver, and / or library to interface, communicate, connect, or otherwise use the display devices 124a to 124n. In one embodiment, the video adapter may include multiple connectors to interface to the display devices 124a to 124n. In other embodiments, the computing device 100 may include multiple video adapters, each of which is connected to the display devices 124a to 124n. In some embodiments, any portion of the operating system of the computing device 100 may be configured to use multiple displays 124a to 124n. In another embodiment, the I / O device 130 may be a bridge between the system bus 150 and an external communication bus (such as a USB bus, an Apple Desktop bus, an RS-232 serial connection, a SCSI bus, a FireWire bus, a FireWire 800 bus, an Ethernet bus, an AppleTalk bus, a Gigabit Ethernet bus, an Asynchronous Transfer Mode bus, a FibreChannel bus, a Serial Attached Small Computer System Interface bus, a USB connection, or an HDMI bus).

[0034] Figure 1B and 1CA computing device 100 of the type depicted in the may operate under the control of an operating system that controls the scheduling of tasks and access to system resources. The computing device 100 may run any operating system, such as any version of the MICROSOFT WINDOWS operating system, different versions of the Unix and Linux operating systems, any version of the MAC OS for Macintosh computers, any embedded operating system, any real-time operating system, any open source operating system, any proprietary operating system, any operating system for mobile computing devices, or any other operating system capable of running on a computing device and performing the operations described herein. Typical operating systems include, but are not limited to, Android produced by Google Inc.; WINDOWS 7 and 8 produced by Microsoft Corporation of Redmond, Washington; MAC OS produced by Apple Computer of Cupertino, California; WebOS produced by Research In Motion (RIM); OS / 2 produced by International Business Machines of Armonk, New York; and Linux, a free-to-use operating system released by Caldera of Salt Lake City, Utah, or any type and / or form of Unix operating system, etc.

[0035] The computer device 100 may be any workstation, phone, desktop computer, laptop or notebook computer, server, handheld computer, mobile phone or other portable telecommunication device, media playback device, gaming system, mobile computing device, or any other type and / or form of computing, telecommunication or media device capable of communication. In some embodiments, the computing device 100 may have a different processor, operating system, and input device consistent with the device. For example, in one embodiment, the computing device 100 is a smart phone, mobile device, tablet, or personal digital assistant. In addition, the computing device 100 may be any workstation, desktop computer, laptop or notebook computer, server, handheld computer, mobile phone, any other computer, or other form of computing or telecommunication device capable of communication and having sufficient processor power and memory capacity to perform the operations described herein.

[0036] Aspects of the operating environment and components described above will become apparent in the context of the systems and methods disclosed herein.

[0037] B. System and method for inter-device networking using intra-device protocols

[0038] Protocols commonly used between device components, such as the PCIe standard (e.g., PCIe 5.0), support high-speed transfer rates. Although not strictly limited to communications within a device and frequently used for communications between devices in a local data center, this high-speed and generally short-range protocol is generally referred to herein as an intra-device protocol. For example, PCIe 5.0 allows transfer rates of up to 32GT / s for a single channel used to connect a device component to one or more peripheral device components. Channels can be combined (e.g., up to 16 channels) to reach 512GT / s. However, if the number of devices becomes too large and / or if the application has a large amount of shared memory resources, PCIe transfers are limited. For example, in a data center, multiple devices share a large pool of memory that extends PCIe to the limits of its capabilities.

[0039] Another intra-device protocol, the CXL standard (e.g., CXL 2.0), facilitates communication between multiple peer processors (and other devices and processors) and shared memory using the PCIe physical layer. Data can be copied from the host processor memory to the memory of another device. However, when a device updates a memory location (e.g., writes to a memory location), all copies of the memory location and / or memory in cache are marked as invalid, and the processor must re-fetch the data from the host memory. CXL is able to use transaction protocols, new handshaking, and auto-negotiation to achieve low-latency communication to replace and improve the PCIe protocol.

[0040] 2, a block diagram of an example system 200 for connecting devices using a PCIe switch for connecting devices is depicted. As shown, CPU 202 is connected to I / O devices 210a and 210b (collectively referred to as I / O devices 210) using PCIe switch 206. PCIe switch 206 contains an upstream port 204 connected to CPU 202 and downstream ports 208a and 208b connected to I / O devices 210a and 210b, respectively. PCIe switch 206 connects CPU 202 to devices 210 within a single network.

[0041] While intra-device protocol improvements such as CXL and PCIe facilitate communication within a single network, CXL and PCIe are limited to a single network and lack features for inter-device protocols that traverse wide area networks such as the Internet. Previous attempts at CXL and / or PCIe network deployment were limited to Ethernet Layer 2 (e.g., within a single chassis or to a single switch). Communication across chassis requires Layer 3 forwarding.

[0042] To address these and other issues, intra-device protocols CXL and PCIe may be employed rather than inter-device protocols such as IP to allow CXL or PCIe devices to cross Ethernet layer 2 boundaries and connect devices together. For example, layer 3 forwarding allows CXL and / or PCIe communications across networks while leveraging existing network infrastructure to implement larger distributed computing systems. For example, PCIe and CXL networks may extend across data center networks and connect devices such as accelerators together across existing IP networks (e.g., IPv4 or IPv6).

[0043] refer to Figure 3A , depicts a block diagram of an example system 300a for connecting devices across an IP network. As shown, an I / O device 302 is connected to a switch 308 in the same (or different) network using Ethernet and / or IP. Port 304 (upstream port or downstream port) interfaces with I / O device 302 using PCIe and / or CXL. If traffic is traveling from I / O device 302 to switch 308, a shim header may be placed (e.g., at Ethernet IP transport shim layer 306) between the Ethernet and IP headers before the traffic from I / O device 302 is transmitted to switch 308.

[0044] refer to Figure 3B , depicts another block diagram of an example system 300b for connecting devices across an IP network. As shown, a CPU 320 interfaces to an upstream port 322 using PCIe and / or CXL. An Ethernet IP transport shim layer 326 (or a multi-protocol label switching (MLPS) header) may apply a shim header to improve data packet delivery by separating service levels, thereby improving traffic flow through a virtual private network (VPN) and / or traffic engineering (TE). The MLPS protocol forwards packets at the switching layer (layer 2) rather than operating at the routing layer (layer 3). The TSHIM header is applied before traffic is transmitted to (or received from) the same network or a different network via Ethernet IP 318. Ethernet IP 318 may be any IP network (e.g., IPv4 or IPv6). Traffic may be directed to I / O devices 332a and / or 332b (collectively, I / O devices 332) using PCIe and / or CXL via downstream ports 330a and / or 330b (collectively, downstream ports 330) and received by I / O devices 332. If I / O devices 332 transmit traffic directed to CPU 320, shim headers may be placed between Ethernet and IP headers (e.g., at Ethernet transport shim layers 328a and 328b, respectively) before traffic from I / O devices 302 is transmitted to switch CPU 320.

[0045] refer to Figure 4, depicts a flow chart of an embodiment of a method 400 for transmitting an intra-device protocol such as PCIe and / or CXL via an inter-device protocol such as IP. The functionality of the method may be used in conjunction with Figures 1A to 1C The components described in detail are implemented or performed by the components described. In a brief overview, in step 402, the business to be transmitted can be received. In step 404, the computing device can map the packets to be transmitted (e.g., transaction layer packets (TLPs)) to the destination IP address. In step 406, the TLPs can be encapsulated into IP packets, and in step 408, the IP packets can be transmitted to the destination IP address via the Ethernet media access control (MAC).

[0046] In step 402, traffic to be transmitted may be received. For example, the CXL and / or PCIe core may receive traffic, such as a request to read memory and / or write to memory (or a response to a request). The traffic may be a transaction layer packet (TLP) including a header and an optional data payload. The TLP packet header may include a transaction type (e.g., request or response to a request), priority, source and / or destination address, routing rules, and other packet characteristics. The destination address may be any IP address (e.g., IPv4 or IPv6).

[0047] Brief advance reference Fig. 9 , a block diagram depicting example TLP fields. PCIe and CXL TLPs may be forwarded based on the requester ID field 902 (or memory address field). There may be a type field (e.g., requester ID field or memory address field) indicating the type of field present in the packet, because switching PCIe / CXL may be based on ID routing (in this case, there is a requester ID field) or memory address routing (in this case, there is a memory address field). The requester ID field 902 may be 16 bits long (e.g., bits 15:0). The bus number 904 may be 8 bits long (e.g., bits 7:0) and identify the number assigned to the PCIe logical bus. The device number (Dev.Num. 906) may be 5 bits long (e.g., bits 7:3) and identify each device on the PCIe logical bus. The function number (Fn 908) may be three bits long (e.g., bits 2:0) and identify the function on the device (where each PCIe device may have up to 8 logically independent functions).

[0048] Return to reference Figure 4 In step 404, the traffic may be mapped to a destination IP address. A lookup table may be used to map PCIe addresses (or CXL addresses) and IP addresses. For example, the lookup table may identify (and map) a memory address and / or a requesting program ID to an IP source and destination. The lookup table may also include UDP and / or TCP source and destination port numbers.

[0049] The lookup table may be statically configured. For example, a text file may be used to configure the lookup table. The lookup table may be copied to each device after the device is connected to the network. For example, the table may be distributed to nodes in the network using, for example, the Secure Copy Protocol (SCP), the Network Configuration Protocol (NetConf), the Yet Another Next Generation (YANG) protocol, or the Representational State Transfer Application Programming Interface (Rest API). The CXL / PCIe to IP functionality may be implemented in a standalone device, or may be integrated into an Ethernet / IP switch device.

[0050] In step 406, the TLP may be encapsulated into an IP packet. Encapsulating CXL and / or PCIe packets into IP packets allows resources across networks to be combined into a system. Encapsulating the TLP may include wrapping layer 2 information into the payload and adding a layer 3 header.

[0051] In step 408, the encapsulated IP packet may be transmitted. For example, an Ethernet MAC implementing a data link layer may be implemented to transmit the IP packet to a destination IP address. The packet may be transmitted wirelessly or connectedly across one or more networks using various routing protocols to reach the destination IP address.

[0052] A reliable transport protocol, such as the Transmission Control Protocol (TCP), may be used to transmit IP packets to a destination IP address. The transport protocol may be selected based on various networking requirements. For example, while TCP may be a commonly used reliable transport protocol, TCP may not be optimal for latency-sensitive processing due to the timing associated with retransmission of lost TCP packets. However, the latency associated with TCP may be improved, for example, using a hardware implementation of TCP.

[0053] In addition, reliable transport protocols such as User Datagram Protocol (UDP), Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE), Scalable Reliable Datagram (SRD), and / or New Data Center Protocol (NDP) may be used to transmit IP packets to the destination IP address. The transport protocol may be selected based on networking requirements. For example, the RoCE protocol may be used for high throughput, low latency communications. However, RoCE may not be preferred in large networks because priority flow control may cause head-of-line blocking, congestion diffusion, and deadlock. In addition, SRD can load balance and recover quickly from link failures and packet losses. However, SRD may deliver packets out of order, leaving packet recovery to higher layers. Similarly, NDP may provide low latency for short data transfers at the expense of packet reordering.

[0054] UDP-based protocols can be made reliable by layering mechanisms on top of them to support ordering and retransmission of lost packets. For example, the Quick UDP Internet Connections (QUIC) protocol, a low-latency transport protocol based on UDP, can be used to transport PCIe and CXL traffic.

[0055] Alternatively, various retransmission controls may be used to guarantee (or improve the likelihood of) transmitted IP packets. In a simple example, an alternating bit protocol (ABP) may be used to alternate numbered packets '0' and '1' so that a receiver may acknowledge a numbered packet by transmitting an acknowledgment back to the sender along with the same numbered packet. If the wrong numbered packet is acknowledged, or an acknowledgment is not received within a predetermined time window, the packet may be retransmitted.

[0056] refer to Figure 5 , depicts a flow chart of an embodiment of a method 500 for receiving an intra-device protocol such as PCIe and / or CXL via an inter-device protocol such as IP. The functionality of the method may be used in conjunction with Figures 1A to 1C The components described in detail are implemented or performed by the components described. In a brief overview, in step 502, traffic is received from a source IP address. In step 504, the computing device can map the source IP address to a TLP. In step 506, the IP packet can be decapsulated into the TLP, and in step 508, the TLP can be routed.

[0057] In step 502, traffic may be received. For example, IP packets may be routed from a source IP address to a computing device across various networks (or the same network). Traffic may include requests to read memory and / or write memory (e.g., responses to requests).

[0058] TLPs that are not sequence-sensitive may be sent out of order, while TLPs that are sequence-sensitive may be buffered and sent in order. In some implementations, packet reordering is performed. For example, out-of-order packets may be placed in a buffer, lost, or forwarded for layer 3 processing depending on the transport protocol used to transmit the packets. For example, TCP corrects out-of-order packets by requesting packet retransmission.

[0059] In step 504, traffic may be mapped to TLPs. Traffic may be IP packets (e.g., standard IPv4 or IPv6 packets). For example, a lookup table may be used to map IP addresses to PCIe (or CXL) addresses. For example, a lookup table may identify (and map) memory addresses and / or requester IDs to IP sources and destinations. The lookup table may also include UDP and / or TCP source and destination port numbers.

[0060] The lookup table may be statically configured. For example, a text file may be used to configure the lookup table. The lookup table may be copied to each device after the device is connected to the network. For example, the table may be distributed to nodes in the network using, for example, the Secure Copy Protocol (SCP), the Network Configuration Protocol (NetConf), the Yet Another Next Generation (YANG) protocol, or the Representational State Transfer Application Programming Interface (Rest API). The CXL / PCIe to IP functionality may be implemented in a standalone device, or may be integrated into an Ethernet / IP switch device.

[0061] In step 506, the IP packets may be decapsulated into TLP packets. Decapsulation may be the process of unpacking (or opening) data as it moves up the protocol stack. For example, depacketizing layer 3 networking information may reveal layer 2 forwarding information (e.g., TLP packets).

[0062] In step 508, TLP packets may be routed. TLP packets (including PCIe and CXL TLPs) are forwarded using, for example, CXL and / or PCIe cores. Routing of TLP packets to PCIe addresses and / or CXL addresses enables memory resources to be shared and / or updated.

[0063] refer to Figure 6 , depicting example packet formats 600a-600d for a given example transport protocol. Similar to standard packet formats, packet formats 600a-600d may include layer 2 information 602. For example, an Ethernet frame may include a sender / receiver MAC address. Packet formats 600a-600d may also include layer 3 information 604 (e.g., sender / receiver IP addresses).

[0064] Packet formats 600a-600d may also include layer 4 information 606. Layer 4 information 606 may differentiate packet formats depending on a transport protocol. For example, as shown, packet format 600d uses TCP (as identified by TCP header 606) as a transport protocol, while packet formats 600a-600c use a UDP-based protocol.

[0065] As discussed herein, an unreliable protocol such as UDP may be used, but reliability mechanisms may be added to support PCIe and / or CXL. Thus, a particular transport protocol 608 may be identified by the packet format. For example, if the RoCE protocol as identified by packet format 600a is used, then the basic transport header (BTH) 608a may be identified by the packet format. If the SRD protocol as identified by packet format 600b is used, then the SRD protocol 608b may be identified by the packet format. If the NDP protocol as identified by packet format 600c is used, then the NDP protocol 608c may be identified by the packet format.

[0066] The packet formats 600a-600d may also include a shim header 610, which contains sequence number and PCIe and / or CXL address information to properly sequence the frame to egress. The message may be included in a CXL and / or PCIe TLP packet 612.

[0067] refer to Figure 7 , depicts a block diagram of an example system 700 that uses CXL to connect devices across an IP network. As shown, a CPU 720 interfaces to an upstream port 722 using CXL and PCIe. A transmitted (or received) message 721 contains CXL data. An Ethernet IP transport shim layer 726 may apply a shim header before a service is transmitted within the same network or to a different network via Ethernet IP 718. When a service is transmitted over Ethernet IP 718, the message may be encapsulated in a packet format 729. Ethernet IP 718 may be any version of an IP network (e.g., IPv4 or IPv6). The service may be directed to I / O devices 732a and / or 732b (collectively, I / O devices 732). When the service is received by I / O device 732, the service has a packet format 729. The service is received by I / O device 732 via downstream ports 730a and / or 730b (collectively, downstream ports 730) using CXL and PCIe. Decapsulation occurs such that traffic 729 is unpacked and CXL message 721 is received by destination I / O device 732 (eg, I / O devices 732a and 732b, respectively).

[0068] refer to Figure 8 , depicting a block diagram of an example system 800 that uses PCIe to connect devices across an IP network. As shown, a CPU 820 uses PCIe to interface with an upstream port 822. A transmitted (or received) message 821 contains PCIe data. An Ethernet IP transport shim layer 826 may apply a shim header before a service is transmitted within the same network or to a different network via Ethernet IP 818. When a service is transmitted over Ethernet IP 818, the message may be encapsulated in a packet format 829. Ethernet IP 818 may be any version of an IP network (e.g., IPv4 or IPv6). The service may be directed to I / O devices 832a and / or 832b (collectively referred to as I / O devices 832). When a service is received by an I / O device 832, the service has a packet format 829. The service is received by the I / O device 832 via downstream ports 830a and / or 830b (collectively referred to as downstream ports 830) using PCIe. Decapsulation occurs such that traffic 829 is unpacked and PCIe message 821 is received by destination I / O device 832 (eg, I / O devices 832a and 832b, respectively).

[0069] Although discussed primarily with respect to PCIe and CXL (as discussed above), any intra-device protocol may be utilized using the systems and methods herein. Similarly, although discussed primarily with respect to network protocols such as IP and TCP, any inter-device protocol or combination of inter-device protocols may be used with these systems and methods.

[0070] It should be noted that certain paragraphs of the present disclosure may reference terms such as "first" and "second" in relation to transmit spatial streams, sounding frames, responses, and subsets of devices for the purpose of identifying or distinguishing one from another or others. These terms are not intended to associate entities (e.g., first device and second device) only in time or according to order, although in some cases, these entities may include such a relationship. These terms also do not limit the number of possible entities that can operate within a system or environment. It should be understood that the system described above may provide several of any or each of those components, and these components may be provided on a stand-alone machine or, in some embodiments, on multiple machines in a distributed system. In addition, the systems and methods described above may be provided as one or more computer-readable programs or executable instructions embodied on or in one or more articles of manufacture (e.g., floppy disks, hard disks, CD-ROMs, flash memory cards, PROMs, RAMs, ROMs, or tapes). The programs may be implemented in any programming language (e.g., LISP, PERL, C, C++, C#) or in any byte code language (e.g., JAVA). A software program or executable instructions may be stored as object code on or in one or more articles of manufacture.

[0071] Although the foregoing written description of the methods and systems enables one of ordinary skill to make and use embodiments thereof, one of ordinary skill will appreciate and understand that variations, combinations, and equivalents of the specific embodiments, methods, and examples herein exist. The methods and systems should therefore not be limited to the embodiments, methods, and examples described above, but rather to all embodiments and methods within the scope and spirit of the invention.

Claims

1. A transmission method via an Internet Protocol (IP) network, comprising: Receiving, by the processor, traffic to be transmitted; mapping, by the processor, a high-speed link CXL address to the destination IP address using a calculation link between the CXL address and the destination IP address, the link between the CXL address and the destination IP address being in a CXL address to IP address lookup table, the CXL address including a requester ID, and the CXL address to IP address lookup table including one or more links between the requester ID of the CXL address and the IP address; encapsulating, by the processor, the traffic to be transmitted into an IP packet addressed to the destination IP address; and The traffic is transmitted by the processor using the IP packets addressed to the destination IP address.

2. The method according to claim 1, wherein transmitting the service using the IP packets addressed to the destination IP address includes using at least one transmission protocol from the group consisting of: Scalable Reliable Datagram (SRD), New Data Center Protocol (NDP), Remote Direct Memory Access (RDMA) based on Converged Ethernet (RoCE), Transmission Control Protocol (TCP), and User Datagram Protocol (UDP). The method of claim 1 , wherein the CXL address to IP address lookup table is statically configured.

4. The method of claim 1, further comprising executing a retransmission control protocol to retransmit the IP packet addressed to the destination IP address.

5. The method of claim 1, wherein the CXL address to IP address lookup table comprises the one or more links between a requester ID of a CXL address and an IP address and one or more second links between a memory address of a CXL address and an IP address.

6. The method according to claim 5, comprising: The CXL address is compared with the one or more links between the requester ID and the IP address of the CXL address instead of comparing the CXL address with the one or more second links between the memory address and the IP address of the CXL address based on the requester ID field type in the transaction.

7. A method for receiving data via an Internet Protocol (IP) network, comprising: Receiving the transmitted business by the processor; mapping, by the processor, a source IP address to a computational express link (CXL) address using a link between the source IP address and the CXL address, the link between the source IP address and the CXL address being in a source IP address to CXL address lookup table, the CXL address including a requester ID, and the CXL address to IP address lookup table including one or more links between the requester ID of the CXL address and an IP address; decapsulating, by the processor, at least a portion of the received traffic into one or more transport layer packets; and The one or more transport layer packets are routed, by the processor, using the mapped CXL address.

8. The method of claim 7, wherein the source IP address to CXL address lookup table is statically configured.

9. The method of claim 7, further comprising reordering, by the processor, one or more packets of the received traffic.

10. A system for transmitting data via an Internet Protocol (IP) network, comprising: a computing device including a processing circuit system and a network interface; wherein the network interface is configured to receive traffic to be transmitted; and wherein the processing circuitry is configured to: mapping the CXL address to the destination IP address using computing a link between the CXL address and the destination IP address, the link between the CXL address and the destination IP address being in a CXL address to IP address lookup table, the CXL address including a requester ID, and the CXL address to IP address lookup table including one or more links between the requester ID of the CXL address and the IP address; encapsulating the service to be transmitted into an IP packet addressed to the destination IP address; and The traffic to be transmitted is transmitted using the IP packets addressed to the destination IP address.

11. The system of claim 10, wherein transmitting the service using the IP packets addressed to the destination IP address comprises using at least one transmission protocol from the group consisting of: Scalable Reliable Datagram (SRD), New Data Center Protocol (NDP), Remote Direct Memory Access (RDMA) based on Converged Ethernet (RoCE), Transmission Control Protocol (TCP), and User Datagram Protocol (UDP).

12. The system of claim 10, wherein the CXL to IP address lookup table is statically configured.

13. The system of claim 10, wherein the processing circuitry is further configured to execute a retransmission control protocol to retransmit the IP packets addressed to the destination IP address.

Images