Reliable coverage based on reliable transport layer
By using a processor to execute instruction set in the communication system to generate and process packets, combining the MPLS protocol and the connection-oriented transport layer protocol, the reliability and coverage problems of packet communication in the communication system are solved, and the stable transmission of packet communication is achieved.
Patent Information
- Application Number
- CN202111188893.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-10-13
- Filing Date
- 2021-10-12
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-10-12
AI Technical Summary
When existing communication systems support packet communication, it is difficult to ensure the effectiveness of reliable coverage and transport layer protocols, especially in the control header processing between the tag switching protocol and the network layer protocol, there is a problem of insufficient discarding and congestion control.
Using an apparatus and method, the instruction set is executed by a processor, and packets are generated and processed, including payloads, headers of tag switching protocols and headers of reliable transport layer protocols, utilizing multi-protocol tag switching (MPLS) protocols and connection-oriented transmissions The layer protocol supports flow control and congestion control to ensure reliable transmission of packets between communication devices.
The reliability and coverage of packet communication in the communication system are realized, packet discarded, packet discarded, the effectiveness of the transmission layer protocol is ensured, and the stability of the network layer protocol and data transmission reliability between communication devices are improved.
Smart Images

Figure CN114422432B_ABST
Abstract
Description
Technical Field
[0001] Various example embodiments relate generally to communication systems and, more particularly, but not exclusively, to supporting reliability of coverage in communication systems. Background Art
[0002] In many communication networks, various communication technologies may be used for various communications. Summary of the invention
[0003] In at least some example embodiments, an apparatus includes at least one processor and at least one memory including an instruction set, wherein the instruction set is configured to, when executed by at least one processor, cause the apparatus to support communication of packets through a communication device, wherein the packets include a payload, a header of a label switching protocol on the payload, and a header of a reliable transport layer protocol on the header of the label switching protocol. In at least some example embodiments, the header of the label switching protocol includes a label set. In at least some example embodiments, the label set is organized as a label stack. In at least some example embodiments, the label switching protocol is associated with a label switching overlay. In at least some example embodiments, the label switching protocol includes a multi-protocol label switching (MPLS) protocol. In at least some example embodiments, the reliable transport layer protocol includes a connection-oriented transport layer protocol. In at least some example embodiments, the reliable transport layer protocol includes a transport layer protocol configured to support at least one of the following: flow control or congestion control. In at least some example embodiments, the reliable transport layer protocol includes a transmission control protocol (TCP), a stream control transmission protocol (SCTP), a fast user datagram protocol (UDP) Internet connection (QUIC) protocol, or a transport layer security protocol (TLS) protocol. In at least some example embodiments, the packet includes a control header between a header of a label switching protocol and a header of a reliable transport layer protocol. In at least some example embodiments, the control header is configured to indicate the size of the payload and the header of the label switching protocol. In at least some example embodiments, the packet includes a header of a network layer protocol on a header of a reliable transport layer protocol. In at least some example embodiments, the network layer protocol includes an Internet Protocol (IP). In at least some example embodiments, the packet includes a header of a data link layer protocol on a header of the network layer protocol. In at least some example embodiments, the data link layer protocol includes at least one of the following: Ethernet or Point-to-Point Protocol (PPP). In at least some example embodiments, in order to support communication of packets, the instruction set is configured to, when executed by at least one processor, cause the apparatus to generate packets through a communication device and send packets to a next hop node through the communication device. In at least some example embodiments, in order to support communication of packets, the instruction set is configured to, when executed by at least one processor, cause the apparatus to receive packets through a communication device and process the packets. In at least some example embodiments, the instruction set is configured to, when executed by at least one processor, cause the apparatus to support communication of an overlay initiation frame through the communication device, the overlay initiation frame being configured to transmit one or more overlay parameters, the one or more overlay parameters being used for overlay supported between the communication device and a remote communication device based on a reliable transport layer protocol. In at least some example embodiments, the overlay initiation frame is sent by the communication device to the remote communication device or received by the communication device from the remote communication device.
[0004] In at least some example embodiments, a non-transitory computer-readable medium stores an instruction set, wherein the instruction set is configured to, when executed by at least one processor, cause the apparatus to support communication of packets through a communication device, wherein the packet includes a payload, a header of a label switching protocol on the payload, and a header of a reliable transport layer protocol on the header of the label switching protocol. In at least some example embodiments, the header of the label switching protocol includes a label set. In at least some example embodiments, the label set is organized as a label stack. In at least some example embodiments, the label switching protocol is associated with a label switching overlay. In at least some example embodiments, the label switching protocol includes a multi-protocol label switching (MPLS) protocol. In at least some example embodiments, the reliable transport layer protocol includes a connection-oriented transport layer protocol. In at least some example embodiments, the reliable transport layer protocol includes a transport layer protocol configured to support at least one of the following: flow control or congestion control. In at least some example embodiments, the reliable transport layer protocol includes a transmission control protocol (TCP), a stream control transmission protocol (SCTP), a fast user datagram protocol (UDP) Internet connection (QUIC) protocol, or a transport layer security protocol (TLS) protocol. In at least some example embodiments, the packet includes a control header between a header of a label switching protocol and a header of a reliable transport layer protocol. In at least some example embodiments, the control header is configured to indicate the size of the payload and the header of the label switching protocol. In at least some example embodiments, the packet includes a header of a network layer protocol on a header of a reliable transport layer protocol. In at least some example embodiments, the network layer protocol includes an Internet Protocol (IP). In at least some example embodiments, the packet includes a header of a data link layer protocol on a header of the network layer protocol. In at least some example embodiments, the data link layer protocol includes at least one of the following: Ethernet or Point-to-Point Protocol (PPP). In at least some example embodiments, in order to support communication of packets, the instruction set is configured to, when executed by at least one processor, cause the apparatus to generate packets through a communication device and send packets to a next hop node through the communication device. In at least some example embodiments, in order to support communication of packets, the instruction set is configured to, when executed by at least one processor, cause the apparatus to receive packets through a communication device and process the packets. In at least some example embodiments, the instruction set is configured to, when executed by at least one processor, cause the apparatus to support communication of an overlay initiation frame through the communication device, the overlay initiation frame being configured to transmit one or more overlay parameters, the one or more overlay parameters being used for overlay supported between the communication device and a remote communication device based on a reliable transport layer protocol. In at least some example embodiments, the overlay initiation frame is sent by the communication device to the remote communication device or received by the communication device from the remote communication device.
[0005] In at least some example embodiments, a method includes supporting communication of packets by a communication device, wherein the packets include a payload, a header of a label switching protocol on the payload, and a header of a reliable transport layer protocol on the header of the label switching protocol. In at least some example embodiments, the header of the label switching protocol includes a label set. In at least some example embodiments, the label set is organized as a label stack. In at least some example embodiments, the label switching protocol is associated with a label switching overlay. In at least some example embodiments, the label switching protocol includes a multi-protocol label switching (MPLS) protocol. In at least some example embodiments, the reliable transport layer protocol includes a connection-oriented transport layer protocol. In at least some example embodiments, the reliable transport layer protocol includes a transport layer protocol configured to support at least one of the following: flow control or congestion control. In at least some example embodiments, the reliable transport layer protocol includes a transmission control protocol (TCP), a stream control transmission protocol (SCTP), a fast user datagram protocol (UDP) Internet connection (QUIC) protocol, or a transport layer security protocol (TLS) protocol. In at least some example embodiments, the packet includes a control header between the header of the label switching protocol and the header of the reliable transport layer protocol. In at least some example embodiments, the control header is configured to indicate the size of the header of the payload and the label switching protocol. In at least some example embodiments, the packet includes a header of a network layer protocol on a header of a reliable transport layer protocol. In at least some example embodiments, the network layer protocol includes an Internet Protocol (IP). In at least some example embodiments, the packet includes a header of a data link layer protocol on a header of the network layer protocol. In at least some example embodiments, the data link layer protocol includes at least one of the following: Ethernet or Point-to-Point Protocol (PPP). In at least some example embodiments, in order to support communication of packets, the instruction set is configured to, when executed by at least one processor, cause the device to generate a packet through a communication device and send the packet to a next hop node through the communication device. In at least some example embodiments, in order to support communication of packets, the instruction set is configured to, when executed by at least one processor, cause the device to receive a packet through a communication device and process the packet. In at least some example embodiments, the instruction set is configured to, when executed by at least one processor, cause the apparatus to support communication of an overlay initiation frame through the communication device, the overlay initiation frame being configured to transmit one or more overlay parameters, the one or more overlay parameters being used for overlay supported between the communication device and a remote communication device based on a reliable transport layer protocol. In at least some example embodiments, the overlay initiation frame is sent by the communication device to the remote communication device or received by the communication device from the remote communication device.
[0006] In at least some example embodiments, an apparatus includes a component for supporting communication of packets through a communication device, wherein the packet includes a payload, a header of a label switching protocol on the payload, and a header of a reliable transport layer protocol on the header of the label switching protocol. In at least some example embodiments, the header of the label switching protocol includes a label set. In at least some example embodiments, the label set is organized as a label stack. In at least some example embodiments, the label switching protocol is associated with a label switching overlay. In at least some example embodiments, the label switching protocol includes a multi-protocol label switching (MPLS) protocol. In at least some example embodiments, the reliable transport layer protocol includes a connection-oriented transport layer protocol. In at least some example embodiments, the reliable transport layer protocol includes a transport layer protocol configured to support at least one of the following: flow control or congestion control. In at least some example embodiments, the reliable transport layer protocol includes a transmission control protocol (TCP), a stream control transmission protocol (SCTP), a fast user datagram protocol (UDP) Internet connection (QUIC) protocol, or a transport layer security protocol (TLS) protocol. In at least some example embodiments, the packet includes a control header between the header of the label switching protocol and the header of the reliable transport layer protocol. In at least some example embodiments, the control header is configured to indicate the size of the header of the payload and the label switching protocol. In at least some example embodiments, the packet includes a header of a network layer protocol on a header of a reliable transport layer protocol. In at least some example embodiments, the network layer protocol includes an Internet Protocol (IP). In at least some example embodiments, the packet includes a header of a data link layer protocol on a header of the network layer protocol. In at least some example embodiments, the data link layer protocol includes at least one of the following: Ethernet or Point-to-Point Protocol (PPP). In at least some example embodiments, in order to support communication of packets, the instruction set is configured to, when executed by at least one processor, cause the device to generate a packet through a communication device and send the packet to a next hop node through the communication device. In at least some example embodiments, in order to support communication of packets, the instruction set is configured to, when executed by at least one processor, cause the device to receive a packet through a communication device and process the packet. In at least some example embodiments, the instruction set is configured to, when executed by at least one processor, cause the apparatus to support communication of an overlay initiation frame through the communication device, the overlay initiation frame being configured to transmit one or more overlay parameters, the one or more overlay parameters being used for overlay supported between the communication device and a remote communication device based on a reliable transport layer protocol. In at least some example embodiments, the overlay initiation frame is sent by the communication device to the remote communication device or received by the communication device from the remote communication device. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] The teachings herein may be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which:
[0008] Figure 1 An example embodiment of a communication system configured to support a reliable MPLS overlay is depicted;
[0009] Figure 2 Described are example embodiments of using a reliable MPLS overlay for network virtualization in a multi-tenant data center environment;
[0010] Figure 3 Described are example embodiments of non-chassis-based network function virtualization (NFV) routers for enabling further understanding of the application of reliable MPLS overlays to chassis-based NFV routers;
[0011] Figure 4 Described are example embodiments of a chassis-based router for enabling further understanding of the application of a reliable MPLS overlay to a chassis-based NFV router;
[0012] Figure 5 Describes an example embodiment of internal structures in a chassis-based router to enable further understanding of the application of reliable MPLS overlays to chassis-based NFV routers;
[0013] Figure 6 Depicted are example embodiments of using a reliable MPLS overlay for an internal fabric in a chassis-based NFV router, where the control plane and forwarding plane are separated through the network;
[0014] Figure 7 Described is an example embodiment of using a reliable MPLS overlay for virtualized Fibre Channel in a storage area network;
[0015] Figure 8 An example embodiment of a communication system configured to support a reliable MPLS overlay is depicted;
[0016] Fig. 9 An example embodiment of an MPLS overlay packet is depicted to illustrate the positioning of reliable transport layer header(s) in an MPLS overlay packet;
[0017] Fig.10 An example embodiment of parsing of a byte stream including an MPLS overlay packet is depicted;
[0018] Fig.11 An example embodiment of an MPLS Control Header (MCH) configured for use on an MPLS overlay packet to support a reliable MPLS overlay is depicted;
[0019] Fig.12 Depicted is an example embodiment of an MPLS Overlay Initial Frame (MOIF) for a new transport connection;
[0020] Fig.13 An example embodiment of a method for use by an initiating router for configuring a reliable transport connection is depicted;
[0021] Fig.14 An example embodiment of a method for use by an initiating router for supporting subsequent actions for a new reliable transport connection is depicted;
[0022] Fig.15 An example embodiment of a method for use by an originating router for constructing a MOIF for a new reliable transport connection is depicted;
[0023] Fig.16 An example embodiment of a method for use by an initiating router when a MOIF response is not received within a predefined time period is depicted;
[0024] Fig.17 An example embodiment of a method for use by an initiating router to handle a MOIF response is depicted;
[0025] Fig.18 An example embodiment of a method for use by a receiving router for configuring a reliable transport connection listener is depicted;
[0026] Fig.19 An example embodiment of a method for use by a receiving router for processing an incoming request for a transport layer connection is depicted;
[0027] Fig. 20 An example embodiment of a method for use by a receiving router for performing post-connect follow-up for a new reliable transport connection is depicted;
[0028] Fig.21 An example embodiment of a method for use by a receiving router for handling a MOIF while waiting for a MOIF response is depicted;
[0029] FIG22 depicts an example embodiment of a method for configuring a reliable MPLS overlay;
[0030] Fig.23 An example embodiment of a method for transporting packets over a reliable MPLS overlay is depicted;
[0031] Fig.24 An example embodiment of a method for transporting MPLS packets over a reliable transport connection is depicted;
[0032] Fig.25 An example embodiment of a method for processing packets received over a reliable MPLS overlay is depicted;
[0033] Fig.26Depicted are example embodiments of an MPLS-in-TCP encapsulation format for implementing a reliable MPLS overlay as MPLS-in-TCP;
[0034] Fig. 27 An example embodiment of a method for configuring a TCP connection for MPLS-in-TCP is depicted;
[0035] Fig.28 An example embodiment of a method for performing post-connect follow-up for a TCP connection for MPLS-in-TCP is depicted;
[0036] Fig.29 Depicted is an example embodiment of a method for use when a MOIF response is not received within a predefined time period for a TCP connection;
[0037] Fig.30 An example embodiment of a method for configuring a TCP connection listener is depicted;
[0038] Fig.31 An example embodiment of a method for processing an incoming TCP connection request is depicted;
[0039] Figures 32A-32B An example embodiment of a method for configuring an MPLS overlay on a TCP connection to form an MPLS-in-TCP overlay is depicted;
[0040] Fig.33 An example embodiment of a method for transporting packets over MPLS-in-TCP is depicted;
[0041] Fig.34 An example embodiment of a method for transmitting MPLS packets over a TCP connection is depicted;
[0042] Fig.35 An example embodiment of a method for receiving and processing MPLS-in-TCP packets is depicted;
[0043] Fig.36 An example embodiment of a method for processing MPLS-in-TCP packets from a TCP segment of a TCP connection is depicted;
[0044] Fig.37 Depicted are example embodiments of an MPLS-in-SCTP encapsulation format for implementing a reliable MPLS overlay as MPLS-in-SCTP;
[0045] Fig.38 An example embodiment of an SCTP control block header for an SCTP packet is depicted;
[0046] Fig.39An example embodiment of an SCTP data chunk header for an SCTP packet is depicted;
[0047] Fig.40 An example embodiment of a method for configuring SCTP associations for MPLS-in-SCTP is depicted;
[0048] Fig.41 An example embodiment of a method for performing post-connect follow-up for an SCTP association of MPLS-in-SCTP is depicted;
[0049] Fig.42 Depicted is an example embodiment of a method for use when a MOIF response is not received within a predefined time period for an SCTP association;
[0050] Fig.43 An example embodiment of a method for configuring an SCTP association listener is depicted;
[0051] Fig.44 An example embodiment of a method for processing an incoming SCTP association request is depicted;
[0052] FIG45 depicts an example embodiment of a method for configuring an MPLS overlay on an SCTP association to form an MPLS-in-SCTP overlay;
[0053] Fig.46 An example embodiment of a method for transporting packets over MPLS-in-SCTP is depicted;
[0054] Fig.47 An example embodiment of a method for transporting MPLS packets over an SCTP association is depicted;
[0055] Fig.48 An example embodiment of a method for receiving and processing MPLS-in-SCTP packets is depicted;
[0056] Fig.49 An example embodiment of a method for processing an MPLS-in-STCP packet from an SCTP packet of an SCTP association is depicted;
[0057] Fig.50 Depicted are example embodiments of an MPLS-in-QUIC encapsulation format for implementing a reliable MPLS overlay as MPLS-in-QUIC;
[0058] Fig.51 An example embodiment of a QUIC long header for a QUIC packet for implementing a reliable MPLS overlay as MPLS-in-QUIC is depicted;
[0059] Fig.52An example embodiment of a QUIC short header for a QUIC packet for implementing a reliable MPLS overlay as MPLS-in-QUIC is depicted;
[0060] Fig.53 An example embodiment of a QUIC frame stack for QUIC packets for implementing a reliable MPLS overlay as MPLS-in-QUIC is depicted;
[0061] Fig.54 An example embodiment of a STREAM (STREAM) frame for QUIC packets for implementing a reliable MPLS overlay as MPLS-in-QUIC is depicted;
[0062] Fig.55 An example embodiment of a general flow frame format for QUIC packets is depicted;
[0063] Fig.56 Depicting an example embodiment of a QUIC packet multiplexing N MPLS overlay packets;
[0064] Fig.57 An example embodiment of a method for configuring a QUIC connection for MPLS-in-QUIC is depicted;
[0065] Fig.58 An example embodiment of a method for performing post-connect follow-up for a QUIC connection for MPLS-in-QUIC is depicted;
[0066] Fig.59 Depicted are example embodiments of a method for use when a MOIF response is not received within a predefined time period for a QUIC connection;
[0067] Fig.60 An example embodiment of a method for configuring a QUIC connection listener is depicted;
[0068] Fig.61 An example embodiment of a method for processing an incoming QUIC connection request is depicted;
[0069] Fig.62 An example embodiment of a method for configuring an MPLS overlay on a QUIC connection to form an MPLS-in-QUIC overlay is depicted;
[0070] Fig.63 An example embodiment of a method for transporting packets over MPLS-in-QUIC is depicted;
[0071] Fig.64 An example embodiment of a method for transporting MPLS packets over a QUIC connection is depicted;
[0072] FIG65 depicts an example embodiment of a method for receiving and processing MPLS-in-QUIC packets;
[0073] Fig.66 An example embodiment of a method for processing an MPLS-in-QUIC packet from a QUIC packet is depicted;
[0074] Fig.67 An example embodiment of a method for supporting reliability of coverage is depicted; and
[0075] Fig.68 An example embodiment of a computer suitable for performing the various functions presented herein is depicted.
[0076] To facilitate understanding, identical reference numerals have been used herein, where possible, to designate identical elements that are common to different figures. DETAILED DESCRIPTION
[0077] Various example embodiments for supporting reliability of overlays are presented herein.
[0078] Various example embodiments for supporting the reliability of overlay can be configured to support the reliable delivery of overlay packets. Various example embodiments for supporting the reliable delivery of overlay packets can be configured to support the reliable delivery of overlay packets of label switching protocols. The label switching protocol can be a multi-protocol label switching (MPLS) protocol or other suitable label switching protocols. The overlay packets of the label switching protocol can be supported using the underlying tunnel of the network layer protocol operating at the network layer. The network layer protocol can be an Internet Protocol (IP), such as IPv4 or IPv6, in which case the overlay packets of the label switching protocol can be supported using the IP underlying tunnel or any other suitable network layer protocol. It can be considered that the overlay packets of the label switching protocol provide label switching coverage, so that the label switching devices that exchange the overlay packets via the label switching path can be considered to be adjacent on the label switching path, even if the label switching devices are separated by the underlying network (e.g., an IP network based on an IP underlying tunnel). It should be understood that in the case where the label switching protocol is MPLS, the reliable overlay can be referred to as a reliable MPLS overlay.
[0079] Various example embodiments for supporting reliability of overlays can be configured to support reliable delivery of overlay packets based on a reliable transport layer. The reliable transport layer can be provided using a reliable transport layer protocol. The reliable transport layer protocol can be a connection-oriented protocol. The reliable transport layer protocol can be configured to support reliability features, such as lossless delivery, flow control, congestion control, etc. and various combinations thereof. The reliable transport layer protocol can include a transmission control protocol (TCP), a stream control transmission protocol (SCTP), a fast user datagram protocol (UDP) Internet connection (QUIC) protocol, a transport layer security (TLS) protocol, or other reliable protocols configured to operate at the transport layer. The reliable transport layer can be arranged between a label switching overlay (e.g., an MPLS overlay or other suitable overlay) and an underlying transport layer (e.g., an IP underlay or other suitable transport layer underlay).
[0080] It should be understood that although this document is primarily presented with respect to example embodiments in which communication protocol layers are based on the Open Systems Interconnection (OSI) model (e.g., a reliable transport layer protocol at the transport layer of the OSI model, a network layer protocol at the network layer of the OSI model, etc.), the various example embodiments presented herein may be used or may be suitable for use with communication protocol layers that may be based on other communication protocol layer models.
[0081] It should be appreciated that advantages or potential advantages of these and various other example embodiments, as well as reliability in supporting coverage, may be further appreciated by reference to the various drawings discussed further below.
[0082] Figure 1An example embodiment of a communication system configured to support a reliable MPLS overlay is depicted. The communication system 100 includes a pair of label switching routers (LSRs) 110-A and 110-B (collectively referred to as LSRs 110) and a communication network 120. LSRs 110 support a reliable MPLS overlay 111 for reliable delivery of MPLS packets between LSRs 110. Reliable MPLS overlay 111 is based on a reliable MPLS overlay 111-A on LSR 111-A and a reliable MPLS overlay 111-B on LSR 111-B. Reliable MPLS overlay 111 may have a reliable connection based on a reliable transport protocol (e.g., TCP, SCTP, QUIC, TLS, etc.) established between the reliable MPLS overlay 111-A on LSR 111-A and the reliable MPLS overlay 110-B on LSR 110-B for supporting reliable delivery of MPLS packets from LSR 110-A to LSR 110-B. Reliable MPLS overlay 111 is configured to support reliable delivery of MPLS packets from LSR 110-A to LSR 110-B based on a set of N label switched paths (denoted as LSP 1-N) running over reliable MPLS overlay 111. Reliable MPLS overlay 111 relies on an underlying tunnel 119 (which may be an Internet Protocol (IP) based tunnel or other suitable type of tunnel) for supporting LSRs from LSR 110-A to LSR 110-B (e.g., LSR 110-A multiplexes LSPs over tunnel 119 because LSR 110-A is not adjacent to LSR 110-B). Reliable MPLS overlay 111 allows LSRs 110 to be adjacent on LSPs in a reliable manner even if LSRs 111 are separated by communication network 120. It should be appreciated that, although primarily presented with respect to a particular transmission direction, a set of LSPs may also be established to support reliable delivery of MPLS packets from LSR 110-B to LSR 110A. It should be appreciated that a reliable MPLS overlay may be used in a variety of contexts, some of which are related to Figures 2 to 7 and is presented.
[0083] Figure 2 Depicted are example embodiments of using a reliable MPLS overlay for network virtualization (NVO) in a multi-tenant data center (DC) environment.
[0084] exist Figure 2In, as indicated above, a reliable MPLS overlay can be used for NVO in a multi-tenant DC environment. Typically, a DC is a pool or collection of cloud infrastructure resources designed specifically for a specific need (e.g., for an enterprise business need or other type of need). The basic resources are servers (e.g., processors (e.g., CPU), memory (e.g., RAM), etc.), storage devices (e.g., disk space), and networks interconnecting servers and storage devices (e.g., bandwidth). A multi-tenant DC can host virtual DCs for multiple customers (referred to as tenants) on a single physical infrastructure. A virtual DC is a virtual representation of a physical data center, including servers, storage clusters, and many network components, all of which reside in a virtual space hosted by a multi-tenant DC. Servers for a virtual DC are virtualized using virtual machines (VMs). One or more VMs can run on top of a physical server, with each VM being allocated a share of the processor resources (e.g., cores) and memory resources (e.g., RAM) of the physical server. VMs in a physical server can belong to the same tenant or to different tenants. A thin layer of software called a "hypervisor" running in the physical server decouples the VM from the physical server and dynamically allocates computing resources to each VM as needed. There are various solutions to virtualize storage devices, which are not described in this article for simplicity. The VMs and virtualized storage devices for tenants are interconnected through tenant-specific virtual networks. The virtual network can be implemented as an overlay network that sits on top of an IP-based underlay that interconnects physical resources in the DC. The NVO solution provides layer 2 and / or layer 3 virtual networks to enable tenant-specific multi-tenancy and VM mobility across virtual networks. Therefore, each tenant is provided with an independent virtualized server and virtualized storage device island, as well as a virtual network that interconnects them. When the MPLS overlay is deployed as an NVO, the MPLS label stack of the MPLS overlay is a demultiplexer that identifies the virtual network.
[0085] exist Figure 2 In FIG. 2 , server 210 and server 220, both of which are physical servers, are connected via IP network 230. Figure 2, servers 210 and 220 are both shown as having been separated into three layers, as follows: (1) a "hardware" layer, which includes processors, memory (e.g., RAM), I / O ports (e.g., network interface cards (NICs), etc.), (2) a "hypervisor" layer, which manages and allocates hardware resources to VMs, and (3) a VM layer, which includes VMs running on top of the hypervisor. Server 210 is connected to IP network 230 via one or more ports in NIC 217 (which may be an Ethernet NIC), and server 220 is connected to IP network 230 via one or more ports in NIC 227 (which may be an Ethernet NIC).
[0086] exist Figure 2 In the example, each physical server hosts two tenants, named Ten-1 and Ten-2, where “Ten” is used as an abbreviation for “tenant.” VM 212 and VM 222 are VMs (or virtual servers) of Ten-1 hosted in physical servers 210 and 220, respectively. VM 213 and VM 223 are VMs (or virtual servers) of Ten-2 hosted in physical servers 210 and 220, respectively.
[0087] exist Figure 2 , a virtual network is created between each tenant's VMs using an MPLS overlay 240. For simplicity, since each tenant has only two VMs, the virtual network in this article is a point-to-point Ethernet link. The MPLS overlay 240 is created between the hypervisors in servers 210 and 220. The MPLS overlay termination point 216 in server 210 is located in the hypervisor in server 210. The MPLS overlay termination point 216 is configured with an IP address to be used for the IP underlying layer so that the address is reachable through the IP network 230. The MPLS overlay termination point 226 in server 220 is located in the hypervisor in server 220. The MPLS overlay termination point 226 is configured with an IP address to be used for the IP underlying layer so that the address is reachable through the IP network 230. Each tenant is assigned a network-wide unique MPLS label to be used as a demultiplexer in its MPLS overlay. For example, assume that Ten-1 is assigned label 100 and Ten-2 is assigned label 200. Note that this example uses a bidirectional LSP as the MPLS overlay for the tenant, and therefore, the same label is assigned for both directions. Note that in terms of operating as an endpoint of an MPLS overlay, Figure 2 The MPLS overlay termination points 216 and 226 may be considered to correspond to Figure 1 LSR 110.
[0088] exist Figure 2In FIG. 1 , VM 212 is connected to MPLS overlay termination point 216 via virtual Ethernet port 214. In MPLS overlay termination point 216, virtual Ethernet port 214 is mapped to MPLS overlay label 100. Similarly, VM 222 is connected to MPLS overlay termination point 226 via virtual Ethernet port 224. In overlay termination point 226, virtual Ethernet port 224 is mapped to MPLS overlay label 100. This completes the establishment of a point-to-point virtual network between the VMs of Ten-1.
[0089] exist Figure 2 In FIG. 2 , VM 213 is connected to MPLS overlay termination point 216 via virtual Ethernet port 215. In MPLS overlay termination point 216, virtual Ethernet port 215 is mapped to MPLS overlay label 200. Similarly, VM 223 is connected to overlay termination point 226 via virtual Ethernet port 225. In overlay termination point 226, port 225 is mapped to MPLS overlay label 200. This completes the establishment of a point-to-point virtual network between the VMs of Ten-2.
[0090] exist Figure 2 2, the payload of MPLS overlay 240 is a layer 2 / Ethernet packet because each VM generates an Ethernet packet. In Ten-1, assume that VM 212 sends a packet to VM 222. First, VM 212 sends an Ethernet packet via virtual Ethernet port 214. When MPLS overlay termination point 216 receives the packet, it pushes MPLS label 100 onto the packet and then pushes the tunnel encapsulation (e.g., MPLS-in-IP, MPLS-in-GRE, or MPLS-in-UDP, etc.). The source address in the IP header of the tunnel is set to the IP address of MPLS overlay termination point 216, and the destination address in the IP header of the tunnel is set to the IP address of MPLS overlay termination point 226. The resulting packet is sent by MPLS overlay termination point 216 to IP network 230. After being routed by IP network 230, the packet eventually arrives at MPLS overlay termination point 226. MPLS overlay termination point 226 decapsulates the tunnel encapsulation and then looks for MPLS label 100 underneath. Since label 100 is mapped to virtual Ethernet port 224 in MPLS overlay termination point 226, label 100 is removed and the resulting Ethernet packet is forwarded on port 224. The packet is then received by VM 220. In the same way, VMs in Ten-2 exchange packets with each other through their virtual networks. Thus, multiple virtual networks are multiplexed between the two tunnel endpoints using their corresponding MPLS overlay labels.
[0091] Various example embodiments for supporting reliable MPLS overlay may be Figure 2The invention is applied within the context of to support reliable delivery of MPLS overlay packets for NVO in a multi-tenant DC environment. For example, a reliable MPLS overlay can prevent the discarding of MPLS overlay packets that might otherwise be discarded in the absence of a reliable MPLS overlay (e.g., preventing the discarding of MPLS overlay packets between MPLS overlay termination points 216 and 226, such as due to buffer overflows in NICs 217 and 227, in any router in IP network 230 (e.g., due to buffer / queue overflows, congestion, etc.), failures in links along IP network 230, etc.). Note that if certain applications require reliability, it is the responsibility of the application to generate the overlay traffic (e.g., a TCP / IP-based HTTP application (providing traditional layer 2 or layer 3 services) that generates traffic on the overlay can recover from losses due to the reliability provided by TCP). It should also be noted that there are also applications that generate traffic on top of an MPLS overlay, but are not based on protocols that provide reliable transport for the application (e.g., TCP / IP, etc.), and some of these applications are various infrastructure protocols / interconnects that are designed to run on reliable media (however, in a multi-tenant DC, the media is virtualized using an MPLS overlay). Such overlays provide infrastructure interconnect services, rather than traditional layer 2 or layer 3 services. Note that since multiple such MPLS overlays are multiplexed on a tunnel between two tunnel endpoints, the tunneling mechanism used by the MPLS overlay can provide reliability.
[0092] Figure 3 An example embodiment of a non-chassis-based NFV router is depicted to enable further understanding of the application of a reliable MPLS overlay to a chassis-based NFV router. The NFV-based router can implement a router platform with one or more VMs on a commercial off-the-shelf physical server (such as an x86-based server platform). There can be two approaches to implementing a NFV router: (1) a non-chassis-based approach, where the entire router is hosted on a single VM, or (2) a chassis-based approach, where each component of the router is implemented by a separate VM. Figure 3 In NFV, non-chassis-based routers run as VMs on a hypervisor on a physical server. Figure 3, a physical server 300 includes hardware 310, a hypervisor 320, and a VM 330 providing a router 331. The hardware 310 includes a set of N NICs (denoted as NIC-1 to NIC-N), the N NICs having N ports associated therewith (denoted as Port 1 to Port N associated with NIC-1 to NIC-N, respectively). The router 331 receives packets from ports on the NICs, processes the packets, and forwards the processed packets to the relevant destinations via ports on the NICs. For optimal performance, the NICs assigned to the router 331 (VM 330) are directly controlled by the router 331 (e.g., using single root input / output virtualization (SR-IOV), PCI pass-through, etc.) without any mediation by the hypervisor 320. The control plane and forwarding plane of the router 331 reside within a single VM 330. Note that since the NICs are Ethernet-based, NFV routers often use Ethernet as the data link layer.
[0093] Figure 4 An example embodiment of a chassis-based router is described to enable further understanding of the application of a reliable MPLS overlay to a chassis-based NFV router. Figure 4 In the embodiment, chassis 400 includes N+1 slots including N slots and one additional slot, the N slots include forwarding plane cards (denoted as SLOT-0 to SLOT-N, which include forwarding plane cards denoted as FWD card 1 to FWD card N, respectively), and the one additional slot includes a control plane card (denoted as SLOT-X, which includes a control plane card denoted as CTRL card). The forwarding plane card provides the forwarding plane of the router. The forwarding plane card includes ports for transmitting and receiving packets (omitted herein for clarity). The forwarding plane card implements the forwarding plane of the router, which receives packets, processes packets, and sends packets to the destination. The control plane card operates the control plane of the router. It should be understood that although there can be multiple control plane cards for redundancy purposes, for simplicity, Figure 4 Only a single control plane card is shown in FIG. 1 . Note that, in general, there are at least two internal networks that can connect the various cards, with the root node (centralized entity) of each network located in the control plane card. An example of an expanded view of the interconnection with two internal networks is shown in FIG. Figure 5 Presented in.
[0094] Figure 5 An example embodiment of internal structures in a chassis-based router is depicted to enable further understanding of the application of reliable MPLS overlays to chassis-based NFV routers.
[0095] exist Figure 5 , the interconnection of two internal networks based on a chassis router is shown.
[0096] exist Figure 5 In , the first network is used for the switching fabric through which packets are switched across the cards. Figure 5 In , the switch fabric is hosted within the control plane card, but it is also possible to have a separate switch fabric card. If a packet arriving on a port in card 1 needs to be sent out by a port in card 4, card 1 sends the packet to a channel leading to the switch fabric after performing ingress processing, and the switch fabric then relays the packet to card 4 for egress processing and final transmission. Each card is connected to the switch fabric through a switch fabric channel (a link in the switch fabric network).
[0097] exist Figure 5 In the , the second network is used for inter-card communication (ICC). ICC control is in the control plane and is connected to each forwarding plane card through an ICC channel (a link in the ICC network). The ICC network is used for all control and management messaging between the control plane and the forwarding plane. For example, the configuration of the forwarding plane by the control plane is performed using the ICC network. Any alarms or warnings generated by the forwarding plane are notified to the control plane through the ICC network. Heartbeats for checking connectivity between cards are exchanged as ICC messages.
[0098] Note that both the switching fabric and the ICC network are lossless, which means that packets / messages can generally be transmitted reliably without any drops.
[0099] It should be understood that although primarily presented with respect to the use of two interconnect networks, more than two interconnect networks may be used.
[0100] It should be understood that Figure 4 and 5 The presented router chassis can be virtualized to provide a chassis-based NFV router, as described in detail in Figure 6 Presented.
[0101] Figure 6 Depicted are example embodiments of using a reliable MPLS overlay for an internal fabric in a chassis-based NFV router, where the control plane and forwarding plane are separated by a network.
[0102] exist Figure 6 As indicated above, a reliable MPLS overlay can be used for chassis-based NFV routers where the control plane and forwarding plane are separated by the network.
[0103] exist Figure 6In , each card is emulated by a VM, and each VM resides in a separate physical server to minimize single points of failure. Here, the term "control plane server" is used to refer to the physical server that hosts the VM for the control plane card, and the term "forwarding plane server x" is used to refer to the physical server that hosts the VM for the forwarding plane card x.
[0104] exist Figure 6 In the example, the control plane and forwarding plane VMs are separated by an IP network. The VMs can be located within a local area network (LAN), for example, within the same IP subnet. However, here, the VMs are separated by an IP network because this is a superset case that can also meet the requirements when the VMs are connected within a LAN.
[0105] exist Figure 6 In , each forwarding plane VM uses NIC-1 to emulate a port for packet reception and transmission. For optimal performance, NIC-1 is directly controlled by the forwarding plane VM (e.g., using SR-IOV, PCI pass-through, etc.) without any mediation by the hypervisor. Each forwarding plane server uses a port on NIC-2 to connect to the IP network. The control plane server uses a port on NIC-1 to connect to the IP network. The ICC and switch fabric channels between the VMs are established as an MPLS overlay across the IP network. The MPLS overlay termination point is located in the hypervisor. There are two MPLS overlays between each forwarding plane server and the control plane server - one MPLS overlay for the ICC channel and one MPLS overlay for the switch fabric channel. Each overlay is established as a bidirectional LSP that uses the same MPLS label in both directions (i.e., from the forwarding plane card to the control plane card and vice versa). For example, assume that the MPLS label of the overlay for the ICC channel is label 100 and the MPLS label of the overlay for the switch fabric channel is label 200.
[0106] exist Figure 6 In order to create an IP bottom layer for the overlay, each overlay termination point creates an IP address on its corresponding port on the NIC connected to the IP network, so that the IP address is routable by the IP network. The IP address on the overlay termination point at the forwarding plane server x is represented as "F-IPx", and the IP address on the overlay termination point at the control plane server is represented as "C-IP". The overlay termination point at each forwarding plane server is configured with an underlying tunnel, and the destination IP address of the underlying tunnel is C-IP. The overlay termination point at the control plane server is configured with N underlying tunnels, each of which leads to a forwarding plane server. For example, the destination IP addresses of the underlying tunnels leading to forwarding plane servers 1, server 2,..., server N are F-IP1, F-IP2,..., F-IPN respectively.
[0107] exist Figure 6 In the example, the forwarding plane VM is connected to its local overlay termination point using two virtual ports - one virtual port for the ICC channel and one virtual port for the switch fabric channel. Within the overlay termination point, the port for the ICC channel is mapped to MPLS label 100 and the port for the switch fabric channel is mapped to MPLS label 200. Both labels are multiplexed on the IP underlay tunnel to the control plane server.
[0108] exist Figure 6 In , for each forwarding plane VM, the control plane VM is connected to its local overlay termination point using two virtual ports - one virtual port for the ICC channel and one virtual port for the switch fabric channel. Therefore, there are a total of 2N virtual ports between the control plane VM and its local overlay termination point. Within the local overlay termination point: (1) the port for the ICC channel to the forwarding plane VM is mapped to MPLS label 100 and the IP underlying tunnel to the forwarding plane server, and (2) the port for the switch fabric channel to the forwarding plane VM is mapped to MPLS label 200 and the IP underlying tunnel to the forwarding plane server. It should be understood that the MPLS overlay for the ICC and switch fabric forms two virtual networks, which is the case of NVO as described above.
[0109] exist Figure 6 In, as an example, any ICC message from the forwarding plane VM of emulated card 1 to the control plane VM will first be sent on its virtual port for the ICC channel. When the local overlay termination point receives the message, it pushes MPLS label 100, followed by a tunnel encapsulation, which uses an IP header in which the source IP address is F-IP1 and the destination IP address is C-IP. The packet is then sent to the IP network via a port in NIC-2. When the overlay termination point in the control plane server receives the packet, it performs the following actions: (1) identifies the source forwarding plane server based on the source IP address in the tunnel encapsulation, (2) removes the tunnel encapsulation, (3) looks up MPLS label 100 and demultiplexes the packet containing the ICC message based on label 100 and removes the label, (4) based on the identified source forwarding plane server, looks up the virtual port of the control plane VM, which is mapped to the ICC channel of the forwarding plane VM, and (6) forwards the ICC message on the virtual port. The ICC message is received by the ICC control module in the control plane VM.
[0110] exist Figure 6, as an example, any packet to be sent from the forwarding plane VM of emulated card 1 to the control plane VM on the switch fabric channel will first be sent on its virtual port for the channel. When the local overlay termination point receives the message, it pushes MPLS label 200, followed by a tunnel encapsulation, which uses an IP header with the source IP address as F-IP1 and the destination IP address as C-IP. The packet is then sent to the IP network via a port in NIC-2. When the overlay termination point in the control plane server receives the packet, it performs the following operations: (1) identifies the source forwarding plane server based on the source IP address in the tunnel encapsulation, (2) removes the tunnel encapsulation, (3) looks up MPLS label 200 and demultiplexes the packet containing the packet for the switch fabric based on label 200 and removes the label, (4) based on the identified source forwarding plane server, looks up the virtual port of the control plane VM, which is mapped to the switch fabric channel of the forwarding plane VM, and (6) forwards the packet on the virtual port. The packet is received by the switch fabric in the control plane VM.
[0111] Note that if a physical server hosts only a single forwarding plane or control plane VM, then for best performance, an overlay termination point can also be implemented within the respective VM to avoid context switching with the hypervisor while transmitting and receiving packets on the channel. In this case, NIC-2 in the forwarding plane server will be directly controlled by the forwarding plane VM (e.g., using SR-IOV, PCI passthrough, etc.). The same is true for NIC-1 in the control plane server.
[0112] Various example embodiments for supporting reliable MPLS overlay may be Figure 6 Applied within the context of to support reliable delivery of MPLS overlay packets for chassis-based NFV routers where the control plane and forwarding plane are separated by the network (e.g., to prevent the NIC from dropping ICC or switch fabric packets (at both the control plane server and the forwarding plane server) or within an IP network).
[0113] Figure 7 An example embodiment using a reliable MPLS overlay for virtualized Fibre Channel (FC) in a storage area network (SAN) is described.
[0114] exist Figure 7In, as indicated above, a reliable MPLS overlay can be used for virtualized FC in a SAN. FC is a high-speed data transmission protocol that provides ordered lossless transmission of raw data blocks, and is primarily used to connect computer data storage to servers in a SAN in a DC. A SAN is a network of block-based storage device pools that can be accessed / shared by multiple servers connected to the SAN. Reliability in FC is provided by its data link layer, which is called FC-1 (FC-0 is the physical layer, which is usually a high-speed optical fiber).
[0115] exist Figure 7 In the example, FC traffic is transmitted over an MPLS overlay 740 in a multi-tenant DC. The example is described using two tenants, denoted as Ten-1 and Ten-2. Server 720 is a physical server directly attached to SAN 750 via FC link 760. Server 720 interfaces with FC link 760 via a port on FC card 724. Server 720 hosts a VM that runs the functions of a multi-tenant SAN controller 722. Any access request to SAN 750 is made through SAN controller 722. In order to implement multi-tenancy, SAN 750 is logically partitioned to appear as multiple independent SANs. The logical SANs of Ten-1 and Ten-2 are mapped to SAN 750 as SAN 751 and SAN 752, respectively. SAN controller 722 maintains a mapping of tenant-specific logical data blocks to physical data blocks in SAN 750. This logical partitioning and mapping is required for security reasons so that any erroneous access by a tenant does not damage the data of another tenant. Server 710 hosts VMs for Ten-1 and Ten-2, denoted as VM 712 and VM 713 respectively. The VMs run some tenant specific server applications. VM 712 accesses SAN 751 and VM 713 accesses SAN 752.
[0116] exist Figure 7 , VMs 712 and 713 and associated SANs 751-752 are located in remote sites that are physically separated by IP network 730. Physical separation is possible due to VM mobility (e.g., VMs can be moved across remote sites via a virtual network connecting the VMs), remote disk access, tape backup, and real-time mirroring. Figure 7 In the model, you can also interconnect the islands of multi-tenant FC SANs through IP networks to form a unified SAN in a single FC fabric. Figure 7 In FIG. 7 , the idea is to interconnect VMs 712 and 713 to associated SANs 751 and 752 by carrying FC traffic in a manner that the FC fabric 750 and FC devices on the SAN 750 are unaware of the presence of the IP network 730 therebetween.
[0117] exist Figure 7 , in order to simulate VMs 712 and 713 being directly connected to their SANs 751 and 752, respectively, the segments of FC link 760 between VMs 712 and 713 and SAN controller 722 need to be virtualized. Such virtualization can be achieved as follows. First, an MPLS overlay 740 is created between the hypervisors in servers 710 and 720 running on top of IP network 730. The hypervisors of servers 710 and 720 host MPLS overlay termination points 717 and 727, respectively, and MPLS overlay termination points 717 and 727 are configured with unique IP addresses routable from IP network 730. MPLS overlay termination points 717 and 727 access IP network 730 via NIC cards 714 and 723, respectively. Second, each tenant is assigned a unique MPLS label (e.g., Ten-1 is assigned label 100, Ten-2 is assigned label 200). Third, VM 712 for Ten-1 is connected to MPLS overlay termination 717 via virtual FC port 715 (which is mapped to label 100 in MPLS overlay termination 717), and VM 713 for Ten-2 is connected to MPLS overlay termination 717 via virtual FC port 716 (which is mapped to label 200 in MPLS overlay termination 717). Fourth, SAN controller 722 is connected to MPLS overlay termination 727 through two virtual FC ports 725 and 726 for Ten-1 and Ten-2, respectively. In MPLS overlay termination 727, FC port 725 is mapped to label 100, and FC port 726 is mapped to label 200.
[0118] exist Figure 7 , VM 712 sends a FC packet to SAN 750 through virtual FC port 715. When MPLS overlay termination 717 receives the FC packet, MPLS overlay termination 717 pushes MPLS label 100 as a demultiplexer for the service belonging to Ten-1. The MPLS packet is then sent to remote MPLS overlay termination 727 by adding an IP underlying encapsulation. When the packet arrives at MPLS overlay termination 727, MPLS overlay termination 727 pops the IP underlying encapsulation as well as MPLS label 100, and forwards the resulting FC packet to virtual FC port 725 mapped to label 100. SAN controller 722 receives the FC packet on virtual FC port 725 assigned to Ten-1, so the controller sends the required FC packet to SAN 751 via FC link 760. In the same manner, VM 713 accesses SAN 752.
[0119] Various example embodiments for supporting reliable MPLS overlay may be Figure 7The invention is applied in the context of to support reliable delivery of MPLS overlay packets for virtualized FC in a SAN (eg, robust to FC packet drops by a NIC or within the IP network).
[0120] It should be understood that although Figure 2-7 A specific context for the use of a reliable MPLS overlay is presented, but a reliable MPLS overlay can be used for various other types of contexts. For example, similar to FC, a reliable MPLS overlay can be used for the reliable transmission of Infiband services across storage systems. For example, a reliable MPLS overlay can be used for the virtualization of a PCI Express (PCIe) bus to support the reliable transmission of PCIe services. For example, similar to FC, a reliable MPLS overlay can be used for the reliable transmission of non-volatile memory express (NVMe) services to access storage on solid-state drives. It should be understood that such applications can be summarized as multi-tenant distributed input and output (I / O) in a DC that can use a reliable MPLS overlay. For example, when a pool of VMs in a tenant forms a server cluster (e.g., grid computing), the VMs are physically separated by a network, and the interconnection in the cluster / grid can be implemented as an MPLS overlay to achieve reliable transmission of server cluster services. Typically, there can be multiple types of distributed applications across tenants' VMs, where application services need to be reliably transmitted between VMs via an MPLS overlay. It should be understood that a reliable MPLS overlay can be used in various other types of contexts.
[0121] Figure 8 An example embodiment of a communication system configured to support a reliable MPLS overlay is depicted.
[0122] The communication system 800 includes a pair of LSRs 810 , including an LSR 810 -A and an LSR 810 -B (collectively referred to as LSRs 810 ), and an IP network 824 configured to support communications between the LSRs 810 .
[0123] LSR 810 supports data link layer 813 (illustratively, based on data link layer 813-A on LSR 810-A and data link layer 813-B on LSR 810-B), IP layer 814 above data link layer 813 (illustratively, based on IP layer 814-A on LSR810-A and IP layer 814-B on LSR 810-B), reliable transport layer 815 above IP layer 814 (illustratively, based on reliable transport layer 815-A on LSR 810-A and reliable transport layer 815-B on LSR 810-B), and MPLS overlay 816 above reliable transport layer 815 (illustratively, based on MPLS overlay 816-A on LSR 810-A and MPLS overlay 816-B on LSR810-B). The data link layer 813, IP layer 814, reliable transport layer 815, and MPLS overlay 816 on LSR 810 may each form part of a communication protocol stack (e.g., based on the OSI model or one or more other suitable protocol stack models) configured on LSR 810. LSR 810 is configured to provide a reliable MPLS overlay using a reliable transport layer over IP, thereby supporting reliable delivery of MPLS overlay packets.
[0124] The MPLS overlay 816 is configured to operate as a reliable MPLS overlay that supports reliable delivery of MPLS packets between LSRs 810. The reliability of the MPLS overlay 816 is based on a reliable transport layer 815. The reliability of the MPLS overlay 816 is based on a reliable connection 825 established at the reliable transport layer 815. The reliable transport layer 815 is connection-oriented and guarantees reliable delivery of MPLS overlay packets between LSRs 810 (e.g., reliable delivery of any MPLS overlay packet between a pair of source and destination addresses in an IP underlying tunnel of an IP layer 814). The reliable transport layer 815 may also support reliability features, such as flow control, congestion control, etc., and various combinations thereof. The reliable transport layer 815 supports a reliable connection 825 established between LSRs 810. The reliable connection 825 is configured to support reliable delivery of MPLS packets between LSRs 810. Reliable connections 825 are configured to support reliable delivery of MPLS packets between LSRs 810 in a reliable manner to support reliable MPLS overlay 816 even if LSRs 810 are separated by IP network 824 .
[0125] The reliable transport layer 815 and the associated reliable connection 825 can be based on a reliable transport protocol. The reliable transport layer 815 and the associated reliable connection 825 can be based on a reliable transport protocol, such as TCP, SCTP, QUIC, TLS, etc. Note that a reliable MPLS overlay using TCP as the reliable transport layer 815 (e.g., TCP is the "tunnel" layer of the MPLS packet) can be represented as "MPLS-in-TCP" in this article. Note that a reliable MPLS overlay using SCTP as the reliable transport layer 815 (e.g., SCTP is the "tunnel" layer of the MPLS packet) can be represented as "MPLS-in-SCTP" in this article. Note that a reliable MPLS overlay using QUIC as the reliable transport layer 815 (e.g., QUIC is the "tunnel" layer of the MPLS packet) can be represented as "MPLS-in-QUIC" in this article. Note that a reliable MPLS overlay using TLS as the reliable transport layer 815 (e.g., TLS is the "tunnel" layer of the MPLS packet, where TLS runs on top of TCP that actually provides a reliable transport channel) can be represented as "MPLS-in-TLS" in this article. It should be understood that although primarily presented with respect to using a particular reliable transport protocol at the reliable transport layer 815 to provide the reliable connection 825 , various other reliable transport protocols may be used at the reliable transport layer 815 to provide the reliable connection 825 .
[0126] It should be appreciated that while primarily presented with respect to supporting a single MPLS overlay on reliable connection 825 (illustratively, between MPLS overlays 816), multiple MPLS overlays may be multiplexed between LSRs 810 over reliable connection 825 and associated IP underlay tunnels through IP network 824.
[0127] Various example embodiments are configured to provide a framework for the operation of a reliable MPLS overlay that is tunneled using connections in a reliable transport layer. Any reliable transport layer designed to tunnel MPLS overlay packets can be configured to follow the specifications of the framework. In at least some example embodiments, the framework can be configured to implement MPLS-in-TCP. In at least some example embodiments, the framework can be configured to implement MPLS-in-SCTP. In at least some example embodiments, the framework can be configured to implement MPLS-in-QUIC. In at least some example embodiments, the framework can be configured to implement MPLS-in-TLS. It should be understood that various other reliable transport protocols can be used to provide a reliable MPLS overlay.
[0128] It should be understood that although primarily presented with respect to using a single reliable transport layer to support a reliable MPLS overlay, a reliable MPLS overlay may utilize multiple reliable transport layers. For example, since an MPLS overlay may carry traffic from multiple applications, one or more of which require reliability and one or more of which do not, one or more of which require reliability may be supported using one or more reliable transport layers, while one or more of which do not require reliability may be supported using one or more other transport layers that are not necessarily implemented as reliable transport layers as presented herein.
[0129] It should be appreciated that application specific packets sent over the reliable MPLS overlay may be adapted independently of the use of a reliable transport layer to provide reliable transport of the MPLS overlay, depending on the requirements or desires of the reliable MPLS overlay.
[0130] It should be understood that while support for a reliable MPLS overlay is primarily presented herein with respect to providing data plane capabilities for supporting a reliable MPLS overlay, support for a reliable MPLS overlay may also be based on various control plane capabilities that may be utilized to support the reliable MPLS overlay. For example, where use of a reliable transport layer to provide a reliable MPLS overlay relies on an LSR being able to access certain information related to the capabilities of a remote LSR (e.g., knowledge of the remote LSR's ability to decapsulate MPLS packets from a reliable transport protocol header, knowledge of MPLS overlay termination, etc., and various combinations thereof), various control plane capabilities may be used to configure such information on the LSR (e.g., manual configuration using information about the LSR, dynamically advertising or sending information between LSRs, etc., and various combinations thereof).
[0131] Fig. 9An example embodiment of an MPLS overlay packet is depicted to illustrate the positioning of (multiple) headers of a reliable transport layer in an MPLS overlay packet. MPLS overlay packet 900 includes a data link header 901, an IP header 902 below the data link header 901, a reliable transport header 903 below the IP header 902, an optional MPLS control header 904 below the reliable transport header 903, an MPLS label stack 905 below the optional MPLS control header 904, and a payload 906 below the MPLS label stack 905. Data link header 901 is a data link layer header of the next hop of a packet in an IP network. IP header 902 is an IP header of an IP bottom layer. Reliable transport header 903 includes one or more headers from a reliable transport layer. MPLS label stack 905 is an MPLS label stack of an MPLS overlay. Payload 1506 is a payload or application packet transmitted over a reliable MPLS overlay. The MPLS Control Header (MCH) 904 is optional and is included when the reliable transport connection is stream-oriented, that is, when the reliable transport connection reliably transports a byte stream. In this case, the receiving LSR needs to find the MPLS overlay packet in the received byte stream. In order to divide the MPLS overlay packets in the byte stream, when the LSR sends the MPLS overlay packet, it pushes the MCH 904 on top of the MPLS label stack 905 in the packet. The MCH 904 includes at least a field indicating the size of the MPLS overlay packet. The receiving LSR parses the first 4 octets (bytes) in the byte stream as the MCH 904 and determines the end byte of the MPLS overlay packet based on its length field. This is done in Fig.10 Shown in.
[0132] Fig.10 An example embodiment of the parsing of a byte stream comprising an MPLS overlay packet is depicted. The byte stream is processed as a sequence of 4 octets, where each word is indexed with the sequence number of its first octet. When the LSR receives this byte stream from a reliable transport connection, it parses word 0 as an MCH, where the size field indicates 20 octets. Therefore, words 4-20 constitute the first MPLS overlay packet. Then, word 24 must be the MCH of the next packet, where the size field indicates 28 octets. Therefore, words 28-52 constitute the second MPLS overlay packet. Then, word 56 must be the MCH of the next packet, and so on.
[0133] Fig.11 Depicted are example embodiments of an MCH configured for use on an MPLS overlay packet to support a reliable MPLS overlay. Fig.11As shown, MCH 1100 is a 4-octet header including a length field, a flag field, and a reserved field. The length field is a 16-bit field that indicates the length of the encapsulated MPLS overlay packet in octets. This field can accommodate MPLS overlay packets of up to 65535B in size. The flag field is an 8-bit field, in which each bit indicates a flag. It is understood that zero or more flags can be defined and used. If the flag is not used, the sender sets the field to 0 and the receiver ignores the field. The reserved field is an 8-bit field that can be used for various purposes. If the field is not used, the sender sets the field to 0 and the receiver ignores the field. Before sending the packet on the stream-oriented reliable transport connection, MCH 1100 is pushed onto the MPLS overlay packet. It should be understood that although it is presented mainly with respect to a specific number, type, and arrangement of fields, MCH 1100 may include various other numbers, types, or arrangements of fields (including various custom fields that can be designed and used).
[0134] Fig.12 Depicted is an example embodiment of an MPLS Overlay Initial Frame (MOIF) for a new transport connection.
[0135] The MOIF 1200 is a special frame that can be sent round-trip when a new transport connection is established, from the LSR initiating the transport connection to the LSR receiving the connection request and back to the LSR initiating the transport connection. The use of the MOIF 1200 is optional. The MOIF 1200 enables the transmission of key parameters (if any) shared by the MPLS overlay layer at both LSRs and supports the robustness of the transport connection (e.g., such as data integrity and performance measurements such as latency, delay, etc.) when sending MPLS overlay traffic over the transport connection. Note that if MOIF is supported and used, the administrator can configure the requirements of MOIF in the LSRs participating in the connection.
[0136] The MOIF 1200 includes an MPLS label field, a sequence number field, an acknowledgment number field, a timestamp field, an optional parameter field, and a checksum field. The MPLS label field is a 32-octet field that is encoded using a value that is reserved to indicate that the frame is a MOIF frame. For example, a label value of 0 (or any other suitable value) can be used to indicate a MOIF frame. The special value used to indicate a MOIF frame will not be assigned for any other purpose, such as a label for MPLS coverage. The sequence number field includes a random number issued by the LSR that initiated the MOIF. The acknowledgment field includes an acknowledgment number encoded by the LSR that receives the MOIF and sends the updated MOIF back to the LSR that initiated the MOIF. For example, the LSR that receives the MOIF and sends the updated MOIF back to the LSR that initiated the MOIF can increment the sequence number received in the sequence number field by one to provide an acknowledgment number. The timestamp field includes a timestamp inserted by the LSR that initiated the MOIF. The timestamp indicates the time when the MOIF was sent. This field is used to measure one or more parameters of the MOIF (e.g., round trip time (RTT), latency, etc.). The Optional Parameters field is a variable length field that can be used to encode various optional parameters. The Checksum field includes the checksum of all previous fields in the MOIF. The checksum is used to verify the integrity of the packet.
[0137] It should be appreciated that while primarily presented with respect to a particular number, type, and arrangement of fields, the MOIF 1200 may include various other numbers, types, or arrangements of fields (including various custom fields that may be designed and used).
[0138] Fig.13 An example embodiment of a method for use by an initiating router to configure a reliable transport connection is depicted. The method 1300 is configured for use by an LSR initiating a reliable transport connection to support configuration of reliable transport to a remote LSR for tunneling of MPLS overlay packets. Note that if the transport protocol is designed to operate in a client-server paradigm, the method 1300 may be performed by an LSR playing the role of a client (e.g., in a client-server paradigm, the client initiates a connection establishment request, the server listens for incoming connection requests and accepts or rejects the connection request in response to the client. It should be understood that while primarily presented as being executed serially, at least a portion of the blocks of the method 1300 may be executed simultaneously or in different Fig.13. The inputs to method 1300 include: (1) one or more remote IP addresses available for connection (e.g., a set of IP addresses of remote LSRs configured to listen for connection requests), and (2) connection parameters, which define a set of parameters for the connection, such as those related to delay / timeout, congestion, segmentation / reassembly of payload (MPLS), etc. At block 1301, method 1300 begins. Block 1302 selects a local IP address in the LSR that can be used to send and receive tunnel traffic, and method 1300 proceeds to block 1304. Block 1304 determines a local identifier to be used for the new connection, and method 1300 proceeds to block 1306. Block 1306 establishes a transport protocol connection from the local IP address to one of the remote IP addresses using the local identifier to specify that the connection will carry MPLS packets, and method 1300 proceeds to block 1308. Block 1308 checks whether the connection is successful. If the connection is successful, then the method 1300 proceeds to block 1310, otherwise the method 1300 proceeds to block 1312. Block 1312 declares a failure to configure the connection, and then the method 1300 proceeds to block 1399, at which the method 1300 ends. Block 1310 performs certain subsequent actions on the new connection (e.g., initiating MOIF, etc.) until the transport connection is ready for use by the MPLS overlay, and then the method 1300 proceeds to block 1399, at which the method 1300 ends. At block 1399, the method 1300 ends.
[0139] Fig.14 An example embodiment of a method used by an initiating router to support subsequent actions for a new reliable transport connection is depicted. Method 1400 may be used to provide Fig.13 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 1400 may be executed simultaneously or in different Fig.14. The inputs to method 1400 include: (1) an operational transport connection (denoted as connection), and (2) parameters for the operational transport connection (denoted as connection parameters). For example, the connection parameters may include parameters related to delay / timeout, congestion, segmentation / reassembly of payload (MPLS), etc. Note that connection parameters may not be applicable to certain transport protocol types, and therefore, in some cases, connection parameters may be considered optional. At block 1401, method 1400 begins. Block 1402 checks whether connection parameters are provided. If connection parameters are provided, method 1400 proceeds to block 1404, otherwise method 1400 proceeds to block 1406. Block 1404 applies the input connection parameters to the new connection, and method 1400 then proceeds to block 1406. Block 1406 may perform various other actions based on the transport protocol type, such as further adjustments to the connection, and method 1400 then proceeds to block 1408. Block 1408 checks whether the requirements of the MOIF are configured in the LSR. If the requirement for MOIF is configured in the LSR, then the method 1400 proceeds to block 1410, otherwise the method 1400 proceeds to block 1499, at which the method 1400 ends. Block 1410 constructs the MOIF and populates the necessary fields, and then the method 1400 proceeds to block 1412. Block 1412 sends the MOIF over the transport connection, and then the method 1400 proceeds to block 1414. Block 1414 waits for a MOIF response from the peer LSR. Note that an implementation may wait at least 90 seconds (or any other suitable length of time) for a MOIF response. To wait, the LSR may start a timer for the duration of the MOIF response it expects, and associate the timer with the transport connection (although it will be appreciated that other mechanisms are contemplated). From block 1414, the method 1400 proceeds to block 1499, at which the method 1400 ends. At block 1499, the method 1400 ends.
[0140] Fig.15 An example embodiment of a method for use by an initiating router to construct a MOIF for a new reliable transport connection is depicted. Fig.14 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 1500 may be executed simultaneously or in different Fig.15. In block 1501, method 1500 starts. Block 1502 creates an empty MOIF, which means that all fields of the MOIF are zeroed, and method 1500 proceeds to block 1504. Block 1504 sets the value of the MPLS label to a special value indicating that the packet is a MOIF packet, sets the S bit to 1 (because no more labels follow), and sets the EXP and TTL values according to the choice of the implementation. Method 1500 then proceeds to block 1506. Block 1506 sets the sequence number field to a random value, and method 1500 proceeds to block 1508. Block 1508 sets the timestamp field to the current time in the LSR, and method 1500 proceeds to block 1510. Block 1510 fills in the optional parameters (if any) (this block is implementation specific), and method 1500 proceeds to block 1512. Block 1512 calculates the checksum of all previous fields in the MOIF and populates the result into the checksum field, and then method 1500 proceeds to block 1599. At block 1599, method 1500 ends.
[0141] Fig.16 An example embodiment of a method used by an initiating router when a MOIF response is not received within a predefined time period is depicted. Method 1600 may be used to provide Fig.14 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 1600 may be executed simultaneously or in different Fig.16 Method 1600 is executed in the order presented in . The input of method 1600 includes a notification about the expiration of the MOIF wait timer. At block 1601, method 1600 starts. Block 1602 retrieves the transport connection associated with the timer, and then method 1600 proceeds to block 1604. Block 1604 closes the transport connection, and then method 1600 proceeds to block 1699. At block 1699, method 1600 ends.
[0142] Fig.17 An example embodiment of a method for use by an initiating router to process a MOIF response is depicted. It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 1700 may be executed simultaneously or in different Fig.17 The input to method 1700 includes a MOIF packet. Note that the determination that a received packet is a MOIF packet can be made as part of the general processing of MPLS overlay packets on a transport connection (e.g., as described with respect to Fig.251702-2514 of method 2500 of ). At block 1701, method 1700 starts. Block 1702 checks whether the LSR is already waiting for a MOIF response from a peer LSR (e.g., whether the MOIF wait timer is running). If the LSR is already waiting for a MOIF response from a peer LSR, method 1700 proceeds to block 1704, otherwise method 1700 proceeds to block 1718 to process as an invalid packet. Block 1718 closes the transport connection. Block 1704 stops the MOIF wait timer, and method 1700 proceeds to block 1706. Block 1706 calculates the checksum of all fields in the MOIF packet except the checksum field, and method 1700 proceeds to block 1708. Block 1708 checks whether the calculated checksum is equal to the value of the checksum field in the packet. If the calculated checksum is equal to the value of the checksum field in the packet, the method 1700 proceeds to box 1710, otherwise the method 1700 proceeds to box 1718 to process the erroneous MOIF packet. Box 1718 closes the transport connection. Box 1710 checks whether the value in the confirmation number field in the MOIF packet is equal to the value in the sequence number field plus 1. If the value in the confirmation number field in the MOIF packet is equal to the value in the sequence number field plus 1, the method 1700 proceeds to box 1718 to process the erroneous MOIF packet, otherwise the method 1700 proceeds to box 1712. Box 1718 closes the transport connection. Box 1712 calculates the RTT of the MOIF packet by decrementing the value in the timestamp field in the MOIF packet from the current timestamp in the LSR. The RTT provides an idea of the delay of the tunnel to the remote LSR. From box 1712, the method 1700 proceeds to box 1714. Box 1714 processes the optional parameters (if any) in the MOIF packet, and then the method 1700 proceeds to box 1716. Block 1716 marks the connection as ready for tunneling MPLS overlay traffic, and method 1700 then proceeds to block 1799 where method 1700 ends. At block 1700, method 1700 ends.
[0143] Fig.18 An example embodiment of a method for use by a receiving router for configuring a reliable transport connection listener is depicted. Note that an LSR may configure a reliable transport layer to listen for requests for incoming connections to be used for an MPLS overlay. Note that this may be used by any LSR configuring an MPLS overlay. It should be understood that while primarily presented as being executed serially, at least a portion of the blocks of method 1800 may be executed simultaneously or in different Fig.18. At block 1801, method 1800 begins. Block 1802 configures a transport layer with the ability to tunnel an MPLS overlay, and then method 1800 proceeds to block 1804. Block 1804 begins listening for incoming connection requests to any IP address in a local IP address pool (e.g., administratively selected). At block 1899, method 1800 ends.
[0144] Fig.19 An example embodiment of a method for use by a receiving router for processing an incoming request for a transport layer connection is depicted. It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 1900 may be executed simultaneously or in different Fig.19. The input to method 1900 includes a connection request with the following parameters: (1) source IP address (IP address of the remote LSR sending the request), (2) destination IP address (local IP address of the LSR), (3) connection parameters (transport protocol specific parameters in the connection request). At box 1901, method 1900 starts. Box 1902 evaluates the connection parameters to check whether the connection can be allowed. If the connection cannot be allowed, method 1900 proceeds to box 1912, otherwise method 1900 proceeds to box 1904. Box 1912 rejects the connection using an appropriate method in the transport layer (e.g., a rejection notification to the remote LSR according to the method in the transport protocol), and then method 1900 proceeds to box 1999, at which box 1999, method 1900 ends. Box 1904 checks whether the connection parameters indicate that the user is MPLS. If the connection parameters do not indicate the user as MPLS, then the method 1900 proceeds to box 1914, otherwise the method 1900 proceeds to box 1906. Box 1914 processes the connection request according to the associated user, and then the method 1900 proceeds to box 1999, where the method 1900 ends. Box 1906 checks whether the connection from the remote LSR (i.e., from the source IP address) is allowed. Note that there are a variety of factors that determine whether the LSR should allow an incoming connection. For example, the LSR can use a method to discover all potential remote LSRs that can participate in the MPLS coverage with the LSR. If the connection from the remote LSR is not allowed, then the method 1900 proceeds to box 1912, otherwise, the method 1900 proceeds to box 1908. Box 1912 rejects the connection using appropriate methods in the transport layer (e.g., a rejection notification to the remote LSR according to the method in the transport protocol), and then the method 1900 proceeds to box 1999, where the method 1900 ends. Block 1908 accepts the connection request (which means that the state of the connection is created based on the semantics of the transport layer, which may use certain elements of the connection parameters and may also include source and destination IP addresses), and method 1900 then proceeds to block 1910. Block 1910 performs certain subsequent actions on the new connection, and method 1900 then proceeds to block 1999, where method 1900 ends.
[0145] Fig. 20 An example embodiment of a method for use by a receiving router for performing a post-connection follow-up for a new reliable transport connection is depicted. Note that method 2000 may be performed by an LSR that receives a connection request for a reliable transport connection. It should be understood that while primarily presented as being performed serially, at least a portion of the blocks of method 2000 may be performed simultaneously or in different Fig. 20. In box 2001, method 2000 starts. Box 2002 checks whether the LSR requires a MOIF. If the LSR requires a MOIF, method 2000 proceeds to box 2004, otherwise method 2000 proceeds to box 2099, where method 2000 ends. Box 2004 waits for a MOIF from a peer LSR. It should be understood that the LSR can wait for the MOIF for any suitable amount of time (e.g., 90 seconds or any other suitable amount of time). It should be understood that the LSR can use various mechanisms to control the wait for the MOIF (e.g., the LSR can start a timer for the duration of its expected MOIF response and associate the timer with the transport connection). From box 2004, method 2000 proceeds to box 2099, where method 2000 ends. If the LSR fails to receive the MOIF within a predefined time period (e.g., the timeout set at box 2004 expires), the LSR performs Fig.16 The method 1600 of FIG. 1600 (thus, the processing is common in both the initiating and receiving LSRs). At block 2099, the method 2000 ends.
[0146] Fig.21 An example embodiment of a method for use by a receiving router for processing a MOIF while waiting for a MOIF response is depicted. Note that the method 2100 may be Fig. 20 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 2100 may be executed simultaneously or in different Fig.21 The input to method 1700 includes a MOIF packet. Note that the determination that a received packet is a MOIF packet can be made as part of the general processing of MPLS overlay packets on a transport connection (e.g., as described with respect to Fig.252502-2514 of method 2500 of ). At box 2101, method 2100 starts. Box 2102 checks whether the LSR is already waiting for a MOIF response from a peer LSR (e.g., checks whether the MOIF wait timer is running). If the LSR is already waiting for a MOIF response from a peer LSR, method 2100 proceeds to box 2104, otherwise method 2100 proceeds to box 2120 to process the packet as an invalid packet. Box 2120 closes the transport connection. Box 2104 stops the MOIF wait timer, and then method 2100 proceeds to box 2106. Box 2106 calculates the checksum of all fields in the MOIF packet except the checksum field, and then method 2100 proceeds to box 2108. Box 2108 checks whether the calculated checksum is equal to the value of the checksum field in the packet. If the calculated checksum is equal to the value in the checksum field in the packet, the method 2100 proceeds to block 2110, otherwise the method 2100 proceeds to block 2120 to process the erroneous MOIF packet. Block 2120 closes the transport connection. Block 2110 sets the value in the acknowledgment number field in the MOIF packet to be equal to the value in the sequence number field plus 1, and the method 2100 proceeds to block 2112. Block 2112 processes the optional parameters (if any) in the MOIF packet, and the method 2100 proceeds to block 2114. Block 2114 calculates the checksum of all previous fields in the MOIF and updates the result into the checksum field, and the method 2100 proceeds to block 2116. Block 2116 sends the updated MOIF packet back to the peer LSR, and the method 2100 proceeds to block 2118. Block 2118 marks the connection as ready to tunnel MPLS overlay traffic, and the method 2100 proceeds to block 2199, where the method 2100 ends.
[0147] FIG. 22 depicts an example embodiment of a method for configuring a reliable MPLS overlay. Note that an LSR may use method 2200 to establish a reliable MPLS overlay to a remote LSR. It should be understood that, although primarily presented as being executed serially, at least a portion of the blocks of method 2200 may be executed simultaneously or in an order different from that presented in FIG. 22. The inputs to method 2200 include: (1) an outgoing MPLS label for an LSP (MPLS overlay), (2) an IP address in a remote LSR for the LSP, and (3) transport parameters that define a set of parameters for the tunnel required by the LSP (e.g., related to delay / timeout, congestion, fragmentation and reassembly, etc.). At block 2201, method 2200 begins. Block 2202 checks whether there is a reliable transport connection with the remote LSR (e.g., used by other MPLS overlays). If there is no existing reliable transport connection with the remote LSR, method 2200 proceeds to block 2214, otherwise method 2200 proceeds to block 2204. Block 2204 obtains a first reliable transport connection with a remote LSR (e.g., used by other MPLS overlays), and then method 2200 proceeds to block 2206. Block 2206 converts the input transport parameters to the equivalent configured in the connection, and then method 2200 proceeds to block 2208. Block 2208 checks whether the parameters requested by the overlay match the parameters of the connection. If the parameters requested by the overlay match the parameters of the connection, then method 2200 proceeds to block 2224, otherwise method 2200 proceeds to block 2210. Block 2210 checks for more existing reliable transport connections with the remote LSR for the MPLS overlay. If there are more connections, then method 2200 proceeds to block 2212, otherwise method 2200 proceeds to block 2214. Block 2212 obtains the next reliable transport connection with the remote LSR, and then returns to block 2206 to repeat subsequent blocks for the next connection. Block 2214, which is reached when a new transport connection is needed, determines the type of reliable transport layer to be used. For example, if the LSR supports multiple types of reliable transport layers that support MPLS overlay, a selection may be made between the multiple types of reliable transport layers. Note that various factors may be evaluated to select between the multiple types of reliable transport layers. From block 2214, method 2200 proceeds to block 2216. Block 2216 converts the transport parameters (input) into the semantics of the selected transport layer (i.e., connection parameters), and method 2200 then proceeds to block 2218. Block 2218 configures a new reliable transport connection with the remote LSR for the MPLS overlay. The transport connection is configured using the connection parameters. Block 2218 may be configured by Fig.16. From block 2218, method 2200 proceeds to block 2220. Block 2220 checks whether the connection has been successfully established. If the connection has not been successfully established, method 2200 proceeds to block 2222, otherwise method 2200 proceeds to block 2222. Block 2222 declares and handles a failure to configure the MPLS overlay. Block 2224 allocates an LSP identifier to uniquely identify the MPLS overlay in the LSR. The LSP identifier binds the transport connection, (multiple) outgoing labels, and other parameters associated with the MPLS overlay. From block 2224, method 2200 then proceeds to block 2226. Block 2226 sets the outgoing MPLS label of the LSP to the outgoing label of the LSP identifier, and method 2200 then proceeds to block 2228. Block 2228 sets the transport connection to a tunnel associated with the LSP identifier, and method 2200 then proceeds to block 2299, where method 2200 ends. At box 2299, method 2200 ends.
[0148] Fig.23 An example embodiment of a method for transmitting packets over a reliable MPLS overlay is depicted. Note that the method 2300 may be performed by an LSR to transmit packets over a reliable MPLS overlay. It should be understood that while primarily presented as being performed serially, at least a portion of the blocks of the method 2300 may be performed simultaneously or in different Fig.23 The input of method 2300 includes: (1) a packet to be transmitted (e.g., Figure 6FC packet, ICC or switch fabric packet, etc. in the example of ), and (2) an LSP identifier that identifies the LSP or overlay. At box 2301, method 2300 starts. Box 2302 retrieves the status of the outgoing label of the LSP through the LSP identifier, and method 2300 proceeds to box 2304. Box 2304 obtains the tunnel configured for the outgoing label, that is, the reliable transport layer connection, and then method 2300 proceeds to box 2306. Box 2306 pushes the outgoing label of the LSP onto the packet, and then method 2300 proceeds to box 2308. Box 2308 sets the S bit, EXP, and TTL fields in the pushed label as required for the packet. For example, if the packet already has another label before the outgoing label of the pushed LSP, the S bit is set to 0, otherwise the S bit is set to 1. The TTL and EXP values can be set by the policy configured for the LSP. For example, if the policy requires the use of a consistent EXP value for all packets on the LSP, the configured EXP value is set. If the policy has a mapping of EXP values of certain fields in the native header of the packet, the mapped EXP value is set. From frame 2308, method 2300 proceeds to frame 2310. Frame 2310 determines the dynamic parameters to be used by the connection when the transport header is pushed onto the packet. Note that such dynamic parameters can be entropy-related fields in the transport header, and the transit routers in the IP network can use these fields to calculate the hash value of the load balancing packet on ECMP. From frame 2310, method 2300 proceeds to frame 2312. Frame 2312 sends the dynamic parameters of the MPLS packet and the transport header on the connection. From frame 2312, method 2300 proceeds to frame 2399, where method 2300 ends. At frame 2399, method 2300 ends.
[0149] Fig.24 An example embodiment of a method for transmitting MPLS packets over a reliable transport connection is depicted. Note that the method 2400 may be used to implement Fig.23 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 2400 may be executed simultaneously or in different Fig.24. The inputs to method 2400 include: (1) an MPLS packet to be transmitted, and (2) dynamic parameters of (multiple) transport layer headers. At box 2401, method 2400 begins. Box 2402 conditions the MPLS packet, which means that the reliable transport layer prepares the packet according to the connection state (e.g., if the packet needs further segmentation, the packet is segmented into multiple segments, congestion control and / or flow control is applied to the segments as needed, etc., and various combinations thereof). From box 2402, method 2400 proceeds to box 2404. Box 2404 checks whether one or more segments of the packet are ready to be sent. For example, due to congestion, flow control, etc., a segment may not be sent. If the segment is not ready to be sent, the segment will be sent later when the congestion is cleared, and then method 2400 proceeds to box 2499, where method 2400 ends. If the segment is ready to be sent, method 2400 proceeds to box 2406. Box 2406 pushes one or more headers of the transport layer. The fields in the header are filled in according to the connection parameters, such as a field indicating that the payload type is MPLS and an identifier of the connection. Dynamic parameters (inputs) are filled into the (multiple) headers as needed. Method 2400 then proceeds to box 2408. Box 2408 pushes the IP header onto the packet. The source address and destination address in the IP header are filled in according to the local and remote IP addresses of the transport layer connection. The protocol field (if it is an IPv4 header) or the next header field (if it is an IPv6 header) is set to indicate the transport layer type. Method 2400 then proceeds to box 2410. Box 2410 performs a route lookup to obtain the destination address in the IP header, which results in the next hop for the packet, and then method 2400 proceeds to box 2412. Box 2412 pushes the data link layer header onto the packet, which is necessary to send the packet to the next hop on the link. Method 2400 then proceeds to box 2414. Block 2414 sends the packet to the next hop, and method 2400 then proceeds to block 2499, where method 2400 ends.
[0150] Fig.25 An example embodiment of a method for processing packets received over a reliable MPLS overlay is depicted. Note that the method 2500 may be used to implement Fig.23 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 2500 may be executed simultaneously or in different Fig.25. The inputs to method 2500 include: (1) a transport layer packet on a connection (e.g., received by the transport layer of an LSR, with (multiple) transport layer specific headers on top), and (2) a connection. At box 2501, method 2500 starts. Box 2502 checks whether MPLS is the user application of the connection. If MPLS is not the user application of the connection, method 2500 proceeds to box 2524, otherwise method 2500 proceeds to box 2504. Box 2524 processes transport layer packets for non-MPLS applications. Box 2504 performs transport layer processing on the transport layer packets in accordance with the specification of the transport protocol, and then method 2500 proceeds to box 2506. Box 2506 checks whether the transport protocol is stream-oriented. In a stream-oriented protocol, the bytes of an entire MPLS packet may not arrive together in a transport layer packet (i.e., blocks of a packet may arrive on multiple transport packets), otherwise the transport protocol may be datagram-oriented, so that a transport layer packet carries the entire MPLS packet. When the method 2500 is implemented for a specific transport protocol, then frame 2506 may not be needed, because whether the protocol is stream-oriented or datagram-oriented is inherent to the protocol. If the check result in frame 2506 determines that the transport protocol is stream-oriented, then the method 2500 proceeds to frame 2508, otherwise the method 2500 proceeds to frame 2514. Frame 2508 parses the first 4 octets of the received stream into MCH, and then the method 2500 proceeds to frame 2510. Frame 2510 checks whether the received stream has at least the number of octets specified in the length field in the MCH, that is, at least the entire next MPLS packet. If the stream does not have the entire packet, then the method 2500 proceeds to frame 2599, at which the method 2500 ends, otherwise the method 2500 proceeds to frame 2512. Block 2512 extracts the number of octets specified in the length field of the MCH plus the 4 octets that make up the MPLS packet encapsulated by the MCH from the received stream, and then removes the MCH to generate the MPLS packet, and then method 2500 proceeds to block 2514. Block 2514 checks whether the MPLS packet is a MOIF. If the MPLS packet is a MOIF, then method 2500 proceeds to block 2516, otherwise method 2500 proceeds to block 2518. Block 2516 processes the MOIF packet. If the LSR is the initiator of the connection, then block 2516 may be connected by Fig.17 Method 1700 is implemented, otherwise box 2516 can be Fig.21. From box 2516, method 2500 proceeds to box 2520. Box 2518 treats the packet as an MPLS packet, and then method 2500 proceeds to box 2520. Box 2520 checks whether the transport protocol is connection-oriented. This check is similar to box 2506. If the transport protocol is connection-oriented, method 2500 proceeds to box 2522, otherwise method 2500 proceeds to box 2599, where method 2500 ends (because the entire MPLS packet is processed). Box 2522 checks whether there are more bytes to be processed in the payload of the transport layer protocol. If there are more bytes to be processed in the payload of the transport layer protocol, method 2500 returns to box 2508 to process the next MPLS packet (if any). If there are no more bytes to be processed in the payload of the transport layer protocol, the method proceeds to box 2599, where method 2500 ends.
[0151] Various example embodiments are configured to use TCP as a reliable transport layer for tunneling of MPLS overlay packets. This provides an implementation of a reliable MPLS overlay as MPLS-in-TCP.
[0152] Fig.26 An example embodiment of an MPLS-in-TCP encapsulation format for implementing a reliable MPLS overlay as MPLS-in-TCP is depicted. The MPLS-in-TCP encapsulation format 2600 includes a payload, an MPLS label stack on top of the payload, an MCH on top of the MPLS label stack, a TCP header on top of the MCH, an IP header on top of the TCP header, and a data link header on top of the IP header. The TCP header (i.e., the TCP header of the TCP connection used as a tunnel by the MPLS overlay) includes a number of fields (at least some of which will be discussed further below).
[0153] The TCP header includes a source port field that includes a local identifier for the TCP connection.
[0154] The TCP header includes a destination port field that includes a new port number to indicate that the TCP tunnel payload is an MPLS packet. For example, the value 6637 can be used to indicate that the payload of the TCP tunnel is an MPLS packet. Note that if standardized in the IETF, this value can be retained in the TCP port number registry maintained by IANA. Note that implementations can use any unused alternative port number to indicate the payload as an MPLS packet. For example, the port number in TCP can be administratively configured across LSRs participating in the MPLS overlay to indicate the payload as an MPLS packet.
[0155] The TCP header includes various other fields that will be set based on the configuration and dynamic parameters of the TCP connection.
[0156] Note that since TCP is a stream-oriented protocol, the MPLS packet (ie, payload and MPLS label stack) is encapsulated by the MCH.
[0157] Fig. 27 An example embodiment of a method for configuring a TCP connection for MPLS-in-TCP is depicted. The method 2700 may be based on Fig.13 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 2700 may be executed simultaneously or in different Fig. 27 . The inputs to method 2700 include: (1) the IP address of the remote LSR, and (2) the TCP connection parameters. Note that when an LSR discovers that a new TCP connection needs to be established to tunnel MPLS overlay packets, the LSR determines the IP address of the remote LSR with which the TCP connection is to be established, and determines the following information about the new TCP connection (e.g., TCP connection parameters, QoS information, etc.). At box 2701, method 2700 begins. Box 2702 selects a configured local IP address that can be used to send and receive tunnel traffic, and then method 2700 proceeds to box 2704. Box 2704 allocates a local TCP port number to be used for the new connection so that the number is not currently used by an existing TCP connection, and then method 2700 proceeds to box 2706. Box 2706 binds the local TCP port to MPLS as its payload / application, and then method 2700 proceeds to box 2708. Box 2708 generates a TCP connection request from the local port at the local IP address to the port number MPLS (e.g., 6637) at the remote IP address. If the TCP connection request is rejected, the LSR should take action to limit unnecessary repeated attempts to establish a connection. For example, the LSR may wait 60 seconds before attempting to reestablish a connection. From box 2708, method 2700 proceeds to box 2710. Box 2710 checks whether the remote LSR accepts the TCP connection request (i.e., successfully). If the remote LSR accepts the TCP connection request, method 2700 proceeds to box 2712, otherwise method 2700 proceeds to box 2714. Box 2714 declares a failure to configure the connection, and then method 2700 proceeds to box 2799, where method 2700 ends. Box 2712 performs a post-connection follow-up. Note that box 2712 can be completed by Fig.28 This is achieved by the method in Fig.14 From block 2712, method 2700 proceeds to block 2799, where method 2700 ends. At block 2799, method 2700 ends.
[0158] Fig.28 An example embodiment of a method for performing a post-connection follow-up for a TCP connection of MPLS-in-TCP is depicted. Note that the method 2800 may be performed by an LSR that initiates a TCP connection. The method 2800 may be used to provide Fig. 27 The method 2800 may be based on block 2712 of the method 2700. Fig.14 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 2800 may be executed simultaneously or in different Fig.28 . The input to method 2800 includes: (1) an operational TCP connection, and (2) TCP connection parameters for the TCP connection. At box 2801, method 2800 begins. Box 2802 applies the input TCP connection parameters to the new connection, and then method 2800 proceeds to box 2804. Box 2804 checks whether the MOIF requirements are configured in the LSR. If the MOIF requirements are configured in the LSR, method 2800 proceeds to box 2806, otherwise method 2800 proceeds to box 2899, at which method 2800 ends. Box 2806 constructs the MOIF and fills in the necessary fields. Note that box 2806 can be completed by Fig.15 . From box 2806, method 2800 proceeds to box 2808. Box 2808 sends the MOIF over the TCP connection, and then method 2800 proceeds to box 2810. Box 2810 waits for a MOIF response from the peer LSR. Note that an implementation may wait at least 90 seconds (or any other suitable length of time) for a MOIF response. To wait, the LSR may start a timer for the duration of the MOIF response it expects, and associate the timer with the transport connection (although it will be appreciated that other mechanisms are contemplated). From box 2810, method 2800 proceeds to box 2899, where method 2800 ends. At box 2899, method 2800 ends.
[0159] Fig.29 An example embodiment of a method for use when a MOIF response is not received within a predefined time period for a TCP connection is depicted. Note that a situation in which the initiating LSR fails to receive a MOIF response within a predefined time period may be determined where Fig.28 The timeout set at block 2810 of 2810 expires if a MOIF response is not received. It should be noted that the determination that a packet received on a TCP connection is a MOIF response can be used as a Fig.36 Note that when the initiating LSR receives a MOIF response from a peer LSR, the initiating LSR may perform Fig.17It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 2900 may be executed simultaneously or in different Fig.29 . The input of method 2900 includes a notification about the expiration of the MOIF wait timer. At block 2901, method 2900 starts. Block 2902 retrieves the TCP connection associated with the timer, and then method 2900 proceeds to block 2904. Block 2904 closes the TCP connection, and then method 2900 proceeds to block 2999. At block 2999, method 2900 ends.
[0160] Fig.30 An example embodiment of a method for configuring a TCP connection listener is depicted. Note that an LSR may use method 3000 to listen for TCP connection requests for MPLS-in-TCP (e.g., listen for new TCP connection requests for MPLS as a user protocol). Note that the MPLS-in-TCP connection listener may be implemented by any LSR that configures an MPLS overlay. Method 3000 may be based on Fig.18 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 3000 may be executed simultaneously or in different Fig.30 . At block 3001, method 3000 begins. Block 3002 opens a TCP port (e.g., 6637) for MPLS, and then method 3000 proceeds to block 3004. Block 3004 binds the port to a local IP address in the LSR, and then begins listening for incoming connection requests for the port on that address. From block 3004, method 3000 proceeds to block 3099, where method 3000 ends. At block 3099, method 3000 ends.
[0161] Fig.31 An example embodiment of a method for processing an incoming TCP connection request is depicted. Note that the incoming connection request may be received on a TCP port of an MPLS. The method 3100 may be based on Fig.19 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 3100 may be executed simultaneously or in different Fig.31. The input to method 3100 includes a connection request with the following associated parameters: (1) a source IP address, which is the IP address of the remote LSR that sends the connection request, (2) a destination IP address, which is the IP address of the local LSR to which the connection request is sent, (3) a source port, which is the TCP port at the remote LSR, and (4) a destination port, which is the TCP port of the LSR to which the connection request is sent. At box 3101, method 3100 begins. Box 3102 checks whether the destination port is open in TCP in the LSR. If the destination port is not open, method 3100 proceeds to box 3112, otherwise method 3100 proceeds to box 3104. Box 3112 rejects the connection using an appropriate TCP mechanism (e.g., a rejection notification in TCP to the remote LSR according to the method). From box 3112, method 3100 proceeds to box 3199, at which box 3199, method 3100 ends. Frame 3104 checks whether the TCP destination port is MPLS (e.g., 6637). If the TCP destination port is not MPLS, the method 3100 proceeds to frame 3114, otherwise the method 3100 proceeds to frame 3106. Frame 3114 processes the connection request according to the type associated with the port, and then the method 3100 proceeds to frame 3199, at which the method 3100 ends. Frame 3106 checks whether the connection from the remote LSR (e.g., from the source IP address) is allowed. It should be understood that various factors can be used to determine whether the LSR should allow incoming connections. For example, the LSR can adopt a method to discover all potential remote LSRs that can participate in the MPLS coverage with the LSR. If the connection from the remote LSR is not allowed, the method 3100 proceeds to frame 3112, otherwise the method 3100 proceeds to frame 3108. Frame 3112 uses an appropriate TCP mechanism to reject the connection (e.g., a rejection notification to the remote LSR in TCP according to the method). From block 3112, method 3100 proceeds to block 3199, where method 3100 ends. Block 3108 accepts the TCP connection request, which means that the state of the TCP connection is created with the tuple {source IP address, destination IP address, source port, destination port} as the key, and method 3100 then proceeds to block 3110. Block 3110 performs the post-connection follow-up, and method 3100 proceeds to block 3199, where method 3100 ends. Note that block 3110 can be created by Fig. 20 2000, if configured to receive a MOIF, causes the LSR to wait for a MOIF from a peer LSR. Note that the determination that a packet received on a connection is a MOIF is made as part of the normal processing of MPLS overlay packets on a TCP connection (e.g., using Fig.36Note that if the LSR fails to receive the MOIF within a predefined period of time (e.g., within Fig. 20 The timeout set at box 2004 in the example expires), the LSR may execute Fig.29 method 2900 (i.e., the handling is common in both the initiating and receiving LSRs). Note that when a MOIF is received from a peer LSR, then the LSR may perform Fig.21 Method 2100.
[0162] It should be understood that although the example embodiments are presented primarily with respect to an example embodiment in which the LSR accepts and establishes TCP connections by listening for new TCP connection requests using a well-known TCP port for MPLS (e.g., 6637 or other suitable port), the LSR may also accept and establish TCP connections with other TCP port numbers other than the well-known TCP port for MPLS (e.g., a port number configured by a network administrator for MPLS).
[0163] Note that simultaneous TCP connection establishment can be supported. If two peer LSRs send TCP connection requests to each other at the same time, two TCP connections are formed. Here, the connection establishment is performed on two connections, resulting in the formation of two TCP tunnels between the LSRs.
[0164] Figures 32A-32BAn example embodiment of a method for configuring an MPLS overlay on a TCP connection to form an MPLS-in-TCP overlay is depicted. Note that when an LSR prepares to establish an MPLS-in-TCP overlay with a remote LSR, it determines the IP address of the remote LSR with which the TCP connection is to be established, and then determines the TCP connection parameters and QoS information for the TCP connection. Method 3200 may be based on the general method 2200 presented in FIG. 22 . It should be understood that, although primarily presented as being executed serially, at least a portion of the blocks of method 3200 may be executed simultaneously or in an order different from that presented in FIG. 32 . The inputs to method 3200 include the outgoing MPLS label of the LSP / overlay, the IP address of the remote LSR of the LSP / overlay, and the TCP connection parameters required for the overlay. At block 3201 , method 3200 begins. Block 3202 checks whether there is an existing TCP connection established with the remote LSR for the MPLS overlay. If there is no existing TCP connection established with the remote LSR for MPLS overlay, then the method 3200 proceeds to block 3214, otherwise the method 3200 proceeds to block 3204. Block 3204 obtains a TCP connection with the remote LSR that is established for MPLS overlay, and then the method 3200 proceeds to block 3206. Block 3206 retrieves TCP connection parameters for the TCP connection, and then the method 3200 proceeds to block 3208. Block 3208 checks whether the parameters requested by the overlay match the TCP connection parameters of the TCP connection. If the parameters requested by the overlay match the parameters of the TCP connection, then the method 3200 proceeds to block 3218, otherwise the method 3200 proceeds to block 3210. Block 3210 checks whether there are more existing TCP connections with the remote LSR for MPLS overlay. If there are more TCP connections, then the method 3200 proceeds to block 3212, otherwise the method 3200 proceeds to block 3214. Block 3212 acquires the next TCP connection with the remote LSR that has been established for the MPLS overlay, and method 3200 then returns to block 3206 to repeat the subsequent blocks for the next TCP connection. When a new TCP connection is needed, block 3214 is reached. Block 3214 configures a new TCP connection with the remote LSR for the MPLS overlay. The TCP connection is configured using the input TCP connection parameters. Block 3214 passes Fig. 27The method in is implemented. The method then proceeds to box 3216. Box 3216 checks whether the TCP connection is successfully established. If the TCP connection has not been successfully established, the method 3200 proceeds to box 3224, otherwise the method 3200 proceeds to box 3218. Box 3224 declares and handles the failure to configure the MPLS overlay. Box 3218 allocates an LSP identifier to uniquely identify the MPLS overlay in the LSR. The LSP identifier binds the TCP connection, (multiple) outgoing labels, and other parameters associated with the MPLS overlay. From box 3218, the method 3200 proceeds to box 3220. Box 3220 sets the outgoing MPLS label of the LSP / overlay to the outgoing label of the LSP identifier, and the method 3200 proceeds to box 3222. Box 3222 sets the TCP connection to a tunnel associated with the LSP identifier, and the method 3200 proceeds to box 3299, where the method 3200 ends. At box 3299, the method 3200 ends.
[0165] Fig.33 An example embodiment of a method for transporting packets over MPLS-in-TCP is depicted. Method 3300 may be based on Fig.23 (except that block 2310 may not be implemented because there are no dynamic parameters of the TCP header in the MPLS-in-TCP packet). It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 3300 may be executed simultaneously or in different Fig.33 . The input of method 3300 includes a packet to be transmitted and an LSP identifier identifying an LSP or MPLS coverage. At box 3301, method 3300 starts. Box 3302 retrieves the status of the outgoing label of the LSP by the LSP identifier, and method 3300 proceeds to box 3304. Box 3304 obtains the TCP tunnel configured for the LSP identifier, that is, a reliable transport layer connection, and method 3300 then proceeds to box 3306. Box 3306 pushes the outgoing label of the LSP onto the packet, and then method 3300 proceeds to box 3308. Box 3308 sets the S bit, EXP, and TTL fields in the pushed label as needed for the packet. For example, if the packet already has another label before the outgoing label of the pushed LSP, the S bit is set to 0, otherwise the S bit is set to 1. The TTL and EXP values can be set by the policy configured for the LSP. For example, if the policy requires a consistent EXP value for all packets on the LSP, the configured EXP value is set. If the policy has a mapping of EXP values based on certain fields in the native header of the packet, the mapped EXP value is set. From box 3308, the method 3300 proceeds to box 3310. Box 3310 sends the MPLS packet on the TCP connection. After box 3310, the method terminates.
[0166] Fig.34 An example embodiment of a method for transmitting MPLS packets over a TCP connection is depicted. Method 3400 may be used to provide Fig.33 Block 3310 of method 3300 and may be based on Fig.24 (except that the TCP header has no dynamic parameters). It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 3400 may be executed simultaneously or in different Fig.34 . The input of method 3400 includes the MPLS packet to be transmitted. At box 3401, method 3400 starts. Box 3402 adjusts the MPLS packet, which means that the TCP layer prepares the packet according to the connection state (for example, if the packet needs to be further segmented, the packet is segmented into multiple segments, congestion control and / or flow control are applied to the segments as needed, etc., and various combinations thereof). From box 3402, method 3400 proceeds to box 3404. Box 3404 checks whether one or more TCP segments of the packet are ready to be sent. For example, due to congestion, flow control, etc., the TCP segment may not be ready to be sent. If the TCP segment is not ready to be sent, the TCP segment will be sent later when the congestion is cleared, and then method 3400 proceeds to box 3499, at which method 3400 ends. If the TCP segment is ready to be sent, method 3400 proceeds to box 3406. Frame 3406 pushes the TCP header to (multiple) segments. The source port and destination port fields in the TCP header are filled in according to the connection parameters. From frame 3406, method 3400 then proceeds to frame 3408. Frame 3408 pushes the IP header onto the packet. The source address and destination address in the IP header are filled in according to the local and remote IP addresses of the TCP connection (the addresses are provided by frame 3406). The protocol field (if it is an IPv4 header) or the next header field (if it is an IPv6 header) is set to a value (e.g., 6 or other suitable values) to indicate that the payload is TCP. From frame 3408, method 3400 proceeds to frame 3410. Frame 3410 performs an IP routing table lookup on the destination address in the IP header, which results in the next hop of the packet, and then method 3410 proceeds to frame 3412. Block 3412 pushes a data link layer header onto the packet, which is required to send the packet to the next hop on the link, and method 3400 then proceeds to block 3414. Block 3414 sends the packet to the next hop on the line, and method 3400 then proceeds to block 3499, where method 3400 ends.
[0167] Fig.35An example embodiment of a method for receiving and processing MPLS-in-TCP packets is depicted. It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 3500 may be executed simultaneously or in different Fig.35 The input of method 3500 includes MPLS-in-TCP packets received on the line.
[0168] At box 3501, method 3500 begins.
[0169] Box 3502 parses and processes the data link header above the packet, and then method 3500 proceeds to box 3504.
[0170] Block 3504 checks whether the data link layer indicates that the packet is local, meaning that the LSR executing method 3500 is the termination point of the data link on which the packet arrives. For example, if the data link header is an Ethernet header, and if the destination MAC address in the Ethernet header is local to the LSR, then the Ethernet link terminates at the LSR. If the packet is local at the data link layer, then method 3500 proceeds to block 3506, otherwise method 3500 proceeds to block 3524. Block 3524 performs the required processing of the packet at the data link layer as a non-local packet, which may result in further forwarding of the packet at the data link layer, and then method 3500 proceeds to block 3599, where method 3500 ends.
[0171] When the LSR is the end of the data link, block 3506 removes the data link header from the packet, and method 3500 proceeds to block 3508. Block 3508 checks whether the payload indicated by the data link header is IP. For example, if the data link header is an Ethernet header, its Ethertype field indicates the payload type. If the Ethernet header has a VLAN tag, the Ethertype field in the last VLAN tag indicates the payload type. If the payload is IP, method 3500 proceeds to block 3510, otherwise method 3500 proceeds to block 3526. Block 3526 treats the packet as a non-IP packet, and method 3500 proceeds to block 3599, where method 3500 ends.
[0172] Frame 3510 processes the IP packet in the IP layer based on its IP header. For example, the destination address of the IP header is looked up in the IP routing table to make a forwarding decision for the IP packet. From frame 3510, method 3500 proceeds to frame 3512. Frame 3512 checks whether the destination address is local (e.g., whether the IP routing table lookup matches the destination address as the IP address locally configured in the LSR). If the destination address is not local, then method 3500 proceeds to frame 3528, otherwise method 3500 proceeds to frame 3514. Frame 3528 processes the packet as a non-local IP packet, such as by forwarding the packet to the next hop associated with the matching routing entry from the IP routing table lookup, and then method 3500 proceeds to frame 3599, where method 3500 ends.
[0173] Block 3514 removes the IP header from the IP packet because it is a local packet, and the method 3500 proceeds to block 3516. Block 3516 checks whether the packet is a TCP segment (e.g., if the IP header is IPv4, then check whether the Protocol field in the IPv4 header is set to 6 (TCP), or if the IP header is IPv6, then check whether the Next Header field in the IPv6 header is set to 6 (TCP)). If the packet is not a TCP segment, the method 3500 proceeds to block 3530, otherwise the method 3500 proceeds to block 3518. Block 3530 handles packets of the corresponding IP protocol type, and the method 3500 proceeds to block 3599, where the method 3500 ends.
[0174] Block 3518 looks up a TCP connection associated with the TCP segment based on the source and destination IP addresses in the IP header (e.g., the addresses are passed along the flow after the IP header is removed in block 3514) and based on the source and destination ports in the TCP header. From block 3518, method 3500 proceeds to block 3520. Block 3520 checks whether a matching TCP connection is found. If a matching TCP connection is not found, method 3500 proceeds to block 3532, otherwise method 3500 proceeds to block 3522. Block 3532 discards the packet as an error packet, and method 3500 then proceeds to block 3599, where method 3500 ends.
[0175] Box 3522 performs TCP layer processing on the TCP segment and, if the TCP connection is bound to MPLS, processes MPLS packets from the byte stream received in the connection. From box 3522, method 3500 proceeds to box 3599, where method 3500 ends.
[0176] At box 3599, method 3500 ends.
[0177] Fig.36An example embodiment of a method for processing an MPLS-in-TCP packet from a TCP segment of a TCP connection is depicted. The method 3600 may be used to provide Fig.35 Block 3522 of method 3500 and may be based on Fig.25 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 3600 may be executed simultaneously or in different Fig.36. The input of method 3600 includes a TCP segment received on a TCP connection and an indication of the TCP connection. Note that the TCP segment received on the TCP connection can be received by the TCP layer of the LSR on top with a TCP header. Note that TCP is a stream byte oriented protocol, so the bytes of the entire packet may not arrive together in the connection (i.e., a block of the packet may arrive on multiple TCP segments). Note that a TCP connection can be indicated by a tuple {source IP address in IP header, destination IP address in IP header, source port in TCP header, destination port in TCP header}. At box 3601, method 3600 starts. Box 3602 checks whether the destination port of the TCP connection is bound to MPLS. If the destination port of the TCP connection is not bound to MPLS, method 3600 proceeds to box 3620, otherwise method 3600 proceeds to box 3604. Box 3620 handles TCP segments for non-MPLS applications, and then method 3600 proceeds to box 3699, and method 3600 ends. Box 3604 performs TCP-related processing on the TCP segment. For example, if this is not the next TCP segment expected for the connection, the TCP segment can be queued internally until all previous TCP segments are received and processed, an acknowledgment can be sent for the processed TCP segment, and so on. Here, for simplicity, it is assumed that the incoming TCP segment is the next segment expected for the connection, the TCP header is removed from the TCP segment, and the payload of the TCP segment is appended to the received byte stream. From box 3604, method 3600 proceeds to box 3606. Box 3606 parses the first 4 octets of the received stream into an MCH, and then method 3600 proceeds to box 3608. Box 3608 checks whether the received stream has at least the number of octets specified in the length field in the MCH (e.g., at least the entire next MPLS packet). If the flow does not have the entire packet, method 3600 proceeds to box 3699, where method 3600 ends, otherwise method 3600 proceeds to box 3610. Box 3610 extracts the number of octets specified in the length field of the MCH plus the 4 octets that constitute the MPLS packet encapsulated by the MCH from the received flow, and then removes the MCH to generate an MPLS packet. From box 3610, method 3600 proceeds to box 3612. Box 3612 checks whether the packet is a MOIF. If the packet is a MOIF, method 3600 proceeds to box 3614, otherwise method 3600 proceeds to box 3616. Box 3616 processes the packet as an MPLS packet, and then method 3600 proceeds to box 3618. Note that if the LSR is the initiator of the connection, box 3614 can be processed by Fig.17 Method 1700 is implemented, otherwise method 3614 can be Fig.21From block 3614, method 3600 proceeds to block 3618. Block 3618 checks whether there are more bytes to be read in the TCP segment. If there are more bytes to be read in the TCP segment, method 3600 returns to block 3606 to process the next MPLS packet in its payload, otherwise method 3600 proceeds to block 3699, where method 3600 terminates.
[0178] Note that whether the top label of an encapsulated MPLS packet is downstream assigned or upstream assigned can be determined based on various criteria. For example, if the tunnel destination IP address is a unicast address, the top label can be downstream assigned. For example, if the tunnel destination IP address is an IP multicast address, then all encapsulated MPLS packets in a particular tunnel have a downstream assigned label at the top of the stack, or all encapsulated MPLS packets in a particular tunnel have an upstream assigned label at the top of the stack. The method of determining this for a particular tunnel can be performed in various ways. Without any knowledge about a particular tunnel, it can be assumed that the label is upstream assigned.
[0179] Note that after receiving TCP encapsulated packets, intermediate routers can balance these packets based on the hash value of the TCP packet's five-tuple (i.e., source IP address, destination IP address, source port, destination port, and protocol). Note that since MPLS overlays multiplexed on TCP tunnels share the same five-tuple, packets across overlays cannot usually be load balanced. If load balancing between MPLS overlays is required, multipath TCP (MP-TCP) can be used as a reliable transport layer.
[0180] Note that in order to provide efficient exchange of MPLS-in-TCP packets, certain TCP connection parameters (e.g., TCP Selective Acknowledgement (SACK) option, TCP Window Scaling option, Protected Packet Sequence Number (PAWS) capability, TCP_NODELAY option, etc., and various combinations thereof) may be used. The Selective Acknowledgement option allows the receiver to acknowledge multiple lost packets in a single ACK, thereby enabling faster recovery. LSRs can negotiate the use of TCP SACK on TCP connections for MPLS and use it to recover faster from lost packets and holes in the TCP sequence number space. The TCP Window Scaling option allows receivers to advertise TCP window sizes larger than the 16-bit limit. It is necessary to allow data in fat networks to fill available pipes. This also means buffering on the TCP sender that matches the (bandwidth * delay) product of the TCP connection. LSRs use a locally available mechanism to set a window size that matches available local buffer resources and expected throughput. The PAWS capability can be used to ensure that TCP sequence numbers do not wrap around within the timeout window. The TCP_NODELAY option can be used to disable the Nagle algorithm. It should be understood that various other TCP connection parameters may be used to provide efficient exchange of MPLS-in-TCP packets.
[0181] Various example embodiments are configured to use SCTP as a reliable transport layer for tunneling of MPLS overlay packets. This provides an implementation of a reliable MPLS overlay as MPLS-in-SCTP. In general, SCTP is a computer network communication protocol standardized by the IETF that runs at the transport layer and acts similarly to TCP. It is understood that SCTP can be characterized as message-oriented, which means that it transmits a sequence of messages (each message is a group of bytes) instead of transmitting an uninterrupted stream of bytes like TCP. In SCTP, as in UDP, the sender sends a message in one operation, and the message is delivered to the receiving application process in one operation. In contrast, TCP is a stream-oriented protocol that transmits byte streams reliably and in order. SCTP applications submit the data they want to transmit in messages (byte groups) to the SCTP transport layer. SCTP puts messages and control information into separate blocks (data blocks and control blocks), each of which is identified by a block header. SCTP can divide a message into multiple data blocks, but each data block only includes data from one message. SCTP bundles blocks into SCTP packets. The SCTP packet submitted to the IP layer includes a packet header, an SCTP control block (as needed), followed by an SCTP data block (when available). In SCTP, the connection between two endpoints is called an "association", so this standard term is used in this article. The term "stream" in SCTP refers to the ability of SCTP to transmit several independent streams of an application in parallel (for example, transmitting a web page image together with the web page text). In essence, the multi-stream transmission capability of SCTP involves bundling several independent data streams within an application into a single SCTP connection. Independent block streams can be bundled together in a single SCTP message, where the payload of the stream can be packaged into one or more data blocks in the message, and each data block is encoded using a stream identifier, based on which the stream of its payload is identified. Also note that in SCTP, an endpoint can use multiple IP addresses, which enables the injection of SCTP packets across multiple available paths between endpoints. When SCTP is used to tunnel MPLS overlay packets, the endpoint is an LSR.
[0182] Fig.37 An example embodiment of an MPLS-in-SCTP encapsulation format for implementing a reliable MPLS overlay as MPLS-in-SCTP is depicted.
[0183] The MPLS-in-SCTP encapsulation format 3700 includes a payload, an MPLS label stack on top of the payload, an SCTP header on top of the MPLS label stack, an IP header on top of the SCTP header, and a data link header on top of the IP header. Note that since SCTP is a datagram-oriented protocol, the MCH is not required to encapsulate the MPLS payload and therefore the MCH is not part of the stack of the MPLS-in-SCTP encapsulation format 3700. The SCTP header is used for the SCTP association that the MPLS overlay uses as a tunnel. Note that in Fig.37 In , SCTP packets are described only in conceptual form. There is no single SCTP header, but rather there are several SCTP-specific headers interspersed throughout the SCTP packet. Fig.37 In the SCTP transport, the payload, MPLS label stack, and SCTP header constitute an "SCTP packet" for reliable tunneling of MPLS packets.
[0184] Generally speaking, an SCTP packet consists of two basic parts: a common header (occupying the first 12 bytes) and a bunch of blocks (occupying the rest of the packet). The common header includes a source port field, a destination port field, a verification tag field, and a checksum field. The source port is the port number of the sender, which the receiver uses in conjunction with the source IP address to identify the SCTP association to which the SCTP packet belongs. The destination port is the port number of the receiver, which is used to demultiplex the STCP packet to the correct application of SCTP (i.e., the upper layer of SCTP). The verification tag is negotiated between peers during the establishment of the SCTP association to verify the sender of the packet. The checksum uses the CRC32 algorithm on the SCTP packet to ensure data integrity. Each block starts with a block header, which includes a one-byte type identifier (15 block types are defined for SCTP, and at least 5 more are defined by additional extensions), eight flag bits, and a 2-byte length field, and the data makes up the rest of the block. Block types can be roughly divided into control blocks and data blocks, and the header formats of control blocks and data blocks are respectively described in Fig.38 and Fig.39 Given in.
[0185] Fig.38 An example embodiment of an SCTP control block header for an SCTP packet is depicted. The SCTP control block header 3800 carries control information related to the SCTP packet and follows the TLV (Type Length Value) format. The parameter type field indicates the type of control information, and therefore the length and value are specified in the SCTP specification.
[0186] Fig.39An example embodiment of an SCTP data chunk header for an SCTP packet is depicted. The SCTP data chunk header carries the actual data from the application (upper layer) using SCTP. The ability to carry multiple data chunks in a single SCTP packet supports multiplexing of multiple streams within a user application. The payload from a stream can be sent in one or more chunks in the SCTP packet. The stream identifier field in the data chunk header is used to demultiplex data chunks in the SCTP packet into streams in the user application.
[0187] Note that various aspects of SCTP can be configured to support reliable delivery tunneling of MPLS packets over SCTP.
[0188] The destination port field in the common header includes a new port number to indicate that the SCTP tunnel payload is an MPLS packet. For example, the value 6640 may be used to indicate that the payload of the SCTP tunnel is an MPLS packet, but it will be appreciated that any other suitable available value may be used. Note that the value ultimately used to indicate that the payload of the SCTP tunnel is an MPLS packet may be retained in the Name and Transport Protocol Port Number Registry maintained by IANA. Note that if standardized values are not used, the port numbers in SCTP may be administratively configured across LSRs participating in the MPLS overlay to indicate that the payload is an MPLS packet.
[0189] An MPLS overlay on an SCTP association may be assigned a unique flow identifier (ID) by the LSR. In this case, the MPLS overlay packets may be sent as independent data chunks within the SCTP packet. This means that packets from multiple MPLS overlays may be multiplexed on a single SCTP packet. Since the MPLS overlays are independent of each other, configuring each overlay as an independent flow may also eliminate head-of-line (HOL) blocking between overlays that share an SCTP association.
[0190] The flow ID space can be local to the LSRs, so each LSR can independently allocate unused flow IDs from its local space. An LSR sending an MPLS overlay packet over an SCTP association uses its allocated flow ID in the corresponding data block in the SCTP packet. For the receiving LSR, there is no correlation between the flow ID of the received data block and the corresponding MPLS overlay, because the LSR identifies the overlay based on the MPLS label stack of the MPLS packet in the payload of the data block. Flow ID value 0 will not be used for MPLS packets transporting MPLS overlays and can be reserved by two LSRs for batch management of MPLS overlays multiplexed on the SCTP association. One usage of batch management is MOIF, which will be sent in the data block with flow ID 0.
[0191] Note that various other aspects of SCTP may be configured to support reliable delivery tunneling of MPLS packets over SCTP.
[0192] Fig.40 An example embodiment of a method for configuring SCTP associations for MPLS-in-SCTP is depicted. Method 4000 may be based on Fig.13 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 4000 may be executed simultaneously or in different Fig.40. The input to method 4000 includes the IP address of the remote LSR. Note that when the LSR discovers that a new SCTP association needs to be established to tunnel MPLS overlay packets, the LSR determines the IP address of the remote LSR with which the SCTP association is to be made, and determines information about the new SCTP association (e.g., QoS information, which is used to encapsulate the appropriate QoS marking in the IP header of the SCTP packet). At box 4001, method 4000 begins. Box 4002 selects one or more configured local IP addresses that can be used to send and receive tunnel traffic, and then method 4000 proceeds to box 4004. Box 4004 assigns a local SCTP port number to be used for the new association so that the number is not currently used by an existing SCTP association, and then method 4000 proceeds to box 4006. Box 4006 binds the local SCTP port to MPLS as its payload / application, and then method 4000 proceeds to box 4008. Box 4008 generates an SCTP association request from a local IP address in a local port and a local IP address for an MPLS (6640) port number at a remote IP address. The SCTP association request is a message of the INIT block carrying SCTP. The INIT block mainly includes information on the maximum number of inbound and outbound flows allowed for association, and a list of local IP addresses to be used for association. The maximum number of inbound or outbound flows can be set to the maximum number of MPLS covers to be transmitted through the SCTP association tunnel. If the maximum number of MPLS covers to be supported is not known a priori, both the inbound flow number field and the outbound flow number field can be set to the maximum allowed value, i.e., 65535 (because the field is 16 bits in size). Note that if the SCTP association request is rejected by the remote LSR, the LSR should take action to limit unnecessary repeated attempts to establish an SCTP association. For example, the LSR can wait 60 seconds before trying to re-establish an association. From box 4008, method 4000 proceeds to box 4010. Box 4010 checks whether the remote LSR accepts the SCTP association request (meaning success). If the remote LSR accepts the SCTP association request, the LSR will receive an INIT ACK from the remote LSR. If the remote LSR accepts the SCTP association request, the method 4000 proceeds to block 4012, otherwise the method 4000 proceeds to block 4014. Block 4014 declares failure of the SCTP association configuration, and the method 4000 then proceeds to block 4099, where the method 4000 ends. Block 4012 performs post-association follow-up as needed, and the method 4000 then proceeds to block 4099, where the method 4000 ends. At block 4099, the method 4000 ends.
[0193] Fig.41An example embodiment of a method for performing a post-connection follow-up for an SCTP association of MPLS-in-SCTP is depicted. Note that the method 4100 may be performed by an LSR that initiates the SCTP association. The method 4100 may be used to provide Fig.40 Block 4012 of method 4000. Method 4100 may be based on Fig.14 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 4100 may be executed simultaneously or in different Fig.41 4100. The input to method 4100 includes an SCTP association. At block 4101, method 4100 begins. Block 4102 checks whether the MOIF requirement is configured in the LSR. If the MOIF requirement is configured in the LSR, method 4100 proceeds to block 4104, otherwise method 4100 proceeds to block 4199, where method 4100 ends. Block 4104 builds the MOIF and populates the necessary fields. Note that this block can be Fig.15 4104, method 4100 proceeds to block 4106. Block 4106 sends the MOIF over the SCTP association. The MOIF is sent as a payload of a data block in an SCTP packet, where the data block is encoded with stream ID 0. From block 4106, method 4100 proceeds to block 4108. Block 4108 waits for a MOIF response from the peer LSR. Note that an implementation may wait at least 90 seconds (or any other suitable length of time) for a MOIF response. To wait, the LSR may start a timer for the duration of its expected MOIF response and associate the timer with the SCTP association (although it will be appreciated that other mechanisms are contemplated). Note that when the initiating LSR receives a MOIF response from the peer LSR, it may perform Fig.17 Note that the determination that a packet received on a connection is a MOIF can be made as part of the general processing of associated MPLS overlay packets (e.g., based on Fig.49 From block 4108, method 4100 proceeds to block 4199, where method 4100 ends. At block 4199, method 4100 ends.
[0194] Fig.42 An example embodiment of a method for use when a MOIF response is not received within a predefined time period for an SCTP association is depicted. Note that a situation in which the initiating LSR fails to receive a MOIF response within a predefined time period may be determined where Fig.41The timeout set at block 4108 of SCTP expires if a MOIF response is not received. It should be noted that the determination that a packet received on an SCTP association is a MOIF response can be performed as part of the general processing of MPLS overlay packets received on an association (e.g., based on Fig.49 Note that when the initiating LSR receives a MOIF response from a peer LSR, the initiating LSR may perform Fig.17 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 4200 may be executed simultaneously or in different Fig.42 . The input to method 4200 includes a notification regarding the expiration of the MOIF wait timer. Block 4202 retrieves the SCTP association associated with the timer, and method 4200 proceeds to block 4204. Block 4204 closes the SCTP association, and method 4200 proceeds to block 4299, where method 4200 ends. At block 4299, method 4200 ends.
[0195] Fig.43 An example embodiment of a method for configuring an SCTP association listener is depicted. Note that an LSR may use method 4300 to listen for SCTP association requests for MPLS-in-SCTP (e.g., listen for new SCTP association requests for MPLS as a user protocol, such as by listening on an SCTP port assigned for MPLS (6640)). Note that the MPLS-in-SCTP association listener may be implemented by any LSR that configures an MPLS overlay. Method 4300 may be based on Fig.18 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 4300 may be executed simultaneously or in different Fig.43 . At block 4301, method 4300 begins. Block 4302 opens an SCTP port (e.g., 6640) for MPLS, and method 4300 proceeds to block 4304. Block 4304 binds the port to the set of local IP addresses in the LSR selected for SCTP association, and then begins listening for incoming association requests (e.g., SCTP messages with INIT blocks) on the port at any IP address. At block 4399, method 4300 ends.
[0196] Fig.44 An example embodiment of a method for processing an incoming SCTP association request is depicted. Note that the incoming association request may be received on an SCTP port of an MPLS. The method 4400 may be based on Fig.19It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 4400 may be executed simultaneously or in different Fig.44 . The input to method 4400 includes an association request having the following association parameters: (1) a source IP address, which is the IP address of the remote LSR that sends the association request, (2) a destination IP address, which is the IP address of the LSR to which the local association request is sent, (3) a source port, which is the SCTP port at the remote LSR, and (4) a destination port, which is the SCTP port of the LSR to which the association request is sent. At box 4401, method 4400 starts. Box 4402 checks whether the destination port is open in the SCTP in the LSR. If the destination port is not open in the SCTP in the LSR, method 4400 proceeds to box 4414, otherwise method 4400 proceeds to box 4404. Box 4414 rejects the association request using an appropriate SCTP mechanism (e.g., using a rejection notification in SCTP to the remote LSR according to the method). For example, the LSR may send an SCTP message back to the sender with an ABORT block to reject the SCTP association request. From block 4414, method 4400 proceeds to block 4499, where method 4400 ends. Block 4404 checks whether the SCTP destination port is MPLS (e.g., 6640). If the SCTP destination port is not MPLS, method 4400 proceeds to block 4412, otherwise method 4400 proceeds to block 4406. Block 4412 processes the SCTP association request according to the payload protocol type associated with the port, and then the method proceeds to block 4499, where method 4400 ends. Block 4406 checks whether the SCTP association request from the remote LSR (e.g., from the source IP address) is allowed. Note that various factors can be used to determine whether the LSR should allow an incoming SCTP association. For example, the LSR can adopt a method to discover all potential remote LSRs and their local IP address sets that can participate in the MPLS coverage with the LSR. If the SCTP association request is not allowed, method 4400 proceeds to block 4414, otherwise method 4400 proceeds to block 4408. Block 4414 rejects the association request using appropriate SCTP means, i.e., a rejection notification to the remote LSR according to the method in SCTP, such as sending an SCTP message to the sender using an ABORT control block. The method terminates after block 4414. Block 4408 accepts the SCTP association request, which means that the state of the SCTP association is created with the tuple {remote IP address set, local IP address set, source port, destination port} as the key, and then method 4400 proceeds to block 4410. Block 4410 performs the post-association follow-up (if any), and then method 4410 proceeds to block 4499, where method 4400 ends. Block 4410 can be completed by Fig. 20 The method 2000 of SCTP is implemented by causing the LSR to wait for a MOIF from a peer LSR if configured to receive a MOIF. Note that the determination that a packet received on an SCTP association is a MOIF can be made as part of the general processing of MPLS overlay packets on the association (e.g., using Fig.49 Note that if the LSR fails to receive the MOIF within a predefined period of time (e.g., within Fig. 20 The timeout set at box 2004 in the example expires), the LSR may execute Fig.42 method 4200 (i.e., the processing is common in both the initiating and receiving LSRs). Note that when a MOIF is received from a peer LSR, then the LSR may perform Fig.21 Method 2100.
[0197] FIG. 45 depicts an example embodiment of a method for configuring an MPLS overlay on an SCTP association to form an MPLS-in-SCTP overlay. Note that to configure an MPLS overlay on an SCTP association, an LSR may assign a unique non-zero flow ID to the MPLS overlay on an SCTP association with a remote LSR. For a transmitting LSR, the flow ID space is local to the LSR, so each LSR may independently assign an unused flow ID from its local space, and the LSR sending the MPLS overlay packet over the SCTP association uses its assigned flow ID in the corresponding data block in the SCTP packet. For a receiving / remote LSR, there is no correlation between the flow ID of the received data block and the corresponding MPLS overlay because the LSR identifies the overlay based on the MPLS label stack of the MPLS packet in the payload of the data block. It should be understood that, while primarily presented as being executed serially, at least a portion of the blocks of method 4500 may be executed simultaneously or in an order different from that presented in FIG. 45. The inputs to method 4500 include the outgoing MPLS label of the LSP / overlay and the IP address of the remote LSR of the LSP / overlay. At block 4501, method 4500 begins. Block 4502 checks for MPLS-in-SCTP whether there is an existing SCTP association with a remote LSR. If there is no existing SCTP association with a remote LSR for MPLS-in-SCTP, method 4500 proceeds to block 4512, otherwise method 4500 proceeds to block 4504. Block 4504 retrieves a first SCTP association with a remote LSR for MPLS-in-SCTP, and method 4500 proceeds to block 4506. Block 4506 checks whether there is at least one unused non-zero stream ID available in the SCTP association. If there is at least one unused non-zero stream ID available in the SCTP association, the SCTP association can be used for overlay, and method 4500 proceeds to block 4516. If there is not at least one unused non-zero stream ID available in the SCTP association, the SCTP association cannot be used for overlay, and method 4500 proceeds to block 4508. Block 4508 checks for MPLS-in-SCTP to see if there are more SCTP connections with the remote LSR. If there are more SCTP connections with the remote LSR for MPLS-in-SCTP, then method 4500 proceeds to block 4510, otherwise method 4500 proceeds to block 4512. Block 4512 retrieves the next SCTP association with the remote LSR for MPLS-in-SCTP, and then method 4500 returns to block 4506 to evaluate the feasibility of the next SCTP association as a tunnel to be used for overlay. Block 4512 configures a new SCTP association with the remote LSR for MPLS-in-SCTP. Note that block 4512 may be configured by Fig.334500. From box 4512, method 4500 proceeds to box 4514. Box 4514 checks whether the SCTP association has been successfully established. If the SCTP association has not been successfully established, method 4500 proceeds to box 4526, otherwise method 4500 proceeds to box 4516. Box 4526 declares and handles the failure to configure the MPLS overlay, and then proceeds to box 4599, where method 4500 ends. Box 4516 allocates an LSP identifier to uniquely identify the MPLS overlay in the LSR. The LSP identifier binds the SCTP association, (multiple) outgoing labels, and other parameters associated with the MPLS overlay. From box 4516, method 4500 proceeds to box 4518. Box 4518 sets the outgoing MPLS label of the LSP / overlay to the outgoing label of the LSP identifier, and method 4500 proceeds to box 4520. Block 4520 sets the SCTP association to the tunnel associated with the LSP identifier, and the method 4500 proceeds to block 4522. Block 4522 assigns an unused stream ID in the SCTP association as an identifier for the stream to be used to exchange packets for the overlay, and the method 4522 proceeds to block 4524. Block 4524 creates a mapping between the LSP identifier and the stream ID of the overlay, and the method 4500 proceeds to block 4599, where the method 4500 ends. At block 4599, the method 4500 ends.
[0198] Fig.46 An example embodiment of a method for transporting packets over MPLS-in-SCTP is depicted. Method 4600 may be based on Fig.23 (except that block 2310 may not be implemented because there are no dynamic parameters of the SCTP header in the MPLS-in-SCTP packet). It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 4600 may be executed simultaneously or in different Fig.46. The input of method 4600 includes a packet to be transmitted and an LSP identifier identifying an LSP or MPLS coverage. At box 4601, method 4600 starts. Box 4602 retrieves the state of the outgoing label of the LSP by the LSP identifier, and method 4600 proceeds to box 4604. Box 4604 obtains the SCTP tunnel configured for the LSP identifier, and then method 4600 proceeds to box 4606. Box 4606 pushes the outgoing label of the LSP onto the packet, and method 4600 then proceeds to box 4608. Box 4608 sets the S bit, EXP, and TTL fields in the pushed label as required for the packet. For example, if the packet already has another label before the outgoing label of the pushed LSP, the S bit is set to 0, otherwise the S bit is set to 1. The TTL and EXP values can be set by the policy configured for the LSP. For example, if the policy requires the use of a consistent EXP value for all packets on the LSP, the configured EXP value is set. If the policy has a mapping based on the EXP value of certain fields in the native header of the packet, the mapped EXP value is set. From block 4608, the method 4600 proceeds to block 4610. Block 4610 sends the MPLS packet in the context of the LSP identifier on the SCTP association. From block 4610, the method 4600 proceeds to block 4699, where the method 4600 ends. At block 4699, the method 4600 ends.
[0199] Fig.47 An example embodiment of a method for transporting MPLS packets over an SCTP association is depicted. Method 4700 may be used to provide Fig.46 Block 4610 of method 4600 and may be based on Fig.24 (except that block 2410 may not be implemented because there are no dynamic parameters of the SCTP header in the MPLS-in-SCTP packet). It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 4700 may be executed simultaneously or in different Fig.47. The input of method 4700 includes the MPLS packet to be transmitted and the LSP identifier identifying the LSP or MPLS coverage. At box 4701, method 4700 starts. Box 4702 obtains the stream ID mapped to the LSP identifier, and then method 4700 proceeds to box 4704. Box 4704 builds an SCTP data block for the stream ID using the MPLS packet as its payload, and then method 4700 proceeds to box 4706. Box 4706 checks whether one or more blocks are pending on the SCTP association. Since SCTP can bundle multiple blocks on a single SCTP packet, previous blocks may have been queued waiting to be sent. Secondly, previous frames may also be pending due to congestion, flow control, etc. If there are no pending frames, method 4700 proceeds to box 4708, otherwise method 4700 proceeds to box 4710. Box 4708 creates an empty stack of SCTP blocks, and then method 4700 proceeds to box 4710. Box 4710 pushes the data blocks (including MPLS packets) into the SCTP block stack, and then method 4700 proceeds to box 4712. Box 4712 checks whether the SCTP block stack is ready to be sent over the SCTP association. Due to congestion, flow control, etc., the SCTP block stack may not be sent immediately, or the LSR may wait for more frames within the time window to send the optimally sized SCTP packets. If the SCTP block stack is not yet ready to be sent, the SCTP block stack will be sent when it is ready later, so method 4700 proceeds to box 4799, where method 4700 ends (e.g., until later method 4700 can be re-executed). If the SCTP block stack is ready to be sent, method 4700 proceeds to box 4714. Box 4714 pushes the associated SCTP common header onto the SCTP block stack. This produces a complete SCTP packet. The source port and destination port fields in the SCTP common header are filled according to the corresponding parameters in the SCTP association. Note that blocks 4704-4714 may be performed by the SCTP layer in the LSR. From block 4714, method 4700 proceeds to block 4716. Block 4716 pushes the IP header onto the packet. The source and destination addresses in the IP header are populated according to a pair of local and remote IP addresses in the SCTP association (the addresses are provided by block 4714). The protocol field (if an IPv4 header) or the next header field (if an IPv6 header) is set to 132 to indicate that the payload is SCTP. From block 4716, method 4700 proceeds to block 4718. Block 4718 performs an IP routing table lookup on the destination address in the IP header, which results in the next hop for the packet. Note that blocks 4716-4718 may be performed by the IP layer in the LSR. From block 4718, method 4700 proceeds to block 4720.Block 4720 pushes the data link layer header onto the packet to send the packet to the next hop on the link, and the method 4700 proceeds to block 4722. Block 4722 sends the packet to the next hop on the line, and the method proceeds to block 4700, where the method 4700 ends. At block 4799, the method 4700 ends.
[0200] Fig.48 An example embodiment of a method for receiving and processing MPLS-in-SCTP packets is depicted. It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 4800 may be executed simultaneously or in different Fig.48 The input of method 4800 includes MPLS-in-SCTP packets received on the line.
[0201] At box 4801, method 4800 begins.
[0202] Box 4802 parses and processes the data link header on the packet, and then method 4800 proceeds to box 4804. Box 4804 checks whether the data link layer indicates that the packet is local, which means that the LSR executing method 4800 is the termination point of the data link on which the packet arrives. For example, if the data link header is an Ethernet header, and if the destination MAC address in the Ethernet header is local to the LSR, then the Ethernet link terminates at the LSR. If the packet is local at the data link layer, then method 4800 proceeds to box 4806, otherwise method 4800 proceeds to box 4824. Box 4824 performs the required treatment of the packet as a non-local packet at the data link layer, which can cause the packet to be further forwarded at the data link layer, and then method 4800 proceeds to box 4899, where method 4800 ends. Note that boxes 4802-4804 and box 4824 can be performed by the data link layer of the LSR.
[0203] When the LSR is the end of the data link, block 4806 removes the data link header from the packet, and method 4800 proceeds to block 4808. Block 4808 checks whether the payload indicated by the data link header is IP. For example, if the data link header is an Ethernet header, its Ethertype field indicates the payload type. If the Ethernet header has a VLAN tag, the Ethertype field in the last VLAN tag indicates the payload type. If the payload is IP, method 4800 proceeds to block 4810, otherwise method 4800 proceeds to block 4826. Block 4826 treats the packet as a non-IP packet, and method 4800 proceeds to block 4899, where method 4800 ends. Note that blocks 4806-4808 can be performed by the data link layer of the LSR.
[0204] Box 4810 processes the IP packet in the IP layer based on its IP header. For example, the destination address of the IP header can be looked up in the IP routing table to make a forwarding decision for the IP packet. From box 4810, method 4800 proceeds to box 4812. Box 4812 checks whether the destination address is local (e.g., whether the IP routing table lookup matches the destination address as an IP address locally configured in the LSR). If the destination address is not local, method 4800 proceeds to box 4828, otherwise method 4800 proceeds to box 4814. Box 4828 treats the packet as a non-local IP packet, such as by forwarding the packet to the next hop associated with the matching routing entry from the IP routing table lookup, and then method 4800 proceeds to box 4899, where method 4800 ends. Note that boxes 4810-4812 and 4828 can be performed by the IP layer of the LSR.
[0205] Block 4814 removes the IP header from the IP packet because it is a local packet, and then method 4800 proceeds to block 4816. Block 4816 checks whether the packet is an SCTP packet (e.g., if the IP header is IPv4, then check whether the Protocol field in the IPv4 header is set to 132 (SCTP), or if the IP header is IPv6, then check whether the Next Header field in the IPv6 header is set to 132 (SCTP)). If the packet is not an SCTP packet, then method 4800 proceeds to block 4830, otherwise method 4800 proceeds to block 4818. Block 4830 processes the packet of the corresponding IP protocol type, and then method 4800 proceeds to block 4899, where method 4800 ends. Note that blocks 4814-4816 can be performed by the IP layer of the LSR.
[0206] Block 4818 looks up an SCTP association for the SCTP packet based on the source and destination IP addresses in the IP header (e.g., the addresses are passed along the stream after the IP header is removed in block 4814) and based on the source and destination ports in the SCTP common header. From block 4818, method 4800 proceeds to block 4820. Block 4820 checks whether a matching SCTP association is found. If no matching SCTP association is found, method 4800 proceeds to block 4832, otherwise method 4800 proceeds to block 4822. Block 4832 discards the packet as an error packet, and method 4800 then proceeds to block 4899, where method 4800 ends.
[0207] Block 4822 performs SCTP layer processing on the SCTP packets and, if the SCTP association is bound to MPLS, processes MPLS packets from the data chunks received in the SCTP packets. From block 4822, method 4800 proceeds to block 4899 where 4800 ends.
[0208] At box 4899, method 4800 ends.
[0209] Fig.49 An example embodiment of a method for processing an MPLS-in-SCTP packet from an SCTP associated SCTP packet is depicted. The method 4900 may be used to provide Fig.48 Block 4822 of method 4800 and may be based on Fig.25 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 4900 may be executed simultaneously or in different Fig.49. The input of method 4900 includes SCTP packets received on an SCTP association and an indication of the SCTP association. Note that the SCTP packets received on the SCTP association may be received by the SCTP layer of the LSR using an SCTP common header on top. Note that the SCTP association may be indicated by a tuple {source IP address in IP header, destination IP address in IP header, source port in SCTP common header, destination port in SCTP common header}. At box 4901, method 4900 starts. Box 4902 checks whether the destination port in the SCTP common header is bound to MPLS. If the destination port in the SCTP common header is not bound to MPLS, method 4900 proceeds to box 4928, otherwise method 4900 proceeds to box 4904. Box 4928 processes SCTP packets for non-MPLS applications. Box 4904 processes the SCTP common header, and then method 4900 proceeds to box 4906. Block 4906 parses the first block in the SCTP packet, and then method 4900 proceeds to block 4908. Block 4908 checks whether the block is a data block. If the block is not a data block (which means it is a control block), then method 4900 proceeds to block 4922, otherwise method 4900 proceeds to block 4910. Block 4922 processes the control block using SCTP, and then method 4900 proceeds to block 4924. Block 4910 reads the data block and hands the data over to the MPLS overlay layer, and then method 4900 proceeds to block 4912. Block 4912 checks whether the flow ID of the data block is 0. If the flow ID of the data block is not 0, then method 4900 proceeds to block 4920, otherwise method 4900 proceeds to block 4914. Block 4920 handles the data as an MPLS packet at the MPLS overlay layer. The MPLS label stack of the MPLS packet in the payload identifies the MPLS overlay to which the MPLS payload belongs. From block 4920, method 4900 proceeds to block 4924. Block 4914 checks whether the data is a MOIF packet. If the data is not a MOIF packet, then the packet is not expected (i.e., a MOIF packet is expected to have a flow ID of 0), and method 4900 proceeds to block 4918, otherwise method 4900 proceeds to block 4916. Block 4918 closes the SCTP association using the appropriate SCTP method (notifying the remote LSR of the reason for the closure, such as by sending a SHUTDOWN control block in an SCTP packet), and method 4900 proceeds to block 4999, where method 4900 ends. Block 4916 processes the MOIF packet. If the LSR is the initiator of the connection, block 4916 may terminate the connection by Fig.17 Method 1700 is implemented, otherwise box 4916 can be Fig.214916, method 4900 proceeds to block 4924. Note that blocks 4914-4920 may be performed by an MPLS overlay, which is an application associated with SCTP. Block 4924 checks whether there are more blocks in the SCTP packet. If there are no more blocks in the SCTP packet, method 4900 proceeds to block 4999, where method 4900 ends, otherwise the method proceeds to block 4926. Block 4926 parses the next block in the SCTP packet and returns to block 4908 to repeat subsequent blocks for the next block. At block 4999, method 4900 ends.
[0210] Note that whether the top label of an encapsulated MPLS packet is downstream assigned or upstream assigned can be determined based on various criteria. For example, if the tunnel destination IP address is a unicast address, the top label can be downstream assigned. For example, if the tunnel destination IP address is an IP multicast address, then all encapsulated MPLS packets in a particular tunnel have a downstream assigned label at the top of the stack, or all encapsulated MPLS packets in a particular tunnel have an upstream assigned label at the top of the stack. The method of determining this for a particular tunnel can be performed in various ways. Without any knowledge about a particular tunnel, it can be assumed that the label is upstream assigned.
[0211] Note that once the intermediate routers receive the SCTP encapsulated packets, they can balance the packets based on the hash value of the SCTP packet's five-tuple (i.e., source IP address in the IP header, destination IP address in the IP header, source port in the SCTP common header, destination port in the SCTP common header, and protocol / next header field in the IP header). If load balancing between MPLS overlays is required, an IP address pool can be used at the two LSRs in the SCTP association, and then the IP address pool can be allocated to send MPLS overlay packets across the IP address pool.
[0212] Various example embodiments are configured to use QUIC as a reliable transport layer for tunneling of MPLS overlay packets. This provides an implementation of a reliable MPLS overlay as MPLS-in-QUIC.
[0213] QUIC is a computer network communication protocol that runs at the transport layer and acts similarly to TCP and SCTP. QUIC is designed to be a high-performance reliable transport protocol and has some fundamental differences from its predecessors TCP, SCTP, etc. QUIC improves the performance of connection-oriented applications by establishing multiple multiplexed connections (called streams) between two endpoints over UDP. The concept of streams in QUIC is similar to SCTP, which allows multiple data streams to arrive at endpoints independently and therefore independently of packet losses involving other streams. This solves the head-of-queue blocking delays encountered in TCP if any TCP packet is delayed or lost. QUIC also reduces connection and transmission delays, as well as bandwidth estimates in each direction to avoid congestion. Note that the fundamental difference between QUIC and TCP or SCTP is that QUIC is a user space transport protocol, which allows the protocol to be modified quickly without waiting for system upgrades. In a QUIC connection, streams can be started at any time. Either endpoint can start a stream, and the stream can be bidirectional or unidirectional. QUIC inherits the TCP byte stream data model, so from the application point of view, it is a stream-oriented protocol. Traditional transport layer protocols (such as TCP and SCTP) identify connections based on the tuple {source IP address, destination IP address, source port number, destination port number}; however, in QUIC, each connection is identified with a numeric connection identifier (ID). Each endpoint independently selects a connection ID and advertises it during connection establishment. When an endpoint sends a QUIC packet to a peer, the endpoint encodes the QUIC header using the connection ID advertised by the peer. The main function of the connection ID is to ensure that addressing changes in the lower protocol layers (UDP, IP) do not cause packets of the QUIC connection to be delivered to the wrong endpoint. Each endpoint uses an implementation-specific (and possibly deployment-specific) method to select a connection ID that will allow packets with that connection ID to be routed back to the endpoint and identified by the endpoint when received. In addition, QUIC provides encryption and authentication of the payload, which makes MPLS-in-QUIC suitable for secure and reliable MPLS overlays. QUIC essentially embeds the TLS (Transport Layer Security) method in its process. During connection establishment, QUIC includes a TLS handshake process to negotiate security keys to encrypt and authenticate the payload in the QUIC packets, and then, using these keys, the payload of each QUIC packet on the connection is encrypted and authenticated using the TLS process. When QUIC is used to tunnel MPLS overlay packets, the endpoints are LSRs.
[0214] Fig.50 An example embodiment of an MPLS-in-QUIC encapsulation format for implementing a reliable MPLS overlay as MPLS-in-QUIC is depicted.
[0215] The MPLS-in-QUIC encapsulation format 5000 includes a payload, an MPLS label stack on top of the payload, an MCH on top of the MPLS label stack, a QUIC header on top of the MPLS label stack, a UDP header on top of the QUIC header, an IP header on top of the UDP header, and a data link header on top of the IP header. For the reference model of reliable MPLS overlay packets, the UDP header and the QUIC header belong to the reliable delivery layer because QUIC runs on top of UDP. Reliability is built into QUIC, and the UDP header provides only the underlying transport service. The destination port number field in the UDP header is set to 80, which is the port number assigned by IANA for QUIC. Note that unless otherwise stated, references to "QUIC packets" refer to packets in which the UDP header is not included in the packet. Fig.50 In the MPLS overlay, the payload, MPLS label stack, MCH, and QUIC header form a "QUIC packet" for reliable tunneling of MPLS packets. The QUIC header is the header associated with the QUIC used as a tunnel by the MPLS overlay. Fig.50 In , the QUIC header is described in conceptual form, because it is not a single header, but a set of headers interspersed across QUIC packets. Note that multiple flows of the tuple {MPLS label stack, payload} are interspersed across QUIC packets.
[0216] Typically, a QUIC packet consists of three basic parts, including the QUIC long header (an example format of which is in Fig.51 ) or the QUIC short header (whose example format is in Fig.52 ) and stacked QUIC frames (whose example format is presented in Fig.53 (presented in ).
[0217] Fig.51An example embodiment of a QUIC long header for QUIC packets for implementing a reliable MPLS overlay as MPLS-in-QUIC is depicted. The QUIC long header 5100 is used for packets sent in the initial stages of QUIC connection establishment. Once the initial stages are completed, the sender switches to sending packets using the QUIC short header. The QUIC long header 5100 includes a header format field, a fixed bit field, a long packet type field, a type-specific bit field, a version field, a DCID length (Len) field, a destination connection ID field, a SCID length (Len) field, and a source connection ID field. The header format field is the most significant bit (0x80) of byte 0 (the first byte) and is set to 1 for the long header. The fixed bit field is the next bit (0x40) of byte 0 and is set to 1. Packets containing a zero value for this bit are not valid packets in the version and will be discarded. The long packet type (T) field is the next two bits of byte 0 (masked with 0x30) and includes the packet type. Currently defined packet types include: (a) 0x0=Initial, (b) 0x01=0-RTT, (c) 0x2=Handshake, and (d) 0x3=Retry. The type-specific bit (X) field includes the lower four bits of byte 0 (masked with 0x0f) and is type-specific. The version field is a 32-bit field following the first byte. This field indicates the QUIC version being used and determines how the remaining protocol fields are interpreted. The DCID Len field is the byte following the version and includes the length in bytes of the destination connection ID field that follows the version. The length is encoded as an 8-bit unsigned integer. In QUIC version 1, this value does not exceed 20. An endpoint that receives a version 1 long header with a value greater than 20 will discard the packet. The server may be able to read longer connection IDs from other QUIC versions in order to correctly form the version negotiation packet. The destination connection ID field follows the DCID Len and is between 0 and 20 bytes in length. The SCID Len field is the byte following the Destination Connection ID and includes the length in bytes of the Source Connection ID field that follows the Destination Connection ID. The length is encoded as an 8-bit unsigned integer. In QUIC Version 1, this value is no more than 20 bytes. An endpoint that receives a Version 1 Long Header with a value greater than 20 will discard the packet. Servers may be able to read longer connection IDs from other QUIC versions in order to correctly form Version Negotiation Packets. The Source Connection ID field follows the SCID Len and is between 0 and 20 bytes in length.
[0218] Fig.52An example embodiment of a QUIC short header for implementing a reliable MPLS overlay as a QUIC packet for MPLS-in-QUIC is depicted. A QUIC short header 5200 is used after the first few stages of connection negotiation. The QUIC short header includes a header format field, a fixed bit field, a spin bit field, a reserved bit field, a key phase field, a packet number length field, a destination connection ID field, a packet number field, and a protected payload field. The header format field is the most significant bit (0x80) of byte 0 and is set to 0 for the short header. The fixed bit field is the next bit (0x40) of byte 0 and is set to 1. Packets containing a zero value for this bit are not valid packets in the version and will be discarded. The spin bit field is the third most significant bit (0x20) of byte 0 and is a delayed spin bit. The reserved bit (R) field is the next two bits of byte 0 (masked to 0x18) and is reserved. These bits are protected using header protection. The value contained before protection is set to 0. An endpoint will receive packets with non-zero values for these bits, after removing packet and header protection, as a connection error of type PROTOCOL_VIOLATION. Dropping such packets only after removing header protection exposes the endpoint to attack. The Key Phase (K) field is the next bit (0x04) of byte 0 and indicates the Key Phase, which allows the recipient of the packet to identify the packet protection key used to protect the packet. The Packet Number Length (P) field is the least significant two bits of byte 0 (masked with 0x03) and includes the length of the packet number, encoded as an unsigned two-bit integer, with a length in bytes that is one less than the Packet Number field. That is, the length of the Packet Number field is the value of this field plus 1. The Destination Connection ID field includes a connection ID selected by the intended recipient of the packet. The length of the Packet Number field is 1 to 4 bytes. The Packet Number has confidentiality protection separate from the Packet Protection. The length of the Packet Number field is encoded in the Packet Number Length field. The Protected Payload field is configured so that packets with a QUIC short header always include a 1-RTT protected payload. Note that the Header Format Bits field and the Connection ID field of the Short Header Packet are version-independent. The remaining fields are specific to the selected QUIC version.
[0219] Fig.53 An example embodiment of a QUIC frame stack for implementing a reliable MPLS overlay as a QUIC packet for MPLS-in-QUIC is depicted. After removing the QUIC long or short header, the payload of a QUIC packet includes a series of complete frames. Note that not all packet types include frames. The payload of a packet that includes frames will include at least one frame, and may include multiple frames and multiple frame types. Note that a frame fits into a single QUIC packet and will not typically span multiple packets. Fig.53As depicted, each frame begins with a frame type, which indicates its type, followed by additional type-related fields. QUIC defines several frame types, one of which is the stream frame (type 8), which carries data for a stream in a QUIC connection. The format of the stream frame is described in Fig.54 Given in.
[0220] Fig.54 An example embodiment of a stream frame for implementing a reliable MPLS overlay as a QUIC packet for MPLS-in-QUIC is depicted. The stream frame 5400 includes a stream ID field, an offset field, a length field, and a stream data field. The stream ID field includes a variable length integer to indicate the stream ID of the stream. The offset field includes a variable length integer to specify the byte offset of the data in the stream frame in the stream. This field exists when the OFF bit is set to 1. When the offset field does not exist, the offset is 0. The length field includes a variable length integer to specify the length of the stream data field in the stream frame. The stream data field includes bytes from the specified stream to be delivered. When the length of the stream data field is 0, the offset in the stream frame is the offset of the next byte to be sent. The offset of the first byte in the stream is 0. The maximum offset (the sum of the offset and the data length) delivered on the stream does not exceed 2^(62-1) because flow control credits for this data cannot be provided. Receipt of a frame exceeding this limit is treated as a connection error of type FRAME_ENCODING_ERROR or FLOW_CONTROL_ERROR.
[0221] Note that during QUIC connection establishment, endpoints negotiate the application protocol that will be transported over QUIC.
[0222] In at least some embodiments, QUIC performs application negotiation by using Application Layer Protocol Negotiation (ALPN) over TLS (because QUIC embeds the TLS handshake during connection establishment). As the name implies, ALPN is a TLS extension that introduces support for application protocol negotiation in the TLS handshake. For instances where multiple application protocols are supported over QUIC, the extension allows the application layer to negotiate which application protocol will be used over the QUIC connection. In TLS, the ALPN extension type ("application_layer_protocol_negotiation(16)") can be defined as follows:
[0223]
[0224] The "extension_data" field of the ("application_layer_protocol_negotiation(16)") extension includes a "ProtocolNameList" value. "ProtocolNameList" includes a list of protocols advertised by the client, arranged in descending order of priority. Protocols are named by opaque, non-empty byte strings registered with IANA. The process is as follows: (1) the client appends a new ProtocolNameList field to the connection establishment message, including a list of supported application protocols, (2) the server examines the ProtocolNameList field and returns a ProtocolName field indicating the selected protocol as part of its response, and (3) the server may respond with only a single protocol name, and if it does not support any of the protocol names requested by the client, it may choose to abort the connection. Thus, once the connection is complete, the secure tunnel is established, and the client and server agree on which application protocol to use, they can begin communicating immediately. The names of the application protocols currently supported are as follows: (a) HTTP / 1.1[HTTP]=>"http / 1.1", (2) SPDY / 1[SPDY / 1]=>"spdy / 1", (3) SPDY / 2[SPDY / 2]=>"spdy / 2", and (4) SPDY / 3[SPDY / 3]=>"spdy / 3".
[0225] In at least some embodiments, QUIC performs application negotiation by using multi-layer protocol negotiation (MLPN). During connection establishment, a negotiation is performed between endpoints to multiplex multiple user protocols over the QUIC connection, regardless of their network layer. This negotiation is performed by the MLPN mechanism introduced in TLS, because QUIC embeds the TLS handshake during connection establishment. The negotiated protocol can be transmitted in the QUIC packet as the payload of a new GENERIC_STREAM frame. Note that the MLPN mechanism can be defined as follows. A new extension type ("multi_layer_protocol_negotiation(20)") is defined for TLS, which can be included by endpoints to negotiate multi-protocol support during connection establishment.
[0226]
[0227] ("multi_layer_protocol_negotiation(16)") The "extension_data" field of the extension contains the "MultiLayerProtocolList" value. "LayerName" includes the name of the protocol layer. The protocol layer is named by an opaque, non-empty byte string. Empty strings are not included and byte strings are not truncated. The following layer names can be used: (1) data link layer = "dlink", (2) network layer = "network". (3) transport layer = "transport", (4) application layer protocol with standardized TCP port = "app / tcp", (5) application layer protocol with standardized SCTP port = "app / sctp", and (6) application layer protocol with standardized UDP port = "app / udp". "ProtocolList" includes a list of 16-bit values, each of which identifies a protocol. The value of the protocol depends on the layer to which the protocol list applies. The following layer-by-layer values of the protocol can be used. For "dlink", the following protocol values may be used: (a) Ethernet = 1, (b) PPP (Point-to-Point Protocol) = 2, (c) HDLC (High-Level Data Link Control) = 3, (d) IEEE 802.11 Wireless LAN = 4, (e) LLDP (Link Layer Discovery Protocol) = 5, and (f) Frame Relay = 6. For "network", standardized Ethernet type values may be used (so that all existing network layer protocols automatically apply to this extension). For "transport", standardized protocol type values may be used (so that all existing transport layer protocols, as well as other user protocols of the network layer, automatically apply to this extension). For "app / tcp", standardized TCP port numbers for application protocols may be used. For "app / udp", standardized UDP port numbers for application protocols may be used. For "app / sctp", standardized SCTP port numbers for application protocols may be used. The use of MLPNs during QUIC connection establishment can be performed as follows: (1) in the connection establishment message, the client sends a MultiLayerProtocolList, which includes a list of protocols it intends to transport over the QUIC connection, (2) the server checks the MultiLayerProtocolList, resets the protocols that the server does not intend to support in the connection, and returns an updated MultiLayerProtocolList to indicate the selected protocols as part of its response, and (3) if the server does not support any of the client's protocol requests, it can choose to abort the connection. Thus, once the connection is completed, the secure tunnel is established, and the client and server agree on which protocols will be used, they can begin communicating immediately.
[0228] Fig.55 An example embodiment of a generic stream frame format for a QUIC packet is depicted. Generic stream frame format 5500 includes a type field, a flag field, a layer field, a protocol ID field, a stream ID field, an offset field, a length field, and a stream data field.
[0229] The type field is an 8-bit field that is common to any QUIC frame and is used to indicate the type of frame. The current QUIC specification defines frame types with values of 0x00-0x1e. It should be understood that any available value (e.g., 0x20, etc.) may be assigned to indicate a GENERIC_STREAM frame. Note that if standardized in the IETF, the value used to indicate a GENERIC_STREAM frame may be retained in the QUIC protocol registry in IANA.
[0230] The flags field is a 4-bit field that can carry multiple flags, including the header bit, OFF (offset) bit, LEN (length) bit, and FIN bit.
[0231] The Header Bits (0x8) field is applicable when the payload (stream data field) is datagram oriented (e.g., Ethernet, IP, etc.). When set, it indicates that the stream data starts at the beginning of the datagram (i.e., the header of the protocol packet). If the Header Bits are not set, it indicates that the stream data carries fragments of the protocol packet. This also means that multiple datagrams are packaged on a single frame. Since streams in QUIC deliver frames in order, there is no need to indicate the offset of the fragment relative to the original packet.
[0232] The OFF bit (0x4) field is set to indicate the presence of the Offset field. When the OFF bit is set to 1, the Offset field is present. When the OFF bit is set to 0, the Offset field is not present and the stream data starts at offset 0 (i.e., the frame includes the first byte of the stream, or does not include the end of the stream data).
[0233] The LEN bit (0x2) field is set to indicate the presence of a length field. If the LEN bit is set to 0, the length field is not present and the stream data field extends to the end of the packet. If the LEN bit is set to 1, the length field is present.
[0234] The FIN bit (0x1) field is set on a frame that contains the final size of the stream. Setting this bit on the frame marks the end of the stream.
[0235] The Layer Type field is a 4-bit field that indicates the layer of the protocol packet carried in the frame. Note that the following values can be defined: (1) 0x01 = Data Link Layer, (2) 0x02 = Network Layer, (3) 0x03 = Transport Layer (Note that this also includes any non-transport protocols running on the Network Layer, (4) 0x04 = Application Layer with TCP Port, which means that it includes any application protocol with a standardized TCP port number, (5) 0x05 = Application Layer with UDP Port, which means that it includes any application protocol with a standardized TCP port number, and (6) 0x06 = Application Layer with SCTP Port, which means that it includes any application protocol with a standardized TCP port number. It will be appreciated that any other suitable value may be used.
[0236] The Protocol ID field is a 16-bit field that indicates the protocol within the layer type. These values depend on the Layer Type field.
[0237] For example, when the layer type is data link layer (0x01), the values in this field may be defined as follows: (a) Ethernet = 1, (b) PPP (Point-to-Point Protocol) = 2, (c) HDLC (High-Level Data Link Control) = 3, (d) IEEE 802.11 Wireless LAN = 4, (e) LLDP (Link Layer Discovery Protocol) = 5, and (f) Frame Relay = 6. It will be appreciated that other data link layer protocols not included in the above list may be assigned values in the range of 7-65535. It will be appreciated that other suitable values may be used.
[0238] For example, when the layer type is the network layer (0x02), the value in this field may be set according to the standardized value of Ethertypes defined in IANA. It should be understood that other suitable values may be used.
[0239] For example, when the layer type is a transport layer (0x03), the value in this field may be set according to a standardized value of an IP number. It should be understood that other suitable values may be used.
[0240] The Stream ID field includes a variable length integer to indicate the Stream ID of the stream.
[0241] The Offset field is a variable-length integer that specifies the byte offset in the stream of the data in this GENERIC_STREAM frame. This field is present when the OFF bit is set to 1. When the Offset field is not present, the offset is 0.
[0242] The Length field is a variable-length integer that specifies the length of the Stream Data field in this GENERIC_STREAM frame. This field appears when the LEN bit is set to 1. When the LEN bit is set to 0, the Stream Data field consumes the remaining bytes in the packet.
[0243] The Stream Data field is a variable length field that includes the bytes to be delivered from the specified stream. If the Header bit is set, it starts with an Ethernet header in the case of an Ethernet packet, an IP header in the case of an IPv4 packet, and so on.
[0244] Various example embodiments may be configured to support the following methods for MPLS-in-QUIC.
[0245] Various example embodiments for supporting MPLS-in-QUIC may be configured such that during connection establishment, an LSR acting as a client negotiates MPLS-in-QUIC using an MLPN by including at least the following protocols in the MultiLayerProtocolList: a LayerProtocolList entry with LayerProtocolList->layer_name "network" and LayerProtocolList->protocols including an Ethertype value for MPLS (e.g., 0x8847).
[0246] Various example embodiments for supporting MPLS-in-QUIC can be configured such that each MPLS overlay is transmitted as an independent stream in a QUIC connection. Packets from the MPLS overlay are carried as the payload of a GENERIC_STREAM frame, where the "Layer" field in the frame is encoded using the value 0x02 (network layer) and the Protocol ID field is encoded using the Ethernet type value for MPLS (e.g., 0x8847). The "Stream ID" field is encoded using the stream ID assigned to the MPLS overlay. The header bits in the Flags field can be set to 0 to indicate that the payload is stream-oriented because the MCH is included in the stream, which divides the MPLS packets in the payload; however, an implementation can also exclude the MCH where the header bits are set according to defined rules to divide the MPLS packets in the payload. Using this approach, multiple MPLS overlays between two LSRs can be multiplexed over a QUIC connection between the two LSRs. Note that the stream ID space is local to the LSR, so each LSR can independently allocate unused stream IDs from its local space. An LSR sending an MPLS overlay packet over a QUIC connection uses its assigned Flow ID in the corresponding data chunk in the QUIC packet. Note that for a receiving LSR, there is no correlation between the Flow ID of a received data chunk and the corresponding MPLS overlay, as the LSR identifies the overlay based on the MPLS label stack of the MPLS packet in the payload of the data chunk. It should also be noted that a Flow ID value of 0 is not expected to be used for MPLS packets transporting an MPLS overlay, as it may be reserved by both LSRs for bulk management of all MPLS overlays multiplexed on a QUIC connection (e.g., for managing a MOIF or performing other management functions).
[0247] Fig.56 An example embodiment of a QUIC packet for multiplexing N MPLS overlay packets is depicted. Note that Fig.56 QUIC packet 5600 implements Fig.50 The conceptual format of Fig.56 In , stream IDs are assigned integer values 1 to N respectively.
[0248] Various example embodiments include methods for configuring a QUIC connection for MPLS-in-QUIC. When an LSR discovers that a new QUIC connection needs to be established with a remote LSR to establish MPLS overlay packet tunneling, the LSR: (1) determines at least one IP address of the remote LSR with which the QUIC connection is to be established (note that during the lifetime of the QUIC connection, QUIC packets may be exchanged between any pair of IP addresses in a pool of IP addresses at the LSR), and (2) determines QoS information that may be used to encapsulate an appropriate QoS marking in an IP header of the QUIC packet.
[0249] Fig.57 An example embodiment of a method for configuring a QUIC connection for MPLS-in-QUIC is depicted. Method 5700 may be based on Fig.16 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 5700 may be executed simultaneously or in different Fig.57 . In box 5701, method 5700 starts. Box 5702 selects one or more configured local IP addresses that can be used to send and receive tunnel traffic, and then method 5700 proceeds to box 5704. Box 5704 allocates a local connection ID to be used for the new connection so that the number is not currently used by another existing QUIC connection, and method 5700 then proceeds to box 5706. Box 5706 constructs an MLPN to be included in a TLS message embedded in the connection request. The MLPN includes support for the protocol ID MPLS (=0x8847) in the "network" layer, and then method 5700 proceeds to box 5708. Box 5708 generates a QUIC connection request with a connection ID from one of the local IP addresses to the remote IP address. The QUIC connection request is a QUIC packet that carries data for an encrypted handshake and the MLPN. The QUIC connection request is sent via a UDP packet. In the UDP packet, the local port number is set to an arbitrary random value and the destination port number is set to 80, 443, or a value administratively selected for the entire network to identify QUIC as a user protocol. Note that if the QUIC connection request is rejected by the remote LSR, the LSR should take action to limit unnecessary repeated attempts to establish a connection. For example, the LSR may wait 60 seconds before attempting to reestablish a connection. From box 5708, method 5700 proceeds to box 5710. Box 5710 checks whether the remote LSR accepts the QUIC connection request (meaning success). If the remote LSR accepts the QUIC connection request (success), the LSR will receive a handshake from the remote LSR, which includes a connection ID assigned by the remote LSR for the connection. If the remote LSR accepts the QUIC connection request (success), method 5700 proceeds to box 5712, otherwise method 5700 proceeds to box 5714. Block 5714 declares a failure to configure the connection, and method 5700 proceeds to block 5799, where method 5700 ends. Block 5712 performs post-connection follow-up (if any), and method 5700 proceeds to block 5799, where method 5700 ends. At block 5799, method 5700 ends.
[0250] Fig.58An example embodiment of a method for performing post-connection follow-up for a QUIC connection of MPLS-in-QUIC is depicted. Note that method 5800 may be performed by an LSR that initiates a QUIC connection. Method 5800 may be used to provide Fig.57 Block 5712 of method 5700. Method 5800 may be based on Fig.14 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 5800 may be executed simultaneously or in different Fig.58 Method 5800 is executed in the order presented in . The input of method 5800 includes a QUIC connection that has been started. At box 5801, method 5800 starts. Box 5802 checks whether the MOIF requirements are configured in the LSR. If the MOIF requirements are configured in the LSR, method 5804 proceeds to box 5804, otherwise method 5800 proceeds to box 5899 and method 5800 ends. Box 5804 constructs the MOIF and fills in the necessary fields. Note that box 5804 can be configured by Fig.60 . From box 5804, method 5800 proceeds to box 5806. Box 5806 sends the MOIF over the QUIC connection. The MOIF is sent as the payload of a GENERIC_STREAM frame in a QUIC packet, where the value in the stream ID field is encoded as 0. The layer type field is encoded as the value 0x02 for the network layer and the protocol ID is encoded as the value 0x8847 for MPLS. From box 5806, method 5800 then proceeds to box 5808. Box 5808 waits for a MOIF response from the peer LSR. For example, the LSR may wait for a MOIF response for at least 90 seconds or any other suitable length of time. To wait, the LSR may start a timer for the duration for which it expects a MOIF response and associate the timer with the QUIC connection. From box 5808, method 5800 then proceeds to box 5899, where method 5800 ends. Note that when the initiating LSR receives a MOIF response from the peer LSR, it may perform Fig.17 It should be noted that the determination that a packet received on a connection is a MOIF can be made as part of the general processing of MPLS overlay packets on a QUIC connection, which is described in Fig.66 The method 6600 is described in blocks 6602-6618 of the method 6600. At block 5899, the method 5800 ends.
[0251] Fig.59 An example embodiment of a method for use when a MOIF response is not received within a predefined time period for a QUIC connection is depicted. Note that a situation in which the initiating LSR fails to receive a MOIF response within a predefined time period may be determined where Fig.58The timeout set at block 5808 of method 5900 expires without receiving a MOIF response. It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 5900 may be executed simultaneously or in different Fig.59 . The input to method 5900 includes a notification that the MOIF wait timer has expired. At block 5901, method 5900 begins. Block 5902 retrieves a QUIC connection with a timer, and method 5900 proceeds to block 5904. Block 5904 closes the QUIC connection, and method 5900 proceeds to block 5999, where method 5900 ends. At block 5999, method 5900 ends.
[0252] Fig.60 An example embodiment of a method for configuring a QUIC connection listener is depicted. Note that an LSR may use method 6000 to listen for QUIC connection requests for MPLS-in-QUIC (e.g., listen for new QUIC connection requests for MPLS as a user protocol). Note that the MPLS-in-QUIC connection listener may be implemented by any LSR that configures an MPLS overlay. Method 6000 may be based on Fig.18 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 6000 may be executed simultaneously or in different Fig.60 . At block 6001, method 6100 begins. Block 6002 opens UDP port 80, 443, or an administratively configured value for the network range to identify QUIC as a user protocol, and then method 6000 proceeds to block 6004. Block 6004 binds the port to the set of local IP addresses in the LSR selected for QUIC connections, and then begins listening for incoming QUIC connection requests on the port on any IP address. From block 6004, method 6000 proceeds to block 6099, where method 6000 ends. At block 6099, method 6000 ends.
[0253] Fig.61 An example embodiment of a method for processing an incoming QUIC connection request is depicted. Note that the incoming connection request may be received on a UDP port of QUIC. The method 6100 may be based on Fig.19 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 6100 may be executed simultaneously or in different Fig.61. The input to method 6100 includes a UDP packet with the following associated parameters: (1) a source IP address, which is the IP address of the remote LSR that sent the request, (2) a destination IP address, which is the address of the LSR to which the local IP packet is sent, (3) a source port, which is the UDP port at the remote LSR, and (4) a destination port, which is the UDP port of the LSR to which the packet is sent. At box 6101, method 6100 begins. Box 6102 checks whether the UDP in the LSR is ready to receive packets on the destination port at the destination IP address. If the UDP in the LSR is not ready to receive packets on the destination port at the destination IP address, method 6100 proceeds to box 6124, otherwise method 6100 proceeds to box 6104. Box 6124 discards the packet and method 6100 then proceeds to box 6199, where method 6100 ends. Block 6104 checks whether the UDP destination port indicates the user protocol as QUIC (e.g., the port is 80, 443, or an administratively configured value, if any). If the UDP destination port does not indicate the user protocol as QUIC, method 6100 proceeds to block 6118, otherwise method 6100 proceeds to block 6106. Block 6118 processes the payload of the UDP packet according to the user protocol associated with the port. From block 6118, the method proceeds to block 6199, where method 6100 ends. Block 6106 checks whether QUIC packets are allowed from a remote LSR (e.g., from a source IP address). Note that various factors may be used to determine whether an LSR should allow QUIC packets from an IP address. For example, an LSR may employ a method to discover all potential remote LSRs and their local set of IP addresses that can participate in a QUIC connection with the LSR's MPLS overlay. If the QUIC packet from the remote LSR is not allowed, the method 6100 proceeds to box 6124, otherwise the method 6100 proceeds to box 6108. Box 6124 discards the QUIC packet, and the method 6100 proceeds to box 6199, where the method 6100 ends. Box 6108 checks whether the QUIC packet carries a connection request (e.g., this determination can be made according to the QUIC specification). If the QUIC packet carries a connection request, the method 6100 proceeds to box 6110, otherwise the method 6100 proceeds to box 6120. Box 6120 processes the packet as a QUIC packet for an existing connection, and the method 6100 proceeds to box 6199, where the method 6100 ends. Box 6110 checks whether the connection request includes an MLPN, and if the connection request is made by the remote LSR according to various embodiments presented herein (e.g., implemented by the LSR making a connection request Fig.57If the connection request does not include an MLPN, then method 6100 proceeds to block 6122, otherwise method 6100 proceeds to block 6112. Block 6122 processes the connection request. Block 6112 checks whether the MLPN includes support for MPLS (e.g., layer type "network" and protocol 0x8847). Note that if the connection request is made by a remote LSR in accordance with various embodiments presented herein (e.g., by an LSR making a connection request), Fig.57 If MPLS is not supported in the MLPN, method 6100 proceeds to box 6122, otherwise method 6100 proceeds to box 6114. Box 6122 processes the connection request. Box 6114 accepts a request for a new QUIC connection. Note that box 6114 can be implemented in accordance with the server process for new connections in the QUIC specification, such as where a local connection ID is assigned to the connection, the state of the connection is created using the tuple {local connection ID, connection ID received in the QUIC packet of the remote connection request}, and the LSR sends a response to the remote LSR, the response having the negotiated encryption, authentication keys, and other connection parameters. From box 6114, method 6100 proceeds to box 6116. In box 6116, post-connection follow-up (if any) is performed. Note that 6116 can be accomplished by Fig. 20 The method 2000 is implemented by causing the LSR to wait for a MOIF from each LSR if configured to receive a MOIF. Note that the determination that a packet received on a connection is a MOIF can be made as part of the general processing of an associated MPLS overlay packet (e.g., using Fig.66 Note that if the LSR fails to receive the MOIF within a predefined period of time (e.g., within Fig. 20 The timeout set at box 2004 in method 2000 expires), the LSR may execute Fig.59 Method 5900 of (e.g., so that the handling is common in both the initiating and receiving LSRs). Note that when a MOIF is received from a peer LSR, the LSR may perform Fig.21 From block 6116, method 6100 proceeds to block 6199, where method 6100 ends. At block 6199, method 6100 ends.
[0254] Various example embodiments include a method for configuring an MPLS-in-QUIC overlay from an LSR to a remote LSR. To configure an MPLS-in-QUIC overlay, the LSR assigns a unique non-zero flow ID to the overlay on a QUIC connection with the remote LSR. Note that the flow ID space is local to the LSR, so each LSR can independently allocate unused flow IDs from its local space. LSRs that play the role of a client in the QUIC connection are assigned even-numbered flow IDs, and LSRs that play the role of a server in the QUIC connection are assigned odd-numbered flow IDs. An LSR that sends an MPLS overlay packet over a QUIC connection uses its locally assigned flow ID in the corresponding GENERIC_STREAM frame in the QUIC packet. For the receiving / remote LSR, there is no correlation between the flow ID of the received GENERIC_STREAM frame and the corresponding MPLS overlay because the LSR identifies the overlay based on the MPLS label stack of the MPLS packet in the payload of the data block.
[0255] Fig.62 An example embodiment of a method for configuring an MPLS overlay on a QUIC connection to form an MPLS-in-QUIC overlay is depicted. Note that when an LSR prepares to establish an MPLS-in-QUIC overlay with a remote LSR, it determines the IP address of the remote LSR with which to establish a QUIC connection, and then determines the QUIC connection parameters and QoS information for the QUIC connection. Method 6200 may be based on general method 2200 presented in FIG. 22. It should be understood that while primarily presented as being executed serially, at least a portion of the blocks of method 6200 may be executed simultaneously or in different Fig.62. The input to method 6200 includes the outgoing MPLS label of the LSP / overlay and the IP address of the remote LSR of the LSP / overlay. At box 6201, method 6200 starts. Box 6202 checks whether there is a QUIC connection with the remote LSR for MPLS-in-QUIC. If there is no existing QUIC connection with the remote LSR for MPLS-in-QUIC, then method 6200 proceeds to box 6212, otherwise method 6200 proceeds to box 6204. Box 6204 retrieves the first QUIC connection with the remote LSR for MPLS-in-QUIC, and then method 6200 proceeds to box 6206. Box 6206 checks whether there is at least one unused non-zero flow ID available in the connection. If there is at least one unused non-zero flow ID available, then the connection can be used for overlay and method 6200 proceeds to block 6216, otherwise the connection cannot be used for overlay and method 6200 proceeds to block 6208. 6208 is directed to checking whether there are more QUIC connections with the remote LSR. If there are more QUIC connections with the remote LSR for MPLS-in-QUIC, then method 6200 proceeds to block 6210, otherwise method 6200 proceeds to block 6212. Block 6212 retrieves the next QUIC connection with the remote LSR for MPLS-in-QUIC and method 6200 returns to block 6206 to evaluate the feasibility of the next connection as an overlay tunnel. Block 6212 configures a new QUIC connection with the remote LSR for MPLS-in-QUIC. Note that block 6212 may be configured by Fig.57. From box 6212, method 6200 proceeds to box 6214. Box 6214 checks whether the connection has been successfully established. If the connection has not been successfully established, method 6200 proceeds to box 6226, otherwise method 6200 proceeds to box 6216. Box 6226 declares and handles the failure to configure the MPLS overlay. Box 6216 allocates an LSP identifier to uniquely identify the MPLS overlay in the LSR. The LSP identifier binds the QUIC connection, (multiple) outgoing labels, and other parameters associated with the MPLS overlay. From box 6216, method 6200 then proceeds to box 6218. Box 6218 sets the outgoing MPLS label of the LSP / overlay to the outgoing label of the LSP identifier, and method 6200 then proceeds to box 6220. Box 6220 sets the QUIC connection as a tunnel associated with the LSP identifier, and method 6200 then proceeds to box 6222. Block 6222 allocates an unused flow ID in the connection as an identifier for the flow to be used for switching packets for the overlay. From block 6222, method 6200 proceeds to block 6224. Block 6224 creates a mapping between the LSP identifier and the flow ID of the overlay. From block 6224, the method proceeds to block 6299, where method 6200 ends. At block 6299, method 6200 ends.
[0256] Fig.63 An example embodiment of a method for transporting packets over MPLS-in-QUIC is depicted. Method 6300 may be based on Fig.23 (except that block 2310 may not be implemented because there are no dynamic parameters for the QUIC header in the MPLS-in-QUIC packet). It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 6300 may be executed simultaneously or in different Fig.63. The input of method 6300 includes a packet to be transmitted and an LSP identifier identifying an LSP or MPLS overlay. At box 6301, method 6300 starts. Box 6302 retrieves the state of the outgoing label of the LSP by the LSP identifier, and then method 6300 proceeds to box 6304. Box 6304 obtains the QUIC tunnel configured for the LSP identifier, that is, a reliable transport layer connection, and then method 6300 proceeds to box 6306. Box 6306 pushes the outgoing label of the LSP onto the packet, and then method 630 proceeds to box 6308. Box 6308 sets the S bit, EXP, and TTL fields in the pushed label as needed for the packet. For example, if the packet already has another label before the outgoing label of the pushed LSP, the S bit is set to 0, otherwise the S bit is set to 1. The TTL and EXP values can be set by the policy configured for the LSP. For example, if the policy requires a consistent EXP value for all packets on the LSP, the configured EXP value is set. If the policy has a mapping based on the EXP value of certain fields in the native header of the packet, the mapped EXP value is set. From block 6308, method 6300 proceeds to block 6310. Block 6310 sends the MPLS packet on the QUIC connection, and method 6300 then proceeds to block 6399, where method 6300 ends. At block 6399, method 6300 ends.
[0257] Fig.64 An example embodiment of a method for transporting MPLS packets over a QUIC connection is depicted. Method 6400 may be used to provide Fig.63 Block 6310 of method 6300 and may be based on Fig.24 (except that the QUIC header has no dynamic parameters). It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 6400 may be executed simultaneously or in different Fig.64. The input of method 6400 includes an MPLS packet to be transmitted and an LSP identifier identifying an LSP or MPLS overlay. At box 6401, method 6400 starts. Box 6402 retrieves the stream ID mapped to the LSP identifier of the MPLS overlay, and then method 6400 proceeds to box 6404. Box 6404 constructs a GENERIC_STREAM frame for the stream ID using the MPLS packet as its payload, and then method 6400 proceeds to box 6406. Box 6406 checks whether one or more frames are waiting to be sent on the QUIC connection. Since QUIC can bundle multiple frames on a single QUIC packet, previous frames may have been queued waiting to be sent. Secondly, previous frames may also be pending due to congestion, flow control, etc. If there are no frames to be processed, method 6400 proceeds to box 6408, otherwise method 6400 proceeds to box 6410. Box 6408 creates an empty frame stack, and then method 6400 proceeds to box 6410. Box 6410 pushes the GENERIC_STREAM frame (including the MPLS packet) into the frame stack, and then method 6400 proceeds to box 6412. Box 6412 checks whether the frame stack is ready to be sent over the QUIC connection. Due to congestion, flow control, etc., the frame stack may not be sent immediately, or the LSR may wait for more frames within the time window to send the optimal size QUIC packet. If the frame stack is not yet ready to be sent, the frame stack will be sent when it is ready later, so method 6400 proceeds to box 6499, where method 6400 ends (e.g., until later method 6400 can be re-executed). If the frame stack is ready to be sent, method 6400 proceeds to box 6414. Box 6414 pushes the QUIC header of the connection (e.g., a QUIC long header or a QUIC short header deemed appropriate for the connection) onto the frame stack. This produces a complete QUIC packet. The destination connection ID field in the QUIC header is populated based on the connection ID negotiated in the connection. Note that boxes 6404-6414 can be performed by the QUIC layer in the LSR. From box 6414, method 6400 proceeds to box 6416. Box 6416 pushes a UDP header onto the QUIC packet. The source port in the UDP header can be a random value or a value based on an equal-cost multi-path (ECMP) hash. The destination port in the UDP header is set to indicate the QUIC packet as its payload, for example, it can be 80, 443, or a network range value selected in an administrative manner. From box 6416, method 6400 proceeds to box 6418. Box 6418 pushes an IP header onto the UDP packet. The source address and destination address in the IP header are populated based on a pair of local and remote IP addresses in the QUIC connection (the addresses are provided by box 6414).The protocol field (if it is an IPv4 header) or the next header field (if it is an IPv6 header) is set to 17 to indicate that the payload is UDP. From box 614, method 6400 proceeds to box 6420. Box 6420 performs an IP routing table lookup on the destination address in the IP header, which results in the next hop of the packet, and then method 6400 proceeds to box 6422. Note that boxes 6418-6420 can be performed by the IP layer in the LSR. Box 6422 pushes the data link layer header onto the IP packet, which is used to send the packet to the next hop on the link. From box 6422, method 6400 proceeds to box 6424. Box 6424 sends the packet to the next hop on the line, and then method 6400 proceeds to box 6499, where method 6400 ends. At box 6499, method 6400 ends.
[0258] FIG65 depicts an example embodiment of a method for receiving and processing MPLS-in-QUIC packets. The method 6500 may be based on Fig.25 65. It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 6500 may be executed simultaneously or in an order different from that presented in FIG. 65. The input to method 6500 includes a packet received on a line. At block 6501, method 6500 begins.
[0259] Block 6502 parses the data link header on top of the processed packet and method 6500 proceeds to block 6504. Note that block 6502 may be performed by the data link layer of the LSR.
[0260] Block 6504 checks whether the data link layer indicates that the packet is local, which means that the LSR executing method 6500 is the termination point of the data link on which the packet arrives. For example, if the data link header is an Ethernet header, and if the destination MAC address in the Ethernet header is local to the LSR, then the Ethernet link terminates at the LSR. If the packet is local at the data link layer, then method 6500 proceeds to block 6506, otherwise method 6500 proceeds to block 6526. Block 6526 treats the packet as a non-local packet at the data link layer, which can cause the packet to be further forwarded at the data link layer, and then method 6500 proceeds to block 6599, where method 6500 ends. Since the LSR is the end of the data link, block 6506 removes the data link header from the packet, and then method 6500 proceeds to block 6508. Note that blocks 6504-6506 and block 6526 can be performed by the data link layer of the LSR.
[0261] Block 6508 checks whether the payload indicated by the data link header is IP. For example, if the data link header is an Ethernet header, its Ethertype field indicates the payload type. If the Ethernet header has a VLAN tag, the Ethertype field in the last VLAN tag indicates the payload type. If the payload is IP, then method 6500 proceeds to block 6510, otherwise method 6500 proceeds to block 6528. Block 6528 treats the packet as a non-IP packet, and then method 6500 proceeds to block 6599, where method 6500 ends. Note that blocks 6508 and 6526 can be performed by the data link layer of the LSR.
[0262] Box 6510 processes the IP packet in the IP layer based on its IP header. For example, the destination address of the IP header is looked up in the IP routing table to make a forwarding decision for the IP packet. From box 6510, method 6500 proceeds to box 6512. Box 6512 checks whether the destination address is local (e.g., whether the IP routing table lookup matches the destination address as an IP address locally configured in the LSR). If the destination address is not local, method 6500 proceeds to box 6530, otherwise method 6500 proceeds to box 6514. Box 6530 treats the packet as a non-local IP packet, for example by forwarding the packet to the next hop associated with the matching routing entry from the IP routing table lookup, and then method 6500 proceeds to box 6599, where method 6500 ends. Note that boxes 6510-6512 and box 6530 can be performed by the IP layer of the LSR.
[0263] Frame 6514 removes the IP header from the IP packet because it is a local packet, and then method 6500 proceeds to frame 6516. Frame 6516 checks whether the packet is a UDP packet. For example, if the IP header is IPv4, frame 6516 can check whether the protocol field in the IPv4 header is set to 17 (UDP), or if the IP header is IPv6, frame 6516 can check whether the next header field in the IPv6 header is set to 17 (UDP). If the packet is not a UDP packet, then method 6500 proceeds to frame 6532, otherwise method 6500 proceeds to frame 6518. Frame 6532 handles the packet of the corresponding IP protocol type, and then method 6500 proceeds to frame 6599, where method 6500 ends. Note that frames 6514-6516 can be performed by the IP layer of the LSR.
[0264] Block 6518 checks whether the destination port in the UDP header indicates QUIC as the payload (e.g., the value is 80, 443, or a network range value assigned in an administrative manner to identify QUIC as the payload). If the payload is not QUIC, then method 6500 proceeds to block 6534, otherwise method 6500 proceeds to block 6520. Block 6534 processes the packet for the corresponding port, and then method 6500 proceeds to block 6599, where method 6500 ends.
[0265] Block 6520 looks up the QUIC connection for the QUIC packet based on the destination connection ID in the QUIC long header or the QUIC short header, and method 6500 proceeds to block 6522. Block 6522 checks whether a matching QUIC connection is found. If a matching QUIC connection is not found, method 6500 proceeds to block 6536, otherwise method 6500 proceeds to block 6524. Block 6536 discards the packet as an error packet, and method 6500 proceeds to block 6599, where method 6500 ends.
[0266] Block 6524 performs QUIC layer processing on the QUIC packets. For each GENERIC_STREAM frame in the packet indicating MPLS as its payload, it processes the MPLS packets from the stream data in the frame. From block 6524, method 6500 proceeds to block 6599, where method 6500 ends.
[0267] At box 6599, method 6500 ends.
[0268] Fig.66 An example embodiment of a method for processing an MPLS-in-QUIC packet from a QUIC packet is depicted. The method 6600 may be used to provide block 6524 of the method 6500 of FIG. 65 and may be based on Fig.25 It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 6600 may be executed simultaneously or in different Fig.66. The input to method 6600 includes a QUIC packet received for a potential existing connection. At box 6601, method 6600 begins. Box 6602 processes the QUIC short header or QUIC long header (whichever exists) in the received packet, although it will be appreciated that it is most likely that it will be a QUIC short header). In this box, an existing QUIC connection is looked up based on the connection ID in the header. From box 6602, method 6600 proceeds to box 6604. Box 6604 checks whether a connection is found. If no connection is found, method 6600 proceeds to box 6630, otherwise method 6600 proceeds to box 6606. Box 6630 discards the packet, and then method 6600 proceeds to box 6699, where method 6600 ends. Box 6606 parses the first frame in the QUIC packet, and then method 6600 proceeds to box 6608. Box 6608 checks whether the frame is a GENERIC_STREAM frame. If the frame is not a GENERIC_STREAM frame, the method 6600 proceeds to box 6624, otherwise the method 6600 proceeds to box 6610. Box 6624 processes the frame using the QUIC mechanism, and then the method 6600 proceeds to box 6626. Box 6610 reads the stream data (payload) from the GENERIC_STREAM frame, and then the method 6600 proceeds to box 6612. Box 6612 checks whether the payload determined by the layer type and protocol ID fields in the GENERIC_STREAM indicates that the packet is an MPLS packet. If the payload type is MPLS, the method 6600 proceeds to box 6614, otherwise the method 6600 proceeds to box 6632. Box 6632 processes the payload according to the payload type. Box 6614 checks whether the stream ID in the GENERIC_STREAM is 0. If the flow ID in the GENERIC_STREAM is not 0, the method 6600 proceeds to box 6620, otherwise the method 6600 proceeds to box 6616. Box 6620 provides the payload layer to the MPLS overlay layer. The MPLS label stack of the MPLS packet in the payload identifies the MPLS overlay to which the MPLS payload belongs. From box 6620, the method 6600 proceeds to box 6626. Box 6616 checks whether the payload is a MOIF packet. If the flow ID of the GENERIC_STREAM frame is 0 and the payload is MOIF, then this determination is made. If the payload is not a MOIF packet, then the packet is not expected and then the method 6600 proceeds to box 6622, otherwise the method 6600 proceeds to box 6618. Block 6622 closes the QUIC connection using an appropriate method (e.g., notifying the remote LSR of the reason for the closure, such as by sending a CONNECTION_CLOSE frame in a QUIC packet in the context of the connection), and method 6600 then proceeds to block 6699, where method 6600 ends.Block 6618 processes the MOIF. Note that if the LSR is the initiator of the connection, block 6618 may be passed. Fig.17 The method 1700 is implemented. Fig.17 Otherwise, box 6618 can be Fig.21 6618, method 6600 proceeds to block 6626. Note that blocks 6614-6620 may be performed by an MPLS overlay, which is a user of the QUIC connection. Block 6626 checks whether there are more frames in the QUIC packet. If there are no more frames in the QUIC packet, method 6600 proceeds to block 6699, where method 6600 ends, otherwise method 600 proceeds to block 6628. Block 6628 parses the next frame in the QUIC packet, and then method 6600 returns to block 6608 to repeat subsequent blocks for the next frame. At block 6699, method 6600 ends.
[0269] Note that whether the top label of an encapsulated MPLS packet is downstream assigned or upstream assigned can be determined based on various criteria. For example, if the tunnel destination IP address is a unicast address, the top label can be downstream assigned. For example, if the tunnel destination IP address is an IP multicast address, then all encapsulated MPLS packets in a particular tunnel have a downstream assigned label at the top of the stack, or all encapsulated MPLS packets in a particular tunnel have an upstream assigned label at the top of the stack. The method of determining this for a particular tunnel can be performed in various ways. Without any knowledge about a particular tunnel, it can be assumed that the label is upstream assigned.
[0270] Note that after receiving UDP encapsulated QUIC packets, intermediate routers can balance these packets based on the hash value of the 5-tuple of the UDP packet (i.e., source IP address in IP header, destination IP address in IP header, source port in UDP header, destination port in UDP header, and protocol / next header field in IP header). If load balancing between MPLS overlays is required, an IP address pool can be used at the two LSRs in the QUIC connection, and then allocated to send MPLS overlay packets across the IP address pool.
[0271] It should be understood that, although omitted for purposes of clarity, the various example embodiments presented herein for supporting MPLS-in-TCP, MPLS-in-SCTP, and MPLS-in-QUIC may be used or adapted for use to provide MPLS-in-TLS.
[0272] Fig.67An example embodiment of a method for supporting reliability of coverage is depicted. It should be understood that although primarily presented as being executed serially, at least a portion of the blocks of method 6700 may be executed simultaneously or in different Fig.67 At block 6701, method 6700 begins. At block 6710, communication of a packet is supported by a communication device, the packet including a payload, a header of a label switching protocol on the payload, and a header of a reliable transport layer protocol on the header of the label switching protocol. At block 6799, method 6700 ends.
[0273] Various example embodiments for supporting reliability of overlays may provide various advantages or potential advantages. For example, various example embodiments for supporting reliability of overlays may avoid reliance on relatively unreliable overlays (e.g., in the case of MPLS overlays, relatively unreliable overlays such as MPLS-in-IP overlays, MPLS-in-GRE overlays, MPLS-in-UDP overlays, etc.); however, it should be understood that at least some such overlays may still be used for various purposes. Various example embodiments for supporting reliability of overlays may provide various other advantages or potential advantages.
[0274] Fig.68 An example embodiment of a computer suitable for performing the various functions presented herein is depicted.
[0275] The computer 6800 includes a processor 6802 (e.g., a central processing unit (CPU), a processor, a processor having a processor core set, a processor core of a processor, etc.) and a memory 6804 (e.g., a random access memory, a read-only memory, etc.). The processor 6802 and the memory 6804 may be communicatively connected. In at least some example embodiments, the computer 6800 may include at least one processor and at least one memory including computer program code, wherein the at least one memory and the computer program code are configured to cause the computer to perform various functions described herein together with the at least one processor.
[0276] The computer 6800 may also include a cooperating element 6805. The cooperating element 6805 may be a hardware device. The cooperating element 6805 may be a process that may be loaded into the memory 6804 and executed by the processor 6802 to implement the various functions presented herein (e.g., in this case, the cooperating element 6805 (including associated data structures) may be stored in a non-transitory computer-readable storage medium, such as a storage device or other suitable type of storage element (e.g., a magnetic drive, an optical drive, etc.)).
[0277] The computer 6800 may also include one or more input / output devices 6806. The input / output devices 6806 may include one or more user input devices (e.g., keyboard, keypad, mouse, microphone, camera, etc.), user output devices (e.g., display, speaker, etc.), one or more network communication devices or elements (e.g., input ports, output ports, receivers, transmitters, transceivers, etc.), one or more storage devices (e.g., tape drive, floppy disk drive, hard disk drive, compact disk drive, etc.), etc., and various combinations thereof.
[0278] It should be understood that the computer 6800 may represent a general architecture and functionality suitable for implementing the functional elements described herein, portions of the functional elements described herein, etc., and various combinations thereof. For example, the computer 6800 may provide a general architecture and functionality suitable for implementing one or more elements presented herein (such as a network device (e.g., a router, etc.), a network controller, etc.), and various combinations thereof.
[0279] It should be understood that at least some of the functionality presented herein may be implemented in software (e.g., by implementing the software on one or more processors for execution on a general-purpose computer (e.g., via one or more processors) to provide a special-purpose computer, etc.) and / or may be implemented in hardware (e.g., using a general-purpose computer, one or more application-specific integrated circuits, and / or any other hardware equivalents).
[0280] It should be understood that at least some of the functionality presented herein can be implemented in hardware, for example, as a circuit system that cooperates with a processor to perform various functions. Portions of the functions / elements described herein can be implemented as a computer program product, wherein computer instructions, when processed by a computer, regulate the operation of the computer so that the methods and / or techniques described herein are invoked or otherwise provided. Instructions for invoking various methods can be stored in fixed or removable media (e.g., non-transitory computer-readable media), transmitted via a data stream in a broadcast or other signal-bearing medium, and / or stored in a memory within a computing device that operates in accordance with the instructions.
[0281] It should be understood that the term "or" as used herein refers to a non-exclusive "or" unless stated otherwise (eg, use of "otherwise" or "or in the alternative").
[0282] It should be understood that although various embodiments that incorporate the teachings presented herein have been shown and described in detail herein, those skilled in the art can readily devise many other varied embodiments that still incorporate these teachings.
Claims
1. A device for communication, include: at least one processor; as well as at least one memory storing instructions; wherein the instructions, when executed by the at least one processor, cause the device to: Communication of packets is supported through a communication device, wherein the packets include a payload, a header of a label switching protocol encapsulating the payload, a control header of the label switching protocol encapsulating the header of the label switching protocol and the payload, and a header of a reliable transport layer protocol encapsulating the control header of the label switching protocol, the header of the label switching protocol, and the payload.
2. The apparatus of claim 1, wherein the header of the label switching protocol comprises a label set. The apparatus of claim 2 , wherein the set of labels is organized as a label stack.
4. The apparatus according to any one of claims 1 to 3, wherein the label switching protocol is associated with a label switching overlay.
5. The apparatus of any one of claims 1 to 3, wherein the label switching protocol comprises a Multi-Protocol Label Switching (MPLS) protocol.
6. The apparatus according to any one of claims 1 to 3, wherein the reliable transport layer protocol comprises a connection-oriented transport layer protocol.
7. The apparatus according to any one of claims 1 to 3, wherein the reliable transport layer protocol comprises a transport layer protocol configured to support at least one of the following: flow control or congestion control.
8. The apparatus according to any one of claims 1 to 3, wherein the reliable transport layer protocol comprises a transmission control protocol (TCP), a stream control transmission protocol (SCTP), a quick user datagram protocol (UDP) Internet connection (QUIC) protocol, or a transport layer security (TLS) protocol.
9. The apparatus of claim 1, wherein the control header is configured to indicate the size of the payload and the header of the label switching protocol.
10. The apparatus according to any one of claims 1 to 3, wherein the packet includes a header of the network layer protocol encapsulating the header of the reliable transport layer protocol, the control header of the label switching protocol, the header of the label switching protocol, and a header of the network layer protocol of the payload.
11. The apparatus of claim 10, wherein the network layer protocol comprises Internet Protocol (IP).
12. The apparatus according to claim 10, wherein the packet includes the header of the network layer protocol, the header of the reliable transport layer protocol, the control header of the label switching protocol, the header of the label switching protocol, and the header of the data link layer protocol of the payload.
13. The apparatus of claim 12, wherein the data link layer protocol comprises at least one of: Ethernet or Point-to-Point Protocol (PPP).
14. The apparatus of any one of claims 1 to 3, wherein to support communication of the packet, the instructions, when executed by the at least one processor, cause the apparatus to: generating the packet by the communication device; and The packet is sent to a next-hop node through the communication device.
15. The apparatus of any one of claims 1 to 3, wherein to support communication of the packet, the instructions, when executed by the at least one processor, cause the apparatus to: receiving, by the communication device, the packet; and The packet is processed.
16. The apparatus of any one of claims 1 to 3, wherein the instructions, when executed by the at least one processor, cause the apparatus to: Communication of an overlay initiation frame is supported by the communication device, the overlay initiation frame being configured to convey one or more overlay parameters for overlay supported between the communication device and a remote communication device based on the reliable transport layer protocol. 17 . The apparatus according to claim 16 , wherein the coverage initialization frame is transmitted by the communication device to the remote communication device, or is received by the communication device from the remote communication device.
18. A non-transitory computer-readable medium storing instructions that, when executed by an apparatus, cause the apparatus to: Communication of packets is supported through a communication device, wherein the packets include a payload, a header of a label switching protocol encapsulating the payload, a control header of the label switching protocol encapsulating the header of the label switching protocol and the payload, and a header of a reliable transport layer protocol encapsulating the control header of the label switching protocol, the header of the label switching protocol, and the payload.
19. A method for communication, include: Communication of packets is supported through a communication device, wherein the packets include a payload, a header of a label switching protocol encapsulating the payload, a control header of the label switching protocol encapsulating the header of the label switching protocol and the payload, and a header of a reliable transport layer protocol encapsulating the control header of the label switching protocol, the header of the label switching protocol, and the payload.
