Method for connecting devices via a global virtual network across a network fabric

Abstract

Systems and methods are disclosed for using a network carpet to connect devices via a virtual global network spanning a network fabric. The network system includes a first access point server in communication with a first backbone switch server, a second access point server in communication with a second backbone switch server, and a network carpet including a first communication path connecting the first and second access point servers and a second communication path connecting the first and second backbone switch servers.

Description

[0001] This patent application is a divisional application of patent application with application number 201680032657.3, filed on June 13, 2016, and entitled "System and method for network carpet multi-protocol integration".

[0002] This application claims priority to U.S. Provisional Application No. 62 / 174,394, filed June 11, 2015, which is hereby incorporated by reference herein. TECHNICAL FIELD

[0003] The present disclosure relates generally to networks, and more specifically to traffic flow through a global virtual network that spans various network fabrics integrated into a larger network carpet. BACKGROUND

[0004] The first deployment of "networks" was typically composed of a topology with a large central computer core, such as a mainframe, along with slave terminals connected directly to it in the same facility. This instantiation of mainframe and terminals had the particular advantage of allowing distributed physical access, but in the past, all users had to be in close proximity to the core. As long distance network transmission improved, slave terminals were able to be located at remote locations further from the mainframe. Today, this topology can be referred to as a central server and thin client devices connected to it.

[0005] Power and storage then shifted to personal computers (PCs), whose local CPUs, RAM, and storage devices allowed computation to be contained within the PC. Today, the pendulum is swinging back. The rise of the personal computer was the driving force behind the development of wired networking technology, then laptops (portable computers) were the driving force for wireless networks, then mobile phones, smart phones, tablets, phablets, and other types of mobile and wireless devices were the driving force for improvements in both wired and wireless network infrastructure.

[0006] Improved mobile device and last mile Internet connectivity has spurred growth in services where host clients store, access, and retrieve their data through servers in the cloud. The Internet of Things (IoT) means more and more connected devices - many of these in LANs, PANs, piconets, etc. - and most of these devices must be discoverable not only in the upstream connectivity, but also in the Internet.

[0007] Device power over the Internet is changing. Some can tolerate less than perfect connectivity, while others have absolute requirements for low latency, zero packet loss, and high bandwidth to function properly. As devices continue to grow, a large number of devices will present problems that need to be solved. These problems include how to reliably connect all of these devices, how to efficiently find all of these devices, and how to transfer large amounts of data between them and large data sinks.

[0008] The Internet is made up of connected devices that make up networks, networks that make up networks. As networking continues to evolve, core protocols and network types continue to mature, they have expanded to the point where network types can be referred to as network fabrics. Public network fabrics build on top of standard protocols, such as IPv4 and IPv6, on top of Ethernet standards, Fibre Channel, InfiniBand, and various other network protocols and types.

[0009] Network fabrics can be defined as networks under the management of a single entity that peer with other networks on a one-to-one basis defined as a single backbone, or one-to-many network relationships via multi-backbone peering relationships. Network fabrics can also define the scale and scope of network protocol types from end-to-end. Ethernet defines a network, but this can be further categorized by Internet Protocol on top of Ethernet, then by IP version such as IPv4 for Internet Protocol version 4 or IPv6 for Internet Protocol version 6, and other network types. On top of Internet Protocol (IP) are protocols such as Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). TCP / IP is more verbose, having excessive built-in error checking and handling for reliability of data sent relative to UDP, which has strict error checking without more fluid flow control combined. This makes UDP more suitable for streaming data, such as audio or video broadcasts, where lost packets will have no significant adverse impact on the consumer experience.

[0010] In addition to different protocols and IP versions built on top of Ethernet, Ethernet itself has different versions, such as Ethernet, Gigabit Ethernet (available at 1 or 10 or 40 or 100 gigabits) plus other versions expected to be introduced as carrying capacity technology improves.

[0011] InfiniBand (IB) is an alternative to Ethernet, IB utilizes different physical NIC ports, wires, and plugs, and IB works in a similar but different manner than IP.

[0012] Ethernet is currently the most popular protocol for connecting various computing devices together so that they can communicate with each other or at least pass data. InfiniBand (IB) is a preferred choice for connecting many nodes into a high performance computing (HPC) environment. IB allows natural remote direct memory access (RDMA) between nodes, thus bypassing network authentication and the lifting process and operating system (O / S) stack of connecting RDMA storage (or other) devices. This facilitates hosting a parallel file system (PFS) providing simultaneous and fast access to many devices.

[0013] To further define the scope, each network infrastructure protocol, such as Ethernet or InfiniBand, and the subsequent network protocols running on top of them can be defined as a fabric. At the point of interconnection between fabrics, successful cross-connection requires a technology such as network address translation (NAT) or equivalent. A network protocol such as IPv4 can be encapsulated so that its packets run on top of another protocol, such as IB via a "wrapper" protocol, for example, IP over IB (IPoIB). If one wishes to connect the various distributed nodes of a parallel file system (PFS) over a network consisting of some non-IB segments, such as Ethernet, a wrapper can be utilized, such as RDMA over converged Ethernet (RoCE).

[0014] While RoCE can allow RDMA access, there is a downside in that the underlying Ethernet does not support the true benefits of IB, thus will exhibit performance lag compared to RDMA over native IB.

[0015] Different types of clients and their users have different expectations and requirements for using the current Internet. These expectations also define quality of service (QoS) requirements for each of these various uses. At the maximum requirement end of the QoS spectrum are client users requiring high quality systems characterized by 100% reliability and availability, highest bandwidth, lowest latency. Some examples are: high performance computing (HPC) - one of the most demanding conditions is HPC where the amount of data is huge, distributed in globally dispersed locations, and needs to be transmitted 100% lossless with possibly the lowest latency. A parallel file system (PFS) is often used with UPC for clients to access a central or distributed data store from local and remote locations.

[0016] financial industry - while the traditional communication needs of the financial industry conducting trades utilize smaller sized packets, the required bandwidth must be congestion free, with absolutely the lowest possible latency, with 100% reliability. A few nanoseconds can be problematic, there can be no loss. Round trip time (RTT) is very critical as trade messages not only have to go through, but must return a confirmation of successful transmission as soon as possible.

[0017] Mass media - high definition coverage of live video streams of sporting events, news broadcasts and other purposes require high bandwidth and low latency.

[0018] At the other end of the Qos spectrum are client users running applications that can tolerate some degree of packet loss and where latency and / or bandwidth requirements are not mission critical. Some examples are:

[0019] Streaming audio - such as internet radio, the bandwidth requirements are moderate, a little periodic loss will not matter, only presenting a momentary static bit.

[0020] RSS text streams - these require very little bandwidth but lossless transmission, in most cases latency is not a significant factor.

[0021] Data backup (off hours) - requires good enough bandwidth and latency to allow data to be sent and acknowledged, but it is not reasonable to spend extra cost for premium lines.

[0022] Voice calls - where two-way audio consumes lower bandwidth, a little loss represents a momentary little line static noise.

[0023] Email sending / receiving - requires moderate bandwidth and "good enough" latency to allow messages to pass through. Higher capacity servers and commercial grade message transmission require better QoS.

[0024] At the lowest QoS requirement needs, bandwidth effectiveness and latency can go up or down, but users can tolerate this fluctuation because they are not willing to pay more for better service.

[0025] In the middle of the two extremes are mainstream client users with various levels of QoS expectations and requirements. Within the mainstream, there is also granularity within the range of low to high level expectations. Some examples are:

[0026] The high end of the mainstream is made up of banks, corporations and various other types of organizations that require WAN connectivity between office and / or centrally distributed applications, with many distributed "thin clients" connected to larger central systems.

[0027] The middle of the mainstream - cloud servers in IDC / CND / etc. that act as consumers and SME clients.

[0028] The low end of the mainstream - budget conscious home users.

[0029] In summary, QoS requirements often drive which type of network to employ, budget constraints are a factor in the quality criteria affecting "online" purchases.

[0030] Ethernet is a combination of networking technologies and is the most widely used type of network, deployed from office LANs, data centers and other clusters of devices to the global backbone of the Internet.

[0031] Ethernet became the dominant network type, with its widespread adoption common in LANs and the broader Internet, because it was a standard that was relatively easy to implement and globally deployed. As more and more devices utilized one protocol or network type, network effects began to come into play, because it made it easier for others to decide to adopt similar technology for compatibility and other reasons.

[0032] In data centers, where concentrated computing, storage, processing and other operations are distributed across various rack-mounted servers, faster transmission than Ethernet is needed to connect these servers together behind the channel to share data.

[0033] Fiber Channel and InfiniBand (IB) are two such technologies that provide ultra-low latency and high capacity bandwidth. The lossless and parallel transmission of IB provides strong advantages, allowing the use of Remote Direct Memory Access (RDMA), and also provides the opportunity to deploy and utilize a globally distributed Parallel File System (PFS). The limitation of IB is that it is deployed only at short distances measured in meters. This was later extended to a few kilometers. Until recently, IB "long distance" links were limited to within a city or between two close metro areas connecting data centers to each other via superfast IB over dedicated lines. Now some technologies allow extending the distance of IB and sending up to 20,000 kilometers between two devices over dark fiber lines. For example, the innovation in the physical layer developed by Bay Microsystems and Obsidian Research provides various advantages, such as low latency of IB, and the ability to do long distance RDMA over IB via dark fiber between remote areas.

[0034] The Ethernet Internet uses TCP / IP, UDP / IP and IPv4, IPv6 addressing protocols from LAN to Internet to LAN. The last mile connection refers to connecting the LAN to the ISP's network to the Internet via POP.

[0035] Ethernet has a store-and-forward model, where a packet is received, studied and then forwarded only after the payload has been completely received and studied. The delay in processing Ethernet traffic packets within computer / router network equipment network devices is approximately 100 microseconds (μs).

[0036] InfiniBand (IB) - extremely low latency compared to Ethernet. It is also much more concise than TCP / IP or UDP / IP. It runs over dark fiber connections. It is still faster than Ethernet over dark fiber if using native IB / RDMA over IB, which can measure one-way for effective transfer, as opposed to double- measured latency like RTT for Ethernet. IB bandwidth under load reaches 90 to 96 percent of the theoretical BW maximum, close to true wire speed. IB is characterized by near-end switching, where it receives the header of a packet, uses logic for forwarding decisions, and passes on the payload of the packet. Although IB is traditionally used within data centers, IB has evolved to burst as a true global transport due to technologies that extend IB over long distances. These new technologies extend IB over dark fiber to reach great distances, up to 20,000 km.

[0037] Remote Direct Memory Access (RDMA) over IB takes advantage of zero-copy networking, where packets can be sent directly through the IB NIC. This reduces CPU load and reduces latency for packets to 1 microsecond (μβ).

[0038] Parallel File System (PFS) takes advantage of RDMA to provide distributed files and folders between various devices, and when combined with long-distance IB, PFS clusters provide fast file access to / from remote file repositories at near wire speed.

[0039] When comparing network types, reliability is the most important. The main drivers that affect network type, network protocol, and physical pathway are time and distance. Latency is the time it takes for data to travel in one direction or the round-trip time (RTT) to travel a specified distance between two points.

[0040] In computing, the main measure of time for networking is in milliseconds (ms), and for processing is in microseconds (μβ) or nanoseconds (ns). Thus, the granularity of a time can be measured in fractions or decimals. For example, 1 / 20 or 1 / 10 or 1 / 100 of a millisecond.

[0041] # Description second millisecond (ms) microsecond (μβ) 1 1 / 10 of a second 0 .10000 100 100，000 2 1 / 20 of a second 0 .05000 50 50，000 3 1 / 100 of a second 0 .01000 10 10，000 4 10 microseconds 0 .00001 0 .010 10 5 100 microseconds 0 .00010 0 .100 100 6 1000 microseconds 0 .00100 1 .000 1，000

[0042] Table 1 - Measures of Time

[0043] How fine the granularity of a moment is determined by the processing power of the device and other factors. Latency is typically measured in milliseconds, and is affected by network type, protocol, distance, network load, congestion, and other factors.

[0044] miles per second in a vacuum miles per second in a fiber light efficiency speed of light 186，282 .34 126，759 .88 68 .05％

[0045] Table 2 - Fiber line speed considering fiber latency drag

[0046] Table 2 compares the speed of light in a vacuum and the speed of light inside the core of fiber optic glass. This represents the physical limit of fiber optic efficiency and establishes a baseline for the theoretical best speed achieved through fiber optics. Although the refractive index of the cable can vary slightly, an average of approximately 203 to 204 m / ps is used, with an efficiency of 68.05%, the speed of light is 299.792 m / ps.

[0047] The maximum number of available IPv4 IP addresses is limited by the 32-bit IP address, which actually has a maximum of 4294967296 (two to the thirty-second power) IPv4 addresses. Of this total, there are approximately 588514304 reserved addresses, leaving only 3706452992 public addresses available. Although the Internet Protocol version four (IPv4) is widely deployed, it can be characterized as a victim of its own success, as the available IPv4 addresses are almost completely exhausted. Although techniques such as NATing specifically address this problem for devices in a LAN, the problem remains unsolved, with few unallocated IPv4 addresses.

[0048] IPv6 IP addresses provide an apparently inexhaustible supply in the event that the IPv4 addressing system reaches a point of exhaustion where there are few, if any, available IPv4 addresses. IPv6 IP addresses are 128 bits, and thus, the number of available IP addresses is enormous, approximately 340 undecillion or 340282366920938463463374607431768211456 possible IPv6 addresses. Although the number of available IP addresses under IPv6 is almost unlimited compared to the availability of IPv4 addresses, the technology has spread slowly on a global scale, which limits the practicality of its deployment.

[0049] Many legacy networks are built with devices that can still only handle IPv4 addresses, which presents a problem. IPv6 appears to have an abundant supply of available IP addresses at its core, but IPv6 has not been deployed universally due to several factors, one of which is the capital investment in legacy equipment that only handles IPv4 and not both IPv4 and IPv6. The address constraints of IPv4 remain until the legacy systems are replaced or upgraded to accommodate both IPv4 and IPv6.

[0050] Ethernet protocols have high latency, poor efficiency, and low utilization over long distances, with less than 25% efficiency in terms of line capacity when compared to infinite bandwidth. The problem is exacerbated in the case of long distance transmission of data that is negatively affected by the performance deficiencies of IP-based network protocols and the subsequent backwash of bandwidth delay product (BDP) at uneven peers and other inherent deficiencies in the native functionality of the protocol.

[0051] Internet connectivity is shared by the public over ISP lines and therefore is not as reliable as a private line such as MPLS or DDN. Ethernet bandwidth (BW) drops to a very low percentage of the theoretical maximum BW under load and over long distances.

[0052] There are also known connectivity issues for peering at the network edge, across multiple long distance network boundaries, and across network hetero fabric. These issues and challenges are addressed by a global virtual network, described in U.S. Provisional Patent 62 / 108987, the contents of which are incorporated by reference.

[0053] TCP / IP is very verbose, utilizing a store-and-forward model that requires acknowledgements. It is prone to congestion slowdowns and bottlenecks through Internet hops between non-equivalent network segments. The result is higher latency and / or packet loss due to congestion or other factors. When a TCP / IP packet is lost or otherwise not delivered, the sender attempts to resend to ensure delivery. This can place a high demand on hardware resources including RAM and processor usage. The inevitable result of this demand is the need for more hardware to push the amount of traffic (relative to the equivalent amount of traffic that can be handled by an infinite bandwidth), resulting in increased cost and physical space requirements. Additionally, this results in higher levels of energy consumption. UDP / IP is unidirectional and does not require the receiving side to send an acknowledgement packet to the sender. This provides a significant speed advantage over TCP / IP, however, the tradeoff for this speed gain is that there is no way for the sender or receiver to discover a loss of a packet during network congestion or other factors affecting reliability if the packet is lost in transit.

[0054] InfiniBand (IB) over dark fiber has advantages, but it requires expensive equipment at both ends of a dedicated point-to-point fiber. In addition to the need for expensive HW edge equipment at each end, dark fiber requires a high monthly operating cost. If the line is cut or fails, there is no automatic failover. It is also an IB-only network, so expensive IB cards must be installed on every device within the network that will utilize the facility. Specialized skills are also required for installation and subsequent normal operation. Therefore, integration technology is required to realize the full benefits of RDMA over IB, which requires upfront and ongoing investments in equipment and labor.

[0055] If a global IB-only network is to be built, the hardware and integration effort requires a large capital investment. For point-to-multipoint topology integration, technical personnel are required to build the architecture and remain responsible for monitoring and maintenance. Despite the advantages of IB multi-homed to the last mile, the upfront cost of hardware endpoint devices and recurring operating costs of dark fiber between each point, and the point-to-point topology present price and technical barriers that only the largest and most well-funded organizations can overcome.

[0056] Today, organizations have the flexibility to deploy many types of networks within their LANs and WANs under their direct control, including IPv4, IPv6, InfiniBand, Fibre Channel, and other network types. If they wish to have an end-to-end network fabric over long distances, current solutions require them to have dedicated lines and invest in central appliances to enhance these WAN connections.

[0057] In summary, TCP / IP provides reliability at the cost of verbosity, and as a result, it is slower. It requires sending a packet and returning an acknowledgement. Thus, the time it takes for a packet to reach its destination and for the acknowledgement to return to its source is measured as the latency of the round-trip time (RTT). UDP / IP does not require returning an acknowledgement. However, UDP cannot tolerate errors and losses as well as TCP. Without flow control, UDP does not suffer from the same degree of congestion problems as TCP, but it can still suffer from the inefficiencies of the IP protocol. Thus, if a UDP packet is lost, neither the sender nor the receiver can know. The advantage of IB is ultra-low latency with parallel transmission, but it is not widely deployed and requires its own hardware NICs, cables, routers, and other appliances to work. IP and IB are not plug-and-play compatible. In order to send IP over IB, it must be encapsulated as IP over InfiniBand (IPoIB) because this is not inherent to the IB protocol. IB has many advantages, but it is relatively more expensive. SUMMARY

[0058] Systems and methods are disclosed for using a network fabric to connect appliances via a virtual global network spanning a network fabric. In one embodiment, the network system can include a first access point server in communication with a first backbone switch server, a second access point server in communication with a second backbone switch server, and a network fabric including a first communication path connecting the first and second access point servers and a second communication path connecting the first and second backbone switch servers. In one embodiment, the first communication path is IP over the Internet. In another embodiment, the second communication path is InfiniBand over dark fiber.

[0059] In other embodiments, the network system further includes a first parallel file storage in communication with the first backbone switch server, a second parallel file storage in communication with the second backbone switch server, and the first backbone switch server is capable of writing directly to the second parallel file storage using the second communication path without using the first communication path.

[0060] In other embodiments, the network system further comprises a first firewall in the communication path between the first access point server and the first backbone switch server, the first firewall isolating the first backbone switch server from threats present on the first communication path. In yet another embodiment, the network system further comprises a second firewall in the communication path between the second access point server and the second backbone switch server, and the second firewall isolates the second backbone switch server from threats present in the second communication path.

[0061] In another embodiment, the network system further comprises an endpoint device in communication with the first access point server and a host server in communication with the second access point server. The communication protocol between the endpoint device and the host server can be one of Infiniband, RDMA, IPv4, and IPv6 or others. The communication protocol can be encapsulated in a different protocol between the endpoint device and the first access point server. The communication protocol can be encapsulated in a different protocol between the second access point server and the host server. The communication protocol can be encapsulated in a different protocol between the first backbone switch server and the second backbone switch server. BRIEF DESCRIPTION OF DRAWINGS

[0062] For the purposes of this disclosure, like numbers in the figures indicate like structural elements. These figures are not to scale, and are merely intended as illustrative, and are not intended to limit the scope of the disclosure in any way.

[0063] Figure 1 The basic logic of a sequentially chained network path is shown.

[0064] Figure 2 The topology of a multi-link segment with failover is shown.

[0065] Figure 3 The global node topology for a global virtual network is shown.

[0066] Figure 4 The framework for defining and describing network fabrics or segment properties within the fabric is shown.

[0067] Figure 5 The global node and performance zones are shown.

[0068] Figure 6 The global node and performance zones are shown.

[0069] Figure 7 The simple network topology of a global virtual network arranged in a hub and spoke configuration is shown.

[0070] Figure 8A simple network topology of a global virtual network arranged in a hub and spoke configuration is also shown.

[0071] Figure 9 A trunk and network segment in two regions connected by an Internet long haul segment is shown.

[0072] Figure 10 A GVN tunnel between two LANs is shown.

[0073] Figure 11 Combining various network fabrics into an end-to-end path is shown. Figure 12 Potential problems with a bottleneck through a trunk between two network segments are shown.

[0074] Figure 13 Equations for calculating bandwidth delay product (BDP) for a connected network segment are shown.

[0075] Figure 14 Combining various network fabrics into an overall network carpet is described.

[0076] Figure 15 Logic for enhancing algorithms for advanced smart routing (ASR) within a global virtual network (GVN) is described.

[0077] Figure 16 Total potential bandwidth relative to line bearing capacity compared to actual usage is shown.

[0078] Figure 17 A simple topology of a global virtual network (GVN) made up of endpoint devices (EPD) connected to a service access point server (SRV_AP) and the like is shown.

[0079] Figure 18 A simple topology of a global virtual network (GVN) made up of endpoint devices (EPD) connected to a service access point server (SRV_AP) and the like is also shown.

[0080] Figure 19 A topology of endpoint devices (EPD) connected via a plurality of tunnels to a plurality of service access point servers (SRV_AP) is shown.

[0081] Figure 20 A simplified wide area network (WAN) built by combining networks of two endpoint devices (EPD) connected to each other via a global virtual network (GVN) is shown.

[0082] Figure 21 A simple network topology of two LANs connected via a WAN is shown.

[0083] Figure 22 IP versus infinite bandwidth latency is compared.

[0084] Figure 23 A simple topology of a global virtual network (GVN) is shown, made up of endpoint devices (EPD) connected to access point servers (SRV_AP), etc.

[0085] Figure 24 Possible paths that a passenger can take if they walk or ride a train from the ticket check to the gate are shown.

[0086] Figure 25 Possible configurations of the physical backplane of various devices working in a network like a global virtual network (GVN) are shown.

[0087] Figure 26 Two types of network paths through a global virtual network (GVN) are shown.

[0088] Figure 27 Four different network channels between two access point servers (SRV_AP) are shown.

[0089] Figure 28 How multiple endpoint devices (EPD) can connect to an access point server (SRV_AP) in a region is shown.

[0090] Figure 29 The logical construction of links between devices in a global virtual network (GVN) is shown.

[0091] Figure 30 The logical construction of links between devices in a global virtual network (GVN) is also shown.

[0092] Figure 31 An example topology of devices within a GVN, including a backbone switch server (SRV_BBX) topology and a gap API sequence, is shown.

[0093] Figure 32 A series of API calls between a GVN device and a SRV_CNTRL within a GVN are shown.

[0094] Figure 33 Information flow between a device in a GVN and a central control server (SRV_CNTRL) is shown.

[0095] Figure 34 Locating a device into various internet data centers (IDC) is shown.

[0096] Figure 35 The three layers of a GVN and how they interact are shown.

[0097] Figure 36Fabric and tunneling within fabric are shown.

[0098] Figure 37 Logical visual representation of different network fabrics that are woven into a global virtual network (GVN) network carpet.

[0099] Figure 38 Base connectivity showing one end as Ethernet fabric, middle as fiber optic InfiniBand, and other end as Ethernet or InfiniBand.

[0100] Figure 39 Two network paths are shown, base network connection path at tier one of GVN and tunnel at tier three of GVN.

[0101] Figure 40 Multiple tunnels between devices within a global virtual network (GVN) spanning multiple regions are shown.

[0102] Figure 41 Framework for running parallel tunnel tests to measure latency, bandwidth, packet loss, and other measurements is shown.

[0103] Figure 42 Algorithm for running a series of tests in parallel for mid-path connectivity is shown.

[0104] Figure 43 Diagram for describing network options.

[0105] Figure 44 Also a diagram for describing network options.

[0106] Figure 45 Flowchart for running tests and for algorithms for remedial action to be taken when a problem is detected.

[0107] Figure 46 Topology of a global virtual network (GVN) is shown, showing paths from an endpoint device (EPD) to the Internet in the same region.

[0108] Figure 47 End-to-end cross-region network path is shown.

[0109] Figure 48 How a GVN is built as the top tier of an over-the-top network connection (OTT1) is shown.

[0110] Figure 49 One possible topology of a GVN is shown, where traffic has more than one option for long haul between regions.

[0111] Figure 50 Cross-region traffic lanes between SRV APs are shown.

[0112] Figure 51 is an algorithmic flow chart depicting how path information is collected, saved and used to determine the best path for traffic to take through the GVN.

[0113] Figure 52 shows how end-to-end native RDMA can be provided using the topology of the global virtual network (GVN).

[0114] Figure 53 shows how a globally distributed parallel file system (PFS) can allow seamless access to three parallel file system (PFS) storage nodes, allowing native RDMA access over the GVN carpet, top-of-the-variety non-native network fabric (OTT).

[0115] Figure 54 also shows how a globally distributed parallel file system (PFS) can allow seamless access to three parallel file system (PFS) storage nodes, allowing native RDMA access over the GVN carpet, top-of-the-variety non-native network fabric (OTT).

[0116] Figure 55 shows how devices connected via the GVN can have direct RDMA access to parallel file system (PFS) devices in various regions.

[0117] Figure 56 shows how files are stored, cataloged, found and accessed in a distributed parallel file system.

[0118] Figure 57 shows the operation of the global file manager (GFM) on each device in the GVN and the operation of the central global file manager (CGFM) on the central control server (SRV CNTRL).

[0119] Figure 58 shows the geographic destination mechanism, where various modules are distributed among devices such as endpoint devices (EPD), access point servers (SRV AP), central control servers (SRV CNTRL) and backbone exchange servers (SRV BBX).

[0120] Figure 59 shows the geographic destination mechanism within the GVN.

[0121] Figure 60 also shows the geographic destination mechanism within the GVN.

[0122] Figure 61 shows bridging two LANs into a wide area network (WAN).

[0123] Figure 62Multiple path options are shown for transferring files between an endpoint device (EPD) connected to an access point server (SRV_AP) in one region and another EPD connected to an access point server (SRV_AP) in another region.

[0124] Figure 63 Full isolation of the IBB path is shown such that internal communications are on a clean and secure path.

[0125] Figure 64 A topology of sequential linear point-to-point connections over large distances from region A to / from region B is shown.

[0126] Figure 65 Logical structure of physical and virtual interfaces on an endpoint device (EPD) and their corresponding connections to devices outside the EPD are shown.

[0127] Figure 66 A conceptual model is shown to describe the layers of the Global Virtual Network (GVN) hierarchy level one and the layers of hierarchy level three built on and integrated with level one.

[0128] Figure 67 Level one of the IP model of the GVN is shown in comparison to the IP model of hierarchy level three of the GVN in a stacked top organization.

[0129] Figure 68 The base Internet layer and first and second level top layers (OTT1 and OTT2) are shown.

[0130] Figure 69 System diagram for some example devices in the GVN utilizing a network blanket. DETAILED DESCRIPTION

[0131] Abbreviations used herein include:

[0132] Abbreviation Abbreviation Expansion

[0133] API Application Programming Interface

[0134] ASR Advanced Smart Routing

[0135] BW Bandwidth

[0136] CAPEX Capital Expenditure

[0137] CDA Content Delivery Agent

[0138] CPA Content Pull Agent

[0139] CPU Central Processing Unit

[0140] DMA Direct Memory Access

[0141] E IP egress / ingress point

[0142] EPD endpoint device

[0143] Geo-D geographic destination

[0144] GFM global file manager

[0145] HFS hierarchical file system

[0146] HPC high performance computing

[0147] IAB Internet Architecture Board

[0148] IB InfiniBand

[0149] IETF Internet Engineering Task Force

[0150] IOPS input / output operations per second

[0151] IoT Internet of Things

[0152] IPv4 Internet Protocol version four (4)

[0153] IPv6 Internet Protocol version six (6)

[0154] ISP Internet Service Provider

[0155] MPLS Multiprotocol Label Switching

[0156] NAPIM neutral API mechanism

[0157] NetTap network tap

[0158] OTT over-the-top

[0159] OTT1 first level OTT

[0160] OTT2 second level OTT

[0161] PEDP portable endpoint device

[0162] PFS parallel file system

[0163] RAM random access memory

[0164] RDMA remote direct memory access

[0165] RFB remote grabber bot

[0166] SFS secure file storage

[0167] SNO secure network optimization

[0168] SRV_AP Access Point Server

[0169] SRV_BBX Backbone Exchange Server

[0170] SRV_CNTRL Central Server

[0171] Tapestry Network Tapestry

[0172] TCP / IP Transmission Control Protocol / Internet Protocol

[0173] UDP / IP User Datagram Protocol / Internet Protocol

[0174] ps microsecond.

[0175] A network tapestry is a union of one or more network fabrics. The prior art can automatically connect together and integrate various fabrics into a tier three within or over the top (OTT) of a global virtual network (GVN) that itself is over the base Internet or fiber, into a peer-to-peer seamless end-to-end network that is parallel to each other. This effective union of fabrics can also be seen as combining various network segments in between the longer network paths (ITM). The problems and issues solved by the global virtual network (GVN) and general GVN description and operation can be found in U.S. Provisional Patent Application No. 62 / 089113.

[0176] ISP-provided local Internet connections are designed for best connectivity within their network. This is why locally hosted and locally CDNed websites perform the best. They naturally perform better because they are closer and they also have strong peering relationships in the same region without one or more parties controlling the network from outside the region peering edge.

[0177] A GVN with broad SRV_AP coverage provides EPD or PEPD a "local" access point into the GVN, over the top of the existing Internet connection provided at the point of connection via their ISP, most commonly a point of presence (POP), that extends to all points on the global Internet. The GVN leverages from LAN to the nearest SRV_AP, then over the top (OTT) of shared high performance network links, with the hub connections separating large distances and connecting to individual regions in the destination hubs. This consumption model provides low barriers to entry via low cost equipment, and a fractional and proportional pay-for-use model of large capacity fiber. The GVN is easy to deploy and operate, and can include advanced smart routing (ASR). The end-to-end network is configured to automatically generate connections and make automatic adjustments to changing conditions as needed.

[0178] The network blanket offered by GVN is enabled by providing an end-to-end solution that offers the most efficient security network optimization (SNO) services in an automated fashion. The network blanket is easy to install, easy to configure and easy to use. The network blanket results in cost savings because no dedicated lines are required, bandwidth models or consumption models can be used, there are low entry barriers and access to advanced connection features that are otherwise unavailable or unaffordable to most customers is provided.

[0179] The figures are grouped in the following sections.

[0180] Simple network topologies: These figures show simple networks, one with redundancy and one without.

[0181] Global network, node and distance related performance and other factors: These figures show the impact of distance on the network and define the performance versus proximity ratio.

[0182] About GVN - topology and features: These figures provide a simple introductory explanation of the hub and spoke topology of the devices within the global virtual network (GVN) to show the end-to-end performance enhancement and optimization.

[0183] Path - characteristics of the relay segments, network segments, issues at fabric junctions: These figures show the network devices, network segments between relay segments at peering points, how the GVN sits on top of the base path (OTT), how a typical path is made up of network segments each with different specifications, the impact of the bandwidth delay product and other descriptions of network conditions.

[0184] GVN overview of example topologies and options: These figures show some example topologies of the GVN and how it can connect various fabrics together and the subsequent base routing options that are offered.

[0185] Show how to build an infinite bandwidth network as a fabric in the blanket: These figures describe how to build a simple IB WAN between two LANs. It further shows how long distance IB fabrics can be integrated into the GVN at the physical layer.

[0186] Blanket topology - mixing Ethernet IP and IP IB and IB native fabrics into the blanket: These figures describe the logic used to integrate the various network fabrics into the GVN, including device connectivity, failover, load balancing, resource sharing, device to device communication and other aspects of integration.

[0187] API information exchange between devices for integrated performance: These figures describe the logic for API and other device to device links.

[0188] Three tiers of GVN, how L3 adjusts to L1 conditions to stretch internal fabric: These figures describe the logical tiers of GVN, and how they are managed across various network segments to expand the end-to-end network fabric.

[0189] Fabric and tapestry range of ASR: These figures show the advanced intelligent routing (ASR) at both the underlying connectivity tier (GVN L1) and the OTT internal lane tier (GVN L3). Figure 47 Further described are the different network segment types that are logically mapped to known options for traffic to spill over in GVN.

[0190] Tapestry topology - example - LAN stitched together in fabric / cloud as OTT2 over GVN OTT1: These figures show how OTT GVN can facilitate options to build constructs on top of its internal lanes, which exist as a second level top tier (OTT2). These can allow OTT1 GVN to handle routing, QoS, and other optimizations of the underlying tier, allowing the OTT2 construct to be used as a fabric through which to run.

[0191] Example file mapping, transfer, availability of tapestry via PFS appliance: These figures show how the OTT2 tier of GVN can be used as an RDMA fabric to facilitate the use of a globally distributed parallel file system (PFS) from LAN to cloud and backbone.

[0192] Fast transfer of GVN geographic destinations from remote to local: These figures describe how the integration of IB fabric into GVN can enhance the workings of GVN's geographic destination mechanism.

[0193] Example WAN of tapestry: These figures describe how various fabrics can be woven together to deliver high performance WAN connectivity between LANs.

[0194] Tapestry logic: These figures describe the logical, physical, and other properties of the network tapestry.

[0195] System diagram - tapestry: These figures describe the logical structure and organization of the GVN network tapestry tiers, modules, and elements.

[0196] The present invention automatically weaves together various network fabrics to form a network tapestry. This can be a component of a global virtual network (GVN) that provides top tier (OTT) services to customers in a plug-and-play fashion, effectively providing low-cost hardware and pay-for-use services on top of the existing Internet connections that customers currently have with their ISPs.

[0197] Simple network topology

[0198] Figure 1The basic logic of a sequentially chained network path is shown. SRV 1-A is connected to SRV 1-B via path 1-P0. SRV 1-B is connected to SRV 1-C via path 1-P2. The connection between SRV 1-A and SRV 1-C must go through SRV 1-B via path segments 1-P0 and 1-P2. There is no direct link between SRV 1-A and SRV 1-C, so if SRV 1-B fails or is otherwise unavailable, there is no redundancy. Thus, without redundancy, SRV 1-A cannot connect to SRV 1-C.

[0199] Figure 2 A topology diagram of a multi-link network segment with failover is shown. This diagram depicts multiple links between servers for direct connections between each pair, regardless of distance, location, or any other factor. As such, there is a sequentially chained network path between SRV 2-A and SRV 2-C through SRV 2-B. Figure 1

[0200] There is also a direct connection network segment 2-P4 between SRV 2-A and SRV 2-C, so this connection does not have to be relayed through intermediate server SRV 2-B. This provides redundancy and ease of operation. It provides different routing options from one SRV to another, which can be used to compare QoS and speed, among other factors.

[0201] Thus, the example connection between SRV 2-A to SRV 2-C through SRV 2-B and SRV 2-A, as well as the basic logic of SRV 2-A to SRV 2-C, provides redundancy directly. If one server fails, then the other two are still able to communicate with each other. If one path between two of the servers fails, then traffic can be passed via the two path segments through the servers.

[0202] Global network, nodes, and distance-related performance, among other factors

[0203] Figure 3 A global node topology for a global virtual network is shown. This diagram shows backbone connections between several example global nodes and their corresponding service areas in North America, South America, Europe, and Asia.

[0204] As described in the legend box in the lower right corner, the center of each area noted herein is a global node from a networking perspective. Around each global node are two rings, representing the types of connection quality zones based on radial distance from the center of the node. This is simplified only, as many factors determine the size and shape of these zones. However, the two zones can be distinguished from each other, with the closest one being a high performance zone and the other being an optimal service zone. ​

[0205] Global nodes are connected to each other by long distance high performance network links.

[0206] The further away a querying client or server or other type of device is from a global node, the higher the latency, at some point the distance is so great that the reduction in QoS causes the device to be outside the optimal service area.

[0207] Devices outside the optimal service area are expected to experience poor QoS.

[0208] Here, geographic areas are indicated, for example, SJC 3-02 for San Jose, California, USA, JFK 3-08 for New York, New York, USA, AMS 3-12 for Amsterdam, Netherlands, NRT 3-22 for Tokyo, Japan, HKG 3-28 for Hong Kong Special Administrative Region, China, and GIG 3-30 for Rio de Janeiro, Brazil.

[0209] There are many other places in the world where important global nodes can be set up, but for simplicity, only a few are shown for illustrative purposes.

[0210] Between each global node, paths are also shown, for example, leg 3-P0812 between JFK 3-08 and AMS 3-12. In reality, there are numerous path options, representing undersea cables, land cables, and other types of communication lines or links between two points. Those illustrated are meant to simplify the example of illustration. The shorter the distance combined with the speed of the line, the lower the latency between points, enabling faster information transfer.

[0211] Figure 4 A framework for defining and describing network fabric or characteristics of network segments within that fabric is shown. It describes the device network stack 4-100 and network lines and links to the backhaul 4-200.

[0212] Within the device 4-100, the physical characteristics 4-110 describe the sockets, network plugs, and cables, the physical pros and cons of the line, network interface cards (NICs), and the like. The data link 4-120 describes the nature of the data on the line, for example, the number of bits per byte, frame size, parameters, and others. The network 4-130 describes the nature of the protocols, encapsulators, packets or frames, and other elements. The transport 4-140 describes what can define and configure flow control, error correction codes (ECCs) or forward error correction (FEC), algorithms, optional compression, maximum transmission unit (MTU), addressing, peering, identity, security, and other elements.

[0213] The network lines and links to the backhaul 4-200 define the physical properties and operational characteristics of the network links from the subnetwork 4-210 to the core network 4-220 or backhaul. This can also be referred to as the uplink, last mile to the backhaul, or by various other names. The characteristics defining this line potential can also be used as a benchmark to measure performance of such factors as bandwidth (BW), latency, jitter, and others.

[0214] Figure 5 Global nodes and performance zones are shown. Figure 5 Global nodes 5-10 are shown and various rings representing levels of quality of service are shown. The high performance zone 5-20 has a radius of 5-D00, representing the best "last mile" connection between the client and the global node. The next level of quality is the optimal service zone 5-30, which has a radius from the center of 5-D00 plus 5-D02, which represents the next level of service. Within the sub-optimal functionality 5-40 ring, the network will still work, but not as optimally as the closer zones.

[0215] The radius 5-D10 indicates the distance immediately adjacent to the global node 5-10, such as co-located in the same data center.

[0216] Figure 6 Global nodes and performance zones are also shown. This example embodiment is based on Figure 5 is a simpler representation of global nodes and performance zones. 6-20 corresponds to 5-20, 6-30 corresponds to 5-30, and 6-40 corresponds to 5-40. A fifth ring 6-50 is included here, where the network can or can not work when connected to the center 6-10.

[0217] QoS is based on line distance and quality from the origin center point to each device. The farther the destination is from the origin, the more prevalent and significant latency and bandwidth issues become. Quantifying these distances and understanding the relative distances of client devices provides an understanding of the expected QoS.

[0218] Regarding GVN-topology and features

[0219] Figure 7 A simple network topology of a global virtual network arranged in a hub and spoke configuration is shown.

[0220] Two example hub and spoke clusters are described, one in each of two regions, Region A RGN-A 7-000 and Region B RGN-B 7-020. Each hub exhibits end point devices (EPD), such as 7-102 through 7-112 in RGN-A 7-000, and 7-122 through 7-132 in RGN-B 7-020, which are capable of connecting to access point servers (SRV_AP), such as 7-302, 7-306, or 7-308 in RGN-A 7-000, and SRV_AP 7-322, 7-326, or 7-328. End point devices (EPD) 7-302 through 7-132 will connect with one or more SRV_AP servers through one or more parallel tunnels. The SRV_AP in each region connects to a local corresponding backbone exchange server; i.e. (SRV_BBX) 7-500 in RGN-A 7-000 and 7-520 in RGN-B 7-020. The connection path 7-P 510 between SRV_BBX 7-500 and 7-520 is via a fast backbone connection over fiber or other network segment. The linked SRV BBX devices provide global connectivity. The SRV BBX can be a high performance server in one or more of the regions acting as a global link load balanced.

[0221] Figure 8 A simple network topology of a global virtual network arranged in a hub and spoke configuration is also shown.

[0222] This example embodiment is based on Figure 7 and its equivalent, in each region, multiple egress- ingress points (EIP) 8-400, 8-410, 8-420, and 8-430 are added as additional spokes to the hub and spoke topology model, with paths to and from the open internet.

[0223] Not shown in this example embodiment is a central control server (SRV_CNTRL) which can serve all devices within the region, the SRV_CNTRL can be one or more master servers.

[0224] This topology can provide routes for EPD through the GVN to EIP in a remote region. Or EIP in the same region. Or EPD to EPD in the same region or EPD to EPD in another region, or many other possibilities. These connections are secured and optimized through the GVN.

[0225] This topology provides a top of the (OTT) GVN layer from which to enter a hub point for traffic to flow through the various network fabric via a unified network blanket.

[0226] Path-relay segment, network segment characteristics, fabric connection point issues

[0227] Figure 9 The figure shows the trunk segments and network segments in two regions connected by Internet long haul network segments. This figure is a visual representation of trunk segments 9-H010, 9-H020, 9-H030, and 9-H040 plus the network segments between trunk segments 9-P1000, 9-P1020, 9-P3040, 9-P4000 in two regions connected by a string of network segments between Internet long haul network segment 9-2030 or a regional trunk segment. Path P2030 represents many trunk segments along the long haul of the Internet - this figure is not to scale. Each of these network segments can have different specifications and can be considered individual fabrics if different from adjacent network segments.

[0228] Figure 10 The figure shows a GVN tunnel between two LANs. The various described elements in this figure are:

[0229] 1 D device 5 TH tunnel interior hop 2 B boundary 6 EH exterior hop 3 P path 7 BP base path 4 ISP Internet Service Provider 8 PP peer

[0230] For example, 10-TH02 on EPD 010-D0 is an internal trunk segment inside the tunnel between LANs, and is also a path inside L3 of the GVN between LAN 010-TH00 and LAN 2 10-TH10.

[0231] The path made of network segments from 10-EH00 to 10-EH32 is the base path of the network at GVN LI. This figure shows the global virtual network tunnel GVN tunnel from LAN 10-TH0 to EPD-0 10-00 to SRV_AP AP-0 10-D4 to SRV_AP AP-2 10-D6 to EPD-2 10-D2 to LAN 2 10-TH10, showing the peering points between ISPs and network edges.

[0232] EDGE-00 10-B0 is the demarcation point for network access connections between LAN 0 10-TH00 and the devices of ISP-0 10-FAB0.

[0233] PP-00 is the point where peering occurs between the networks of ISP-0 and ISP-2. PP-02 is the peering point between the networks of ISP-2 and ISP-4.

[0234] EDGE-2 10-B2 is the demarcation point for network access connections between LAN-2 10-TH10 and the devices of ISP-4's network.

[0235] Some advantages can be realized by placing SRV AP-0 10-B4 at PP-0 10-B4 so that the SRV AP can directly peer with both ISP-0 and ISP-2. More advantages can be realized by placing SRV AP-2 at PP-2 so that this SRV AP can directly peer with ISP-2 and ISP-4. If the network of ISP-2 is not ideal, the GVN can alternatively route traffic through another route or line or ISP or carrier around ISP-2.

[0236] The internal hop count through the neutral third layer of the GVN is six hops from LAN to LAN. The distance between ISPs is not proportional. In addition, there can be more relay segments within the networks of the ISPs, but for simplicity the illustrated quantities are simplified.

[0237] The relay segment through the Internet from 10-EH00 to 10-EH32 is seventeen hops. Although this illustration shows connecting tunnels at the AP relay segments, the client devices within the path between LAN1 and LAN2 will see this as a single tunnel. This single tunnel represents the neutral third layer of the GVN in which all traffic can run that is normally transmitted through the Internet, including TCP, UDP, and other protocols, plus other tunnels such as IPSec, Open VPN, PPTP, or others. The third layer of the GVN also realizes other advantages. Some include lower TTL and the ability to have more control over routing, plus others. Figure 11 This illustration shows the various different network segments being joined into an end-to-end path. The elements described in this illustration include:

[0238] 1 BW bandwidth 2 CP communication path

[0239] From client 11-000 to server 11-300, the traffic transmission is through a local area network (LAN) 11-010, to an end point device (EPD) 11-100, to the network of an Internet service provider (ISP) 11-200, to a backbone 11-220 of the Internet 11-250 to a point of presence (POP) 11-320 of an Internet data center (IDC), to an internal network 11-310 of the IDC, and then to the server 11-200.

[0240] As shown in this example, it is important to understand the characteristics of each network segment, and how that network segment affects traffic flow relative to a complete end-to-end path. The internal network or LAN 11-N100 will typically have a reasonable amount of bandwidth (BW) for internal use, such as a 10 GigE BW 11-B100. The bandwidth for the network 11-N202 for the ISP will also typically be quite large, as shown by the 40 GigE BW 11-B202. Between those two networks, the last mile connection 11-N200 between the customer location and the ISP is a smaller 100 Mbps 11-B200 BW. There are numerous driving forces behind this, but the primary one is cost. The ISP will lay a pipe of a certain size of bandwidth to the neighborhood, and then will typically share this amount with many different users to the last mile. These upstream paths are the beginning network segments toward the wider general Internet. The backbone 11-N220 connects ISPs to each other, connects regions to regions, etc., and the backbone provides very deep high bandwidth connections, such as a 100 GigE 11-B220. This can represent the carrying capacity of a string of fiber between two points, and / or the rated capacity size of a switch or other factors.

[0241] The Internet 11-N250 in this diagram has dual pipes of BW 11-B250 and 11-B252, each of which is 40 GigE. This is an example of multi-homed connectivity in the Internet. There can be many other large pipes at the core of the Internet that are connected together. The ISP peering 11-N320 between the Internet 11-N250 and the IDC network 11-N310 is again represented by a multi-homed connection BW of 10 GigE, each for 11-B320, 11-B322, and 11-B328. This represents a dedicated last mile for this data center. There can be many more communication links for the IDC.

[0242] The internal IDC network 11-N310 will typically have a very high BW 11-B310, which is distributed among various internal networks, each rated to a certain speed, such as 100 GigE. The notation 2*100 GigE represents that this is a network of twice 100 GigE BW.

[0243] Figure 12The potential problem of a bottleneck through the relay segment 12-300 between the two network segments 12-100 and 12-500 is shown. For example, during the service 12-900 of a file from a server to a client, a particular algorithm dictates the transmission bandwidth based on the end-to-end line bearer capacity. If the burst traffic is too high, losses due to congestion, the server throttles the bandwidth to enable the most efficient transmission while mitigating losses. This enables a good server and responsible citizens of the pipe usage, but it can also result in over dictating the bandwidth, significantly slowing down the transmission far below the actual end-to-end line bearer capacity.

[0244] When the server starts to service the data or file stream, it will boost many packets per second based on the assumed high bandwidth 11-BW220 of the network segment such as 11-N220. This server connects to this large pipe network segment.

[0245] If the data stream is constrained at 12-300, the losses force the server to actively throttle the stream transmission, slowing down the transmission, the server can over reduce the transmission rate, over slowing down the overall process, due to the need to retransmit the lost packets.

[0246] Figure 13 The equation for calculating the Bandwidth Delay Product (BDP) is shown for a connected network segment or path, considering various connection attributes. The bandwidth 13-000 is in Megabits per second (Mbps), the granularity 13-002 is in seconds, and for this example the ratio of bytes 13-020 to bits 13-022 is eight bits, so 1 / 8 and the delay is a measure of the RTT (Round Trip Time).

[0247] The significance of the BDP is that it provides a deterministic measure of how much data can be transmitted along the line from the time the server starts to burst data, and it reaches the bottleneck until the receiving device discovers the loss and sends back an acknowledgement packet to the sending server.

[0248] GVN Overview of Example Topology and Options

[0249] Figure 14 The combining of various network fabrics into an overall network tapestry is described, and specifically the placement of various connection paths connecting various perimeter locations is pointed out. This embodiment illustrates that various network fabrics can be combined into a larger network tapestry. These fabrics can be seamlessly woven together as described in U.S. Provisional Patent Application No. 62 / 174394 to form a Global Virtual Network (GVN), its various devices, communication paths, and other embodiments topology. It demonstrates how various geographic regions or areas or territories can be linked together through various paths.

[0250] LAN Zone 14-ZL00 describes a typical Local Area Network (LAN) including a firewall placed with respect to an End Point Device (EPD) 14-100 between the LAN and an external network GVN OTT 14-202 and the Internet 14-30. There is a hardware FW 14-40 between the LAN 14-04 and the EPD 14-100. Another HW or SW FW 14-42 is between the EPD 14-100 and an egress entry point (EIP) 14-20 to protect from external threats from the Internet 14-30.

[0251] LAN Zone 1 14-ZL10 has a topology similar to LAN Zone 0 14-ZL00 except that there is no FW placed between the EPD 14-110 and the LAN 14-46. Internet Zone 0 14-ZI00 describes an example Internet topology in a region very close to 14-ZL00. Internet Zone 1 14-ZI10 describes an example Internet topology in a region very close to 14-ZL10. Internet Zone 2 14-ZI20 describes an example Internet topology in a region very close to 14-ZD20. Internet Zone 3 14-ZI30 describes an example Internet topology in a region very close to 14-ZD30.

[0252] Internet Data Center 2 Zone 14-ZD20 describes a topology and placement of a cloud-based firewall CFW 14-46 including virtualized FW devices behind a cloud FW load balancer. Internet Data Center 3 Zone 14-ZD30 describes a topology and placement of a cloud-based firewall CFW 14-48 including virtualized FW devices behind a cloud FW load balancer. SRV_BBX 14-72 in a region or zone 14-ZD20 can be connected to SRV_BBX 14-80 in other regions or zones 14-ZD30 via dark fiber connections 14-P220 through dark fiber 14-220.

[0253] SRV_BBX 14-72 uses the invention to write files directly into parallel file storage device PFS 14-82 via remote direct memory access (RDMA) over 14-P220, bypassing the stack of SRV_BBX 14-80 via path 14-P82. SRV_BBX 14-80 uses the invention to write files directly into parallel file storage device PFS 14-74 via remote direct memory access (RDMA) over 14-P220, bypassing the stack of SRV_BBX 14-72 via path 14-P74.

[0254] Path 14-P210 can be IPv4 or some standardized Internet protocol through which traffic flows from SRV_AP 14-300, via path 14-P210 at the top of the GVN, via a tunnel or other type of communication path, to or from SRV_AP 14-310.

[0255] Although the topology described herein does not have FW or traffic monitoring devices within the GVN channel, these devices can be placed as needed to further secure the flow of data.

[0256] Figure 15 Logic is described for an algorithm to enhance Advanced Smart Routing (ASR) within a Global Virtual Network (GVN). The first main process is to identify a target zone 15-100 with its corresponding sub-process identifying zones 15-110, and to identify potential EIPs 15-120 to use. This establishes subsequent processes to reside on the target egress ingress point (EIP) to exploit.

[0257] The next main process, Route Options (ASR) 15-200, utilizes a sub-process Server Availability List 15-210 and a routing list that is ordered 15-220 to determine the optimal server to build a tunnel, if one does not exist.

[0258] The next main process checks network segments 15-300 and utilizes a sub-process Measure Segments 15-310 and network statistics 15-320 for each path to evaluate the viability of the path to use to send the required traffic type. For example, for very small data that requires a faster path, then the shortest distance and lowest latency are most important, low bandwidth can be tolerated. Conversely, for large amounts of data that are not time sensitive in delivering the first bit, the path that provides the highest bandwidth is optimal, even though the first bit is delivered slower than other paths, the last bit is expected to arrive faster due to the higher bandwidth.

[0259] The next main process checks Routing Status 15-600, whose sub-processes Compare Routes 15-610 and test: Is the total path completely secure 15-620 to transmit data along that path. The final main process, Best Route for Traffic 15-700, whose sub-processes sub-algorithms: Which is the best path? 15-710, is this path the best for the traffic type? 15-720 are used to determine and set the best end-to-end route.

[0260] Each main process and sub-process is used to ensure that each type of traffic is best carried by the tunnel that is best suited for that traffic type.

[0261] Figure 16The total potential bandwidth is shown relative to the line's carrying capacity compared to actual usage. Based on an example office location, during the work week, Monday through Friday, when most workers are doing most of their work, there is a direct correlation to BW consumption. The peaks and valleys illustrated are an example of the daily cycle. Actual work usage will be similar, but unique for each usage scenario.

[0262] On this graph, the left or vertical axis is for bandwidth in percentage measure. The range is from 0% to 120%. The bottom or horizontal axis represents twenty-four hours a day, seven days a week.

[0263] This example shows that workdays have higher bandwidth usage than weekends, so it is likely that the office is only open on weekdays. Other usage scenarios will have their own periodic curves. Some will use the full bandwidth all the time, while others will have times of heavy BW usage and times of lower BW usage.

[0264] The point is that a fixed dedicated line is expensive and can be underutilized for a large amount of time. OTT services take advantage of less expensive lines to provide similar quality to a dedicated line, and are more reasonable and cost efficient. In addition, OTT services based on data traffic consumption rather than bandwidth capacity can be the most fair way.

[0265] Assume that for a business, the bandwidth to provide a certain potential is 100% of the carrying capacity twenty-four hours a day, seven days a week. If the line is used at full potential all the time, the average cost per GB of traffic is low. The capital cost of the equipment, plus the operating cost of maintenance, TI personnel, and dedicated dark fiber can be very expensive. If an organization pays for only the BW capacity that the organization can afford, it can shape the peak to cut, resulting in restricted times, limiting usage.

[0266] By providing a service based on the actual usage of the line, the full carrying capacity is utilized when necessary, and based on the consumed usage, the customer is only charged for what it uses.

[0267] Figure 17 A simple topology of a global virtual network (GVN) is shown, made up of endpoint devices (EPDs) 17-100, etc., connected to a service access point server (SRV_AP) 17-300. The last mile is from the network edge where the EPD 17-100 is located to the point of presence (POP) of the service provider to the Internet, which will link to the Internet and the optimal connection to the SPvV_AP 17-300. A secure tunnel is built over (OTT) this last mile connection to the Internet between the EPD 17-100 and the SRV_AP 17-300.

[0268] Quality of Service (QoS) 17-102 for both the underlying Internet path and the connection through the tunnel can be tested, analyzed, and adjusted for various conditions at all times. The underlying connection can be optimized, EPDs can form multiple connections to one or more SRV APs and be able to use multiple IP addresses and ports. In the event that the IPv4 Internet underlying path between the EPD and the SRV AP can be congested, the KPv6 alternate path can be a better choice. Or different routing through either protocol can be able to bypass the problem.

[0269] There can be bridges from the SRV AP 17-300 to other regions or to other protocols or other such options. For example, the internal path 17-P100 of the tunnel can be IPv6 encapsulated over the underlying IPv4 network path 17-P100. After the SRV AP 17-300, the path 17-P110 can be IPv4, so the IPv6 tunnel contents will still have to be encapsulated to run over IPv4, to get to the SRV AP 17-110. However, the path 17-112 can be native IPv6, meaning that IPv6 does not have to be encapsulated over IPv6.

[0270] Any protocol that can be encapsulated or otherwise "carried" can pass through the GVN via almost any other protocol or fabric.

[0271] The results of constant testing are stored and mapped to compare to other options through the fabric, as well as to understand to weave fabric properties peer-to-peer or stitched into a tapestry.

[0272] Figure 18 A simple topology of a Global Virtual Network (GVN) is also shown, made up of End Point Devices (EPDs) connected to Access Point Servers (SRV APs), etc. This figure is similar to Figure 17 , with additional elements, such as a Local Area Network (LAN) 18-000, an egress entry point (EIP) 18-302, presence points (POPs) 18-012, 18-022, an IPv4 cloud 18-010, and an IPv6 cloud 18-020.

[0273] The LAN 18-000 is both IPv4 and IPv6 as is the underlying network segment 18-P800. The remote Internet segments are either IPv4 only 18-P804 or IPv6 only 18-P806.

[0274] The key point is that for traffic entering the GVN, as it enters the EIP 18-302, it can enter as either IPv4 or IPv6, one or the other, and both through the GVN to their corresponding fabric, and will egress in the LAN 18-000. Address translation and mapping are key elements at the peer points.

[0275] Figure 19 A topology is shown of an endpoint device (EPD) 19-100 connected to multiple access point servers (SRV APs) 19-300 and 19-302 via multiple tunnels 19-P300 and 19-P302, respectively. The fabric of a local area network (LAN) 19-110 is stitched to the fabric of 19-200. The blanket 19-500 is the stitching together of fabrics implemented by a population of devices working together as a GVN component.

[0276] The tunnels between EPD 19-100 and SRV AP 19-300 and SRV AP 19-302 are TUN 100-300 and TUN 100-302. They are examples of multiple tunnel options between an EPD and the best current access point server (SRV AP) based on server availability and other factors such as destination, traffic type, QoS of each fabric segment between the origin and the destination.

[0277] The blanket 19-500 allows protocol bearers to "thread" through various GVN paths to egress and / or ingress at the egress ingress point (EIP) of the GVN. The population of GVN devices 19-600 represents individual GVN devices working at the physical layer that are composed of routing options through the GVN.

[0278] The GVN global network OTT Internet via other links 19-700 is the GVN 2 layer logic with modules such as geographic destination, DNS service, advanced smart routing (ASR), global ASR (GASR), server availability, tunnel building module, tester, analyzer, etc.

[0279] The GVN 1-800 can be described as a construct of what a client user sees through the GVN utilizing various protocols for available network paths to various EIP points to different locations.

[0280] Figure 20 A simplified wide area network (WAN) is shown constructed by combining two endpoint devices (EPDs) connected to each other via a global virtual network (GVN). This figure shows a wide area network (WAN) constructed by combining two endpoint devices (EPDs) 20-100 and 20-150 connected to each other via a global virtual network (GVN) 20-300 through tunnels TUNO 20-PTO and TUN2 20-PT2 into the GVN. Not shown here but assumed is that at least one or more access point servers (SRV APs) are at the other end of each of these tunnels and there can be more intermediate fabric segments in the GVN network path.

[0281] Tunnels TUNO 20-PTO and TUN2 20-PT2 are on top of the underlying network links (OTT). This underlying network link can be one or more of many protocols.

[0282] This figure also shows that there can be various different protocols working as fabric simultaneously on the LAN side of both EPDs, such as Internet Protocol (IP) on Ethernet 20-112 and 20-162, InfiniBand 20-118 and 20-168, or another network protocol 20-116 and 20-166. These can run in parallel on bridges through the GVN, or be stitched together to form a tapestry.

[0283] Any protocol can flow end-to-end through the GVN, regardless of the various underlying fabrics of network protocols in the various intermediate network segments. For example, in Figure 30 IB via paths 30-P 106 to 30-P 116 allows remote direct memory access (RDMA) access to parallel file systems (PFS) with plug-and-play functionality. In addition, another option is to route via 30-P308 to other regions.

[0284] There are various possibilities, one-to-one matching, or one type to another type, or one-to-many, or many-to-one, or others. From the perspective of EPD 20-100 or EPD 20-110, the end-to-end network properties inside the tunnels are perfect for the type of network between the LANs on either end.

[0285] The global virtual network (GVN) tapestry on various fabrics forms seamless WAN circuits between them.

[0286] Figure 25 Various backplanes on different devices are shown. To physically connect different network fabrics in a LAN, the EPD has ETH0 for Internet connectivity, three ETH ports for legacy LANs, plus one IB port for LANs.

[0287] How to establish InfiniBand on long-distance networks as a fabric in the tapestry

[0288] Figure 21 A simple network topology is shown that connects two LANs via WAN 21-102 long distance, which can utilize native InfiniBand (IB) or another high-performance network end-to-end.

[0289] IB device A 21-200 can represent an endpoint device (EPD), such as EPD A, as an enabler between LAN 21-300 and a wider network. IB device B 21-202 can represent an endpoint device (EPD), such as EPD B, as an enabler for another LAN 21-302. The dark fiber C 21-100 can be a switched dedicated circuit, a string of dark fiber, a dedicated line, or a physical network medium.

[0290] This point-to-point connection over dark fiber requires expensive devices running on top of the expensive required dark fiber at each end, which need to be installed at the location of both ends.

[0291] Because of hardware solutions from companies like Bay Microsystems or Obsidian networks, it is possible and reliable to have IB over long distances.

[0292] For improved global transport, IB over long distances is better than IP because it provides low latency high bandwidth transport.

[0293] Figure 22 Latency is compared between IP and IB, and the time spent relative to resource usage and consumption is carefully studied. It is also compared between the two fabrics and the underlying protocols for short, medium, and long distances.

[0294] HW is the time hardware takes to process network operations. This includes the time spent by the CPU, RAM, NIC, and other components.

[0295] HW = CPU + RAM + NIC + Other components

[0296] Where CPU = the time the CPU takes to process network operations. The majority of the time is spent by the CPU processing network operations, but the NIC and RAM do add some drag, thus increasing the processing time.

[0297] In addition to the hardware time, the time taken for network operations includes the time spent by the operating system (OS), hardware drivers, and the software stack including any applications. The total system time (SYS) is:

[0298] SYS = APP | Software Stack | O / S + HW Drivers | HW

[0299] For example, in a GVN use case, such as with the geo-destination mechanism, although IB is faster than Ethernet, over short distances, it is not worth the effort by the CPA / RFB to assemble files into a single group, then transmit the list of files via the side channel API communication, transmit the group via the chained cache, then disassemble the group back into individual files at the CDA in the EPD. This is because of the time it takes to do so. Over medium to large distances, however, the reduction in latency is well worth the extra work of pulling, caching, assembling, transmitting, communicating the list of files in the group, disassembling the group and serving the individual files at the destination from the source zone to the destination zone.

[0300] This analysis includes the group assembly / disassembly and the instant messaging functionality of this set / sequence of actions. When using IB instead of ETH, the time for CPU processing, RAM consumption, internal copy between RAM->SYS->NIC is also reduced, as IB is zero-copy, passing packets directly to / from the NIC by the application.

[0301] Total transmission time = CPU + RAM ↔ SYS ↔ NIC + network latency (RTT)

[0302] The best time is evaluated by the algorithm against the baseline, and also a threshold can be programmed to indicate when it is efficient to use ETH or more efficient to use IB.

[0303] In summary, rather than just being aware, but considering the various elements that increase latency acutely, sensitive to protocol usage, allows the algorithm analysis to analyze the characteristics, in other cases, to predict the expected latency or other conditions.

[0304] Figure 23 A simple topology of a global virtual network (GVN) is shown, made up of endpoint devices (EPD) connected to an access point server (SRV_AP), etc. This figure continues to describe the most basic topology of a GVN, and expands on the EPD connected to the SRV_AP, as Figure 17 shown.

[0305] It also shows the added elements in the GVN network path to the intermediate backbone exchange servers (SRV_BBX). Two BBX servers are connected to each other by a path via the Internet backbone (IBB) 23-800. This path can be IP or IB.

[0306] Figure 24The possible paths that passengers can take if they walk or ride the train from the ticket check 24-000 to the boarding area 24-900 are shown. They all start at 24-010 and can walk along path DA to DF and they can walk directly to 24-100 or can meander. They can decide to take the train at 24-100. If they do, by path Yes 24-P200 they board the train at 24-200 which takes them directly by path 24-P300 to 24-300 where they get off by path 24-P320. From there they re-enter the walking path at DW and proceed by other path relays DX, DY, DZ to the boarding area at 24-090. Those who choose to walk via path No 24-090 will start walking at DG and will likely meander as they walk between various other hop points in their path until they also arrive at the end point 24-090. Although taking the train and getting off the train can add a little extra time, this is more than made up by the high speed attribute of the train transport. Those who take the train will also experience less fatigue and potential stress than the walkers.

[0307] The path from the plane 24-900 to the boarding exit 24-000 starts at the origin 24-910 and again offers the choice of taking the train or walking with similar performance and time advantages for those who choose to take the train. This is analogous to the decision of whether to use a Slinghop between long distance points or to make the packet travel along an elongated Internet path.

[0308] Taking the train and getting off the train takes some time and effort. The train travels according to a fixed or variable schedule with all the passengers of the train traveling together from fixed point A to fixed point B. The walkers on the adjoining paths never stop moving.

[0309] The train transports the passengers more quickly and directly. The walkers can take indirect paths and can be delayed or lost. The train takes them there via the same known guaranteed delivery path.

[0310] Figure 25 The possible configuration of the physical backplane (front of the chassis) of various devices working in a network like a Global Virtual Network (GVN) is shown. The backplane utilizes two types of network fabric physical plugs, Ethernet and Infiniband, and it also indicates some of the possible roles that those plugs can take. There can be more or less or different types of plugs depending on the use, these are provided as examples only.

[0311] The illustration of the backplane of the end point device (EPD) 25-100 shows four RJ45 Ethernet ports, ETHO 25-110 as a WAN working, and three LAN ports ETH1 25-112, ETH2 25-114, ETH3 25-116. The WAN port 25-110 is a plug for cable connection to the base Internet connection via path 25-P100. One InfiniBand (IB) socket IB 025-120 is for IB cable connection to an IB switch in LAN 25-126 via path 25-P122, and also to a parallel file system (PFS) device 25-128 or other devices.

[0312] This example embodiment also shows the backplane for an access point server (SRV_AP) 25-300, a sling node (SRV_SLN) 25-550, and a backbone switch server (SRV_BBX) 25-500. It also illustrates the connection pathways between devices, and from devices to various clouds to other devices, such as remote SRV_SLN 25-558 and remote SRV_BBX 25-552.

[0313] The GVN connection from EPD 25-100 via SRV_AP 25-300 to SRV_BBX 25-500 is on top of the ISP last mile connection path 25-P000 through the Internet 25-000, and on top of LAN 25-032 in the Internet data center (IDC) path 25-302.

[0314] These physical ports, backplane (front of the board), connection paths, and other elements described herein are for illustration only. The absence of IB ports on the SRV_AP 25-300 is illustrated as acting as an "air gap" between end-to-end base protocols, where IB can be encapsulated over Ethernet for end-to-end IB of clients in the LAN of EPD 25-100 such as LAN 25-016. However, the SRV_AP can have IB ports if there is a native IB connection between the SRV_AP and EPD or other devices, or if there is a demand for it.

[0315] Figure 26 Two types of network paths through a global virtual network (GVN) are shown, end-to-end Internet protocol (IP) paths and hybrid IP paths on Ethernet at either end, and InfiniBand (IB) backbone paths in the middle.

[0316] Both paths have local IP segments 26-000 and 26-012. The latency, bandwidth, and other characteristics of these local segments 26-000 and 26-012 are equivalent for both paths. The middle segment of the IP path is 26-P028 to 26-P056, and the latency of this path segment is measured by 26-260.

[0317] The slingshot mechanism has a transmission advantage over segment 26-420, but adds an amount of time at both ends of the slingshot at stages 26-400 and 26-440. In analyzing which is the better path, the net latency of the IB slingshot path 26-260 must be compared directly to the IP path 6-200.

[0318] Rug topology - mixing IP over ETH with IB over IP and IB native fabric into a rug

[0319] Figure 27 Four different network channels between two access point servers (SRV_AP) 25-200 and 25-202 are shown.

[0320] Two Ethernet over IP paths are shown, 27-P420 to 25-P436, which is an IPv4 end-to-end, and 27-P420 to 27-P626 to 27-P636, which is a mix of IPv4 and IPv6 network segments.

[0321] Another base connection described goes from SRV_AP 27-200 to backbone switch server (SRV_BBX) 27-500, which uses a network slingshot to transmit data to remote SRV_BBX 27-510 to SRV_AP 27-202, and return traffic uses a reverse slingshot mechanism, both over a fiber backbone.

[0322] TUN 27-222 is a tunnel channel over the top (OTT) of either of these three connection paths. Algorithmic analysis can be applied to select which transmission type is optimal for the path. This figure does not show EPDs or other devices connected to the SRV_AP, but they can work in it.

[0323] Figure 28 Multiple endpoint devices (EPD) are shown as to how they can connect to access point servers (SRV_AP) in a region. Two regions are shown here. Egress entry points (EIP) to presence points (P0P) 28-004 and 28-024 to interact with various destination servers (which can also be devices) in each region via the local Internet there 28-002 and 28-022.

[0324] There are two types of cross regional connection paths through the GVN shown here. OTT 28-600 to OTT 28-650 to OTT 28-610, which is an Internet Protocol over the top end to end network.

[0325] The alternate path is OTT 28-600 to IBB 28-800 to OTT 28-610, where the IBB portion is a non-OTT path, possibly an IB between two backbone switch servers (SRV_BBX) 28-500 and 28-520.

[0326] Figure 29 The logical construct of links between devices in a Global Virtual Network (GVN) is shown. This figure depicts backbone switch servers SRV_BBX 29-500 and 29-502, each of which acts as a hub for the regions it serves. The SRV_BBX servers 29-500 and 29-502 are connected to each other by a backbone path 29-P500, which can be high performance IP over Ethernet or InfiniBand (IB) or other such technology.

[0327] Each SRV_BBX "hub" serves individual access point servers (SRV_AP). Each endpoint device (EPD) is connected to individual (one or more) SRV_AP servers, so there is redundancy and routing options for traffic to move via the best connection at different times.

[0328] The indicated connection paths can be tunnels over IP Ethernet Internet Top (OTT), or tunnels over direct links through Ethernet, or IB over fiber, or IB (RoCE) over Ethernet, or other types of connections.

[0329] The placement of the SRV_BBX and SRV_AP devices is based on expected demand from client locations, which are in the best IDCs relative to the pipe, interconnected to act as target regions when connecting to global locations.

[0330] The devices are also connected to a central control server (SRV_CNTRL) 29-200 via paths such as 29-EP 112 to EPD 25-112 or path 29-P218 to SRV_AP 29-318, etc. Having these paths allows the devices to connect to the SRV_CNTRL via an API or alternative traffic paths for information transfer.

[0331] Figure 30 The logical construct of links between devices in a Global Virtual Network (GVN) is also shown. This figure continues the depiction of the SRV_BBX and SRV_AP devices, and also depicts the SRV_CNTRL 29-200. Figure 29Connectivity within a Global Virtual Network (GVN) of various devices is described, focusing on endpoint devices (EPD) 30-100, 30-110 to access point servers (SRV_AP) 30-300.

[0332] In some aspects, it simplifies Figure 29 The diagram given in the Summary, augmented with some elements, such as multiple connection paths from each device to other devices or clouds or combinations, such as tunnels (TUN) 30-T00 or 30-T02 from the GVN cloud 30-200 top (OTT).

[0333] The GVN and its components provide services to improve client connectivity and guarantee its security. With multiple "local" presences in multiple locations, controllable and configurable automated systems, optimized connectivity is provided, cost savings are realized, benefits are MPLS replacement, and extended high performance connectivity is provided, such as remote direct memory access (RDMA) via encrypted tunnels, security and privacy, and other benefits.

[0334] A huge benefit is the ability to connect various network fabric types, such as IB in LAN 30-108 of EPD 30-100 to IB LAN 30-118 of EPD 30-110, end to end from the client's perspective, even though some of the underlying network segments in the middle are not native IB but IP. This is achieved through IB encapsulation over IP or through another IB native wire route or other methods.

[0335] The point is that the GVN allows various network fabrics to run on top (OTT) of various other network fabrics at the underlying layer. The overall effect is to weave together various fabrics into a network blanket, enabled and optimized by the GVN for best performance at highest security.

[0336] API information exchange between devices for integrated performance

[0337] Figure 31 is a diagram showing an example topology of devices within a GVN according to embodiments of the disclosure, including a backbone exchange server (SRV_BBX) topology with security and breach API sequences. This example embodiment shows a series of API calls in sequences within an example topology of devices within a typical global virtual network.

[0338] The request 31-A2 to initiate the first API call from the access point server SRV_AP 31-300 to the central control server SRV_CNTRL 31-200 is received, parsed and processed by the SRV_CNTRL 31-200. It then triggers three more API calls, all initiated by the SRV_CNTRL 31-200. Depending on the nature of the communication, these can be sequential or can be handled simultaneously in parallel. The three additional calls of the request 31-A4 to the backbone exchange server SRV_BBX 31-800 and the response 31-A6, 31-A8 request and its response 31-A10 to another SRV_BBX 31-810, and the third additional API call of the request 31-A12 to the SRV_AP 31-302 and its response 31-A14 back to the SRV_CNTRL 31-200. Upon completion of all three of these "internal" calls, the final response 31-A16 is returned to the SRV_AP 31-300, the device that initiated the first request 31-A2.

[0339] The API request 31-A2 and response 31-A16 can be characterized as a gap call, requiring that its internal calls 31-A4 to 31-A6 involving the SRV_BBX 31-800, 31-A8 to 31-A10 involving the SRV_BBX 31-810, and 31-A12 to 31-A14 involving the SRV_AP 31-302 be completed before it is completed. This can be required before the SRV_AP 31-300 can take subsequent action to implement the measurement and integration purposes or other reasons. For example, if an end-to-end tunnel is to be constructed from the SRV_AP 31-300 through the SRV_BBX 31-800 to the SRV_BBX 31-810 to the SRV_AP 31-302 via paths 31-P800 to 31-P808 to 31-P810, then all those devices can need to be configured or triggered with the appropriate information and details. Such an API call can exemplify a request made via 31-A2 to the SRV_CNTRL 31-200, which will then make the internal three API calls 31-A4 to 31-A6, 31-A4 to 31-A10, 31-A12 to 31-A14, the response 31-A16 can include configuration and setup information for the SRV_AP 31-300 to be used, as well as an indication from the SRV_CNTRL 31-200 that the other peer devices are set up and ready.

[0340] Figure 31 The EIP 31-500, via 31-P500, POP 31-600 and 31-P600 to the open Internet 31-700. Figure 31The Open Internet 31-702 includes EIP 31-502, via 31-P502, POP 31-602 and 31-P502 to EIP 31-502. 31-P100 connects EPD 31-100 and SRV AP 31-300. 31-P500 connects SRV AP 31-300 and EIP 31-500. 31-P102 connects EPD 31-102 and SRV AP 31-302. 31-P502 connects SRV AP 31-302 and EIP 31-502.

[0341] In some embodiments, 31-A4 / 31-A6 and 31-A8 / 31-A10 and 31-A12 / 31-A14 are independent API calls in a series / sequence. In other embodiments, 31-A4 / 31-A6 and 31-A8 / 31-A10 and 31-A12 / 31-A14 can be executed in parallel.

[0342] Security elements can be placed in various locations within the GVN topology described herein. For example, firewalls FW 31-400 and FW 31-402 can be placed along 31-P800 and 31-P810. Firewalls FW 31-400 and FW 31-402 can protect SRV_BBX 31-800 and 31-810 from Internet threats, ensuring secure backbone communications.

[0343] Information about secure egress entry points (EIPs) 31-500 and 31-502 can also be a factor in this API exchange.

[0344] Figure 32 A series of API calls between GVN devices and SRV CNTRL within a GVN are shown. The figure shows a gap API call wrapper that encapsulates and envelops the internal API calls. Three internal round trips are required for the correlation of the external API calls to be completed successfully. This exemplary embodiment is based on Figure 31 , which can provide different perspectives of a set of API calls between GVN devices and a central control server SRV CNTRL 32-200 within a global virtual network GVN. A gap call wrapper API#1 (32-A2 to 32-A16) encapsulates and envelops internal API calls API#2 (31-A4 to 31-A6), API#3 (31-A8 to 31-A10), and API#4 (31-A12 to 31-A14).

[0345] Three internal round trips are required for the dependencies to fully constitute the external round trip. The response (RESP) to API #1 (32-A16) will wait for the internal API calls API #2 (31-A4 to 31-A6), API #3 (31-A8 to 31-A10), API #4 (31-A12 to 31-A14) to complete, after which the results are evaluated and sent back as the RESP. Only then can the gap API be closed and the response sent.

[0346] This sequence is similar to a transaction set of SQL statements. All need to complete or no one can. Thus, in the event of a failure in one or more of the calls, the calculation can be re-done.

[0347] Figure 33 Information flow between devices in the GVN and the central control server (SRV CNTRL) 33-200 is shown, according to embodiments of the disclosure. A central repository, made up of databases 33-58 and file storage 33-68, can be coupled to the SRV CNTRL 33-200. In some embodiments, the central repository can store API / action information, in other embodiments, it will contain tunnel and routing information, as well as data to produce context device specific server availability lists, etc. For example, the central repository can store definitions of APIs / actions, scripts associated with the APIs / actions that can be used to process the APIs / actions. In some embodiments, the central repository can also store the peer relationships of the devices. This repository can also store information about known network fabrics, their current and historical dynamic data, characteristics of network fabrics that can be useful in analyzing similar network fabrics, etc.

[0348] 33-P100, 33-P200, 33-P300, 33-P202, 33-P502, 33-P206, 33-P506 represent communications between GVN devices that have a peer-to-peer relationship with each other, and thus have a privileged relationship. EPD 33-100, SRV AP 33-300, other device 33-502 can be coupled to file storage 33-60, 33-62, 33-64 and databases 33-50, 33-52, 33-54.

[0349] A ring pattern of peer-to-peer communications is illustrated, from SRV CNTRL 33-200 to EPD 33-100 via 33-P100, to SRV AP 33-300 via 33-P300, or to other device 33-502 via 33-P502. EPD 33-100 communicates with SRV CNTRL 33-200 via P200, with SRV AP 33-300 via 33-P202, and with other device 33-502 via 33-P502.

[0350] In some cases, there can be a loop of sharing information between devices, for example, in the case where EPD 33-100 can request information from SRV_CNTRL 33-200 via 33-P200, which is sent back to EPD 33-100 via 33-P100.

[0351] In other cases, one device can report information related to other devices, for example, SRV_AP 33-300 reports to SRV_CNTRL 33-200 via 33-P202, which then sends information to EPD 33-100 via 33-P100 or to other device 33-502 via 33-P502.

[0352] In other cases, a full loop can not be required, for example, log information is sent from a device such as EPD 33-100 to SRV_33-200 via 33-P200, there is no need to further forward this information onward. However, the log information can be moved at a later time from a repository on SRV_CNTRL 33-200 to a long term log storage server 33-502 via 33-P502.

[0353] Direct link 33-PT02 is between devices EPD 33-100 and SRV AP 33-300. Direct link 33-PT08 is from SRV_AP 33-300 to other device 33-502. Direct links involve communication between devices that does not necessarily involve SRV_CNTRL 33-200.

[0354] PUSH information 33-208 from SRV_CNTRL 33-200 can be an RSS feed or other type of information pushed via 33-P208. API query 33-206 from SRV_CNTRL 33-200 can be either a traditional API transaction or a RESTful API call with a request, made via 33-P206REQ, with a response received via 33-P206RESP. PUSH 33-206 and API query are given to illustrate devices that do not share a peer-to-peer relationship, action codes or definitions (e.g., action codes and / or definitions not obtained, outdated action codes and / or definitions), privileged status, and / or similar system architecture to GVN devices.

[0355] Data information is stored in databases on DB 33-50 for EPD 33-100, DB 33-52 for SRV_AP 33-300, DB 33-54 for other devices 33-502, DB 33-58 for SRV_CNTRL 33-200, and DB 33-56 for SRV_BBX 33-500. In addition, two types of file storage are described herein, HFS hierarchical file storage for storage hardware hosted on the device for its own internal access, and PFS parallel file storage system that is independent and provides RDMA access. PFS 33-510 represents another PFS file storage on another device at another location accessed via RDMA (remote).

[0356] Figure 34 The positioning of devices into various Internet Data Centers (IDCs) is shown, with IDC1 34-002 and IDC2 34-004 in the same region, IDC3 34-006 in another region, and IDC0 34-000 representing the location of the central server (SRV_CNTRL) 34-200.

[0357] 34-P500 is the regional to regional connection between global nodes through international or cross-regional links to connect IDC1 34-002 with IDC3 34-006. The SRV_CNTRL 34-200 server is a multiple master topology with equivalent operations when interacting with various devices. A key feature is an aggregated topology where the web sockets of SRV_AP 34-200, 34-202, 34-210, 34-212 across multiple data centers in a regional cluster are linked via paths 34-P200, 34-P202, 34-P210, 34-P212 to a common SRV_BBX node 34-500, which is connected to another SRV_BBX 34-506 in another region, which is a long distance transport sink point for SRV_AP 34-220, 34-222 via paths 34-P220 and 34-P222. Device operations and coordination are through API paths, such as from SRV_AP 34-212 via path 34-API-08 to SRV_CNTRL 34-200.

[0358] Three layers of GVN, how L3 adjusts to L1 conditions to expand internal fabric

[0359] Figure 35The three layers of the GVN are shown and how they interact. LAN 35-000 connects to LAN 35-020 via internal tunnel 35-L300 internal relay segment 35-H0 to EPD at relay segment 35-H8. In the tunnel, segments 35-P010 to 35-P016 make up the end-to-end fabric through the GVN.

[0360] The secondary logic layer 35-L200 analyzes and regulates the connections on the primary network layer 35-L100 to weave the layers of a fabric together in the best way to optimize for the GVN. The peers and primary infrastructure connections of a fabric are 35-S00, 35-S02, 35-S04, and 35-S06. The interaction between 35-L200 and 35-L100 is via 35-LC0102 and the interaction between 35-L300 and 35-L200 is via 35-L0203. The seams between the underlying fabric 35-S00, 35-S02, 35-S04, 35-S06 are managed by the secondary 35-L200 so that the traffic of one fabric can spill over onto a different fabric.

[0361] The underlying Internet fabric 35-100 to 35-102 can be IPv4, IPv6, IB, IPv4 / IPv6, or other network types. The path through L300 is the GVN layer that the customer can see. L100 represents the physical network layer for the end-to-end various network segments. L200 is the layer that constructs the fabric carpet through logic, integration, address mapping, routing, and other techniques.

[0362] Figure 36 The fabric of the underlying connections and the fabric within the tunnel (TUN1) 36-T00 are shown. The tunnel runs on top of the underlying connections (OTT). Other embodiments show the communication path between the two devices, an endpoint device (EPD) 36-100 and an access point server (SRV_AP) 36-200.

[0363] The tunnels are on top of other underlying connections (OTT) and these paths represent the network fabric available at the time, for example 36-OTT00 → Internet Protocol version 4 (IPv4), which is the most common, 36-OTT02 → Internet Protocol version 6 (IPv6), 36-OTT06 → InfiniBand (IB), 36-OTT08 → Other - some other network type or combination of fabrics, for example, a fabric enabled on a network segment for IPv4, IPv6.

[0364] TUN 136-TOO represents a tunnel (or bridge) built between two devices on top of the Internet (OTT). Can be one of end-to-end 36-OTT00, 36-OTT02, 36-OTT06 or 36-OTT08, or can be a combination of various different fabrics in a string of network segments.

[0365] 36-P00 is the IPv4 fabric within the tunnel, 36-P02 is the IPv6 fabric within the tunnel, 36-P04 is encapsulated RDMA over RoCE or IP Ethernet, 36-P06 is IB over IP (IBoIP) or other similar protocol, 36-P08 can also be a combination of, for example, IPv4 and IPv6 or other. The point is end-to-end fabric through the blanket on the GVN through any other fabric or series of various other network fabrics. A device in the cloud at LAN or SRV_AP 36-300 or at EPD 36-100 sees the end-to-end network as the fabric through the tunnel regardless of the underlying base connectivity.

[0366] Figure 37 is a logical visual representation of different network fabrics such as a level of a global virtual network (GVN) woven into a network blanket at three levels. The flow can be one fabric in at the top, combined and carried end-to-end by the GVN and out at the other end.

[0367] For example, IPv6 37-102 is able to enter the network blanket 37-300 via 37-P102 and exit the fabric via path 37-P112 to IPv6 37-112 regardless of what type of fabric is in the underlying middle where the GVN runs. These various fabrics through the GVN can run side-by-side in parallel with other fabrics, with entry or ingress points and egress or exit points.

[0368] Figure 38 shows the base connectivity of an Ethernet fabric with Ethernet fabric 38-000 at one end, fiber optic InfiniBand 38-002 in the middle, and Ethernet or InfiniBand 38-006 at the other end. The diagram also shows three top (OTT) tunnels between EPDs 38-110, 38-120, 38-130 and servers 38-116, 38-126, and a parallel file system (PFS) device 38-136 at the other end. EPD 38-110 to TUN 38-210 to server 38-116 is end-to-end InfiniBand (IB). EPD 38-120 to TUN 38-220 to server 38-126 is end-to-end IP. EPD 38-130 is end-to-end remote direct memory access (RDMA) allowing long distance RDMA access to PFS 38-136.

[0369] The path from one point to another over the Internet will typically span more than one fabric. The GVN automatically analyzes and weaves many different network fabrics into a network tapestry. This permits client devices to have parallel sets of their selected compatible end-to-end fabric on top of various different fabric network segments in parallel. The GVN is a first level OTT (denoted as OTT1) on top of an underlying network such as the Internet, on top of which a second level OTT (OTT2) fabric is to be built.

[0370] The network tapestry permits IPv6 for example from EPD 38-120 to server 38-126, but not from EPD 38-120 to SRV_AP 38-320, the underlying connection 38-000 can be over IPv4, as the IPv6 within the tunnel is encapsulated. From the client's perspective it will be end-to-end IPv6 along the network path from origin to destination. The underlying network segments woven together constitute a tapestry of IPv4 and IPv6 fabrics, possibly other protocols like IB woven together.

[0371] Figure 39 Two network paths are shown, the bottom one showing the underlying network connection path at the first level of the GVN, and the top one showing the tunnel at the third level of the GVN. To integrate the various network fabrics into the network tapestry, various devices organized into the GVN topology are involved, as well as various distributed modules, such as EPD / PEDP connected to SRV APs on top of normal Internet connections, advanced smart routing (ASR), geo-destinations, geo-destination mechanism elements such as chain caching, reverse geo-computing and others, NAPIM to enhance information exchange to enhance data transfer, global file manager (GFM), etc. The EPD knows which SRV APs it can connect to with a server availability list, which is generated specifically for that EPD, based on testing, load balancing and server availability mechanisms 39-222 taking into account other EPDs' current and predicted needs, other factors taken into account.

[0372] Thus, each device is to work according to its role, for example an EPD is to connect with an access point server (SRV AP), the EPD should have various options with respect to building or rebuilding tunnels, storm weather mode to help it deal with complex difficult network conditions, the EPD device is to connect with both hosts and peers, plus intermediate devices, core junctions and other needs to coordinate actions based on shared information.

[0373] The key feature for selecting the best path type based on the data being processed is that the Tester 39-118 and Builder 39-110 work with the Tunnel Manager 39-210 and the Advanced Intelligent Routing 39-228. The associated firewalls and security monitors 39-0140 and other modules 39-160 working at the first layer 39-GVN-1 provide some support for the Tester and Builder. The traffic and bandwidth analyzer 39-258 and connection analysis 39-288 provide information used by the traffic and bandwidth recorder 39-328 and others. The EPD has a tunnel tester 39-322 as does the SRV_AP 39-312 as the network path analysis should provide insight into both directions. This approach helps to detect problems with the peer relationship or bottleneck or routing or other issues that can occur in one direction but not the other direction of the data stream.

[0374] In dealing with different types of content streams, for example, clicks versus content services (images) versus video streams or large data files differ slightly in their QoS requirements, all of these can be dealt with in different ways.

[0375] To build a dynamic system as a construct of tunnels or series of connection tunnels 39-T01 through 39-T02 through 39-T03 through the third layer 39-GIV-3, information is used not only to maintain the connections between the EPD 39-100 and the SRV_AP via 39-T01 and the SRV_AP 39-300 and the SRV_AP 39-302 via 39-T02 and the SRV_AP 39-302 and the EPD 39-102 via 39-T03 but also to achieve the best possible bandwidth with the lowest possible latency and to provide other improvements.

[0376] The multiple tunnels automatically built between the EPD and the SRV_AP, the utilization of tunnels within tunnels between other devices, and the automated security bootstrapping at startup, provide enhanced security with the dynamic tunnel manager that can work in configuration, setup, adjustment, and others. These enable productivity improvements with better connectivity and can provide the best secure network optimization, improved routing, and others. Other functions are triggered by the heartbeat cycle, by routine maintenance times, and events. Such functions include testing, recording, and analysis of the connections with automated healing, understanding the stitching together of various types of networks to form a network quilt provides a multiprotocol collection of fabrics woven together at the base internet first layer 39-GVN-1 and any end-to-end path inside the tunnels 39-GVN-3. The testing can analyze LAN to GVN performance at both ends of the tunnels 39-CTN140 and 39-CTN240 and can compare and contrast GVN 39-CTN340 to Internet 39-CPT340 performance and matching across the regional network segment portions.

[0377] ASR of fabric and tapestry

[0378] Figure 40 Multiple tunnels between devices within a global virtual network (GVN) spanning multiple regions are shown. This example embodiment also describes routing options that traffic can take within the GVN 39-GVN-3, at the third layer of the global virtual network (GVN) fabric. The GVN is constructed on top of the underlying Internet fabric (OTT). Each leg will consider the physical network type of the first layer 39-GVN-1, the fabric of the third layer 39-GVN-3 can be another network type. This approach allows for tapestry end-to-end running of multiple network types and various fabric protocols to transport data via the optimal path for that data type, automatically considering account data size, network conditions, and other factors.

[0379] The OTT advantage from the client's location at the EPD 40-100 to the first SRV_AP 40-300 or SRV_AP 40-302 or SRV_AP 40-304 over the underlying Internet connection is that the client is able to use their regular line, at a lower cost relative to a dedicated solution, with multiple options into the GVN. Although the EPD 40-100 is connected through the same Internet line, TUN 40-T00 and TUN 40-T02 and TUN 40-T04 can provide different quality of service (QoS) because of routing factors, congestion, peering relationships, and intermediate pipe capacity, and other factors, so the multiple options improve the overall QoS by providing alternatives. These TUNs are also able to provide different underlying fabrics, on top of which the internal fabric is able to operate OTT. For example, if the top IB is at the first layer 39-GVN-1, the native InfiniBand (IB) at the third layer 39-GVN-3 of the GVN will be most efficient to run.

[0380] The GVN is delivered as a service on top of the underlying connection (OTT) to the aggregation point, to the backbone, to other fabrics on top of which automation is included, including multi-layer, multi-step best path analysis via advanced smart routing (ASR) and more. The more options available, the better.

[0381] The EPD 40-100 is at one location 40-M0, the SRV_AP is at the region 40-M2, SRV_AP 40-300, SRV_AP 40-302, and SRV_AP 40-304, the SRV_AP is at the region 40-M3, SRV_AP 40-310, SRV_AP 40-312, and SRV_AP 40-314.

[0382] Because of the nature of the third tier 39-GVN-3 path fabric, there is a need to mitigate loopback risks to prevent misrouted geographic destinations, ASR remote redirection backtracking, and to test, flag, and resolve SRV_AP, inter-zone link outages, and other issues.

[0383] The figure also shows the mapping of various egress entry points (EIPs) such as 40-510, 40-512, and 40-514, which are destinations for GVN traffic to find the Internet fabric outside the GVN, and routing origins for traffic from those locations to other locations such as LAN 40-000 via third tier 39-GVN-3, via EPD 40-100 to route into the GVN, or other destinations available via the GVN.

[0384] Thus the path selection is based on QoS factors, fabric type of first tier 39-GVN-1, capacity rather than current load, context mapping based on the devices and their path options, and other fixed and dynamic factors.

[0385] Figure 41 A framework is shown for running parallel tunnel tests to measure latency 41-100, bandwidth 41-110, packet loss 41-120, and other measurements 41-150. These processes can be run on the network fabric of first tier 39-GVN-1, on the GVN path or segment of third tier 39-GVN-3, or on other network paths or segments, on the network segment between two devices.

[0386] The tests can be run sequentially or in parallel, starting with joint 41-020.

[0387] After the tests, other processes are run to clean up and release resources 41-300 after the test run. At the end of the test, log test results 41-320 save relevant information for the devices running the tests, and are analyzed by the central control server (SRV_AP). This information can be used in building the dynamic list of contexts for the server, enabling the devices to connect with the list of servers available, taking into account the test results and the mapping of the routing selection for the GVN path fabric.

[0388] Figure 42An algorithm is shown for running a series of tests in parallel for connectivity of path 42-010. Tests are run both on the tunnel of the third tier 39-GVN-3 and on the underlying connection 39-GVN-1. The test 42-110 of the tunnel is currently tested and compared and contrasted with the underlying path 42-120 test between, for example, the EPD and the SRV_AP. Analysis of the results of these two tests can provide insight into the health of the underlying connection and the health of the tunnel. If the tunnel is unhealthy but the underlying connection is good, then the remedial action can be simply to rebuild the tunnel, or to access the AP using a different set of IPs and ports, or other remedial action.

[0389] In the case where the tunnel test 42-110 returns a bad result but the alternate tunnel 42-130 test provides better connectivity, it is possible to simply shift traffic load to the better of the two.

[0390] Also determinative is the monitoring of the current user 42-160 usage of the network for a number of reasons. One reason is that the performance measurements of the test need to take into account the current network load, as the test will share the line bandwidth, so it can appear to produce a false low BW measurement relative to the expected line capacity. So if the connection has a BW of 20 Mbps, and the user is using 15 Mbps of that BW in the test, it makes sense to assume that the test will not produce more than 5 Mbps, as that is all the bandwidth available. Another reason to monitor concurrent usage is to use that information to set parameters for the test so that the test itself does not interfere with, slow down, or otherwise interfere with the QoS of the customers currently using the network.

[0391] All results are shared with the SRV_CNTRL 42-280 so that the granular test results can be aggregated across the system, by device, by region, etc. so that it can be analyzed and utilized in the future.

[0392] Figure 43 Is a diagram to describe network options. 43-100 is the source, and traffic can be divided based on ideal path type or weave or QoS or other criteria. If it is better via other types of paths, the testing and recording of QoS for each path 43-P210, 43-P220, 43-P230, 43-P240, and 43-P250 provides analysis and override potential.

[0393] Class B Bl 43-210, B2 43-220, B3 43-230, B4 43-240 and B5 43-250 are first connections OTT of the basic Internet connection. The performance of the paths 43-P210, 43-P220, 43-P230, 43-P240 and 43-P250 can be compared and contrasted to determine the best path from a set of available paths. QoS can also analyze the fabric and protocol types in determining the best path based on optimal conditions.

[0394] Class C CI 43-302 to C15 43-330 are long distance connections based on data type, QoS, currently available alternative connections through GVN and relative QoS of the path. Class C is via Class B, which are all connected to Class A as a starting point.

[0395] Figure 44 Also a diagram for describing network options. This diagram continues to show Figure 43 The example embodiments described in the previous diagram for A, B and C class routing options. The new elements are the client 100, the sink point D 44-500 immediately ahead of the destination and the server 44-800. The diagram also indicates the connection paths from C class to sink point D, for example 44-CP328 from C14 44-328 to D 44-500. There is also a communication path from the client 100 to 44-100.

[0396] This example embodiment can be used to describe the multi-step options available to the advanced smart routing (ASR) to be used when plotting the best route for a traffic type and also based on the path quality (QoS) from testing to consider the best route.

[0397] There are other embodiments, for example, to a visual mapping of the routing options to be used as a framework for testing and other uses.

[0398] Figure 45 is a flowchart of an algorithm for running the test 45-100 and for remedial action 45-300 to be taken when a problem is detected. This algorithm has a starting point 45-000 and an end point 45-500, so it needs to be triggered when it needs to be run, as it is not an infinite loop.

[0399] The action to be taken can be how to cope with the detected packet loss 45-P310, which calls for multiple streaming of the duplicate content 45-310, or, for example, if the underlying connection 45-P340 has a problem adjusting the settings 45-340 at the first layer of GVN 39-GVN-1, or if there is a segment problem 45-P380, the remedial action would be to adjust the protocol settings 45-390, etc.

[0400] Modifications can also be triggered in at least two cases: first, if a 45-200 problem is detected, but no logical follow-on path 45-P300 is found. If the underlying connection is normal, but the problem remains intractable, then a 45-240 support person can be notified. Another example of a notification is if the use of bandwidth is at or above capacity 45-P350, then a 45-350 management person can be notified of this condition. There are other events that can trigger a notification.

[0401] If a 45-410 problem is detected, then both the test 45-110 and the remedy are logged. These logs can be copied to a central control server (SRV CNTRL) for analysis and future use.

[0402] Figure 46 A topology through a global virtual network (GVN) is shown, showing a path from an end point device (EPD) to the Internet in the same region 46-000. The EPD 46-100 is also connected to an access point server (SRV AP) 46-200 via a tunnel on top of the client's underlying Internet connection. This example embodiment also shows path options for traffic to reach different devices beyond the SRV AP 46-200, such as to a SRV AP 46-700 via path 46-P700, to a SRV AP 46-702 via path 46-P702, and to a backbone exchange server (SRV BBX) 46-500 via path 46-P500.

[0403] This example embodiment also describes the same or different protocols in other regions, showing how various fabrics are woven together to form a network tapestry. The quality of these connections is also measured. The quality of service (QoS) of the connection from the EPD 46-100 to the local Internet 46-000 is measured by QoS ISP 46-802. The performance of the tunnel to the GVN 46-806 is measured by QoS TUN OTT ISP. The connectivity through the GVN beyond the SRV AP 46-200 is measured by QoS GVN 46-808.

[0404] The analysis of the quality of the connections through the various path type options through the GVN can be utilized to determine the best path for traffic based on matching the fabric type to the data type, size, QoS requirements, and other factors. The more fabrics that are understood and woven together, the more fabric type options the tapestry can support.

[0405] Figure 47End-to-end cross regional network path 47-CPT300 is shown. It splits this path into three different parts, a local part in one region 47-CTP310, a local part in another region 47-CPT320 and an intermediate part connecting the two regions through a long distance backhaul 47-CTP330.

[0406] Other features described are the fabric available along this network path 47-CPT300. The segment from 47-P402 to 47-428 exemplifies an Internet Protocol version four (IPv4) path 47-400. The segment from 47-P612 to 47-P628 exemplifies an Internet Protocol version six (IPv6) path 48-600. The combined IPv4 and IPv6 path 47-500 is from segment 47-512 to 47-520. The reciprocal spring mechanism to spring described by path 47-800. The spring integrated into and combined with the IPv4 path is shown by combined path 47-900.

[0407] The automated mapping of segments and understanding paragraphs allows the most efficient weaving of various network fabrics together into a tapestry. Automated testing checks and evaluates all routes, including the segments on the primary base path 39-GVN-1 and also within the tertiary GVN tapestry GVN 39-GVN-3.

[0408] Although there are methods to run one network over another by encapsulation or other methods on another base network segment, they can not be compatible across the various different segments on the Internet, so the GVN secondary 39-GVN-2 must be able to hop between network path fabric types when needed. For example, IPv6 can be encapsulated on 47-P402 to 47-P408, then can run on native IPv6 via 47-P510, then to 47-512 to 47-520, then via 47-P622 to 47-P628.

[0409] Tapestry topology - example - stitched together fabric

[0410] Figure 48 How the GVN is built as the top most layer of the over the top of the base network connection (OTT1) is shown. The GVN also weaves together the various fabrics and connects the layers together, for example, from local area network (LAN) A 48-002 through an egress entry point (EIP) 48-108 to a local cloud node 48-122, which is a second order layer top of the over the top (OTT2) of the local GVN (OTT1) on the EPD 48-100. The complete network path exemplified can be described as an end-to-end cloud bridge path from LAN A 48-002 to LAN B 48-012.

[0411] The multi-dimensional top-of-rack fabric between EPD 48-100 and Access Point Server (SRV_AP) 48-300 is built on top of a combined IPv4 and IPv6 channel, GVN builds IP tunnels 48-112 between them, through which channels are built on top of this 48-122.

[0412] This topology also extends the edge of the LAN beyond the edge of LAN 48-000, into the cloud through EPD 48-100 as an extension of the LAN into the cloud 48-322. This mechanism can also pull cloud nodes into EPD 48-100 which acts as a local node to host cloud services via APP or other GVN functionality.

[0413] Other advantages can be achieved with this carpet construction.

[0414] Figure 49 A possible topology for a GVN is shown in which traffic has more than one option for long haul between regions.

[0415] The tunnel or other type of network path between two Access Point Servers (SRV_AP) can be via SRV_AP 49-300 to SRV_AP 49-310 path 49-P308, IP on top of the underlying Internet or long haul or other type of Ethernet (OTT). This segment is measured and analyzed by part ETH 49-020.

[0416] It also shows the path options between two Backbone Exchange Servers (SRV_BBX) 49-500 and SRV_BBX 49-510 via path 49-P500 to IBX cluster 49-038 to path 49-P510 to SRV_BBX 49-510. This segment is measured and analyzed by part IB 49-028.

[0417] Figure 50 Inter-region traffic channels between SRV_APs are shown. This figure is similar to Figure 49 where multiple path options for long distance backhauls are described, such as 50-P 620 IP path measured by part OTT IP 50-620. Another option is IB path 50-P500 through BBX cluster 50-520 to path 50-P510 between SRV_BBX 50-500 and SRV_BBX 50-510.

[0418] This example embodiment also shows multiple SRV_AP servers in IDCs in Region A 50-608 and Region B 50-618, which provide redundancy, multiple paths, and high-availability "front-line" resources, giving the EPD connection options dictated by server availability.

[0419] In this embodiment, SRV_BBX 50-500 and SRV_BBX 50-510 act as hubs for their respective regions, as well as being cross-region global nodes that provide enhanced connection pathways to another region's global nodes and the devices therein.

[0420] Figure 51 Is an algorithmic flow chart depicting how path information is collected 51-110, saved 51-116, run and assembled 51-120 for testing and used to determine the best path for traffic to take through the GVN, to analyze and save 51-126 these results in a database 51-B010. The protocols and specifications for each path are tested 51-130 and saved 51-136. This algorithm is able to adjust 51-210 as needed to improve connections. It checks to see if the routing is ideal 51-220, and if not 51-P250, builds and tests 51-250 new routing.

[0421] If the connection 51-300 is not ideal, the path check and test is restarted via path 51-P102. If the conditions are ideal 51-P380, the results are recorded 51-380 and then the path 51-P022 is restarted 51-020. It will continue to do this until the next time period 51-040, and if it is time 51-P100, it starts again 51-100.

[0422] Example file mapping, transfer, availability via PFS device's application blanket, GVN-geo-D-fast transfer from remote region to local region

[0423] Figure 52 Shows how end-to-end native RDMA can be provided from within a local area network (LAN) at one or more end point device (EPD) 52-100, 52-110 locations, via paths to parallel file system (PFS) devices 52-608 in the same or remote region, utilizing the topology of a global virtual network (GVN). This is OTT1 on the GVN blanket.

[0424] RDMA on top of IB OTT2 fabric constructs built on top of OTT constructs as OTT1 of the GVN.

[0425] This figure extends the edge of the RDMA fabric to make it native to the local RDMA fabric 52-P638 via the 52-P608 connection. Authentication at the edge can be based on application layer rather than network layer factors. These can switch whether the device is discoverable and whether read and / or write and / or other operations are allowed to the device, driver, folder, file, etc.

[0426] Maximum communication optimization for traffic via the integration point on the GVN to the InfiniBand server switch point (SRV_BBX). The SRV_BBX parallel file system (PFS) allows RDMA to be available to the file manager on the SRV_AP locally and via IB transport.

[0427] Figure 53 This figure shows how a globally distributed parallel file system (PFS) can allow seamless access to one of three parallel file system storage nodes 53-800 or 53-802 or 53-812, allowing native RDMA access over the GVN carpet top of various non-native network fabrics (OTT) to achieve the required quality of service (QoS) and follow the high performance computing (HPC) principles required for such functionality. Path 53-P300 is the underlying Internet connection, 53-TUN00 runs on top of 53-P300. Path 53-P500 is on top of the Internet within an IDC or between IDCs.

[0428] Another embodiment can be that one PFS instance 53-800 in the client's LAN A 53-102 behind the EPD 53-100 is linked to two other PFS instances in the cloud 53-802 and 53-812. The tunnel to connect these three PFS devices through the GVN can be native RDMA as a fabric within the larger GVN carpet, regardless of the underlying network connection, and in parallel with other fabricated fabrics through the GVN.

[0429] Figure 54 This figure also shows how a globally distributed parallel file system (PFS) can allow seamless access to three parallel file system (PFS) storage nodes, allowing native RDMA access over the GVN carpet top of various non-native network fabrics (OTT). This example embodiment is a continuation of Figure 53 , further exemplifying the logical fabric of RDMA tunneling options, the options being bridged paths 54-P600 to 54-P508 and end-to-end paths 54-P610 as second order top of the tunnel (OTT2) within the global virtual network (GVN).

[0430] This example embodiment also shows the application network carpet providing native RDMA between various endpoints through the GVN tunnel top of various network fabrics (OTT).

[0431] Devices in LAN 54-000 are able to access files physically stored on PFS file storage devices such as 54-600 and / or 54-610 via RDMA as if they were local and directly connected to the PFS devices. File synchronization and transfer replication via regions can also be via path 54-P510.

[0432] Figure 55 Built on top of Figure 53 to 54 this illustrates how devices connected via a GVN can have direct RDMA access to parallel file system (PFS) devices in each region.

[0433] It also shows how each server such as access point server (SRV_AP) 55-300 with a hierarchical file system (HFS) attached to it contains an HFS file storage device 55-308, and how a backbone exchange server (SRV_BBX) 55-500 contains an HFS 55-508, etc.

[0434] Two SRV_BBX servers 55-500 and 55-510 are connected via path IBB 55-580, which refers to an Internet backbone or fiber connection or other connection between the two regions. Each SRV_BBX is connected to one or more SRV_APs, for example, SRV_BBX 55-510 is linked with SRV_AP 55-310. Each SRV_BBX is connected to a native InfiniBand (IB) cluster in its region, for example, IB cluster 55-550 is connected with SRV_BBX 55-500 via path 55-P500. This IB cluster 55-550 provides logical network channel access to PFS devices 55-552, 55-556, and 55-558, respectively. IB cluster 55-560 similarly provides access to PFS devices 55-568, 55-566, and 55-562.

[0435] This topology as a second tier top OTT2 allows for native RDMA paths across regions across fabric, regardless of the underlying network fabric.

[0436] Figure 56How files are stored, cataloged, found, and accessed based on the physical layer 56-100, how they are used at the use layer 56-300 by the global file manager (GFM), and how information about the files is stored in a database (DB) 56-220 at the abstraction layer 56-200 are shown. Channels 56-FA108 and 56-FA102 represent file access (FA). Paths 56-DP102, 56-DP108, and 56-DP220 are for database information paths (DP) between the physical files stored on the HFS device 56-102 and / or PFS device 56-108 and the file information in the file table at 56-202. Information about each file is stored in a file table database row, such as 56-222 data row. Example fields for a file table data row can be [Storage_Type] of HFS, PFS, or other, [Device_ID] is a device ID that references a device table, [Server_ID] is a server ID, [Device_Type] can be EPD, SRV_AP, SRV_BBX, or other, [Folder] is a path to the folder where the file is saved. Other fields can be in the structure of the file table.

[0437] File paths (FP) 56-PF102 and 56-FP108 are used for file access to the HFS 56-102 or to the PFS 56-108, respectively, which are combinations of device type, device ID, and folder ID where the physical file is located. Other tables related to the file table 56-202, such as file associations 56-204, servers 56-210, and users 56-206 can be associated with files. There can be more or less tables in an implementation.

[0438] The point is that the GFM 56-302 at the use layer 56-300 has indexed and organized information stored in the tables at the abstraction layer 56-200, including extensive information about each file, and where the files are stored on devices at the physical layer 56-100.

[0439] Figure 57 The operation of the global file manager (GFM) on each device in the GVN and the operation of the central global file manager (CGFM) on the central control server (SRV CNTRL) 57-200 are shown.

[0440] Each GFM is responsible for tracking the files stored on the hierarchical file storage (FIFS) devices contained within them, for example, the SRV AP GFM 57-300 tracks the files stored on FIFS 57-306, the SRV_BBX GFM 57-500 tracks the files stored on HFS 57-506, and so on.

[0441] Each GFM on each device reports information about its files to the CGFM on SRV_CNTRL 57-200 via API paths 57-200300, 57-200500, and 57-200510. Conversely, the CGFM also replicates file storage and location information to all devices using the API paths described above.

[0442] In addition, as files are stored, modified, or otherwise managed on parallel file system (PFS) devices, such as 57-800, 57-802, 57-806, 57-810, 57-812, and 57-816, file information is also transmitted to the CGFM 57-200, which then replicates this information to all devices.

[0443] File transfer paths 57-FP300 between SRV_BBX 57-500 and SRV_AP 57-300 and 57-FP500 between SRV_BBX 57-500 and SRV_BBX 57-510 are also indicated.

[0444] Rug application - example - geo-destination

[0445] Figure 58 A geo-destination mechanism is shown, in which various modules are distributed among devices such as endpoint device (EPD) 58-100, access point server (SRV_AP) 58-300, central control server (SRV_CNTRL) 58-200, and backbone exchange servers (SRV_BBX) 58-D550 and 58-D500.

[0446] Connections between EPD 58-100 and SRV_AP 58-300 can be via path 58-CP02 or 58-TP00 to 58-TP02 or via backbone paths 58-BB0 between SRV_BBX 58-D550 and 58-D500.

[0447] SRV_BBX servers allow the geo-destination mechanism to achieve high-speed long-distance file availability via PFS using network rugs, as opposed to (only) client-server transfer technology for chained caches and / or other methods.

[0448] Figure 59The geographic destination mechanism within the GVN is shown. It also shows the efficiency of the remote fetcher bot (RFB) 59-D328 working with the content pull agent (CPA) 58-D320 to fetch content 58-600, 58-602, 58-606, 58-608, and 58-610 on behalf of the remote client 58-800. The content delivery agent (CDA) 58-D120 working on the EPD 58-100 communicates with the CPA 58-D320 to make it work as if the client 58-800 is located in the remote region where the SRV_AP 58-300 is located. Using the IP address of the remotely located SRV_AP 58-300, the content fetched from the geographic perspective is local to that remote region. However, to enhance performance, the following functions of the geographic destination mechanism are used to speed up and at the same time simplify (from the user's perspective of the client) the process: On modern web pages, there is often a mix of many individual content files from various sources. Fetching each file from a remotely located server has limitations and problems due to routing, bandwidth (BW) bottlenecks, latency, packet loss, and other issues.

[0449] When the client has to fetch many files, such as tens to over a hundred individual files plus managing the data streaming, the distance problem can significantly mix in.

[0450] Figure 60 The geographic destination mechanism within the GVN is also shown, specifically showing how the remote fetcher bot (RFB) 59-D328 on the access point server (SRV_AP) 59-300 in the remote region where the content is located retrieves multiple files 59-600, 59-602, 59-606, and 59-608.

[0451] The retrieved files are passed to the cache manager 59-D330 on the SRV_AP 59-300, where they are sorted and assembled together into one large file 59-700, which can be saved to the parallel file system (PFS) 59-508 or PFS 59-558.

[0452] This list of cataloged files is passed to a Content Delivery Agent (CDA) 59-D120 on the EPD 59-100 for use by a cache manager 59-D130 to group resolve and inspect the files, and upon successful validation, the CDA 59-D120 serves the files to the client. The files 59-610, 59-612, 59-616, and 59-618 are served from the EPD 59-100 to the requesting client as if they were being served by the origin server. This geographic mechanism in combination with other elements of the GVN provide a reverse CDN to deliver local performance such as low latency and high BW to the effect of bringing remote locations to the client.

[0453] Rug application - example - WAN

[0454] Figure 61 It is shown that two LANs 61-000 and 61-010 are bridged via EPDs into a Wide Area Network (WAN), each first connecting to an Access Point Server SRV_AP 61-200 via a base tunnel built over the top of their Internet connection (OTT).

[0455] From EPD 61-100, the base connection path OTT is via path 61-P022 to a Point of Presence (POP) 61-022, to the Internet 61-020, to POP 61-024 of SRV_AP 61-300.

[0456] From EPD 61-110, the base connection path OTT is via path 61-P032 to a Point of Presence (POP) 61-032, to the Internet 61-030, to POP 61-034 of SRV_AP 61-300. This can also point to another SRV_AP not shown here that can be linked to SRV_AP 61-300.

[0457] The transmission path 61-P026 from POP 61-024 via 61-P036 to SRV_AP 61-300 to POP 61-034 can be through the Internet, through the SRV_AP, or bypass the SRV_AP and rely on routing over public networks. If EPD 61-100 wishes to connect to EPD 61-102 via the Internet, it can follow a different route based on policies that are not controlled by the GVN or either EPD. EPD 61-100 builds a tunnel TUN 61-T00 between itself and SRV_AP 61-300. EPD 61-102 also builds a tunnel TUN 61-T10 between itself and SRV_AP 61-300. One or both of these tunnels can or can not be encrypted or secured.

[0458] It can also be another tunnel, an internal tunnel INT TUN 61-T20, run through two other tunnels that converge at SRV_AP 61-300, through which traffic can flow. This tunnel can be the communication path that the WAN that builds the connection of EPD 61-100 to EPD 61-110 goes through.

[0459] The point is that in the tunnels and underlying connections, both can be different network protocols. The network fabric that a GVN can support can be a mix of different network protocols mapped to a string of various network segments, while the GVN can be an end-to-end top fabric of one network type within the internal tunnels.

[0460] Figure 62 Multiple path options are shown for transferring files between an endpoint device (EPD) 62-100 connected to an access point server (SRV_AP) via a tunnel TUN 59-200 in one region and another EPD 62-110 connected to an access point server (SRV_AP) 62-310 via a TUN 59-210 in another region.

[0461] Paths 62-P600 to 62-600 to 62-P602 and 62-P610 to 62-610 to 62-P612 are for IP OTT Internet. The path via 62-600 is for end-to-end file transfer, and the path via 62-610 utilizes chained caching of files to utilize the super-speed of the backbone network to transfer files as fast as possible to storage devices, to requesting pulling clients, or to requesting pushing recipient devices.

[0462] Path 62-P500 connects a backbone exchange server (SRV_BBX) 62-500 to SRV_AP 62-300. Path 62-P510 connects a backbone exchange server (SRV_BBX) 62-510 to SRV_AP 62-310. Paths 62-P800 to 62-800 to 62-P802 and 62-P810 to 62-810 to 62-P810 are for native InfiniBand (IB) or IP and / or RDMA flowable equivalent top dedicated lines on dark fiber. The path via 62-800 is for direct RDMA access to files on a PFS server where the files are stored. The path via 62-810 involves copying files from a source PFS device to another PFS device in another region.

[0463] Traffic selection makes traffic flow decisions based on traffic type via the most appropriate path type. Optimal flow of different data via the best path type then follows the best "current" routing path through the GVN. This is a win-win result.

[0464] Figure 63 The IBB path 63-800 is shown to be fully isolated so that internal communications are on a clean and secure path.

[0465] The FWs 63-400 and 63-410 protect the internal IP communication paths 63-P300 and 63-P310 between the access point servers (SRV APs) 63-300 to 63-310 to the backbone switch servers (SRV BBXs) 63-500 and 63-510, respectively.

[0466] Another protection is that the paths 63-P100, 63-P300, 63-P110, and 63-P310 are Internet Protocol (IP) and the paths 63-P500, 63-P510, and 63-P528 are InfiniBand (IB). This physical protocol hop in addition to the firewall provides a chink so that contamination between IP and IB is logically impossible.

[0467] Figure 64 The topology of sequential linear point-to-point connections from region A 64-000 to region B 64-010, through a large distance 64-020, is shown.

[0468] The SRV BBX 64-500 acts as a public portal for the SRV APs, e.g., SRV AP 64-300, in region A 64-000. The SRV BBX 64-510 acts as a public portal for the SRV APs, e.g., SRV AP 64-310, in region B 64-010. The SRV APs and SRV BBXes in the same region can be located in the same Internet Data Center (IDC), or they can be located in other IDCs in the same region connected by fast links.

[0469] The secure file system layer using RDMA over IB between the SRV BBXes 64-500 and 64-510 is able to provide ultra-fast access to files stored on parallel file system (PFS) devices managed by a global file system (GFS).

[0470] Rug logic and logical structure

[0471] Figure 65 The logical structure of the physical and virtual interfaces on an endpoint device (EPD) 65-100 and their corresponding connections to devices outside the EPD 65-100 are shown.

[0472] Physical ports ETHO 65-100, ETH1 65-106, and ETH2 65-108 correspond to network plugs on the EPD backplane. ETHO 65-102 connects to the last mile connection between the EPD 65-100 and the Internet Service Provider (ISP) provided Internet. ETHO 65-102 connects to a Point of Presence (POP) 65-022 via path 65-P022 and from there to the Internet 65-020 and beyond.

[0473] Tunnels TUNO 65-310 and TUN2 65-312 run over and through the top of the last mile (OTT) of ETHO 65-102.

[0474] ETH1 65-106 connects to LAN A 65-050 and ETH2 65-108 connects to LAN B 65-060.

[0475] Both ETH1 65-106 and ETH2 65-108 are pooled as LAN connections within the EPD 65-100 at bridge BR 065-104.

[0476] Routing is applied at a string of Virtual Interfaces (VIF) between BR0 65-104 to VIFO 65-102 where the routing table matches traffic through TUNO 65-310. For unmatched addresses, they are passed to VTF1 65-122 where the routing table match will push traffic to TUN2 65-312. The remaining unmatched addresses go to VIF2 65-126 which will then egress via path 65-P022.

[0477] The physical fabric is tested and managed at each of the various physical interfaces. A top fabric is constructed on top of these physical interfaces which constitutes the Global Virtual Network (GVN). The various fabrics are woven together to form the network carpet.

[0478] Figure 66A conceptual model is shown to describe the layers of a Global Virtual Network (GVN) level 39-GVN-1 and the layers of level 39-GVN-3 built on and integrated with level 39-GVN-1. It describes the logical constructs for the layers of an End Point Device (EPD) 66-100, an Access Point Server (SRV_AP) 66-200, and a Backbone Switch Server (SRV_BBX) 66-500. It also shows the physical network interfaces (NICs) on each of these devices, such as an Ethernet NIC 66-M0 on EPD 66-100, or an Ethernet NIC 66-M1, an IB NIC 66-N1, an Ethernet NIC 66-M2 on SRV_AP 66-200, or an ETH NIC 66-M3, an IB NIC 66-N2 on SRV BBX 66-500.

[0479] The connection between ETH NIC 66-M0 on EPD 66-100 and ETH NIC 66-M1 on SRV_AP 66-200 is via path Ethernet 66-000. The connection between SRV_AP 66-200 and SRV_BBX 66-500 is via Ethernet path 66-010 or InfiniBand 66-020, providing one or the other as a network connection option. IB NIC 66-N2 can also connect to a SRV_BBX in another region 66-510 via InfiniBand path 66-030. See Figure 67 More details of the conceptual model layers at GVN level 39-GVN-1 and GVN level 39-GVN-3 are obtained.

[0480] Figure 67 A level 39-GVN-1 of an IP model of a GVN is shown in comparison to an IP model of a level 39-GVN-3 of a GVN in a stack top organization. The network interface 67-T1 of the level is an Ethernet protocol 67-R1 for ETH NIC 67-N1. Internet 67-T2 corresponds to IP 67-R2A. Transport 67-T3 corresponds to either protocol TCP 67-R3A or UDP 67-R3B. The application layer 67-T4 can be HTTP 67-R4A or POP3 67-R4B or other, or GVN ETH layer 67-R4C. The GVN stack 67-C3 then relates to the IP layer 67-R5 in GVN Internet 67-G5, GVN transport 67-G6 relates to TCP 67-R6A and UPD 67-R6B. Applications 67-G7 relate to FTP 67-R7A, HTTP 67-R7B, POP3 67-R7C or other.

[0481] The figure also shows how the base layer can be predicted on an InfiniBand (IB) NIC 67-N2. The RDMA layer 67-R2B is related to the Internet 67-T2, the Internet Protocol (IP) over IB IPoIB 67-R3C is related to the transport 67-T3, and the GVN IB 67-R4D is related to the application 67-T4.

[0482] Figure 68 There are the base Internet layer 68-ATOP82 and the first and second over-the-top (OTT1 and OTT2) layers. The Internet and OTT1 layers are combined to provide the best routing and performance options for traffic to flow through the global virtual network (GVN). The OTT2 layer is on top of the OTT1 layer to provide constructs to be built on top of the GVN.

[0483] There are also five levels of the GVN, which correspond to the three layers described above.

[0484] The GVN level 1 68-L100 is the base network layer. The GVN level 3 68-L300 is the internal channel through which traffic is optimized to flow, and the GVN level 2 68-L200 is the logical layer between the level 1 68-L100 and the level 3 68-L300, which is where the testing, analysis, mapping, routing, conditioning, encapsulation, security, and other operations are performed to ensure the best performance of the various options given by the level 3 68-L300 relative to the level 1 68-L100.

[0485] The GVN level 5 68-L500 is the internal channel of the constructs built on top of the GVN internal channel of the level 3 68-L300, which itself is built on top of the base network layer level 1 68-L100. The GVN level 4 68-L400 is the logical layer between the level 5 68-L500 and the level 3 68-L300, which requires an understanding of the options available to it through the GVN, with similar testing, analysis, and other operations. Of particular interest are the peers, the hops between the OTT levels, the mapping, the protocols, and the end-to-end path options relative to stitching together the segments in the path most appropriately and efficiently.

[0486] The present exemplary embodiment can be directly related to Figure 48 where the LAN A 48-200, the Internet 48-000, the Internet 48-010, and the LAN B 48-012 are all at the GVN level 1 68-L100.

[0487] The local GVN 48-112, the GVN on the AP 48-312, and the local GVN 48-116 are all at the GVN level 3 68-L300. This layer is where the performance and routing are focused to provide the options for the GVN.

[0488] Local cloud nodes 48-122, LAN in cloud extension 48-322, and local cloud node 48-128 are all at GVN 5 level 68-L500. These represent the fabric through the GVN.

[0489] Figure 69 is a system diagram for some example devices in a GVN for utilizing a network carpet. The devices described here are end point devices (EPD) 69-100, access point servers (SRV_AP) 69-300, central control servers (SRV_CNTRL), and backbone exchange servers (SRV_BBX) 69-500.

[0490] Two network interface cards are indicated on the SRV_BBX, an Ethernet IP NIC 69-506 and an IB NIC 69-510, to correspond to these different network protocols based on hardware (HW) differences.

[0491] System software 69-130, 69-330, 69-230, and 69-530 make up the fabric logic of the GVN to create the network carpet.

[0492] Communication paths are also indicated, for example:

[0493] 69-P200 ↔ 69-P430 ↔ 69-P500 - API between SRV_BBX 300 and SRV CNTRL_200

[0494] 69-P510 ↔ SRV_BBX 69-510 ↔ 69-P810 - This is the path to other regions. A parallel file storage device PFS 69-810 is indicated here as an example, the BBX 69-510 is able to connect to many other devices.

[0495] 69-P100 ↔ 69-P400 ↔ 69-P300 - Can indicate traffic or API between EPD and SRV_AP, 69-P100 ↔ 69-P410 ↔ 69-P200 - Can represent API or other type of communication path between EPD and SRV_CNTRL.

[0496] 69-P300 ↔ 69-P436 ↔ 69-P500 - Is the path between SRV_AP 69-300 and SRV_BBX 69-500.

[0497] 69-P510 ↔ BBX 69-510 - Represents the traffic path between SRV_BBX servers on the backbone, which spans long distance connecting regional clusters, or simply other connections of SRV_BBX hubs and spoke clusters, including devices such as PFS clusters, other SRV_BBXs, other backbones, etc.

[0498] Global file managers 69-360, 69-260, and 69-560 catalog and manage files on hierarchical file system (HFS) storage 69-630, 69-620, 69-650 and parallel file systems such as 69-800 or 69-810.

[0499] Fabric managers 69-380, 69-280, and 69-580 work independently, sometimes in step, to build first order top (OTT1) and second order top (OTT2) tiers.

[0500] According to embodiments of the present disclosure, the following notes are also disclosed:

[0501] 1. A system for connecting devices via a global virtual network across a network fabric, comprising: a first access point server in communication with a first backbone switch server;

[0502] a second access point server in communication with a second backbone switch server; and

[0503] a network fabric comprising a first communication path connecting the first and second access point servers and a second communication path connecting the first and second backbone switch servers.

[0504] 2. The system of note 1, wherein the first communication path is IP over the Internet.

[0505] 3. The system of note 1, wherein the second communication path is InfiniBand.

[0506] 4. The system of note 1, wherein the first communication path is IP over the Internet and the second communication path is InfiniBand.

[0507] 5. The system of note 1, further comprising:

[0508] a first parallel file storage in communication with the first backbone switch server;

[0509] a second parallel file storage in communication with the second backbone switch server, wherein the first backbone switch server is capable of writing directly to the second parallel file storage using the second communication path without using the first communication path.

[0510] 6. The system of note 5, wherein the first communication path is IP over the Internet.

[0511] 7. The system of note 5, wherein the second communication path is dark fiber.

[0512] 8. The system of clause 5, wherein the first communication path is IP over the Internet and the second communication path is dark fiber.

[0513] 9. The system of clause 5, wherein the first backbone switch server writes to the second parallel file storage using a remote direct memory access (RDMA) protocol.

[0514] 10. The system of clause 1, further comprising:

[0515] a first firewall in the communication path between the first access point server and the first backbone switch server;

[0516] wherein the firewall isolates the first backbone switch server from threats present on the first communication path.

[0517] 11. The system of clause 10, further comprising:

[0518] a second firewall in the communication path between the second access point server and the second backbone switch server;

[0519] wherein the second firewall isolates the second backbone switch server from threats present on the first communication path.

[0520] 12. The system of clause 1, further comprising:

[0521] an endpoint device in communication with the first access point server; and a host server in communication with the second access point server.

[0522] 13. The system of clause 12, further comprising a communication protocol between the endpoint device and the host server is one of infiniband, RDMA, IPv4, and IPv6.

[0523] 14. The system of clause 13, wherein the communication protocol is encapsulated in a different protocol between the endpoint device and the first access point server.

[0524] 15. The system of clause 13, wherein the communication protocol is encapsulated in a different protocol between the second access point server and the host server.

[0525] 16. The system of clause 13, wherein the communication protocol is encapsulated in a different protocol between the first backbone switch server and the second backbone switch server.

Claims

1. A method for connecting devices via a global virtual network across a fabric of networks, comprising: causing a first access point server to communicate with a first backbone switch server and an endpoint device, wherein the endpoint device is coupled to a first file system, wherein the first access point server and the endpoint device communicate via (a) a first communication protocol that uses a store-and-forward model and (b) a tunnel built on top of the first communication protocol, the tunnel using a second communication protocol that utilizes near- way switching; causing a second access point server to communicate with a second backbone switch server, wherein the second backbone switch server is coupled to a second file system, and wherein: the second access point server is coupled to the first access point server via a first communication path, the first communication path using the first communication protocol, and the second backbone switch server is coupled to the first backbone switch server via a second communication path, the second communication path using the second communication protocol; and selecting, by high-level intelligent routing, the tunnel and the second communication path to carry traffic between the endpoint device and the second access point server based on a type of the traffic, quality of service requirements of the traffic, characteristics of the first communication path, and characteristics of the second communication path, wherein carrying the traffic includes writing a file from the endpoint device to the second file system via a local end-to-end remote direct memory access (RDMA) fabric using the second communication protocol, the first communication path is IP over the Internet, and the second communication path is InfiniBand over dark fiber.

2. The method of claim 1, wherein, the first backbone switch server writes directly to the second file system using the second communication path without using the first communication path.

3. The method of claim 2, wherein, the first backbone switch server writes to the second file system using the remote direct memory access (RDMA) protocol.

4. The method of claim 1, further comprising: setting a first firewall in a communication path between the first access point server and the first backbone switch server; wherein the firewall isolates the first backbone switch server from threats that occur on the first communication path.

5. The method of claim 4, further comprising: setting a second firewall in a communication path between the second access point server and the second backbone switch server; wherein the second firewall isolates the second backbone switch server from threats that occur on the first communication path.

6. The method of claim 1, further comprising: causing a host server to communicate with the second access point server.

7. The method of claim 6, wherein, a communication protocol between the endpoint device and the host server is one of InfiniBand, RDMA, IPv4, and IPv6.

8. The method of claim 7, wherein, the communication protocol between the endpoint device and the host server is encapsulated in a different protocol between the endpoint device and the first access point server.

9. The method of claim 7, wherein, the communication protocol between the endpoint device and the host server is encapsulated in a different protocol between the second access point server and the host server.

10. The method of claim 7, wherein, The communication protocol between the endpoint devices and the host server is encapsulated in a different protocol between the first and second backbone exchange servers.

Images