System for global virtual networks

Abstract

Systems and methods for connecting devices via a virtual global network are disclosed. In one embodiment, the network system can include a first device in communication with a first endpoint device and a second device in communication with a second endpoint device. The first device and the second device can be connected with a communication path. The communication path can include one or more intermediate tunnels connecting each endpoint device to one or more intermediate access point servers and one or more control servers.

Description

[0001] This invention patent application is a divisional application of the invention patent application filed on January 28, 2016, with application number 201680007187.5 and invention title "System and Method for Global Virtual Network".

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 108,987, filed January 28, 2015; U.S. Provisional Patent Application No. 62 / 144,293, filed April 7, 2015; U.S. Provisional Patent Application No. 62 / 151,174, filed April 22, 2015; U.S. Provisional Patent Application No. 62 / 174,394, filed June 11, 2015; International Patent Application No. PCT / US2015 / 064242, filed December 7, 2015; U.S. Provisional Patent Application No. 62 / 266,060, filed December 11, 2015; and International Patent Application No. PCT / US2016 / 012178, filed January 5, 2016, all of which are incorporated herein by reference. U.S. Provisional Patent Application No. 62 / 089,113, filed December 8, 2014, and U.S. Provisional Patent Application No. 62 / 100,406, filed January 6, 2015, are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to networks, and more specifically to the configuration and operation of Global Virtual Networks (GVNs). Background Technology

[0004] Although "last-mile connectivity" has improved dramatically in recent years, long-distance connectivity and throughput issues remain due to distance, protocol limitations, peering, interference-related problems, and other issues and threats. GVN provides secure network optimization services to clients on top of their standard internet connections.

[0005] This application provides an overview of the components of GVN and describes the relevant technologies that can be used as GVN elements. GVN elements can operate independently or within the GVN ecosystem, such as adopting a GVN architecture for their own purposes, or they can be deployed to enhance the performance and efficiency of GVN.

[0006] This overview also describes how other technologies can benefit from GVN, either as stand-alone deployments using some or all of GVN's components, or as stand-alone mechanisms built on top of existing GVN, thus taking advantage of its benefits.

[0007] Humans can perceive delays of 200 milliseconds or higher because this is typically the average reaction time of a human to an event. If the delay time is too high, online systems such as thin client to cloud-based servers, customer relationship management (CRM), enterprise resource planning (ERP), and other systems will perform poorly and can even stop functioning due to timeouts. High delay time plus high packet loss can result in a connection being unavailable. Even if the data gets through, at a certain point, it is too slow to result in a good user experience (UX) and in these cases, users can ultimately reject the conditions, which in effect, treat the poorly performing service as useless.

[0008] To address some of these problems, various technologies have been developed. One technology is WAN optimization, which typically involves a hardware (HW) device at the edge of a local area network (LAN) that establishes a tunnel to another WAN optimization HW device at the edge of another LAN, thereby forming a wide area network (WAN) between the two hardware devices. This technology assumes that the two devices are connected to each other via a stable connection. WAN optimizers strive to compress and protect data streams, which often results in speed gains. The business driver for employing WAN optimization is to save the data volume sent, thereby reducing the cost of data transmission. The downside of this technology is that it is typically point-to-point and can struggle when the connection between the two devices is poor, because there is little to no control over the traffic path through the Internet between the two. To address this problem, users of WAN optimizers typically choose to run their WANs over MPLS or DDN lines or other dedicated circuits, resulting in additional expense and often also necessitating a rigid, fixed point-to-point connection.

[0009] In the market at the time of writing this patent, some vendors focus on selling hardware and not on the connection service over the Internet between their hardware devices. Other vendors are service providers who can provide simple endpoint devices or software that can be installed by customers onto their own devices to connect to the service provider's cloud servers as a link to the service provided by the vendor's package, but the main focus of these vendors is the service provision.

[0010] Direct links such as MPLS, DDN, private circuits or other types of fixed point-to-point connections can provide connection quality and Quality of Service (QoS) guarantees. These links are expensive and often take a long time to install due to the need for physical wiring from the POP on each side of the connection. Point-to-point topologies work well when connecting from one LAN to resources in another LAN via a WAN over this direct connection. However, when the gateway (GW) to the general Internet is located at the LAN-end LAN, such as at a corporate headquarters, then traffic from a remote LAN in a subsidiary country can be routed through the GW to the Internet. As the traffic returns through the Internet to a server in the same country / region as the subsidiary, a slowdown occurs. The traffic must then flow from the LAN through the WAN to the LAN where the GW is located, then back through the Internet to the server in the original country, then back through the Internet to the GW, and then back along the dedicated line to the client device in the LAN. In effect, what should only take a small fraction of the global latency to access this nearby site, is doubled or tripled (or worse) in global transmission time to access the nearby site. To overcome this problem, alternative connectivity of another Internet line configured with appropriate changes and additional devices can provide local traffic to the Internet at each end of the system.

[0011] Another option to establish a WAN link from one LAN to another involves building a tunnel between two routers, firewalls or equivalent edge devices, such as an IPSec or other protocol tunnel. They are often encrypted and can provide compression and other logic to attempt to improve connectivity. Control over the routing between the two points is minimal or non-existent as they rely on the policies of various intermediate participants on the Internet who transmit their own traffic through their networks and are in peering relationships with other carriers and / or network operators. Firewalls and routers, switches and other devices from several device vendors often have tunnel options built into the firmware.

[0012] Software (SW) based Virtual Private Networks (VPN) provide privacy via a tunnel between a client device and a VPN server. These have encryption benefits and in some cases also provide compression benefits. But equally there is little or no control over how traffic flows between the VPN client and the VPN server and between the VPN server and the host server, host client or other device at the destination. These are typically point to point connections requiring client software to be installed on each device using the VPN and requiring a certain level of technical ability to maintain the connection for each device. If the VPN server exit point is via a high quality communications path close to the destination host server or host client, then performance will be good. If not, then performance will be significantly constrained and cause dissatisfaction in terms of availability. VPN users often have to disconnect from one VPN server and reconnect to another VPN server to access content from one region optimally or locally relative to content from another region.

[0013] A Global Virtual Network (GVN) is a type of computer network on top of the Internet that provides global secure network optimization services with a network of devices distributed around the world securely linked to each other by advanced tunnels; collaborating and communicating via Application Program Interfaces (APIs), Database (DB) replication, and other methods. Traffic routing in the GVN is always via best communications paths managed by Advanced Smart Routing (ASR) driven by an automated system that combines builders, managers, testers, algorithmic analysis, and other methods to adapt to changing conditions over time and to learn in order to configure and reconfigure the system.

[0014] The GVN provides services on top of one or more regular Internet connections to provide secure, reliable, fast, stable, precise, and centralized parallel connectivity. These benefits are achieved through compression of data streams that are transmitted through multiple wrapped, disguised, and encrypted tunnels between an ETO and an Access Point Server (SRV_AP) close to an EPD. The quality of the connection between the EPD and the SRV_AP is continuously monitored.

[0015] The GVN is a combination of hardware (HW) endpoint devices (EPD) with installed software (SW), databases (DB), and other automated modules of the GVN system such as a Neutral Application Program Interface mechanism (NAP), a Reverse Channel Manager, a Tunnel Manager, and more features that connect EPDs to distributed infrastructure devices such as Access Point Servers (SRV_AP) and Central Servers (SRV_CNTRL) within the GVN.

[0016] The algorithm continuously analyzes the current network state, taking into account subsequent trends plus long-term historical performance, to determine the best traffic routing to take and the best SRV AP or series of SRV APs to push the traffic to. Configuration, communication paths, and other changes are made automatically and on the fly, with minimal or zero user interaction or intervention required.

[0017] The advanced intelligent routing in the EPD and SRV APs ensures that traffic flows from origin to destination via the most ideal path through the GVN's "third layer" as simply as possible. Client devices connected to the GVN see this third layer as a normal Internet path, but with fewer hops, greater security, and in most cases, less latency time than traffic flowing to the same destination via the regular Internet. Logic and automation operate in the GVN's "second layer," where the GVN's software automatically monitors and controls the underlying routing and construction of virtual interfaces (VIFs), multiple tunnels, and the combination of communication paths. The third and second layers of the GVN exist on top of the GVN's operational "first layer," which interacts with the underlying Internet equipment. SUMMARY

[0018] Systems and methods for connecting devices via a virtual global network are disclosed. The network system can include a first device in communication with a first endpoint device. The network system can include a second device in communication with a second endpoint device. The first device and the second device can be connected with a communication path. The communication path can include one or more intermediate tunnels connecting each endpoint device to one or more intermediate access point servers and one or more control servers.

[0019] According to other aspects of the present embodiments, at least one of the first endpoint device and the intermediate access point servers are configured to perform a domain name system (DNS) query to locate the second device.

[0020] According to other aspects of the present embodiments, at least one of the first endpoint device and the intermediate access point servers are configured to perform a domain name system (DNS) query from a cache to locate the second device.

[0021] According to other aspects of the present embodiments, at least one of the intermediate access point servers are configured to cache content.

[0022] According to other aspects of the present embodiments, at least one of the endpoint devices and the intermediate access point servers are configured to perform intelligent routing based on a global virtual network.

[0023] According to other aspects of the present embodiments, the intelligent routing is based on at least one of best bandwidth, lowest latency, least hops, and no packet loss.

[0024] According to other aspects of the present embodiments, the intelligent routing is based on at least one of real-time statistics and historical statistics.

[0025] According to other aspects of the present embodiments, at least one of the endpoint device and the intermediate access point server is configured to perform a firewall service.

[0026] According to other aspects of the present embodiments, the firewall service is between the first device and the intermediate access point server.

[0027] According to other aspects of the present embodiments, the firewall service is between the first device and the intermediate access point server and the second endpoint server. BRIEF DESCRIPTION OF DRAWINGS

[0028] For a more complete understanding of the present application, reference is now made to the following descriptions taken in connection with the accompanying drawings in which like numerals represent like numbers throughout the drawings. These drawings should not be construed as limiting the present application, but are intended to be merely for illustrative purposes.

[0029] Figure 1 A block diagram illustrating the technology used and implemented by a global virtual network ("GVN") is shown.

[0030] Figure 2 A high level block diagram of the Internet is shown.

[0031] Figure 3 is a block diagram illustrating the resolution of a uniform resource locator (URL) to a numeric Internet Protocol (IP) via the Domain Name System (DNS).

[0032] Figure 4 is a diagram showing the upstream and downstream paths taken to transmit data from a host client device (C##) to another host client or host server device (S##).

[0033] Figure 5 is a diagram showing the border exchanges in the paths taken to transmit data from a host client device (C##) to another host client or host server device (S##).

[0034] Figure 6 Some example threats and issues that exist on the Internet are shown.

[0035] Figure 7 Content delivery network (CDN) resolution and delivery of region specific content is shown.

[0036] Figure 8 Operation of a proxy server is shown.

[0037] Figure 9 A point-to-point tunnel established between two gateway devices is shown.

[0038] Figure 10 Relationship of security features between device scope, system-wide scope, communication scope, and device collaboration is shown.

[0039] Figure 11 Information flow between devices of a global virtual network is shown.

[0040] Figure 12 Stacks for supporting automation of some devices in a GVN are described.

[0041] Figure 13 GVN topology including backbone segments over the Internet or dark fiber is shown.

[0042] Figure 14 Distributed firewall (FW) in a cloud implemented by a GVN is shown.

[0043] Figure 15 Multi-perimeter firewall (MPFW) in a cloud driven by a global virtual network is shown.

[0044] Figure 16 Logical view of software architecture of three types of network devices working together as part of a global virtual network (GVN) is shown.

[0045] Figure 17 GVN using a hub and spoke topology with backbone segments and octagonal routing is shown.

[0046] Figure 18 Backbone connections between some GVN global nodes in North America, Europe, and Asia and their corresponding service areas are shown.

[0047] Figure 19 Connectivity between various devices within a GVN is shown.

[0048] Figure 20 GVN module and device interaction is shown.

[0049] Figure 21 Additional details regarding GVN module and device interaction are shown.

[0050] Figure 22 GVN module and device interaction with other devices over the Internet is shown.

[0051] Figure 23Multiple tunnel connectivity between an endpoint device (EPD) and an access point server (SRV_AP) is shown.

[0052] Figure 24 Is a simplified example diagram of how the Internet works today, taking into account hop count or time to live (TTL) and paths taken due to peering relationships and associated routing policies.

[0053] Figure 25 Strategic positioning of the infrastructure to enhance performance is shown.

[0054] Figure 26 The way the GVN incorporates technologies such as Network Slingshot is shown.

[0055] Figure 27 How tables in the databases of various GVN devices relate to each other is shown.

[0056] Figure 28 Collaborative results between various modules, mechanisms, technologies, and other components of the GVN are shown.

[0057] Figure 29 The GVN's advanced smart routing (ASR) feature is shown.

[0058] Figure 30 A series of encrypted tunnels between a client (C) and a server (S) is shown.

[0059] Figure 31 The information flow required by two peers in a peer pair is shown.

[0060] Figures 32-35 The GVN's third layer with respect to neutrality and security of GVN tunnels is shown.

[0061] Figure 36 Multiple network structures are shown being woven together into a Tapestry framework.

[0062] Figure 37 Communication paths in the GVN for automatic device collaboration are shown.

[0063] Figure 38 The problems and challenges of dynamic tunnel establishment are shown.

[0064] Figure 39 Two LANs are shown being bridged into a wide area network (WAN) via two or more EPDs.

[0065] Figure 40 A multi-perimeter firewall mechanism (MPFWM) running on the GVN is shown.

[0066] Figure 41 The GVN stack is shown built on top of the Internet (OTT).

[0067] Figure 42 The Internet Protocol (IP) stack, OSI model, and GVN network stack are compared.

[0068] Figure 43 The global Internet flow between countries via numerous possible routes is shown.

[0069] Figure 44 The Internet Protocol (IP) stack, OSI model, and GVN network stack are compared.

[0070] Figure 45 The tunnel between two LANs via GVN is shown.

[0071] Figure 46 The GNV layer 1, layer 2, and layer 3 operations are shown.

[0072] Figure 47 The elements of the Advanced Smart Routing (ASR) feature and the GVN's geo-destination mechanism within the End Point Device (EPD) are shown.

[0073] Figure 48 An example of multiple parallel traffic paths taken via GVN is shown.

[0074] Figure 49 Automatic Advanced Smart Routing (ASR) from one device to a second device is described.

[0075] Figure 50 The secure perimeter between the perimeter- below BB / Backbone layer and the perimeter- above IP / Internet layer is shown.

[0076] Figure 51 is a flowchart of the Advanced Smart Routing (ASR) within the Global Virtual Network (GVN).

[0077] Figure 52 is a flowchart of the various routes available through the GVN from a starting point to a destination.

[0078] Figure 53 is a flowchart of the algorithm that controls the traffic routing from a starting point device to an end point device.

[0079] Figure 54 The modules required for automatic device collaboration and information exchange in the GVN are shown.

[0080] Figure 55 The communication between the EPD, SRV_CNTRL, and SRV_AP via the Neutral API Mechanism (NAPIM) of the GVN is shown.

[0081] Figure 56 Various types of communications available between GVN devices via NAPIM are shown.

[0082] Figure 57 API call groups between different types of devices within a global virtual network (GVN) are described.

[0083] Figure 58 Steps taken from a client device initiating an API call that is sent to a server device and returned to the client are described.

[0084] Figure 59 is a flowchart showing interactions between an EPD and a SRV_AP for obtaining geo-destination functionality.

[0085] Figure 60 Device collaboration within a geo-destination is described.

[0086] Figure 61 Operation of a globally distributed parallel file system (PFS) within a GVN is shown. DETAILED DESCRIPTION

[0087] SUMMARY

[0088] Figure 1 shows a block diagram of the technology used and implemented by a global virtual network ("GVN") including GVN core element G0, GVN modules G100, and technology implemented by global virtual network GVN G20CL The GVN core includes a mechanism overview G1 and its constituent parts, namely topology G2 layer, fabric G3 layer, logic G4 layer, and control G5 layer. The GVN core G0 also includes GVN elements G6 and relationships between these GVN elements.

[0089] The GVN can include plug-ins and / or standalone GVN modules G100 including, but not limited to: a neutral API mechanism ("NAPIM") module G102 as described in PCT / US16 / 12178; a geo-destination ("Geo-D") module G104 as described in PCT / US15 / 64242; an advanced smart routing ("ASR") module G106, a connectivity module G108, and other modules G110 as described in U.S. Provisional Patent Application US 62 / 151, 174.

[0090] The GVN also provides a platform that can enable other technologies, including but not limited to: Network Tapestry G202; MPFWM G204; Network Slingshot G206; Network Beacons G208, Granularity of a tick G210, and other technologies G212. These are described in U.S. Provisional Patent Application No. 62 / 174,394, U.S. Provisional Patent Application No. 62 / 266,060.

[0091] The GVN modules (G100) and technologies enabled by the GVN (G200) can operate as constituent parts of the GVN on top of existing GVN, or can be independent and employ all or some of the separate parts of the GVN to support its own independent operation.

[0092] Figure 2 A high level block diagram of the Internet is shown. The average user has a very sketchy understanding of how the Internet works. Host source 2100 is the starting point and represents a client device, which can be a computer, a mobile phone, a tablet device, a laptop, or other such client. This client connects via the Internet 2200 to a host server 2300 to send or retrieve content, or to another host client 2303 to send or receive information.

[0093] A less technically knowledgeable user can think that the traffic goes along path 2P002 to the host server, not even realizing that their data will be transited through the Internet. Or, they can think that the traffic flows directly to another client device via path 2P006.

[0094] A more knowledgeable user will understand that the traffic flows via path 2P004 to the Internet 2200, and then either via path 2P102 to the host server target 2300 or via path 2P104 to the host (client) target 2302.

[0095] A more technically knowledgeable user will further understand that when an email is sent, this email will leave its client device 2100, travel via path 2P004 to the Internet 2200 and then via path 2P202 to the email server 2202. The recipient of the email will then request to retrieve this email via its host client 2302, along path 2P104 to the Internet, and then along path 2P204 to the mail server 2202.

[0096] That is about the extent of the average person's understanding of the Internet.

[0097] Figure 3is a block diagram showing the resolution of a uniform resource locator (URL) to a digital internet protocol (IP) via the domain name system (DNS).

[0098] Content request 3000 or push from host client (C) 3100 to host server (S) 3300 as a file or data stream or data block flows to host server (S) 3300 from host client (C) 3100. Response or content delivery 3002 as a file or data stream or data block returns from host S to host C. Host client device 3100 in a client-server (CS) relationship with host server (S) requests access to content from a remote host server (S) or sends data to a remote host server (S) via a uniform resource locator (URL) or other network reachable address.

[0099] Initial connection from host client (C) 3100 to the internet 3206 is shown as 3P02, a connection from the host client (C) to a directly facing point of presence (POP) 3102. In other scenarios, the host client (C) can be in a local area network (LAN) that is then connected to the internet via a point of presence (POP) and can be referred to as a last mile connection. Point of presence (POP) 3102 represents the connection from an end point to the internet provided by service providers (ISPs) via their networks and interconnections. This can be, but is not limited to, cable, fiber, DSL, Ethernet, satellite, dial-up, and other connections. If the URL is a domain name rather than a digital address, the URL is sent to a domain name system (DNS) server 3104 where the domain name is converted to an IPv4 or IPv6 or other address for routing purposes.

[0100] Traffic from host client (C) 3100 to host server (S) 3300 is routed through the internet 3206, representing the transmission between POPs (3102 and 3302), including other transmissions to peers, backbones, or network boundaries.

[0101] Connection 3P04 between POP 3102 and domain name system 3104 to find a digital address from a uniform resource locator (URL) to obtain an IPv4 address or other digital address of a target server (S) can be directly accessible from the POP or via the internet 3206. Connection 3P06 from the ISP's POP 3102 to the internet 3206 can be a single or multi-homed connection. Similarly, connection 3P08 from the internet 3206 to a remote ISP can also be a single or multi-homed connection. This connection is generally an internet-facing POP 3302 to an ISP or internet data center (IDC). Connection 3P10 from the remote ISP's POP 3302 to the host server (S) can be direct or via multiple hops.

[0102] The lookup from a URL or hostname to a numeric address via the Domain Name System is the standard on the Internet today, and the system assumes that the DNS server is integral and that the DNS server results are current and trustworthy.

[0103] Figure 4 is a simplified diagram showing the upstream and downstream paths taken to transmit data from a host client device (C##) to another host client or host server device (S##). The numbers used in the device labels such as C01 or S08 are for identification purposes to locate the individual devices, and the numbers themselves do not mean or imply that one device is larger or has more power than another.

[0104] Figure 4 Host client devices (C##), host server devices (S##), switches (SW##), routers (R##), area routers (RR##), edge routers (ER##), core routers (CR##). The communication paths or pipes (P##) refer to the connections between two devices and the line thickness is used to represent the size or bandwidth capacity of the pipe. The thinner the line, the lower the megabits per second (Mbp). The thicker the line, the higher the amount of Mbp or gigabits per second (Gbp). The distances of the P## are not drawn to scale and do not take into account hop counts or time to live (TTL) and delay time or round trip time (RTT) when referring to the P## between devices.

[0105] A simplified local area network (LAN) is downstream of a switch (SW) SW01. It consists of wire connections P01 and P04 to client devices C01 and C04. Wireless connections are represented by dashed lines P02 and P03 between the wireless hub WLAN01 and the wireless client devices C02 and C03.

[0106] The connection P05 between the LAN and its Internet Service Provider (ISP) point of presence (POP) R01 can also be referred to as the "last mile." This POP R01 is the hub that connects other spokes P06, P07, P08, and P09 to corresponding switches of other clients such as SW02, SW03, SW04, and SW05. There is also an upstream path P16 to an area router (RR) RR02.

[0107] This hub and spoke topology is illustrated for POP R02, R03 and R04, their spoke connections to respective switches (e.g. SW 6, SW 7, SW 8, SW 9, SW 10, SW 11, SW 12, SW 13, SW 14, SW 15, SW 16, SW 17, SW 18, SW 19, SW 20) and their connections to their area router (e.g. RR 02, RR 03, RR 04, RR 05) (e.g. P 17, P 18, P 46, P 28).

[0108] A further upstream connection P 19 from area router RR 02 to edge router ER 02 describes a connection to an edge router of an ISP network. Edge router ER 02 has a link P 20 to core router CR 03. This can be considered the backbone of the internet. Link P 32 between CR 01 and CR 02 can describe a very large backbone, which is referred to as a backhaul network or when connecting multiple national networks can be referred to as an international backhaul network.

[0109] Both POP R01 and R02 are connected to area router RR 02 and this can indicate, but is not limited to, that both POPs are located within the network of the same ISP.

[0110] For connectivity between a device within the network of router R01 and a device within the network of router R04, traffic will take one of many possible paths, such as P 16 -> P 19 -> P 20 -> P 30 -> P 31 -> P 24 -> P 27 -> P 28. This can describe connectivity peering between the networks of two or more different ISPs and potentially other carrier peers in between, depending on the owners of the infrastructure through which the traffic is transmitted. Traffic through the backbone will be transmitted by the potentially highest capacity pipe. Traffic between router R01 and router R04 can also be transmitted via path P 16 -> P 41 -> P 44 -> P 23 -> P 27 -> P 28. Although this path can appear shorter, due to pipe size, intermediate devices, peering relationships and policies of the intermediate ISPs, this second path can be the least efficient in terms of control edge router ER 03 to transmit traffic between the two other ISPs. There can also be choke points between them.

[0111] Another feature illustrated is the connectivity of the host servers S08 to S12 connected to the switch SW13. This can be in an Internet Data Center (IDC) or in a LAN. The switch SW13 is connected to the router R03 via P53 and to the regional router RR04 via P46. The connection P46 can describe a leased line or a direct digital connection for enhanced connectivity.

[0112] Another feature shown in this diagram is that P32 is at the most upstream point of reach of the path and the independent host device is at the most downstream point of reach of the path. Downstream of the core processors CR04, CR06 and CR07 are edge routers ER## connected to regional routers RR## which connect down to routers R## located in POPs.

[0113] There can be other possibilities not described herein and in fact, each router R## has multiple spokes to a switch SW## and there are a lot more pipes P## between devices. There can also be more tiers of equivalent regional routers RR## or edge routers ER## devices or other devices in the sequence.

[0114] Figure 5 is a simplified diagram showing the border switching in the path taken to transmit data from a host client device (C##) to another host client or host server device (S##). This can be very similar to Figure 4 Very similar, but with one exception. On the backbone between core router CR01 and core router CR02, at a specific point on the peer-to-peer path between them, there is a series of border switches 400, each of which is limited in capacity relative to the backbone as a whole, and there can be congestion events between these switches.

[0115] Figure 6 Some example threats and issues that exist on the Internet are shown. The network data paths have been simplified in the diagram to outline the connectivity and to highlight the threats from endpoint devices (EPD) and other threats from intermediary devices.

[0116] A content request from host client device C002 to retrieve from host server device 207 should take the path P109->P105->P103->P102->P101 and transmit via the Internet 101 to CP01->CP02->P205->P207. A legitimate Internet Data Center (IDC) can have a load balancer that sends traffic to either a healthy host server 207 (via P207) or an infected host server 206 (via P206). The infected host server can send malware or viruses or other bad content back to the client device C002.

[0117] Another threat is to redirect legitimate traffic to a phony host server 114. Traffic should take the path between CO2 and 207 as described above, however the phony server can suck in the legitimate traffic. The traffic will still take a path such as P109->P105->P103->P102->P101 and pass through the Internet 101, but instead of being transmitted to the legitimate server via CP01, the traffic is transmitted to the phony server 114 via P113 to P114.

[0118] The phony server can be designed to phish for confidential information or credentials or other data by appearing to the Internet user as a real server. The average user cannot distinguish between a legitimate server and a phony server. A third party can also use a phony server to stop legitimate traffic from being transmitted to a client by sending back invalid traffic or altered content.

[0119] Public Domain Name System (DNS) servers can be used on the Internet to be queried by client devices to convert a Uniform Resource Locator (URL) such as a domain name www.thisdomain.com to a numerical IP address such as an IPv4 or IPv6 address so that traffic from a host client device can find a path to a host server device.

[0120] If a DNS server such as 212 or 116 is poisoned 112 or phony 114, the converted numerical IP address can become incorrect directed traffic sent to an illegal or compromised destination device. Another way a DNS can be compromised on the Internet is for a device to not deliver results or deliver incorrect results to operate improperly. Propagation of changes from a master DNS registry server to a DNS server also requires clear and valid connectivity, otherwise indexed results can become stale or incorrect. An example of how to protect and secure DNS lookups will be illustrated by a secure DNS (DNSSEC) server 110 and its connectivity via P19. This relies on the ability of client devices to connect to the DNS server 110 and their "handshakes" not to be interrupted.

[0121] Even when both the host client and host server devices are operating correctly, there is a very real risk that a sniffer or intercept device 204 inserted into the communication path to a host server such as mail server 203 can intercept and capture data since the Internet is not encrypted. Although traffic destined for mail server 203 should flow from the Internet 201 via P 202 to POP 202 to P 203 path to mail server 203, a sniffer or intercept device 204 will cause the traffic to pass through 204 and to P 222 via P 204. It is very difficult to detect such tampering unless the IP address of the hop in the communication path can be positively identified as belonging to a malicious device rather than another router that is part of the Internet infrastructure.

[0122] One growing threat comes from a BOT network of infected devices 213, 215, 216 controlled by a command and control (C&C) server such as 214. These devices can collectively perform a bulk attack such as a distributed denial of service (DDoS) in which a host server device can be overwhelmed with excessive requests beyond their capacity causing requests from legitimate host client devices to become slow or completely unresolvable.

[0123] A BOT network can also be used to perform stealth hacking attacks coordinated by the C&C server so that a large number of different source IP addresses attempting a dictionary password attack will be more difficult to completely block than the same attack from a single IP address.

[0124] A BOT network is also a distribution mechanism for spam email, phishing email, malware distribution, and other malicious purposes.

[0125] National firewalls such as 304 can block the free flow of information. These firewalls can be used as a censorship tool to block traffic that a nation deems undesirable. It can also be used as an intercept device to covertly steal industrial, commercial, or other secrets. Depending on the time of day, the overall Internet traffic, and the health of these national firewalls, traffic transmitted through them can suffer from latency or packet loss, or be shaped to a maximum bandwidth thereby creating a bottleneck, or a combination of all of the above or even other problems.

[0126] The example embodiments mentioned above only describe some of the problems and threats. There are many other threats and new ones are appearing from time to time.

[0127] Figure 7Content delivery network (CDN) resolution and regional specific content delivery is shown. Content delivery networks (CDNs) can provide significant advantages in speed and flexibility and provide load balancing when providing content to clients. A content request 7REQ000 flows from a host client (C) 7100 to a host server (S) and a response flow 7RESP002 of the content delivery returns from the host server (S) to the host client (C) 7100 as a file or data stream or data blocks.

[0128] The host client (C) 7100 can be a device such as a laptop, desktop computer, phone, tablet or other device that functions as a client in a client-server (CS) relationship with the host server (S). The host client (C) requests content provided by the host server (S) via a uniform resource locator (URL).

[0129] The POPs 7102, DNS servers 7104, Internet 7300 operate in a conventional manner as described above.

[0130] In the case of a CDN infrastructure, the CDN mapping tag 7200 operates in coordination with a CDN control server 7202. The CDN mapping tag 7200 and the CDN control server 7202 determine the region in which the host client device is located and to which IN server the host client should connect for the content being provided. For example, if the host client 7100 is in region A, it will be routed via a server POP 7404 in region A to an IN server 7504 in region A. A host client 7100 in region B will connect via a server POP 7402 in region B to an IN server 7502 in region B. A host client 7100 in region C will connect via a server's POP in a server POP 7400 in region C to an IN server 7500 in region C.

[0131] The initial CDN mapping tag 7200 lookup via 7P00, via POP 7102, via POP 7404 can be very fast or can take a relatively high lookup time if the CDN mapping tag server is located in a region far from the client device. Once the lookup is complete, the traffic will flow via 7P008 to the nearest and or best available IN server.

[0132] To illustrate this figure, a region is defined as a geographic region that is different from another geographic region. It does not necessarily represent a large area but can have a large area and it can also represent a large distance from one region to another or they can be very close to each other. The key is that a client in one region will receive content via a CDN server from that region and not from another region.

[0133] In the present example embodiment, the content of each region is different from the content of the other regions. CDN servers 7500, 7502 and 7504 are in communication with origin servers 7600, and content region servers 7700, 7702 and 7704, which publish region-specific content to the CDN servers in each region, and in turn to the clients in their corresponding regions.

[0134] When a client 7100 in a region, for example region C, wants to obtain content provided by a server 7502 or 7504 from another region, whatever they do, they are only provided with content from the server 7500 in their own region. They cannot access other content, even if they attempt to force a connection to the content server in the region from which they desire to receive content. They are constantly obtaining content from their own region without choice. The local DNS lookup 7104 resolves to the IP of the CDN server 7500 in the own region only. This can be due to the global IP address mapping to the CDN in the own region only (in the case of a global IP), or another reason. The result is that the client can be geo-blocked at 7404 or 7402.

[0135] Normal connection via 7408 based on the current geographic location is not blocked, and traffic flows in a manner that causes the host client 7100 to receive content for that geographic location via host server 7500.

[0136] For a target different from the current geographic location 7502 and 7504, traffic stops at 7402 and / or 7408 and the host client is rejected by the content from the remote geographic destination. They can be forced to connect to the server in their current location 7500, or not receive any content or receive an error message or just not the desired content, depending on the configuration and policy of the CDN control system 7202.

[0137] Figure 8The operation of the proxy server is shown. A content request or push 8REQ000 flows from the host client (C) to the host server (S) as a file or data stream or data block. The content delivery 8RESP002 flows from the host server (S) back to the host client (C) as a file or data stream or data block. The host client 8100, a client device in a client-server (CS) relationship with the host server 8500, requests access to content from a remote server (S) via a uniform resource locator (URL). This request will pass through a gateway (GW) device 8102 running proxy client software. In other cases, the proxy client software can run directly on the host client 8100. The proxy client software connects to a proxy server 8306 via an encrypted or unencrypted tunnel, connects from the gateway GW 8102 to a point of presence (POP) 8200 via path 8P02, connects to a WAN 8308 (part of the Internet) via path 8P04, connects to a proxy server 8306 in a remote region via path 8P6. Traffic exits the proxy server 8306, enters the open Internet 8300 via path 8P16 and connects to the POP 8302 and then to the host server 8500 in the target region via path 8P10.

[0138] The host server sees this traffic as coming from the IP address and geographic location of the proxy server. If the IP is in the same region as defined by the server in the target region, the desired content will be provided. To assist in this localization, the proxy server will typically connect to a DNS server 8404 in the same region as the proxy server.

[0139] Figure 9 A point-to-point tunnel TUN established between two gateway devices 9A1 and 9B1 is shown. Each device 9A1 and 9B1 is located at the edge 9EDGE-1 and 9EDGE-2 between the Internet EH3 to H115 and their corresponding local area network (LAN) 9A2 and 9B2.

[0140] The baseline from H11 to EH17 describes the number of point-to-point hops. The number of hops from H13 to EH15 is assumed and provided for illustrative purposes, and the number of hops in a real connection path can be more or less. The number of hops for a client taking the tunnel 9TUN from 9A2 to 9A1 to 9TUN to 9B1 to 9B2 will be about four or five visible hops.

[0141] This example embodiment describes a scenario where LAN 9A2 is connected through its gateway 9A1 to the network of one Internet Service Provider 9ISP-1 and LAN 9B2 is connected through its gateway 9B1 to another Internet Service Provider 9ISP-3. This example embodiment further illustrates that 9ISP-1 does not directly peer with 9ISP-3. Both 9ISP-1 and 9ISP-3 require that their network traffic in both directions must be transmitted through the network of another Internet Service Provider 9ISP-2. The interconnection between 9ISP-1 and 9ISP-2 is defined as peering point 9PP-01 and the interconnection from 9ISP-3 to 9ISP-2 is defined as 9PP-02.

[0142] This point of the example embodiment serves to illustrate that on the Internet, third party Internet Service Providers or equivalent providers such as backbone or backhaul providers will often transmit the traffic of other Internet Service Providers. 9ISP-1 or 9ISP-3 have little to no control over how 9ISP-2 transmits its own traffic. Although 9A2, a customer of 9ISP-1, can directly complain to their provider 9ISP-1 about service issues and 9B2 can directly complain to 9ISP-3, if the issue is with 9ISP-2, 9A2 or 9B2 can do little to directly affect 9ISP-2.

[0143] Potential points of congestion can occur on any device, but since 9PP-01 and 9PP-02 are peering points, they are areas of concern. Control of routing and quality of service for all connections is limited. Therefore, point-to-point tunnels can have difficulty maintaining high quality, stable connections over distance, especially when there is partial traffic transmission through a third party network.

[0144] Figure 10 The relationship of security features between device scope 1080 and system-wide scope 1090 is shown. It also points out the communication scope 1098 and device cooperation 1089.

[0145] With respect to device scope 1080, GVN protects the client privacy of its data, network data flow, credentials, peer-to-peer information, and protects the physical device from suffering intrusion, with proprietary code included from suffering tampering or theft, and other threats.

[0146] System-wide scope 1090 requires protection from intrusion or other malicious traffic such as DDoS attacks, protection from misoperation, routing around suboptimal devices or paths, balancing and dispersing load and preventing exhaustion of resources, IP addresses or other global issues.

[0147] The communication range 1098 focuses on the path of traffic through the GVN primarily pushed through the traffic tunnel TUN. It also covers the entry and exit points (EIP) between the outside network and the internal network of the GVN. It can prevent traffic hijacking, man-in-the-middle attacks, poisoned information sources such as bad DNS, and other threats. In addition, the quality and its nature of the test of the various network segments enables the GVN to understand the complete path QoS and bypass problems.

[0148] The device collaboration 1089 security features are in place to protect the operational integrity of the various devices within the GVN. The secure return channel, intrusion resistant mechanisms, DNS secure web, various database protections such as rotating keys, neutral API mechanisms (NAPIM), automated testing, updating, peer-to-peer relationships, authentication, and other modules can ensure that the system integrity is maintained.

[0149] Figure 11 The information flow between the devices of the global virtual network is shown. The central repository, made up of the database B200 and the file storage HFS200, resides on the central server (SRV CNTRL) 200.

[0150] The communication paths between the devices labeled P### can represent API calls, database replication, direct file transfer, combinations such as database replication through API calls, or other forms of information exchange. The thicker lines 11P200100, 11P200300, 11P200500, 11P100200, 11P100300, 11P10011500, 11P300200, 11P300500, and 11P500200 represent the communication between the GVN devices with peer-to-peer pairs and privileged relationships between each other.

[0151] The looped mode of peer-to-peer communication is shown in the figure from SRV CNTRL 200 via 11P200100 to EPD100, from SRV CNTRL 200 via 11P200300 to SRV AP 300, or from SRV CNTRL 200 via 11P200500 to other device 11500. EPD 100 communicates with SRV CNTRL 200 via 11P100200, with SRV AP 300 via 11P100300, and with other device 11500 via 11P1001500.

[0152] In some cases, the devices will share information loops, such as EPD 100 can request information from SRV CNTRL 200 via 11P100200, and the request will be sent back to EPD 100 via 11P200100.

[0153] In other cases, one device can report information related to other devices, such as SRV_AP 300 reporting to SRV_CNTRL 200 via 11P300200, while SRV_CNTRL 200 subsequently sends information to EPD 100 and SRV_AP 300 via 11P200100, and to other SRV_AP 300 than the reporting SRV_AP 300 via 11P200300, and to other device 1 1500 via 11P200500.

[0154] In other cases, no full loop is needed, such as sending log record information from a device such as EPD 100 to SRV_CNTRL 200 via 11P100200, without further forwarding of this information. However, log record information can later move from a repository on SRV_CNTRL 200 to a long-term log record storage server 1 1500 via 11P200500.

[0155] There is a direct link 11P100300 between device EPD 100 and SRV_AP 300. Direct link 11P300500 is from SRV_AP 300 to other device 1 1500. Direct links involve communication between devices that does not require SRV_CNTRL 200 participation.

[0156] Push information from SRV_CNTRL 200 can be RSS feed information or other types of information published via 11P306. APIs from SRV_CNTRL 200 can be traditional API transactions, or RESTful API calls via 11P302REQ to make requests and 11P302RESP to receive responses. The push information and API elements presented are to illustrate a similar system architecture of devices that do not share peer-to-peer relationships, privileged status, and / or have GVN devices.

[0157] Figure 12 A stack is described for supporting automation of some devices in a GVN. Specifically, this figure illustrates the modules needed for automated device collaboration and networking and operating system (0 / S) management.

[0158] EPD 100 is an endpoint device. SRV_AP 300 is an access point server located in a target destination area. SRV_CNTRL 200 is a central control server accessible by both EPD and SRV_AP, and by other devices or other GVN modules, components, or servers that can support graphical destination mechanisms.

[0159] Each device EPD 100, SRV_AP 300 and SRV_CNTRL 200 stores information about itself in lists, files, database tables and records and in other ways in a local information store. This store also includes information about peer device relationships, stored log records and other related operational information. SRV_CNTRL 200 also has additional storage functions and its role is to provide information to other devices associated with it and / or to peer devices that can be connected to it in order to assess current status and provide guidance similar to a centralized control, such as publishing server availability lists and other functions. A neutral API mechanism (NAPM) can send information between devices and their connected peers and can also be used to update the API itself.

[0160] The database S293 on SRV_CNTRL 200 serves as a repository for related information for that device itself as well as a central repository for other devices. There can be many different SRV_CNTRL 200 servers in many locations to act as a multi-master device. Each database can store specific information, including tunnel information, peer information, traffic information, cache information and other information. Security and other aspects are managed independently by each device, including heartbeat functions, trigger scripts and other mechanisms.

[0161] GVN software D196, D296, D396 includes tunnel builder / manager, virtual interface manager, automatic intelligent routing, test modules, security, logging and other functions. Figure 11 Operating system (0 / S) level packages D195, D295, D395 are also shown and include hardware and software drivers, drivers, installed packages, including their dependent software packages, and other items built on top of system hardware components.

[0162] Figure 13 The GVN topology is shown including backbone segments on the Internet or dark fiber. International Patent Application No. PCT / US15 / 64242, entitled SYSTEM AND METHOD FOR CONNTENT RETRIEVAL FROM REMOTE NETWORK REGIONS, discloses a feature in which multiple files are aggregated into larger files and sent from one geographic region to another via "chain caching" over file transfer. To / for this advantageous feature, file transfer needs to be as fast as possible. As a transfer method for groups of various data payloads "files", the information slingshot method of the present invention moves larger chunks of data from one end of the world to the other more quickly than prior art methods.

[0163] Referring now to Figure 13 illustrates a plurality of zones: LAN Zone 0 (ZL00), LAN Zone 1 (Z110), Internet Zone 0 (ZI00), Internet Zone 1 (ZI10), Internet Zone 2 (ZI20), Internet Zone 3 (ZI30), Internet Data Central Zone 2 (ZD20), and Internet Data Central Zone 3 (ZD30).

[0164] SRV_BBX 1372 in zone or region ZD20 can connect via dark fiber connection 13P220 to SRV_BBX 1380 in another zone or region ZD30 via dark fiber 13220. SRV_BBX 1372 via 13P220, bypassing SRV_BBX stack 1380 and via path 13P82 via Remote Direct Memory Access (RDMA) writes files directly to parallel file storage PFS 1382. SRV_BBX 1380 uses the present application to via 13P220, bypassing SRV_BBX stack 1372 and via path 13P74 via Remote Direct Memory Access (RDMA) writes files directly to parallel file storage PFS 1374.

[0165] Path 13P210 can be IPv4 or some standardized internet protocol through which traffic flows from SRV_AP 13300 to SRV_AP 13310 and / or from SRV_AP 13310 to SRV_AP 13300 via path 13P210 over GVN via tunnel or other type of communication path.

[0166] This shows that various types of network structures can be combined into a larger network tapestry. These structures can seamlessly be woven together as described in U.S. Provisional Patent Application No. 62 / 174,394. This can be a standalone approach or integrated as a network segment within a larger network path made up of multiple network segments. The present example embodiment illustrates the topology of a global virtual network (GVN), its plurality of devices, communication paths, and other embodiments. It shows how various geographic regions or zones or areas are linked together through various types of paths.

[0167] Figure 14 A distributed firewall (FW) in the cloud implemented by GVN is shown. Due to the nature of the topology of GVN, device-to-device communication, and secure traffic paths, the firewall mechanism can be cloud-based and also virtualized. With the face firewall hop 144 to and from GVN via an egress internet point (EIP) via open internet 14000, there can be a cloud firewall (CFW) load balancer 144LB that can distribute cloud firewall resources such as 144-2, 144,3, etc.

[0168] This on-demand provided scalability provides numerous advantages to GVN clients. By mitigating the attack hit rate of threats in the cloud that are about to be subjected to, the "last mile connectivity" of the clients is not affected. This cloud firewall in conjunction with the control node and analyzer enables the FW in the area that is subjected to the attack to perceive the nature, source, signature and other characteristics of the attack so that the cloud firewall can be aware and prepared to defend against the attack as the target shifts. In addition, information about past and current attacks can be shared to other CFW instances via the GVN's Neutral API Mechanism (NAPM) to enable perception of global threats. This also provides the advantage of running multiple types of FW mechanisms simultaneously, as described in reference Figure 15 .

[0169] Figure 15 A multi-perimeter firewall (MPFW) AVN tunnel 15TUN0 in the cloud driven by a global virtual network is shown over the top (OTT) of the Internet between an endpoint device (EPD) 15100 and an access point server (SRV_AP) 15300 in close proximity to the EPD 15100.

[0170] The three perimeters indicated in this example embodiment are: 15M1, which represents the boundary between the client location and its link to the Internet; 15M2, which is the boundary at the data center in the cloud in close proximity to the SRV_AP 15300; and 15M3, which is another boundary at another location in the same data center as the SRV_AP 15300 or in close proximity to the SRV_AP 15302.

[0171] Tunnels 15TUN2 and 15TUN0 are similar, with one difference in one aspect, S, which is the personal endpoint device (PEPD) 15130 that it connects to, which can be a mobile device, thus connecting through a public access wireless or wired or other network to the SRV_AP 15300 to integrate into the GVN.

[0172] Each SRV_AP 15300 and SRV_AP 15302 can represent one or more SRV_AP devices that can be connected to the EPD 15100 and / or EPD 15130 simultaneously via one or more tunnels.

[0173] This example embodiment describes three types of firewalls. The FW Local 15442 is an example firewall that clients can use to protect their local area network (LAN) from internet-based threats. This is typically located between the EPD15100 and LAN15000. This FW15442 provides features such as IP address and port blocking, forwarding, and other functionalities. The other two types of firewalls shown are the FWSPI15446 located on 15M3, which provides Stateful Packet Inspection (SPI), and the FWDPI15444 located on 15M2, which provides Deep Packet Inspection (DPI).

[0174] The difference between SPI and DPI involves a trade-off between performance and visibility. SPI examines the packet header for malicious information or patterns, or matches IP addresses, ports, or other information from a list of known threats against the current packet flow. As the name suggests, DPI examines the entire packet more deeply, and in the case of multi-part, multi-packet transmissions, it examines a compilation of a series of packets to gain further insight into the transmitted data.

[0175] All firewalls can be configured for investigation and application of rules to incoming and outgoing traffic, and provide other related functionality. In many cases, clients will have to choose between the efficiency of SPI and the thorough but resource-intensive and time-consuming requirements of DPI.

[0176] GVN provides the opportunity to distribute these firewalls across multiple points in the cloud. Furthermore, it does not impede traffic flow for various types of firewalls that need to operate sequentially.

[0177] By positioning the FWSPI15446 at 15M3, the nearest edge of Internet 15302, via remote EIP15310, it is possible to defend against large volumes of attack traffic originating from known source IP addresses or with identified malicious headers. Traffic flows from SRV_AP15302 to FWSPI15446 via 15T10 and returns via 15T12. FWSPI15446 can be a CFW load balancer with high resource demands (see [link to relevant documentation]). Figure 14 The SRV_AP at 15113 could be a multi-homed backbone with enormous capacity. Therefore, attacks can be captured at the first perimeter, thus protecting the bandwidth in GVN.

[0178] At the next perimeter 15M2, the FWDPI 15444 can have all traffic flow through or just receive traffic copies from the SRV_AP 15300 via 15T20 and can or can not return traffic via 15T22. The point is that the DPI feature can be a trailing edge indicator that allows certain traffic to pass but analyzes and records the results. This FWDPI 15444 can also be a CFW that load balances as needed with resources provided as needed to cope with large scale events when needed without requiring each client to have to deal with or bear the cost burden for maintaining the infrastructure during normal periods.

[0179] Information from the FWSPI 15446 and the FWDPI 15444 are shared with each other via internal communication paths 15P6 which can be transported by the GVN's NAPM or through the GVN tunnel or through the GVN return tunnel or via other communication pathways. Each of the FW mechanisms also share information with the GVN's central control server (SRV_CNTRL) 15200. This information can be relayed to other FWSPIs and FWDPIs around the world so that databases can provide attack vectors, sources, payloads and other related information so that the SPI and DPI checks can have reference points for comparison. This enables scale efficiency to be improved as the global distribution of information provides an additional safety net.

[0180] Capturing malicious traffic outside of the client LAN and in the cloud can protect the last mile internet connectivity of the client from saturation by unwanted traffic. Offloading traffic to a scalable CFW also provides numerous advantages to the client.

[0181] The local FW 15442 can be a standalone device, a software application (APP) running inside the EPD 15100 or other type of FW device.

[0182] The FffSPI 15446 and FWDPI 15444 devices and related devices such as load balancers, cloud firewalls or other devices can be custom or off the shelf from other vendors so that the best combination is selected for the client. These devices must be able to receive and forward traffic, identify threats and most importantly be able to communicate threat findings and receive threat profiles and other information from other devices.

[0183] As threat data accumulates, analysis can be performed on the content, patterns, attack vectors and other information collected by the FW. This analysis can provide the basis for applying heuristics analysis on new potential threats.

[0184] This can be implemented by the GVN's Security Network Optimization (SN0) service alone or by a similar network of related devices connected by both secure tunnels and communication pathways.

[0185] Figure 16 A logical view of the software architecture of three types of network devices working together as part of a global virtual network (GVN) is shown. As shown, the software and hardware can be distributed within the network devices and can be distributed across different circuit boards, processors, network interface cards, memory and storage devices.

[0186] One of the network devices is an endpoint device (EPD) 100. Another of the network devices is a central server (SRV CNTRL) 200 and a third device is an access point server (SRV AP) device 300.

[0187] The EPD 100 is connected to the SRV AP 300 via an encrypted tunnel described as a communication path, which can be via an encrypted tunnel SYSC04 to a point of presence (POP) SYS 406, through a communication path SYS 06 to a WAN SYS 400 to a communication path SYSCPlO to a POP SYS 402 to a communication path SYSCPl 2. The path through the WAN SYS 400 can also be through a conventional unencrypted Internet.

[0188] Each of the devices, EPD 100 and SRV AP 300, can also be connected to the SRV CNTRL device 200 via a communication path SYSCP08.

[0189] The software architecture of the EPD 100 and the SRV AP 300 are very similar to each other, with the difference being that each device has a different role in operation and some of the modules are different.

[0190] The lowest level of each device is the memory (RAM) 106, 206, 306 and the processor (CPU) 102, 202, 302 and the network interface (NIC) 108, 208, 308. All of these are on the hardware level. The operating system (0 / S) 110, 210, 310 can be a LINUX system or an equivalent such as Debian or other system. This operating system describes the data packets and configurations for routing, hosting, communications and other system level operational software.

[0191] Above the operating system 110, 210, 310 is the system software level 112, 212, 312 of the global virtual network (GVN) operating system. Custom commands, system modules, managers and other components operate here, along with other components of the GVN. Each type of device in the GVN can have some or all or different parts of the system software level, depending on their role.

[0192] The database modules Db 120, 220, 320 and hosting modules 122, 222 and 322 are configured in the present example embodiment for GVN neutral API mechanism (NAPM), listening, sending, processing, storing, retrieving and other related infrastructure level operations for Graphical User Interface (GUI) and other server side script hosting sites. The database 120, 220, 320 modules can be MySQL or equivalent such as MariaDb and the hosting modules 122, 222 and 322 can be Apache and PHP scripts or other type hosting languages. Command line scripts are also used and can be written in Bash, C, PHP, Pearl, Python or other languages.

[0193] The billing modules can cooperate and share information billed through the consumption model, such as data volume of tunnel traffic consumption. The accounting module ACC 132, 232, 332 operates on the EPD 100 and the SRV_AP 300 has a corresponding billing module. Both modules can provide financial information to a reporting screen, provide payment forms, reports sent by email and other financial data generated by the GVN.

[0194] The SRV_CNTRL 200 has a repository manager 238 that handles billing information, tunnel manager information and other data that can be employed by various devices in the GVN. The repository manager 238 also handles coordination of peer information, credentials and other information with independent devices connected to other API peers through the GVN's neutral API mechanism (NAPM) peering.

[0195] The EPD 100 has an API module 130, the SRV_CNTRL has an API module 230 and the SRV_AP 300 has an API module 330. For simplicity of explaining the present example embodiment, only one API module is described for each device. In practice, depending on the device's function in the GVN, the device can act in a combined client and server role.

[0196] The cache manager on the SRV_CNTRL 200 manages the master index of multiple chained caches distributed across many devices in the GVN. The compression engine 136 on the EPD 100 and the compression engine 336 on the SRV_AP 300 manage compression and decompression of data stored on files, in DB tables or for streaming data.

[0197] The advanced smart routing (ASR) 150 module on the EPD 100 handles traffic routing from the EPD 100 to the destination best egress point via the GVN.

[0198] Remote retriever B0T 311 on SRV_AP 300 is a core component of the Geo_Destination mechanism (Geo_D).

[0199] DNS manager 254 on SRV_CNTRL 200 manages a master DNS index that can seed DNS servers on various GVN devices, such as DNS 154 on EPD 100.

[0200] Log manager on SRV_CNTRL 200 manages local logging and logging shared by devices to a repository via API calls. The log manager in the present example embodiment is given the function of recording operational events, API behavior, and transactions, and the logger also has other roles and processes for various aspects of GVN operations.

[0201] Local cache 152 on EPD 100 and local cache 352 on SRV_AP 300 locally cache data.

[0202] GVN manager 272 operates on SRV_CNTRL 200 to control the operation of various components of the system on SRV_CNTRL 200 and other devices of the GVN.

[0203] Local DNS server and cache 154 on EPD 100 and cache 354 on SRV_AP 300 allow for caching DNS lookups for fast local retrieval. DNS 154 and 354 can be fully flushed, purge individual items, or set a timeout for deleting retrieved lookups after a certain time.

[0204] A content delivery agent (CDA) 158 is provided on EPD 100, which is a component of Geo-D. A content pulling agent (CPA) 358 is provided on SRV_AP 300, which is also a component of Geo_D. CPA 358 works with B0T 311 on SRV 300 to pull content from remote regions using local DNS 354 seeded from the region. CPA 358 sends the scraped content to CDA 158 using tunneling, caching, and other improved functions of the GVN

[0205] Firewalls (FWs) (not shown) on EPD 100, on SRV_CNTRL 200, and on SRV_AP 300 operate to protect access to the devices and the communication paths between the devices and others.

[0206] The connectivity managers (not shown) on the EPD 100 and on the SRV_AP 300 manage the tunnels and other device-to-device communication paths between devices. The compression manager on 215 of the SRV_CNTRL 200 manages local compression and also cooperates with the compression engine 136 on the EPD 100, the compression engine 336 of the SRV_AP 300, and compression engines on other devices of the GVN. The routing on the EPD cooperates with the ASR 150, Geo-D, and other elements to manage traffic routing.

[0207] The structure of the database tables in the SDB 100, SDB 200, and SDB 300 is equivalent for device operation, while the data of each database table is specific to the device type, and each device has an identification specific to the device. On the SRV_CNTRL 200, the repository database SDB 202 is used to store unique information for all devices, and the repository management library 238 can use this information to communicate API credentials, tunnel information, or other information to the devices.

[0208] Each device stores information about the device itself and the identification and API peer information of the device's peers partners, transaction lists and queue data, and other information. The methods and databases have other uses besides those described, but for simplicity of explanation, this example covers only a few exemplary core functional elements.

[0209] Topology

[0210] Figure 17 A GVN using a hub and spoke topology with a backbone segment and octagonal routing is shown. Figure 17 The network topology of the GVN in two different regions 17-RGN-A and 17-RGN-B and how the regions are connected via global connection 17-RGN-ALL through paths 17-POA and 17-POB is shown. In addition, Figure 17 The hub and spoke connections in each of the two regions is shown. Figure 17 With Figure 15 Multiple entry and exit points (EIPs) are added in each region, similar and in the form of additional spokes of the hub and spoke model.

[0211] SRV_BBX 17-280 and SRV_BBX 17-282 are backbone switch servers and provide global connectivity. SRV_BBXs can be one or more load balanced servers used as global links in a certain region. Access point servers (SRV_AP) 17-302, 17-304, and 17-306 in 17-17-RGN-A connect to SRV_BBX 17-280. Central control server (SRV_CNTRL) 17-200 serves all devices in the region, and it can be one or more multi-master SRV_CNTRL servers. End point devices (EPD) 17-100 through 17-110 will connect to one or more SRV_AP servers through one or more parallel tunnels.

[0212] This figure also shows multiple entry and exit points (EIP) 17-EIP420, 17-EIP400, 17-EIP430, and 17-EIP410 in each region as additional spokes of the hub and spoke model, which have paths to and from the open internet. This topology can provide EPD connectivity to EIPs in remote regions through the GVN. In an alternative, this topology also supports EHN connectivity to EIPs in the same region, to EPDs in the same region, or to EPDs in remote regions. These connections are optimized through GVN security.

[0213] Figure 18 The backbone connections between some GVN global nodes in North America, Europe, and Asia and their corresponding service areas are shown. As Figure 18 As described in the legend box in the lower right, each region pointed out here from a networking perspective is described as a global node. Global nodes are connected to each other via high performance network links. The lower the latency time between points, the faster the information transfer.

[0214] The two rings around the global nodes represent the quality of connectivity area types, for example, within a radius from the center where the source information is located. This is for simple illustration purposes only, as the size and shape of these areas are determined by many factors. However, these two areas can be distinguished from each other in that the closest area is a high performance area, while the other area is a best service area.

[0215] The farther away a query client or server or other type of device is from a global node, the longer it takes for information to flow, and at some point the QoS drops so much due to the distance that the device is no longer in a high performance area, but now in a best service area.

[0216] If the QoS drops below a certain threshold, then the device is outside of the best service area, and therefore the distance between the device and the global node is too great for the advantages provided by the GVN, except for security, to be certain.

[0217] Figure 18 District SJC18-01 in San Jose, California, USA, District JFK18-02 in New York City, New York, USA, District AMS18-11 in Amsterdam, Netherlands, District NRT18_21 in Tokyo, Japan, and District HKG18-22 in Hong Kong Special Administrative Region, China are shown. Many other locations around the world require the placement of important global nodes, but for simplicity of illustration, only a few locations are shown for illustrative purposes.

[0218] Figure 18 Representative paths between various global nodes are also shown, such as between JFK18-02 and AMS18-11. In reality, many paths exist between the two points, representing undersea cables.

[0219] Figure 19 Connectivity between various devices within the GVN is shown, with multiple connection paths from devices in the spokes to the hub device being indicated. The placement of SRV_BBX (backbone switching server) 19-800 and 19-810 points is based on client locations with respect to optimal Internet Data Center (IDC) interconnections to the pipe for serving the target region, while connecting to global locations via paths 19-BB2 and 19-BB6.

[0220] SRV_BBX serves as the hub for the regions it serves. The hubs are connected to each other by tunnels over the top of the Ethernet links in the Internet (OTT), tunnels over direct Ethernet links, fiber over unlimited bandwidth, Ethernet over unlimited bandwidth, or other forms of connectivity between regions. Each hub serves multiple SRV_AP servers, such as 19-302, 19-306, 19-308, which serve one region within the global region. 19-312, 19-316, and 19-318 can serve another region of the global region.

[0221] Endpoint devices (EPDs) such as 19-100 through 19-128 will connect to the most appropriate SRV_AP server with respect to their location, network connectivity, peering, and other relevant factors. These factors change constantly, and thus multiple tunnels to multiple SRV_AP servers are always maintained by the Ero. Each Ero is simultaneously connected to various (one or more) SRV_AP servers.

[0222] Exit points (EIPs) are provided at the EPD, at the SRV_AP, and other locations where traffic can exit the GVN into the Internet or enter the GVN from the Internet, and the GVN protects and optimizes traffic as far as possible.

[0223] SRV_AP devices such as SRV_AP 19-308 and SRV_AP 19-318 are also connected to each other through tunnel paths such as 19P60 so that two EPDs such as EPD 19-110 can connect to EPD 19-128 via paths 19P22 to 19P60 to 19P58.

[0224] Central control server (SRV_CNTRL) 19-200 links to multiple devices, for example, to SRV_AP 19-302 via path 19P62 for neutral API mechanism (NAPIM) information exchange. EPDs also connect to SRV_CNTRL 19-200 via NAPM paths. To keep the example embodiment relatively simple, the NAPM to SRV_CNTRL paths are not shown.

[0225] The NAPM information exchanged between the SRV_CNTRL and the various devices can be used to share usage statistics, tunnel establishment information such as IP addresses, ports, protocols, security credentials, certificates, keys, and to share other information to enable automatic and secure operation of the GVN.

[0226] Figure 20 The manner in which the GVN modules and devices interact is shown. The global virtual network (GVN) is made up of various devices that operate independently as well as in cooperation with other devices. Although each role can be different based on their type and underlying function, they follow similar code base, database schema, and other architectural elements.

[0227] The infrastructure is installed in a region to support the operation of EPDs and PEPDs. Devices such as endpoint devices (EPDs) 100, portable endpoint devices (PEPDs), and endpoint hubs (EPHs) connect various LANs, PANs, and other networks to the GVN via tunnels to access point servers (SRV_AP) 300. Each device has its own locally hosted database.

[0228] Redundancy is provided by having multiple primary SRV_CNTRLs and other server types in each region with multiple servers of each type. The central data repository is located on the central control server (SRV_CNTRL) 200. The job of the SRV_CNTRL is to connect to the various devices via the neutral API mechanism of the GVN. API calls via the NAPM API of the GVN of the EPHs allow for device_10 and registration / region mapping in the Db repository on the SRV_CNTRL for communication between devices, for example, EPD 100 to SRV_BC 20-502 communication, allow for API peer relationship management, generate appropriate server availability lists (SALs), and accept logging. This allows for efficient management of relationships and connections to SRV_APs and GW servers.

[0229] GVN's back-end servers and infrastructure devices include a back-channel server (SRV_BC) 20-502; a secure boot server (SRV_SB) 20-504; an authentication, authorization, accounting server (SRV_AAA) 20-508 and a logging server (SRV_LOG) 20-516, among others.

[0230] Gateway servers and other devices are connected to the SRV_CNTRL 200 via connector 20AD0 and to gateway devices via the "all devices" hub 20AD2. This can include a gateway email server (SRV_GW_EMAIL) 20-510, a gateway server for financial transactions (SRV_GW_FIN) 20-518 and / or a gateway server for third-party connections (SRV_GW_TPC) as a class of other SRV_GW_* 20-512.

[0231] A gateway server that is specially tasked can be adjusted and protected in ways that are specific to that function. By authorizing an email gateway server, it can be set up as a secure email sender and receiver. This will require configuration and maintenance and observation of its operation. At the same time, however, no other servers are required to handle email, freeing the management burden of those devices. All devices can forward email via data payloads sent to the API by the action of requesting that an email be sent. Flags in the payload can indicate whether the email is to be sent immediately or at a specific time, or should be sent with what priority. Other settings can govern how it is sent. The SRV_GW_EMAIL will receive these data payloads, add them to its email sending queue, and the email manager will handle the time and manner of delivering the email and will log the event accordingly. Bounces, replies and other incoming email can also be handled by a point server type SRV_GW_EMAIL.

[0232] Logging servers and other devices can also be accessed by GVN devices via 20AD4.

[0233] Figure 21Additional details are shown regarding the manner of interaction between GVN modules and devices. These additional details include communication paths, such as from SRV_BC4-502 to 2100 of 200, for reporting of information from the back channel server to the central control server. The point is that while the GVN devices will need information about themselves, their peers, their connectivity options, and other information to operate, sharing performance and other data to the SRV_CNTRL 200 and / or other devices allows the larger system to be understood in its entirety. Constant feedback loops allow for automatic adjustment and learning in the course of transmission, to make better decisions.

[0234] Figure 22 The topology and connectivity of GVN modules and devices are shown, and how they interact with other devices on the Internet. Figure 22 The communication paths shown include external paths (PE), tunnel paths (for traffic) (PT), control paths (CP), encryption system paths (ES), and API communication paths (PA) between GVN devices, among others.

[0235] The central server (SRV_CNTRL) 200 includes file repositories and databases that hold important system information. The SRV_CNTRL is able to connect with all GVN devices via the PA path for API communication. The endpoint device (EPD) 100 is a network access point between a local area network (LAN) and the Internet, via various parallel potential communication paths.

[0236] The advanced smart routing (ASR) within the EPD can send local traffic via path 22-PE00 to a point of presence (POP) 22-020 to 22-PE02 to the closest Internet 22-010. The back channel server (SRV_BC) 22-502 connects to the EPD 100 via a back channel connection from 22ES04 through 22-010 via 22ES02 to 201 to 22ES020 into the EPD 100. The path is an encrypted control path and is independent of the tunnel path for transmission traffic.

[0237] The EPD 100 maintains multiple tunnels to each of multiple access point servers (SRV_AP), namely via 22PT00 and 22PT02 to SRV_AP 300, via 22PT04 and 22PT08 to SRV_AP 22-302, via 22PT10 and 22PT12 to SRV_AP 22-306, and via 22PT14 and 22PT16 to SRV_AP 22-308.

[0238] The figure is not drawn to scale, but for example SRV_AP 22-302 and SRV_AP 300 are in the same region and exit the GVN into the Internet 22-012 via path 22PE04 to POP 22-022 to 22PE08 to Internet 22-012 and path 22PE16 via POP 22-026 to 22PE12 to Internet 22-012. TM Both can do a local DNS lookup to Domain Name Service (DNS) server 22-402.

[0239] Both SRV_AP 22-302 and SRV_AP 300 maintain API communication paths to SRV_CNTRL 200 via 22PA02 and 22PA08 respectively.

[0240] Gateway device (SRV_GW) 22-514 is in the same region as SRV_AP 22-302 and SRV_AP 300. This can send email, handle financial transactions and other functionality of the GVN's SRV_GW device.

[0241] SRV_AP 22-306 connects to SRV_CNTRL 200 via 22PA10 and the egress point in its region to the Internet 22-014 is via 22PE20 to POP 22-024 to 22PE22 to Internet 22-014.

[0242] SRV_GW server 22-516 connects to SRV_CNTRL 200 via 22PA24 and to the Internet 22-014 via 22PE26 to POP 22-024 to 22PE22 to Internet 22-014.

[0243] SRV_AP 22-304 connects to SRV_CNTRL 200 via 22PA18 and the egress point in its region to the Internet 22-016 is via 22PE26 to POP 22-028 to 22PA30 to Internet 22-016.

[0244] SRV GW 22-512 connects to SRV_CNTRL via 22PA14 and to SRV_AP via 22PA16. Local traffic from SRV_GW 22-516 exits via 22PE28 to POP 22-208 to 22PA30 to Internet 22-016.

[0245] There are other devices within the GVN and they assume specific roles such as backup server SRV_Backup 22-522 and logging server SRV_Logging 22-516. These are connected to SRV_CNTRL via 22PA20 and 22PA22 respectively. They can accept data relayed from SRV_CNTRL 200 or from other devices via PA## paths to SRV_Backup 522 or SRV_Logging 22-516.

[0246] The described topology of the GVN allows traffic from EPD 100 to have multiple options of per-region traffic through multiple tunnels to multiple SRV_AP servers. Other devices ensure that information is distributed to the various devices for efficient utilization.

[0247] Figure 23 Multiple tunnel connectivity between endpoint device (EPD) 100, 23_102, 23_158 and access point servers (SRV_AP) 300, 302 is shown. These tunnels can be used for client data traffic, internal system data or other transmissions. This figure further illustrates the connection of global virtual network (GVN) infrastructure devices such as central server (SRV_CNTRL) 200 and back channel management server (SRV_BC) 23-502 to other devices in the GVN.

[0248] SRV_BC 23-502 establishes and maintains tunnels to EPD 100 23PA02, to EPD 102 23P018, to EPD 23_158 23PA06, to SRV_AP 23-302 23TP50, and so on. There can be more SRV_BC servers within the GVN in order to provide redundancy in case one SRV_BC does not operate, and also to ensure optimal performance by placing SRV_BC servers in strategic locations close to the devices they connect to.

[0249] EPD 100 connects one LAN 23-002 to various paths that data taken through the GVN such as to SRV_AP 300 via one of three multiple tunnels 23TP00, 23TP02 or 23TP04, to an exit point to the internet 23-410 via path 23PE00.

[0250] Another path is from SRV_AP 300 to SRV_AP 23-302 via one of three multiple tunnels 23TP10, 23TP12 or 23TP14.

[0251] The path option from SRV_AP 23-302 is via 23-382 to the internet 23-412 exit point.

[0252] The external entry point X-IP 305 into the GVN from the Internet 23-412 allows connection by non-GVN devices to address and access the devices through the GVN to enhance the GVN during the duration of traffic passage by the GVN.

[0253] Another benefit achieved by the GVN is to provide secure tunnel connections with EPDs 23-158 at the locations of service provider partner organizations in the cloud to enable secure tunnels to their servers and related services at the locations of LANs 23-152 via the GVN.

[0254] LAN-WAN-LAN bridges from LAN 23-002 to LAN 23-012 can be via the communication path from 23-002 to 23CP02 to GWD 23-004 to 23CP04 to EPD 100 to 23TP002 23TP0223TP04 to SRV_AP 300 to 23TP1023TP1223TP14 to SRV_AP 23-302 to 23TP2023TP2223TP24 to EPD 23-102 to 23CP14 to GWD 23-014 to 23CP12 to LAN 23-012. All traffic transmitted by this bridge is protected and improved by the GVN mechanisms.

[0255] Multiple tunnels between two devices such as 23TP002 23TP0223TP04 or 23TP1023TP1223TP14 or 23TP2023TP2223Tp24 can provide a single communication path by sending traffic along one tunnel, or two or more tunnels can be aggregated together where two or more bound tunnels can transmit traffic as if it were one tunnel.

[0256] SRV_CNTRL 200 with API communication paths between pairs of peers and tunnels to other devices can be used for file transfers and data exchange via paths such as 23PA00 to EPD 100 or 23TP30 to 23-302 to 23TP22 to EPD 23-102 or 23PA04 to 23-302 to 23TP60 to EPD 23-158 and other potential options.

[0257] There are other possible communication paths in this example embodiment and there are more options for communication paths through the GVN. In this example embodiment, all tunnels represent links via layer three of the GVN, which are each built on the GVN layer one over the Internet.

[0258] Figure 24is a simplified example diagram of how the Internet works today, taking into account hop counts or time to live (TTL) and paths taken due to peering relationships and associated routing policies.

[0259] A0 represents a network of an Internet Service Provider (ISP). A1 through A06 represent points of presence (POPs), and these POPs are further connected to switch equipment or client devices in order to link them to the Internet. This hub and spoke structure shows a network cluster within a wider ISP network. Lines with a circle in the form of a line cap indicate this connectivity. For simplicity, in this example embodiment, the structure of A1, A2, A3, and other POPs do not show links of the last mile network, but these links should be implied. Each POP has its own hub and spoke connectivity to the network, such as a local area network (LAN) or an Internet data center (IDC) that enables Internet connectivity via the POP.

[0260] H0 is an example of a single-homed ISP, indicating that it relies on one path between it and the Internet. If this path is severed or fails, then connectivity from this ISP to the wider Internet is severed.

[0261] B0 is an example of a multi-homed ISP that shows five connections between it and other ISP networks, meaning that traffic can flow through the Internet even if one path is unavailable, but is done so through a less direct path.

[0262] 1X1 and 1X2 are examples of Internet exchanges (IXs), which can be linked independently of each other through backbones or backbone- dedicated connections. IXs are where ISPs and other ISPs can connect to each other in a “meet-me room” or equivalent arrangement for direct network-to-network peering connections.

[0263] There are also communication paths between the networks of ISPs and the networks of other ISPs, or there are IXs or intermediate routers between them. These backbone communication paths are shown by lines with arrow caps on both ends. Intermediate equipment is shown by circles between lines with arrow caps. Backhaul connectivity between IXs is shown by dashed lines with arrow caps on both ends. A pagination connector IBH1 is used to show an international backhaul (IBH), that is, 1X2 also has connectivity to another IX that is not shown in this example embodiment.

[0264] To show a direct and efficient connection between ISPs, there are only four intermediate hops from A0 to G0 via the path AX1-1 -> AX1-2 -> IX1 -> GX1-1 and should be the most efficient route.

[0265] To show the detour path due to path failure, if path GX1-1 fails, then traffic from H0 or A0 destined for G0 will not be able to go via 1X1 through GX1-1. The alternative is for the traffic to go via B0 and E0 to G0. What used to be 4 intermediate hops from A0 via AX1-1-» AX1-2-» IX1-» GX1-1 now requires more hops AX1-1 to AX1-2 to IX1 to BX1-4 to BX1-3 to BX1-2 to BX1-1 to B0 to EB-5 to EB-4 to EB-3 to EB-2 to EB-1 to E0 to GE-3 to GE-2 to Ge-1 to G0. If GX101 fails, then traffic from A0 to G0 now requires 17 intermediate hops and corresponding higher latency times.

[0266] At the same time, traffic from G0 to IX1 that should have gone through the single intermediate hop of GX1-1 will have to go from G0 to E0 to B0 and then to IX1.

[0267] This extra traffic can exhaust the connections and can cause higher latency times and congestion related packet loss. The IX peering will typically have much more capacity and ability to handle large volumes of traffic. When the single intermediate hop GX1-1 from G0 to IX1 is not available, the extra hops (TTL) and round trip times (RTT) through the alternative routing can result in too many hops or too much time, which in turn can result in packets being marked as undeliverable or internet based services timing out.

[0268] The best connectivity between the two ISP networks via IX and through the backhaul implementation is represented by the path H2 to H0 to HX1-1 to HX1-2 to IX1 to X1X2-1 to X1X2-2 to IX-2 to DX2-2 to DX2-1 to D0 to D2. This is a total of 12 hops from POP to POP.

[0269] The next direct path should be via B0, a total of 16 hops. The path is H2 to H0 to HX1-1 to HX1-2 to IX1 to BX1-4 to BX1-3 to BX1-2 to BX1-1 to B0 to DB-4 to DB-3 to DB-2 to DB-1 to D0 to D2.

[0270] The next direct path would be via A0 via C0, a total of 19 hops. The path is H2 to H0 to HX1-1 to HX1-2 to IH to AH-2 to AH-1 to A0 to AC-1 to AC-2 to AC-3 to AC-4 to AC-5 to C0 to CD-1 to CD-2 to CD-3 to D0 to D2.

[0271] Due to routing policies and peering relationships, an indirect but possible path can be 30 hops, for example via G9 via E0 via B0 via F0. The path is H2 to H0 to HX1-1 to HX1-2 to 1X1 to GX1-1 to G0 to GE-1 to GE-2 to GE-3 to E0 to EB-1 to EB-2 to EB-3 to EB-4 to EB-5 to B0 to FB-5 to FB-4 to FB-3 to FB-2 to FB-1 to R) to DF-5 to DF-4 to DF-3 to DF-2 to DF-1 to D0 to D2.

[0272] A loop occurs when traffic cannot reach a destination because of bad or incorrect routing policies governing the intermediate devices between the origin and the destination. For example, if traffic from C0 expects to route to G0, then C0 will choose to go to B0 thinking that B0 will send traffic to E0 because C0 can think that B0 and E0 are close to each other and this is the best path. However, B0 can not be directly peered with E0 but has a strong peering relationship with F0. F0 also does not have a peering relationship or a path to E0 and so it can send traffic to D (D0 only has two choices to send traffic to C0 or to B0, in both cases the end result is that traffic loops, is not deliverable. There are other reasons for this looping, such as routing table failures, devices compromised, intrusion and other misbehaviors, or other reasons.

[0273] The end result of too many hops and too much latency is timeouts or packets being dropped.

[0274] Figure 25 The strategic positioning of the infrastructure to enhance performance is shown. There are three or four key points within this example where the strategic positioning of the SRV_AP servers and other GVN infrastructure will ensure the best peering and performance between all points on the example network topology shown.

[0275] The SRV_AP servers installed and operating at IX1_IDC, B5 and IX2-IDC and possibly at D5 to include optional routing options and failover will provide peering with all other networks and stable paths between the SRV_APs by providing the option to route around any compromised paths. This strategic positioning provides the flexibility and possibility to implement other performance enhancements.

[0276] Figure 26 It is shown how the GVN can incorporate technologies such as Network Slingshot to seamlessly achieve many advantages across distances. Network Slingshot is further described in US provisional patent 62 / 266,060.

[0277] The first boundary is the GVN EIP 26-322 between the Internet and the GVN. The next boundary is the security perimeter 26-182. This layered security approach protects the core infrastructure on which the GVN is built.

[0278] The security perimeter 26-182 between the GVN and the GVN backbone protects the high-speed global network. The GVN portion above the perimeter 26-822 has traffic that flows over the open Internet on top of the tunnel (OTT) via a secure GVN tunnel. Below the security perimeter 26-182, GVN connections employ various protocols over dark fiber or other connections that are not directly reachable from the Internet.

[0279] Supercomputer nodes 26-538 can operate inside (below) the security perimeter 26-832, which can operate with advanced features such as a real internal network with remote direct memory access (RDMA) to parallel file system (PFS) 26-602 devices.

[0280] Figure 27 How tables on the databases of various GVN devices relate to each other and how they interact is shown. For example, the repository database DB_2300 on SRV_CNTRL has various tables on it about devices and their interactions via the GVN's neutral API mechanism (NAPIM) between devices. Tables in the database DB_2300 such as the device registry DBT_2310 are designated as REP0_ACTIVE, which means that the table receives information from many sources, is read / written to, and can be queried as a source of information for selectively or completely replicating the table such as the device identification DBT_2102 as part of a database EH) local Db DB_2100. This table DBT_2101 has the designation SEL_REP+W, which allows selective replication from DBT_2310 and allows relevant identifications to be reported back to the device registry.

[0281] Control and release of information is governed by the data manager. Database table type designators include normal read / write tables "regular" (REGULAR), read-only replication tables REP_INFO, read-only partial replication tables SEL_REP with only relevant rows, and merged tables REP0S_ACTIVE from all sources on a repository such as the device registry DBT_2310. Other possibilities include "logging" (LOGGING) from source tables to be merged on the database DB_2800 on SRV_LOGS. These designations of tables are for example purposes only and can differ in real use and there are more tables and other types based on use.

[0282] Figure 28The collaboration between the various modules, mechanisms, techniques and other components of the GVN is shown.

[0283] There are 3 layers in the GVN, layer 1 is the physical network layer on top of which the GVN is built (OTT), for example the Internet. Layer 3 is the GVN network layer that is seen by the client devices as part of or complete path to the destination. Layer 2 is the logical layer in between.

[0284] There are components that interact with the physical conditions 28-00. The dynamic construction module at 28-20 works to maintain the connectivity of the GVN. The joint action described herein links the relevant modules of the GVN to the physical 28-00 and dynamic 28-20 elements. For example, in order for the Advanced Smart Routing (ASR) module G106 to function properly, multiple Access Point Servers (SRV_AP) GP106 must be placed in multiple locations with routing and peering GR106. In order for the EPD to be able to select the most appropriate SRV_AP to establish a connection with, information is needed about which SRV_AP is best. The ASR Server Availability module SA106 ranks the servers for this particular EPD based on information provided by the ASR Test Manager TM106 and when the EPD needs to establish a new tunnel, it employs the server availability list SA106 to establish the new tunnel. Subsequently, tests are run on the tunnel via TM106.

[0285] As another example, in order to operate the NAPIMG102, both the host client and the host server need an API listener and processor HL102AAPIM on the host server and the operations manager OM102 runs on both to handle the preparation of API requests and responses, then sending, handling, processing. The dynamic construction of the NAPIMG requires a peer manager PM102, a related NAP action manager AM102 and transactions at the physical TP102 and dynamic TM102.

[0286] Construction

[0287] Figure 29 The Advanced Smart Routing (ASR) feature of the GVN is shown. Specifically, the diagram shows the Advanced Smart Routing (ASR) feature of the GVN within an End Point Device (EPD) 103 to multiple exit points in multiple regions of the world.

[0288] Traffic in this example embodiment starts in LAN A 102 from a connected device such as host client 101. The target traffic zones shown in this example embodiment are: 1) local traffic stays local via POP 401 where GVN tunneling will not necessarily improve performance; 2) local traffic is carried in encrypted tunnel TUN1 to the Internet 203; 3) traffic to another zone goes via TUN2 to SRV_AP 301 in that zone to access the Internet 303; and 4) traffic goes via TUN3 to other remote zones where there is some ASR on SRV_AP 501.

[0289] The DNS cache 103-4 within EPD 103 does DNS lookups from the DNS servers at each target zone, including DNS 404 for the Internet 402, DNS 204 for the Internet 203, and DNS 304 for the Internet 303, and DNS 504 for the Internet 503. The internal DNS cache 103-4 is accessible via path DP4.

[0290] The physical network interface controller (NIC) hardware devices of EPD 103 include four ports. ETH0 103-9 is a WAN port that connects EPD 103 to the Internet via a network access point (NAP) to POP 401 to ISP P401 to the Internet 402. All traffic from ETH0 goes through this connection as the first layer of the GVN network. The TUN tunnels above this connection are the third layer of the GVN. ETH1 103-1 is a local area network (LAN) port that connects to LAN A 102 via path P102. ETH2 103-2 is another physical LAN port that connects to LAN B 104 via path P104. Finally, there is a virtual interface (VIF) that acts as a bridge BR0 103-3 for communicating LAN interfaces 103-1 and 103-2 via internal paths DPI and DP2, respectively.

[0291] Traffic from LAN bridge BR0 103-3 is sent via device path DP3 to a chain of virtual interface (VIFs). Advanced smart routing (ASR) is applied at each VIF, with a routing table of IP addresses to direct traffic flows from each VIF to one of two or more egress points. The last VIF can have only one possible egress point for "all others" remaining traffic.

[0292] For example, at VIF 0103-5, local traffic exits via P401. All other traffic through VIF 0103-5 is sent via DP5 to the next VIF in the chain, VIF 1103-6. Traffic from VIF 1103-6, destined for the Internet 203, exits EPD 103 via path P201, through encrypted tunnel TUN1 to SRV_AP 201, then to path P202 to POP 202 to P203 to the Internet 203. From this location, a regional DNS lookup can be queried via path P204 through SRV_DNS 204. Host client 205 or host server 206 can be connected to from this location via P205 and P206, respectively.

[0293] Any remaining traffic from VIF 1103-6 is sent via path DP6 to VIF 2103_7. Based on the routing table applied to VIF 2103-7, all traffic destined for the Internet 303 as well as connected devices at this location, such as host server 306, exits VIF 2 via path P301 to TUN2 to SRV_AP 301, and continues through the Internet 303 and to other places beyond the Internet.

[0294] Any further remaining traffic from VIF 2103-7 will be sent to VIF 3103-8. All traffic from VIF 3103-8 is sent via encrypted tunnel TUN3 to SRV_AP 501. At SRV_AP 501, ASR routing is applied such that traffic destined for IP addresses within the Internet 503 is sent via path P502 to POP 502 to the Internet 503.

[0295] Traffic from SRV_AP 501, destined for the Internet 603, is sent via connected encrypted tunnel TUN4 to SRV_AP 601 to path P602 to POP 602 to P603 to the Internet 603, and to other places beyond the Internet.

[0296] A DNS lookup in the region of the Internet 603 can be made to SRV_DNS 604, and devices at this location can be connected to, such as via P605 to host server 605 or other devices.

[0297] This ASR mechanism can be used at various traffic junctions in order to optimally send traffic to the best egress point traffic on the Internet located in multiple target regions, to achieve geo-destination traffic, and to obtain other advantages realized by GVN.

[0298] Figure 30A series of encrypted tunnels are shown being established between a client (C) and a server (S). Steps 30-0 through 30-18 show a series of simplified communications between C and S.

[0299] The first step is to open a connection from C to S 30-0. The next step is for S to accept the connection handshake 30-2. If the handshake data is malformed or does not match the expected format, the process can stop here.

[0300] After receiving and accepting the handshake 30-4, C provides a certificate to S for S to use along with the required security information to establish a Secure Socket Layer (SSL) connection 30-8 between the two. The certificate received from C is compared to the corresponding certificate key on S. If the certificate is expired or incorrect, then the SSL connection cannot be established and the process stops.

[0301] This connection will be used to send information about the tunnel from C to S 30-10, including a pass phrase, a metric, and other information about the tunnel metric, including which IP address and port each device will use for tunnel traffic, among other information.

[0302] S will validate this information against its own version of the tunnel metric and pass phrase and other information 30-12. If the information is not accurate, the process stops at this step.

[0303] Upon successful validation, S will send a response back to C so that C can begin initiating or building the process of the tunnel with the configuration settings provided 30-14.

[0304] After the tunnel is established, routing can be applied at C or S or both 30-16. While the tunnel has been established, during the process of adding routing to it, traffic can not flow through the tunnel, or even if it can, there is a risk of data leakage. This risk occurs because, before all routing is applied, traffic destined for a target IP address can leave the default egress path to the internet without being encrypted or traveling through the tunnel. After routing has been added to the tunnel, subsequent traffic will be protected because it will be transmitted through the tunnel. Depending on the size of the routing table to be applied to the tunnel, this delay can be a significant amount of time.

[0305] When all routing has been applied to the tunnel, the tunnel can be used to push traffic through it 30-18.

[0306] Figure 31The information flow required for two peers in a peer pair is shown. The peers can be a client (C) to a server (S), or one peer to another in a P-2-P topology. To simplify the labeling and description in this example embodiment, C to S and P-2-P represent two peer relationships of the same type, and the C to S relationship is described herein. GVN primarily uses the C to S relationship between devices, but the methods and techniques can also be applied to P-2-P peer pairs for tunnel construction.

[0307] An encrypted tunnel is a secure communication path over which data can flow. When a client and server are separated by some distance and the connection between them is over an open unencrypted Internet, an encrypted tunnel is the ideal conduit for securely exchanging data. If there is a human network administrator at either end, they can program the devices. However, there are challenges regarding how to relay secure information such as passphrases, keys, and other information. Some can be coordinated using voice telephone, some can be shared using a series of posts through a secure website, or other methods can be used. There can be a need to perform the task of manually setting up individual tunnels. Managing multiple tunnels can become cumbersome.

[0308] To automatically construct a series of encrypted tunnels between two devices in a peer pair, information needs to be securely shared. The tunnel information also needs to be current and securely stored on the devices. In addition, there are threats that must be addressed during the setup process. While the tunnels are established, there are other threats that will need to be addressed.

[0309] SRV CNTRL 31D00 is the central server that includes repositories that manage information in database tables, files stored in a secure file storage system, lists located in storage, and other related information. SRV CNTRL also has algorithms and mechanisms to evaluate certain data to generate information reports.

[0310] Client device 31D02 represents a device that will initiate tunnel construction by "dialing" a connection to a server device via a specific IP address and port. Many client 32D02 devices can employ similar software and configuration to connect to the GVN simultaneously, the distinguishing factor between devices being a unique device identification UUID, and unique information per client per channel.

[0311] Server device 31D06 represents the device that will listen for client connection attempts on a specific IP address and port. If the client follows the correct protocol and setup sequence, and provides the correct credentials and other security information, the server will allow the client to build a tunnel to the server. Many server 31D06 devices can employ similar software and configuration to connect to the GVN simultaneously, with the distinguishing factor being the unique device identification UUID and unique information.

[0312] Tunnel information 31S2 shows the information stored on the client device 31D02 and server device 31D06. Each device can establish multiple tunnels, and each tunnel will have its own set of tunnel information and security information. Some sets of tunnel information can be used to build a currently active tunnel, and other sets of tunnel information can be saved in a repository for use in future tunnels.

[0313] Some information between the C and the S will be identical, such as one will present to the other a password phrase, other information will vary depending on availability. The information requirements to build a tunnel between two points can include: client / server topology and settings; IP and port of each endpoint that the tunnel will use; tunnel metrics, including MTU size, protocol, and other information for its operation; keys, passphrases, and other information about security protection for tunnel use; SSL certificates and other information for protecting the exchange of information before tunnel setup; and other information. This information is shared between devices using specific API action calls of the neutral API of the GVN.

[0314] Pre-tunnel 31S0 describes the process of receiving and sharing information between the devices 31D02 31D06 and the repository 31D00 on SRV_CNTRL, and returning it to the devices 31D02 31D06. API communication paths API-31CP0, API-31CP2, API-31CP4, and API-31CP6 represent request-response information exchanges, and the arrows represent the direction of information flow from one device to another.

[0315] Server 31D06 reports information to the Receive Info 31C-0 module of the SRV_CNTRL 31D00 device via path API-31CP0. The SRV_CNTRL 31D00 receives the information from the server and stores the relevant identification, tunnel, current load and other information in its repository. For example, algorithms and AI logic on the SRV_CNTRL 31D00 analyze the server load and, based on current and anticipated demand from the client 31D02 device, update the server availability C-l matrix. Server availability C-l information can be transmitted by the Share Info 31C-6 module copying the database to the client 31D02 via API call path API-31CP6 through the GVN's API; via direct file sharing through the GVN; or other methods.

[0316] Client 31D02 reports information to the Receive Info 31C-0 module of the SRV_CNTRL 31D00 device via path API-31CP2. This information will be stored in the SRV_CNTRL 31D00's repository. Specific tunnel information from the client 31D02 can be shared with the server 31D04 via path API-31CP4 by the Share Info 31C-6 module.

[0317] SRV_CNTRL 31D00 compiles a current client 31C-4 list per server, which is published to the server 31D06 via the Share Info 31C-6 module via path API-31CP4.

[0318] If the client 31D02 or server 31D06 detects a problem with establishing a tunnel with current tunnel information, one device or the other can request that a new set of tunnel information be generated by the SRV_CNTRL via API-31CP2 or API-31CP0, respectively. The new set of tunnel information can be shared with both peers in the peer pair via the Share Info 31C-6, with client 31D02 information sent via API-31CP4 and server D02 information sent via API-31CP6.

[0319] The client 31C-4 list and the current status of the server 31D06 will directly affect the server availability 31C-2.

[0320] Each server 31D06 needs to curate, protect and coordinate its client 31C-4 list, which will attempt to establish new tunnels for the server 31D06's shared resources. This information will be fluid and needs to be updated periodically via secure API calls to the SRV_CNTRL 31D00.

[0321] The need to securely coordinate information between devices is necessary to protect the integrity of the tunnels between them.

[0322] Tunnel construction 31S4 describes the process of tunnel establishment via the sharing of information 31C-6. Reference is made to Figure 30 for an understanding of the steps taken to construct a tunnel between a client and a server. Path 31TP0 represents the path between the client 31D02 and the information exchange 31C-10 and from the information exchange 31C-10 to the server 31D06 via path 31TP2.

[0323] Establishing a threat 31C-8 refers to a threat to the information exchange 31C-10 during tunnel establishment. If the signature of the tunnel type is visible, then there can be a threat 31CC-8 during tunnel establishment such as a fake transport layer security (TLS) handshake from an intermediate illegal operator, TLS errors at the handshake, ports and IP identities that cause blocking or hindering, timeouts caused by filtering devices, reset packets sent by intermediate ISPs or firewalls or devices or other threats.

[0324] If the information exchange 31C-10 is successful, then the step of constructing the tunnel 31C-12 is performed in which the application of the routing and other related operations is made to enable the secure construction of the tunnel TUN between the client 31D02 and the server 31D06.

[0325] Tunnel establishment 31S6 describes the stages during normal traffic flow through the tunnel. Information must be communicated between the devices and SRV_CONTROL 300 is needed to manage the unique information of the various client 31D02 and server 31D06 devices and the unique information of the multiple tunnels constructed between them.

[0326] Information exchange between devices must occur on a regular basis, as often new dynamic tunnels need to be formed. Some ports on IP addresses can be blocked or become blocked, and as long as the port of that IP address changes, it will allow the tunnel to be built and data to flow. In addition, each tunnel needs one or more unique ports per IP address in order to avoid conflicts between tunnels. When a client 31D02 device requests creation of new tunnel information, a random port number is generated, and the port availability on the target server 31D06 for that particular IP address is checked against two or more of the following factors: whether the port is already in use by an existing tunnel (an operational port or a standby port that can be brought into an operational state); and whether the port has been used in the past by the particular client 31D02 / server 31D06 peer pair and has been blocked. In both cases, a new random number is generated. There are 65,536 available ports per IP address, of which a certain number are reserved for specific services. A lower limit value of, for example, 5,500 leaves 60,036 available ports, which can be used by a random number generator with a minimum of 5001 and a maximum of 65536. When a tunnel is torn down and the port is marked as blocked for a certain peer pair, it is available for use by other peer pairs. This port release is necessary in order to avoid port exhaustion. Therefore, tracking by the SRV CNTRL 31D00 of IP and port combinations is necessary.

[0327] Tunnels can help their own establishment by steps, but this also has limitations. Most tunnels, while secure, are visible during establishment. The handshake and signature about the type of the tunnel are visible during operation. Manually setting keys is cumbersome and does not change often, and if used for too long, there is a risk that they can be compromised; therefore, keys should be replaced often with new keys.

[0328] An automated system needs to ensure that information such as new keys, ports of IP addresses, and other information can be created and used by both sides of a peer pair to enable the building and rebuilding of tunnels. Both sides must be configured and ready to be able to establish tunnels. Therefore, the exchange of information between peer pairs needs to be secure, otherwise the overall security of the tunnels themselves will be compromised.

[0329] While the tunnel is established and push traffic, there is an operational threat 31C-14. The tunnel signature can be visible (e.g., if the tunnel is sniffable without being obfuscated). If the tunnel type can be discovered, then the tunnel structure is known. This creates the risk that the packet stream is captured and the tunnel content is decrypted using brute force key cracking. If the reset code or other tunnel control code is known, then the reset signal can disrupt the tunnel. Thus, to maintain tunnel security and integrity between the client 31D02 and server 31D06 devices in a peer pair, updates and sharing of information need to be done automatically and securely.

[0330] The GVN structure enables devices to automatically and securely establish tunnels between peer pairs based on recent information. The combination of security features and methods provide self-hardening protection.

[0331] Figures 32-35 The third tier of GVN neutrality and security with respect to GVN tunnels is shown, comparing the number of hops to the underlying Internet connection. The term LAN is used in these figures intentionally and can represent a home or office or Internet Data Center (IDC) network. The devices can be clients or servers connected to the LAN. Figure 32 GVN tunnel from LAN to EPD to SRV_AP to Internet is shown. Figure 33 GVN tunnel from LAN to EH) to SRV_AP to Internet to LAN is shown. Figure 34 GVN tunnel from LAN to EPD to SRV_AP to SRV_AP to EH) to LAN is shown. Figure 35 GVN tunnel from LAN to EH) to SRV_AP to SRV_AP to EH) to LAN is shown. Figure 34 GVN tunnel from LAN to EH) to SRV_AP to SRV_AP to EH) to LAN is shown.

[0332] All four figures include common baseline elements from EH1 to EH17, which represent the external hops of the underlying Internet connection. The distances between each hop are not to scale and do not represent anything other than the number of hops. Other common elements include a local area network LAN1 with a gateway device GWD1 at one end and another LAN2 with a GWD2 at the other end. Each variation of the example embodiment also has a GVN endpoint device EPD-1 connected to an access point server AP-1. There is a tunnel between these devices and each device NH1 and NH2 within the third tier of the GVN has a neutral hop.

[0333] Figure 32GVN tunnel from LAN to EPD to SRV_AP to Internet. The tunnel can also function in the other direction, providing ingress access from Internet to GVN tunnel back to LAN. Between AP-1 and Internet is a point of presence POP-1. Between Internet and GWD2 is another POP-2, which represents the network access point (NAP) for the connection of this LAN.

[0334] Figure 33 GVN tunnel from LAN to EPD to SRV_AP to EPD to LAN. This variant shows an end-to-end GVN tunnel between the edges of two LANs via one SRV_AP. This variant is similar to Figure 32 The difference between and is that the tunnel extends through the entire transport from EH3 through Internet to H115. A second EPD-2 is shown.

[0335] Between EPD-1 and AP-1 is one tunnel. This is coupled to a second tunnel between AP-1 and EPD-2. Within the third tier of the GVN are three neutral hops represented by NH1, NH2, and NH3, as compared to 13 hops on the underlying Internet between H13 and H115.

[0336] Thus, the total hop count from LAN1 to LAN2 is a minimum of seven hops from LAN1 to GWD1 to NH1 to NH2 to NH3 to GWD2 to LAN2. The end-to-end count includes the two internal hops at both ends from EH1 to EH17, and totals a minimum of 17 hops.

[0337] Figure 34 GVN tunnel from LAN to EPD to SRV_AP to SRV_AP to EPD to LAN. This variant shows an end-to-end GVN tunnel between the edges of two LANs via two (or possibly more) SRV_APs. This variant is similar to Figure 33 The difference between and is that a second AP-2 is inserted into the path to represent another coupling of tunnel AP-1 to AP-2 and tunnel AP-2 to EPD-2. Adding another internal neutral hop brings the hop count within the third tier of the GVN to eight.

[0338] Figure 35 GVN tunnel from LAN to EPD to SRV_AP to SRV_AP to EPD to LAN. This variant shows an end-to-end GVN tunnel between the edges of two LANs via two (or possibly more) SRV_APs. This variant is similar to Figure 34the additional elements of the GVN tunnel from LAN to EPD to SRV_AP to SRV_AP to Era to LAN, which includes a peer that peers the point between the ISP and the network edge. This variant shows an end-to-end GVN tunnel between the edges of two LANs via two SRV_APs, and further shows more information about different Internet Service Providers (ISPs) carrying traffic through certain parts of the Internet between EH-3 and EH-15.

[0339] The difference between this variant and Figure 34 is that additional elements have been indicated. As shown in Figure 9 the following elements have been covered in this variant of the example embodiment: a) EDGE-1 is the demarcation point of the network access connection between the device of LAN-1 and the POP of ISP-1; b) PP-01 is the point where peering occurs between the networks of ISP-1 and ISP-2; c) PP-02 is the point where peering occurs between the networks of ISP-2 and ISP-3; and d) EDGE-2 is the demarcation point of the network access connection between the device of LAN-2 and the POP of ISP-3.

[0340] Certain advantages can be realized by placing SRV_AP_1 at PP-1 so that this SRV_AP can directly peer with both ISP-1 and ISP-2. Further advantages can be realized by placing SRV_AP-2 at PP-2 so that this SRV_AP can directly peer with both ISP-2 and ISP-3. If the network of ISP-2 is not ideal, then the GVN can alternatively route traffic around ISP-2 through another route or line or ISP or carrier.

[0341] The hop count through the neutral third layer of the GVN remains as eight as in Figure 34 . The distances between the ISPs are not to scale. Also, there can be more hops within the networks of the ISPs, but the numbers shown have been simplified for simplicity.

[0342] While Figure 33 , Figure 34 and Figure 35 all show the joining of the tunnels at the AP hops, this is considered a single tunnel for the client devices within LAN1 and LAN2. This single tunnel represents the neutral third layer of the GVN, within which all traffic that would normally be transmitted over the Internet can be run, including TCP, UDP, and other protocols, in addition to other tunnels such as IPSeC, OpenVPN, PPTP, and so on. The third layer of the GVN also realizes other advantages. Some advantages include lower TTL and the ability to have more control over routing, in addition to other advantages.

[0343] Figure 35 Various network structures are shown incorporated together in a network blanket framework. This example embodiment shows various network structures incorporated together at the physical layer, with the global virtual network (GVN) operating over-the-top (OTT) of the physical layer. These structures at the physical layer 36102 constitute a series of network segments, which can be, for example, IPv4 and IPv6 aware, or just one or the other protocol. End point devices (EPD) 36100 to LAN (36000) can be IPv4 and / or IPv6. Tunnel TUN 36P2 can be one or the other protocol or both between EPD 36100 and access point server (SRV_AP) 36300.

[0344] Egress / ingress point (EIP) 36302 indicates the egress and ingress points from the GVN to the network structures at the internet level. Path 36P04 indicates a connection to an IPv4 internet network 36400, and path 36P06 indicates a connection to an IPv6 internet network 36600.

[0345] The key point is that the blanket framework of the GVN allows end-to-end links of structures such as IPv4 internet 36408 to IPv4 in LAN 3600 or end-to-end IPv6 36608 from internet 36600 to LAN 36000, even though there can be some different segments at the physical level 36102.

[0346] Figure 37 Communication pathways in the GVN for automation device collaboration are shown. This example embodiment shows communication pathways such as P37202-C used by the neutral API mechanism (NAPIM) API 37202 37206 37208 to enable automated interactions between various devices working together to constitute a global virtual network (GVN), such as P37202-C.

[0347] Key operational aspects can be automated to facilitate fast system response. These include infrastructure operations, heartbeat routines, connections, testing and diagnostics, and other functions.

[0348] Infrastructure operations such as there are predictably updating device operating system software and data packets from reliable sources, maintaining GVN modules and databases, and other operations. For example, end point device (EPD) 100 can query central control server (SRV_CNTRL) 200 via API 37202 along path P37202-B to 37202-C. In another example, email gateway server (SRV_GW_Email) 37310 can update system data packets from SRV_CNTRL 200 as a reliable source of trusted system software.

[0349] Other items running via daemons or other repetitive periodic operations such as heartbeat functions include keeping service initiation, operation, and health by reporting from devices such as access point servers (SRV_AP) 300 via API 37202 through paths P37202-A to P37202-C to SRV_CNTRL 200. There are also redundant paths such as via API 37208 through paths P37208-A to P37208-C. Other heartbeat functions can keep queues running and clear queues, can replicate log data, and can perform other such operations.

[0350] For connections such as tunnel P37206-C between EPD 100 and SRV_AP 300, the two ends of the tunnel, the listener at SRV_AP 300 and initiator EPD 100 require related information. This information can include peer pair information related to each device or related to the tunnel. Both are communicated to SRV_CNTRL 200 via separate paths via API 37202.

[0351] With multiple tunnels being hung off of virtual interfaces and the option of more than one tunnel between devices such as between EPD 100 to SRV_AP 200 or SRV_AP 200 to SRV_AP 20x, various different API calls are required to manage multiple tunnels, routes, and other information.

[0352] Power server availability algorithms rely on system analysis of various information to provide EPDs with a list of SRV_AP servers they can connect to via tunnels. Since each tunnel requires IP addresses and ports at either end of the mapping to GVN constructs for routing clarity, information that is changing needs to be updated. Automated device coordination helps with this.

[0353] A key component for information sharing is for test and diagnostic data from layer 1 physical networks, from GVN construct layer 3, and from logic at GVN layer 2. This connection information provides more information for analysis to SRV_CNTRL 200. Replication of this data can also be sent to a logging server via API 37208 or other communication paths. Analysis results can also be stored on the logging server.

[0354] APIs can also be used to update information about each peer in a pair itself such as peer pair credentials, IDs, and other information, queues on each peer, transaction logs for mediation, updates through internal security audits, and for updating or adding or deprecating action functions of the API mechanisms themselves.

[0355] System and resource monitoring and reporting are equally crucial for automatically communicating information about service startup and operation, host functionality, database engine startup and operation, security system operation, and more.

[0356] Figure 38 This example illustrates the problems and challenges of establishing dynamic tunnels. It uses the transfer of files, database structures, and other data from GVN repository 38R-00 to device 38D-00 to demonstrate these issues. In most cases, repository 38R-00 will reside on GVN's central server (SRV_CNTRL). Device 38D-00 can be an endpoint device (EPD), access point server (SRV_AP), gateway server (SRV_GW_XX), or other GVN devices.

[0357] Depending on the device type, a newly created device can load a copy of the master disk to be configured during the initial boot, or, as in the case of a remote server, the first boot script will be securely transferred to the server to run and pull the basic system files. Other possible scenarios may combine pre-loaded files with files to be loaded remotely.

[0358] When the first boot script runs, most of the current database structure is copied from repository 38R-00 (from DB structure repository 38R-06-A) to Db38D-04 on device 38D-00. Data populating this database is sent from DB data repository 38R-06-B to identification information module 38S-00 via 38P06. Some data passed through 38S-00 can be filtered and modified to incorporate identification information (such as Device_ID and other UUID elements) and other data passed through in a direct copy without modification.

[0359] Based on the device type and the device's Universally Unique Identifier (UUID), data applicable to device 38D-00 is sent to database 38D-04 via path 38P16. Some information may also be populated into a template configuration file, which can be cloned to the software and configuration file storage 38D-02 on device 38D-00. Unique identification information for the device may include: device attributes, naming and UUID information, credentials / keys, key regulator, and other information.

[0360] Setup files for system data packets and other modules will be cloned from the setup file store 38R-02-B on the repository 38R-00 and sent via path 38P02 to the software and configuration file memory 38D-02 on the device 38D-00. Some "factory default settings" and other files can also be copied via path 38P10 to the secure file memory 38D06 on the device 38D-0. The secure file memory 38D-06 is managed by the file and folder manager of the GVN. Files from 38D-06 can also be cloned to 38D-02 via 38P12 if needed, such as in the case where a return to factory settings is necessary.

[0361] Code base files 39R-02-A from the repository 39R-00 can be copied via path 38P00 to the software and configuration file memory 38D-02, and can also be copied via path 38P8 to the secure file memory 38D-06.

[0362] The above shows the loading of files and data from the repositories to the devices during first boot, updates, periodic data exchanges, and other operations.

[0363] Figure 39 This figure shows two LANs 39-000 and 39-010 bridged into a wide area network (WAN) via two or more EHs. More specifically, this figure shows two LANs 39-000 and 39-010 bridged into a wide area network (WAN) via two EHs 39-020 and 39-030. Each EPD first connects to an access point server SRV_AP 39-200 via a base station tunnel established over its internet connection.

[0364] From EPD 39-100, base station connectivity path 0TT is via path 39-P002 to point of presence (POP) 39-002 to internet 39-004 to POP 39-006 of SRV_AP 39-200. From EPD 39-110, base station connectivity path 0TT is via path 39-P012 to point of presence (POP) 39-012 to internet 39-014 to POP 39-016 of SRV_AP 39-200.

[0365] Transmission path 39-P06 from POP 39-006 to POP 39-016 can be a path through the internet, by way of the SRV_AP and relying on routing over public networks. If EPD 39-100 wants to connect to 39-102 via the internet, it can follow a different route based on policies that cannot control the GVN or EHs.

[0366] EPD 39-100 establishes a tunnel TUN39-P100 between itself and SRV_AP 39-200. EPD 39-102 also establishes a tunnel TUN39-P12 between itself and SRV_AP 39-200. One or both such tunnels can or can not be encrypted or protected. There can also be another tunnel, an internal tunnel INTTUN39-P20, which passes through both other tunnels, joining at SRV_AP 39-200 where traffic can flow. This tunnel can be the communication path that establishes the WAN.

[0367] The tunnels and base station connectivity connectivity can use different network protocols. The network blanket framework provided by the GVN can be a mix of different network protocols mapped to a range of various network segments, while the GVN can be end-to-end of one network type within the internal tunnels.

[0368] Figure 40 A multi-perimeter firewall mechanism (MPFWM) running on a GVN is shown. This example shows how a second level 40TOP88 can exist over the elements of a global virtual network (GVN) (OTT). At a first level OTT 40TOP86, the GVN 40-86 operates OTT, base station internet connectivity 40-82. In the case of a multi-perimeter firewall mechanism 40-88 construct, the GVN operates OTT and from this can be constructed as a second level 40TOP88 over the top elements.

[0369] Figure 41 A GVN stack built over the top of the internet (OTT) is shown. This example describes a GVN 41-800 stack built over the internet 41-000. The figure illustrates connectivity between an EPD 100 and two SRV_AP servers 300 and 302 via tunnels TUN41-100-300 and TUN41-100-302. Such two tunnels are an example of multiple tunnel options between an EH) and best current access point server (SRV_AP) based on server availability and other factors such as destination, traffic type, QoS of various network segments between the origination point and the destination, and others.

[0370] The blanket framework 41-500 weaves together various network protocols of independent network segments and end-to-end protocols that can be "through" the GVN path.

[0371] The cluster GVN device 41-600 represents the physical layer of routing between GVN devices.

[0372] GVN Global Network 0TT Internet+ via other links 41-700 is a GVN layer 2 logic where modules such as geo destination, DNS management, advanced smart routing (ASR) / global ASR (GASR), server availability, tunnel management and builder modules, etc. operate.

[0373] GVN 41-800 represents the network as seen by the client user.

[0374] Figure 42 Compare the Internet Protocol IP stack B2, OSI model C2, and GVN stack C3.

[0375] The IP stack is composed of network interface T1, Internet T2, transport T3, and application T4.

[0376] For non-GVN traffic and for physical tunnels that are not visible to the client that flow out through ETHNICN 1, the IP stack as seen by the client follows elements R1 at the network interface T1 layer, R2A at the Internet T2 layer, R3A or R3B at the transport T3 layer, and R4A, R4B, or R4C at the application T4 layer.

[0377] For traffic through GVN tunnels and networks, the client will observe R4C at the network interface T1 layer, R5 at the Internet T2 layer, R6A or R6B at the transport T3 layer, and R7A, R7B, or R7C at the application T4 layer for its GVN traffic.

[0378] While the OSI model can be used by the client for IP traffic through tunnels, GVN has its own stack of network interface G1, Internet G2, transport G3, GVN routing and logic G4, GVN Internet G5, GVN transport G6, and application G7.

[0379] Logic

[0380] Figure 43 The global Internet flow between countries via numerous possible routes is shown. Traffic on the global Internet flows between countries via numerous possible routes that transmit different interconnections between peers.

[0381] The Internet within a region such as countries in Asia is mostly connected to each other by terrestrial and submerged ocean links. Often they link at third or other countries in the middle of the traffic transmission from one country to another.

[0382] 43-X01 represents the most direct route from Asia to Europe. The latency time from Hong Kong to Paris via 43-X01, for example, will be between 180 ms and 250 ms depending on the route taken.

[0383] 43-X02 is an indirect longer path where traffic is pushed naturally over the Internet. Here traffic goes from Asia via link 43-P400 to the US West Coast 43-400 then via link 43P402 to the US East Coast 43-402 and then via link 43P600 to the landing point in Europe 43-600. The latency time via 43-X02 will be approximately 396ms to 550ms or more depending on the destination in Europe.

[0384] Before leaving the region, traffic can have to be relayed from one country to one or more other countries (etc.) before it can access the international backbone. Such additional in-region hops can add 50ms to 150ms or more to the RTT even before the traffic leaves the region.

[0385] Once in the destination region, traffic will land from the trans-Atlantic link 43P600 in one country, for example in the UK 43-600. From the UK 43-600, traffic will run via link 43-600 to France 43-602 and then via link 43P606 to Germany 43-606. Such additional in-region hops can add 30ms to more ms to the RTT depending on the destination.

[0386] International backhaul quality can also vary between peers, with each having various RTT QoS times. Routing and corresponding speeds on the normal Internet are determined middlemen participants, and these are in the majority of cases based on lowest cost that usually passes slow RTT speeds. High latency is not the only problem that lower quality networks have to contend with. These usually have higher levels of congestion and corresponding high packet loss. Loss and slow links significantly reduce performance.

[0387] Figure 44 Comparing the Internet Protocol, IP, stack, OSI model, and GVN network stack again. This example compares various conceptual network models such as the TCP / IP stack B2, the Open Systems Interconnection model (OSI) A2C2, also variations such as the TCP / IP model in GVN stack A3, and the model of GVN C3.

[0388] Two perspectives are presented. The client perspective Al compares A2 and A3 side by side. The global virtual model architecture Cl compares C2 with C3. There is also the tree-like connection layer of B2.

[0389] In TCP / IP model B2, there is a network interface Tl that corresponds to the Ethernet protocol Rl. The Internet T2 corresponds to the Internet Protocol (IP) R2. The transport T3 layer corresponds to the TCP protocol R3A and the UDP protocol R3B. Other protocols can exist and operate at this layer. Above this layer is the application layer T4, where there is the Hypertext Transfer Protocol HTTP R4A, the mail service POP3 R4B and the GVN application. Other applications such as File Transfer Protocol (FTP) or other services can exist in this layer.

[0390] To compare the TCP / IP model with the OSI model in the B2 range, the OSI data link S9 and physical link S8 are parallel to Tl. The OSI network S10 is parallel to T2. The OSI transport Sll is parallel to T3.

[0391] The OSI session S12, presentation S13 and application S14 layers are in the range of R4C, the GVN application.

[0392] The extension of the network tree to the top of R4C is established by the TCP / IP model through the GVN B3.

[0393] From the client perspective, the layers Tl, T2, T3, T4 combine into a single TCP / IP model layer T5, which becomes the network interface layer for the neutral third layer of the GVN. This is compared to the OSI model A2 physical S 1 and data link S2 layers.

[0394] Above R4C, there is a presentation of the Internet layer in the third layer. The Internet IP layer is at R5 and this corresponds to the A3 grade of the Internet T6 and the A2 network grade S3.

[0395] The TCP protocol R6A and the UDP protocol R6B and this grade correspond to the A3 grade transport 17 and the A2 grade transport S4. Other protocols can exist and operate at this layer.

[0396] The application layer from the client perspective T8 corresponds to Internet protocols such as FTP R7A, HTTP R7B and POP3. The OSI model splits the application layer T8 into three layers, the session S5, the presentation S6 and the application S7.

[0397] In the three-layer model of the GVN, Al describes the operation in the third layer while B 1, B2 describe the operation in the first layer. The GVN application at T4 R4C and the operation under Cl describe how the second layer can be used to allow the third layer to operate above the first layer.

[0398] There is a similarity between the network operation in the third layer and the first layer of the GVN.

[0399] Network connectivity N0 can be other network connectivity on periodic Internet N1 via WANN2, private circuit N3, MPLS line N4, or other links to the Internet.

[0400] Figure 45 Tunneling via GVN between two LANs is shown. In particular, this figure depicts an internal path from LAN 45-000 to LAN 45-002 through GVN path 45P00 to 45P10, which segments through internal tunnel 45L300. There are five hops 45H0 to 45H8 visible to the client in either direction between the two LANs. The path through 45L300 is the GVN layer visible to the client.

[0401] GVN1 level network layer 45L100 represents the physical network layer end-to-end for various different types of network segments. Although the number of hops is not indicated in this figure, and the network segments are at least equal and most likely greater than those network segments visible to the client in internal tunnel 45L300.

[0402] Logical layer 2 level logic 45L200 is the logic where various network segment integration, routing, and other GVN operations occur.

[0403] If the client path is IPv6 through a tunnel, for IPv4 segments only as in 45-104, internal IPv6 traffic can be packaged in this way so that it can remain inherently IPv6 end-to-end independent of the network type of network layer 45L100.

[0404] Figure 46 The network via paths P01 to P13 at the base station level is compared to the network through GVT01 to T03.

[0405] A number of measurements at the base station Internet level CTN140 are LAN to GVN via EPD 46-100 to SRV_AP 46-300, and connectivity metrics of bandwidth BW, latency time At = Ams, packet loss, and other factors are evaluated for this measurement. On the other end of the connection, similar measurements of BW, At = Cms, packet loss, and other factors measure the traffic rise from EPD 46-102 to GVN in CTN142. Through GVN between SRV_AP 46-300 and SRV_AP 46-302, various Internet segments CTN340 measure BW, At = Bms, packet loss, and other factors are evaluated for GVN across region 0 TT. The total path latency time through GVN layer three GVN4-3 can be calculated as the sum of latencies A + B + C, all in milliseconds.

[0406] At GVN Layer Three, GVN4-3, ASR and other features dictate how and where traffic flows through the GVN. This entails determining the best tunnels and traffic types to send traffic based on target regions, QoS and other factors for segments through the GVN.

[0407] At GVN Layer One, GVN4-1, the physical conditions of base station network connectivity are monitored and tested to determine the best routing options upon which to build GVN tunnels and paths therethrough. GVN paths can be transmitted through connected tunnels that pass through SRV APs, SRV BBXs and other GVN hardware devices. This also determines which tunnels to continue using and which to discard.

[0408] The mechanisms, modules and components of GVN Layer Two, GVN4-2, facilitate setting up, testing, managing and otherwise operating the pipelines between Layer Three, GVN4-3, and GVN Layer One, GVN4-1. Tunnel testing 46-310 can be done at the EPD 4100 and in Layer Three via its tunnel tester 46-312 at the SRV AP 46-300.

[0409] Figure 47 Elements of the Advanced Smart Routing (ASR) feature and the GVN’s geo-destination mechanism within the endpoint device (EPD) are shown. This includes using multiple DNS sources to send traffic via multiple paths to egress points in various regions of the world. The target traffic regions shown in this example embodiment are: 1) local traffic from VIF3 47-118 stays local to the Internet 47-004; 2) traffic to other regional Internet 47-002 will go from VIF1 47-112 through TUN1 102-6 to path 47P48 to SRV AP 47-300 and then via path 47P50 to Internet 47-002; 3) traffic for other regional Internet 47-006 will go from VIF2 47-116 through TUN2 102-8 to path 47P52 to SRV AP 47-302 and then via path 47P54 to Internet 47-006; and 4) traffic for other regional Internet 47-008 will go from VIF3 47-118 through TUN3 102-10 to path 47P56 to SRV AP 47-304 and then via path 47P62 to Internet 47-008.

[0410] The SRV AP 47-304 includes more detail to show some functionality of its components, AP logic 47-314 and content pull agent 47-318. In addition, the EPD 100 includes a more detailed flowchart to show its internal functional components.

[0411] Tunnels TUN1 102-6, TUN2 102-8, TUN3 102-10 and traffic flows through VJF with routing tables applied at each of virtual interfaces VIF1 47-112, VIF2 47-116, VIF3 47-118 operate in a similar manner to the virtual interfaces and tunnels.

[0412] DNS cache 47-114 is seeded to Internet 47-004 to SRV DNS 47-104 via path 47P38 via 47P34 from multiple DNS sources via local DNS query mechanism 47-110. Remote DNS query mechanism 47-108 can cause DNS requests to SRV DNS 47-114 via content pull agent (CPA) 47-318 via 47P44.

[0413] Geo-destination mechanism (Geo-D) pushes routing information to routing manager 47-104 via 47P04 connecting content delivery agent (CDA) 47-106 with CPA 47-318. Path 47P30 to 47P40 via JO1 is an abstraction representing coordination of the operation of CPA 47-318 with CDA 47-106. Communication between CPA & CDA is still via tunnels and or API calls, or via linked cache transfers, through tunnels, or can be via other mechanisms.

[0414] In this example embodiment, by Geo-D, CPA 47-318 pulls all regional content from Internet 47-008 via 47P62 to SRV AP 47-304 to pull content from host server 47-110 that hosts the destination content via 47P66 and in the content CPA 47-318 can find links for other content and CPA 47-318 will subsequently pull content streams from host server 47-108 via 47P64. Other content can be pulled from host server 47-112 via 47P68. Typically many websites host web pages on one server, video files stream from another server and graphics are served from another server.

[0415] Figure 48 An example is shown of multiple parallel traffic paths taken via GVN. Left side of EDGE-1 represents LAN side. Right side represents Internet facing side. Right side of EDGE-2 represents LAN side and left side represents Internet facing side.

[0416] Traffic from devices in LAN 001 causes EPD 101 to exit via P 002 through an encrypted tunnel P 003 to SRV_AP 102 and can flow out to the general Internet 106 to reach host client or server devices D 005 via path H 005. Traffic from devices in LAN 201 causes EPD 301 to exit via P 103 to SRV_AP 302 and can flow out to the Internet 106 via P 106 to reach host client or server devices D 005 via path H 005.

[0417] EPD 101 can link to EPD 301 via the Internet 106 through P 003 to SRV_AP 102 to P 006 to the Internet 106 to P 106 to SRV_AP 302 to P 103 to EPD 301. There is a secure tunnel between the EPDs via paths P 003 and P 103. To ensure full security, the path between the EPDs for an end-to-end secure tunnel is EPD 101 to P 005 to SRV_AP 103 to P 007 to WAN 107 to P 107 to SRV_AP 302 to P 105 to EPD 301.

[0418] EPD 101 can build a secure tunnel via P 003 to SRV_AP 102 and from there link to another secure tunnel via P 201 to WAN 103 to P 202 to SRV_AP 104 and then out in the remote area via path P 203 to the Internet 105 and via path H 002 to host client or server devices D 002.

[0419] EPD 301 can build a secure tunnel via P 103 to SRV_AP 302 and from there link to another secure tunnel via P 301 to WAN 303 to P 302 to SRV_AP 304 and then out in the remote area via path P 303 to the Internet 305 and via path H 004 to host client or server devices D 004.

[0420] EPD 101 is also able to reach devices in the Internet 305 via a secure tunnel between EPD 101 to SRV_ 102 to SRV_AP 302 to SRV_AP 304 and from there out to the Internet 305.

[0421] EPD 301 is also able to reach devices in the Internet 105 via a secure tunnel between EPD 301 to SRV_ 302 to SRV_AP 102 to SRV_AP 104 and from there out to the Internet 105.

[0422] There are numerous other options via end-to-end tunneling, tunnels to egress points on the open internet, tunnels via multiple SRV_AP devices, and other options.

[0423] The important point illustrated by this example is that client traffic carried by the GVN is passed through the GVN third layer from the client's perspective the same as a path through the internet and thus can carry any type of traffic through it, although the improved benefits and higher security provided by the GVN are still recognized.

[0424] For example, path P008 illustrates a WAN optimized connectivity between firewall GW002 device and firewall GW202 device to create a LAN-WAN-LAN bridge. The communication between the devices is carried within the third layer of the GVN and is transparent to GW002 and GW202.

[0425] For simplicity purposes, point of presence (POP) network access points are not shown in this figure. The path to device D002 to and from the internet such as internet 105 has a POP in H002.

[0426] WAN in this example embodiment represents a secure tunnel between GVN devices over the internet and thus any reference to WAN is in the third layer of the GVN where all GVN traffic is still transported in the first layer.

[0427] Figure 49 Automatic advanced intelligent routing (ASR) is described from a starting device 49-000 to an end point device 49-800. If a route is not available, the automatic advanced intelligent routing can build a route including but not limited to building a new tunnel and updating internal routing for an optimized path.

[0428] Tables 1 through 5 are used by this algorithm as data points to use for routing purposes such as determining the best egress point for traffic from the GVN through an access point server to the open internet. This data can also be used by the algorithm to help differentiate which route is more preferred over another route.

[0429] Table 1 lists various available paths from a starting point to a destination and includes a rating of the ranking of the path.

[0430] Table #1 - QoS of various routes through the GVN

[0431]

[0432] EPD [EIP] and SRV_AP2 [EIP] indicate the egress / ingress point (EIP) from a device to the internet or from the internet to the device. The double headed arrow symbol Indicates the routing path between two devices. This can be as a network segment over the Internet, as a tunnel or other mechanism (possibly as part of the GVN) directly through the Internet or via other network paths between devices. The starting point is on the left and the destination means the traffic will be routed to or from the last location of this route.

[0433] The rating is a calculated value for routing based on several factors. A rating of 0.00 means that it is not possible to route. A rating of 1.00 means the best route with the highest bandwidth at the wireline speed latency time. RTJD is the routing ID number that distinguishes one route from another for utility, testing and record purposes. This is used to determine the quality of various routes through the GVN. RTJD is the identifier of a particular route from the route list.

[0434] Table 2 describes the server availability matrix.

[0435] Table #2 - Server Availability Matrix

[0436]

[0437] The information maintained in the server availability matrix includes the serverJD, server IP_addressJD, port number, EPDJD field, parameter field (including security and configuration settings, status flags and time stamps).

[0438] PRI is the weighted priority order of the server to connect with the EPD. Priority 1 is the absolute lowest priority. 0 indicates that the server is currently unreachable. This is different from Flag_State which indicates whether the record is current or not. PRI can be maintained in the same table or in another related table since PRI is a continuously changing value and another table would allow for historical records and analysis.

[0439] Flag_State of 0 indicates that it is a backup entry. Flag_State of 1 indicates that it is active and it can be used. Flag_State of -1 indicates that it has been retired, not usable.

[0440] Table 3 shows the latency time of the complete path as well as the latency time of the constituent network segments.

[0441] Table #3 - Route -> Path Latency Time Evaluation

[0442]

[0443] The path from the LAN via the EH) to the GVN and or the internet or combination of various network segments has a total path latency, otherwise known as RTT, round trip time. The time is measured in milliseconds (ms) and is used for the ICMP ping from the starting point to the destination and its return to the starting point.

[0444] To evaluate the best route, it can be broken down into groups of network segments that make up the constituent parts of the total network path. The evaluation of each segment can provide information about the route and provide data points that can be used. The path rating will always give additional priority weighting to the traffic to transmit the GVN0TT versus the traffic to transmit the open internet.

[0445] The total path latency is the sum of the following latencies: LAN to EPD plus EPD to SRV_AP plus GVN transmission plus GVN egress to destination.

[0446] Table 4 lists the measured quality of the service attributes of the route.

[0447] Table #4 - Route -> Measured QoS Factors (Current and Historical)

[0448]

[0449] This table is kept as a log of the current and historical QoS (Quality of Service) results of the route between the source peer and another peer in another location and or region. It can be used in real time to make QoS expectation decisions based on real time conditions. This table is located on each starting point device and indicates the route performance.

[0450] Various factors are used to evaluate the line quality comparison. Such factors include system load (LOAD), security (SEC), round trip latency (RTT), packet loss (R-RELIABILITY), bandwidth (BW), hop count (EFF-EFFICIENCY), and other factors (array of values that can be used to evaluate line parameters).

[0451] A baseline for each point and the network segment in between is taken so that comparisons can be made between resources with different hardware configurations and network speeds, bandwidth, and other ratings.

[0452] L_ID indicates the row ID for the recorded route information.

[0453] RT_ID is the path id. The path can indicate a path through the base station internet, through a tunnel, a stitched tunnel, or other GVN related route.

[0454] Reg_ID is the target region ID.

[0455] RTT is the Round Trip Time or latency based on historical standards. A value of 1.0 is standard, while greater than 1.0 indicates lower than typical latency and less than 1.0 indicates greater than typical latency.

[0456] SEC is the Security Rating. A value of 1.0 is secure, and a value of 0.0 indicates completely insecure and a completely compromised resource. This is based on security testing, performance logging, and other data points. Any value less than 1.0 is of concern.

[0457] R is Reliability and relates to packet loss on the route. For example, R = 0.97 indicates 3% packet loss on the route. A value of R = 1.0 indicates 0% packet loss and 100% reliability. Ratings greater than one indicate parallel replication of packets sent along the route. R = 2.0 indicates 100% reliability for replicated packets sent.

[0458] EFF indicates the line efficiency in terms of hop count relative to the route length and is based on its historical average. An EFF value of 1.0 means standard hop count and less than 1 means greater than typical hop count. Values greater than one mean less than typical hop count.

[0459] BW (Bandwidth) is based on the line rating for the base station connection in conjunction with the full network segment between two points. A value of 1.0 for BW means 100% of the BW is available. A value of 0.5 means only 50% of the BW is available based on the route BW rating. And if the value is greater than one, such as 2.0, this means 200% of the BW capacity rating of the route is available and can be employed. For example, a rating of 0.55 for a 1 GigE base station link between two points indicates 550 Mbps is available. A rating of 2.0 would not be available for a 2 GigE, etc.

[0460] In the case of RT_ID = 1, a SEC (Security) value of 1.0 indicates it is 100% secure, and values greater than one of RTT = 1.1 and BW = 2.0 indicate the connectivity of the route RT_ID from one point to another has 10% lower latency and is twice the bandwidth of the average route between the points comparable baseline performance.

[0461] For example, where RT_ID = 5, a security rating of 0.80 indicates there is an ongoing security risk, and an associated available BW rating of 0.30 shows the server is under attack such as DDoS or Brute Force, where multiple security threats such as an onslaught of multiple parallel requests, the requests saturate the available BW (Bandwidth) while degrading the SEC (Security).

[0462] Flag State = 1 indicates the current active route. And Flag State = 0 indicates a historical route performance that is no longer used. The timestamp indicates the start time in UNIX timestamp (seconds since the epoch).

[0463] L_ID = 3 * L_ID = 5 indicates a comparison between the two different UNIX timestamps 1448674238 and 1448848558 from the start point to the region Reg_ID = 44. It shows that the subsequent performance has improved from the previous rating. The load = 0.9 is better than load = 0.7, and the underlying network connectivity has also improved.

[0464] This table can also be used to determine the better route of the two routes from the start point device to the target region by comparing the QoS factors of each route. For example, L_ID = 5 and 1_10 = 6 both indicate the current (Flag State = 1) route from the start point to Reg_ID = 44, although the routes of RT_ID = 5 and RT_ID = 9 are different. The better route of the two across this range is RT_ID = 9 and should be weighted with higher priority in the server availability list.

[0465] Table 5 evaluates and ranks the entry / exit points (EIPs) in the target region.

[0466] Table #5 - EIPs in Region

[0467]

[0468] The ATR field is the attribute field. This is an array of attributes to describe the EIP specifications (RAM, core, storage space, other factors, etc.). The S_ID field holds the server ID dPJD field holds the IP address ID. Bandwidth (BW) is measured in GigE. For example, 20 Mbps is 0.02, 100 Mbps is 0.1 and 1 GigE is 1, and 40 GigE is 40.

[0469] QoS (Quality of Service) indicates the current EIP (entry / exit point) suitability of the server to handle connections and traffic. QoS of 1.0 indicates an ideal state of the server with acceptable available BW (bandwidth) and little to no load (server's resource load, combination of RAM, CPU, NIC and other factors) and EH) connections,

[0470] QoS less than 1.0 means the server being employed. If QoS is close to zero, this means it is close to all useless due to capacity saturation. As a baseline and for system health, QoS less than 0.40 will indicate that servers will be prioritized with lower ratings so that servers with healthier QoS will be weighted to present higher on the list and thereby will attract connections and not overload any current servers.

[0471] This evaluation and rating mechanism can also be used as a determining factor as to how to support the build-out of the physical infrastructure.

[0472] Figure 50 A security perimeter 50-182 is shown between the BB / Spine layer below perimeter 50-832 and the IP / Internet layer above perimeter 50-822.

[0473] There are two layers of natural protection in place. The first layer of protection is that the only way to join the two layers together is via paths 50-TR6B22 and 50-TR6B32 and must pass through the security perimeter. Only valid GVN traffic can be transmitted in either direction via two logical checks. The other security protection in place is that the network types above and below the security perimeter 50-182 are different.

[0474] Figure 51 is a flowchart of the Advanced Smart Routing (ASR) within a Global Virtual Network (GVN).

[0475] From the host client 101 device's point of origin in a Local Area Network (LAN) 102 connected to an End Point Device (EPD) 103, the GVN provides a multitude of connection paths for the EPD to a multitude of potential end points. This is a high level illustration of the routing logic that the flowchart is a grouping can be thought of as it employs ASR to transmit the GVN for optimized performance. From the host client 101 point of view, its traffic will flow through an Internet Protocol (IP) network with a minimal number of hops and the best possible latency time due to the third layer of the GVN. The first layer of the GVN is a base station Internet with automatic configuration of constructs with virtual interfaces, tunnels, routing, and other network policies. The second layer of the GVN is

[0476] The algorithms, software, and logic govern the operation of the layer between layer three and layer one.

[0477] The first major routing decision is at logic gate 104 within EP0D, where traffic flows out to the local internet 107 (here Ero is located via path P104) or if it is to go via P107 through a secure wrapped and obfuscated tunnel to the access point server (SRV_AP) 110, then to the best connectivity to the region where the SRV_AP 110 is located. Before the traffic flows out of the SRV_AP 110, it passes through the routing logic gate 111. Traffic that flows locally to the internet 113 will go via path P111 to the host client 115 or host server 116 there. If the traffic is not local but is relayed to another region, then it will go via path P116 through tunnel 118 to the next SRV_AP 119.

[0478] At SRV_AP 119, three of the many possible routing options are shown by the paths that traffic can take. Logic gate 126 determines whether the traffic should remain and flow out to the local internet 129 or whether the traffic should go through a tunnel via P126 to a SRV_AP in another region 127. Another possibility is shown via path P119, which indicates a tunnel from SRV_AP 119 to another EPD 121 in a remote region. This is EPD 103 bridged to EPD 121 via multiple bridging tunnels.

[0479] A further possibility is that the traffic reaches a client device 125 126 in LAN 122, where EPD 121 is located by the EPD's connection P121.

[0480] Figure 52 is a flow chart of the various routes available from the starting point C 52-002 to the destination S 52-502 through the GVN. There can be more possible combinations not shown or discussed.

[0481] The path 52CP00 from client C 52-002 to EPD 52-108 can be used to measure the performance of a client through a LAN to an ETO. The matching of the best route is achieved after testing and evaluating real-time data of the available paths. The GVN enters the access point server (SRV_AP) 52-102, 52-104, 52-106, 52-202, 52-204 from the EPD via the first hop 52CP00.

[0482] The path from the EPD to the first SRV_AP can be defined as from the EPD to the entry point in the GVN and measured from there. The internal hops from SRV_AP to SRV_AP are along internal routes that always try to maintain the best path connectivity. These can be 0TT internet, over a backbone, over dark fiber, or other relevant routes.

[0483] The best egress point outside of the GVN also maintains a local trace in that remote region and also overall for the complete network segment from the origination point to the destination.

[0484] Tests can take into account various factors of evaluation running on individual segments, combinations of segments, and overall network paths from one end to the other. Traffic types and path determinations can be based on data attributes and profile QoS requirements. Primary path selection is always based on the best factors of traffic on the path. The function of this mechanism is to match paths between the destination and origination point to flow for the best possible bi-directional routing.

[0485] Table 6 is a list of IP addresses to be locally saved based on IP address, protocol (etc.) and port (etc.).

[0486] Table #6 - IP addresses to be locally saved

[0487]

[0488] This table saves which IP addresses are to be locally saved so that EIPs (egress / ingress points) are transmitted directly on the EPD or via SRV APs in the same region as the EPD. The

[0489] The LRIJD field keeps the local route IP address ID. Region value 0 indicates that IP addresses (etc.) to be locally saved should go from the EPD directly to the Internet from its local EIP. Region values 1 to 300 indicate countries and regions. Region values of higher region IDs represent finer granularity. The IP4J; also address field keeps the IPv4 address.

[0490] Under a column such as protocol or port, an asterisk ("*") means that a wildcard covers all possible values in the allowed range or in the allowed list of values. If one or more values are in a column and separated by a comma, then it indicates that more than one port, or protocol, or other column value can be used. Then only those values explicitly indicated will be affected by the table's provisions, other values not specified follow the default behavior.

[0491] Table 7 is a list of IP address ranges, their target geographic destination IDs, and the EPD IDs to which such rules apply.

[0492] Table #7 - IP address table to be routed via geographic destination

[0493]

[0494] The GDRegJD field holds the geographic destination ID. The region value of 0 indicates that the IP address (etc.) to be saved locally should go from the EPD to the Internet directly from its local EIP. Region values of 1 through 300 indicate countries and regions. Region values of higher region IDs represent finer granularity. The IP4_Start and IP4_End fields hold the start and end IPv4 addresses.

[0495] Table 8 is a reference of IP addresses for countries and other geographic regions employed by the geographic destination mechanism. Due to the large number of IP addresses employed, the CIDR notation is employed.

[0496] Table #8 - Reference of IP addresses per region

[0497]

[0498] This table defines the IP address ranges for the nationwide blocks or regional blocks according to the granularity employed for regional routing. Addresses for geographic destination routing are routed in order before the regional IP address table and are thus routed first.

[0499] The CIPBJD field holds the country IP address block IDX The IDXR4 column indicates the range of IPv4 addresses in CIDR4 notation. IDXR represents classless inter-domain routing, which is a notation describing a range of IP addresses. For example, the slash eight ( / 8) notation represents 16,780,000 IP address blocks. The slash twenty ( / 20) represents 4,096 IP addresses. The Total_IP4 column indicates the total number of IPv4 addresses covered by the CIDR4.

[0500] Figure 53 is a flowchart of an algorithm that controls the routing of traffic from a start point device to an end point device.

[0501] In the GVN, there are routing tables for the basic level of Internet between the GVN device and each of the other devices such as the EPD and SRV_AP over which tunnels can be built. The routing tables control the routing of level 1 (Internet level) traffic through the GVN at level 3. Sometimes, tunnels can not exist or if they do exist, they can not be optimal. The GVN routing can be mapped to existing and possible GVN routing according to the topology database. All information about the basic network segments and links between devices is stored in the GVN database.

[0502] The algorithm begins by identifying the destination region for a particular GVN traffic. Next, a check is made to see if the path exists through the GVN 5306. If the path does not exist, a new tunnel is built 5310. The next step is to check the health of the tunnel 5312. If it is not healthy, a new alternate tunnel will be built 5310. Once a healthy tunnel is available, the route health is checked 5320.

[0503] If a route exists to the EIP in the target zone 5322 and the route is checked to see if it is the best route for the traffic type. If it is the best route, then the route is used 5360.

[0504] If the route is not ideal for the data type, then a check is made to see if there is an alternative 5350. If there is an alternative, then the best route for the traffic type is taken 5352 and the best route is used 5360. As routes are used, a process evaluates the route performance 5365. Before the algorithm is complete, another process saves the performance data in a log about server availability, a list about EIPs 5322 and a log about mapped paths to target zones used by 5302 via P 5328.

[0505] If the test at 5350 determines that the route is not ideal for the data type and there is no alternative, then a new tunnel is constructed 5310 via path P 5314.

[0506] Control

[0507] Figure 54 The modules required for automatic device cooperation and information exchange in the GVN are shown.

[0508] EPD 100 is an endpoint device. SRV_AP 300 is an access point server located in a target destination zone. SRV_CNTRL 200 is a central server accessible by the EHP and SRV_AP and by other devices that can support graphical destination mechanisms.

[0509] Each device, EPD 100, SRV_AP 200 and SRV_CNTRL 300 stores information about itself in a local information store in the form of lists, files, database tables and records and other ways. This store also includes information about peer device relationships, stored log records and other related operational information. SRV_CNTRL 200 also has additional storage functions and its role is to provide information to other devices associated with it and / or to peer devices that can be connected to it in order to assess the current state and provide guidance similar to centralized control, such as publishing server availability lists and other functions. A neutral API mechanism (NAPM) can send information between devices and their connected peers and can also be used to update the API itself.

[0510] The database on SRV_CNTRL 200 serves as a repository for information about itself and as a centralized repository for other devices. There can be many different SRV_CNTRL 200 servers in many locations to act as a multi-master device. Each database can store specific information, including tunnel information, peer information, traffic information, cache information, and other information. Security and other aspects are managed independently by each device, including heartbeat functions, trigger scripts, and other mechanisms.

[0511] Figure 55

[0512] Figure 55 Communication between EPD 100, SRV_CNTRL 200, and SRV_AP 300 via the Neutral API Mechanism (NAPIM) of the GVN via paths API-55A1-55A2, API-55A3-55A2, and API-55A1-55A3 is shown.

[0513] Each device in a peer pair requires specific information for each tunnel that will be constructed between EPD 100 and SRV_AP 300, TUN55-1, TUN55-2, and TUN55-3, and for tunnels via TUN55-5 from EPD 100 to other SRV_AP servers such as TUN55-4 and from other EPDs to SRV_AP 300.

[0514] The NAPIM stores relevant credentials, coordinates, and other information for each side of a peer pair when a new tunnel is constructed via tunnel managers 55110 and 55310. Server availability mechanism 55222 on SRV_CNTRL 300 evaluates the performance of various tunnels, which are tested on the EH) side via tunnel tester 55112 and on the SRV_AP side by tunnel tester 55312. Information from the tests is relayed to connectivity analyzer 55288 on SRV_CNTRL 200. Test results include assigned IP address and port combinations, ports used, results from historical combination usage, results from port spectrum testing, and other relevant information.

[0515] The server availability list represents EPD 100 with a list of IP addresses and ports that can be used by the tunnel manager to construct new tunnels. The SRV_AP 300 and other SRV_AP servers mentioned on the list will be notified of the expectation 55320 and listen for connection attempts made by EPD 100.

[0516] Server availability prioritizes the list of SRV_APIP address and port combinations according to the expected best performance of the tunnel constructed, while also looking at the current load of available SRV_AP servers, balancing the allocation list given to other EPDs, and other available information.

[0517] Figure 56 Various types of communications available between GVN devices via the NAPIM are shown.

[0518] Closed loops are available as NAPIM REQ / RESP communications between known peer pairs and there are two main types; device to repository 56-P2C and device to device 56-P2P.

[0519] RESTful URL publication is open access to unknown peers (such as generic or generally non-sensitive information that can be shared) if allowed that particular action.

[0520] Each defined API action has flags controlling access via path type with possible values, another flag as to whether authentication is required, plus other controls. For example, an EPD 100 can request a list of available servers via 56REQ100200 and the corresponding IP address and port and receive the list from SRV_CNTRL 200 via response path 56RESP100200. Meanwhile, a SRV_AP 300 can be informed by the EPD 100 via 56REQ100300 or can receive the information from SRV_CNTRL 200 via NAPM, by database replication, via a back channel, or other messages.

[0521] Figure 57 API call groups 57202, 57206, and 57208 between different types of devices within a global virtual network (GVN) are described. Each API call is essentially a loop, where a request is sent from a client to a server and a response is sent back to the client. In most cases, the client can be one end or the other of a peer pair, provided the other peer has enabled a listening function to act as a server.

[0522] API call group 57202 represents calls from a central server (SRV_CNTRL) 200 via path P57202-C, to an endpoint device (EPD) 100 via P57202-B, and to an access point server (SRV_AP) 300 via P57202-A. This type of communication can exchange information between the repository databases and file stores on SRV_CNTRL 200 and EPD 100 and SRV_AP 300 regarding tunnel information, log information, billing information, device peer pair data, and other forms of related information.

[0523] There are two types of communication paths between EPD 100 and SRV_AP 300. Direct tunnel between them can push layer 3 traffic, information and binaries as packets via path P57206-C. There is also an API call architecture 57206 implemented via paths P57206-B to 57206 to P57206-A between EPD 100 and SRV_AP 300.

[0524] The direct connection between EPD 100 and SRV_AP 300 implemented via API 57206 can be used for information sharing, collaboration and verification and other information. For example, an attempt to restart a tunnel can typically be triggered by one side, the other side automatically responds and rebuilds it. However, in the case where the tunnel is blocked and cannot be rebuilt, the API can be used to send a command to attempt to force a restart of the tunnel at both ends, and if still unsuccessful, information can be shared between the devices. This information can trigger a need to build a different tunnel between the two devices using new tunnel information, or cause both devices to send a query to SVR_CNTRL 200 to obtain new tunnel building information. Thus, it is very useful to have a communication path between them via API 57206.

[0525] API call group 57208 represents calls from CNTRL_SRV 200 and internal backend infrastructure devices and other infrastructure support devices of GVN via path P57208-C. For simplicity of illustration, some gateway devices are shown in this example embodiment, and other types of infrastructure devices in GVN that can be present via this path to SRV_CNTRL are not shown here.

[0526] SRV_GW_email 57310 represents an email server and is linked via P57208-B1 to 57208, which is linked to P57208-C, which is linked to CNTRL_SRV 100. Emails can be sent and received via an email network access point (NAP) 57401. A dedicated email server enables other devices to focus on their own functionality, and also provides simplified management, as it is the only type of device that needs to be maintained in terms of email server management.

[0527] SRV_GW_FIN 57318 represents a financial gateway server, using which credit card and other financial related transactions can be conducted with third parties via external APIs 57501 NAP. As with the example SRV_GW_EMAIL, the device type role focused on a single function enables other devices to focus on their core functions, and provides for simplified management, as only the SRV_GW_FIN server requires additional management to secure financial transactions with third parties.

[0528] SRV_GW_OTHER 57315 represents other types of gateway between the GVN and other services on the internet. Communication between these types of gateway servers and the SRV_CNTRL 200 is achieved via P57208-B3 to 57208 to P57208-C.

[0529] The secondary API path between the SRV_AP 300 and the SRV_CNTRL 200 is via P57208-A to 57208 to P57208-C, and exists for redundancy purposes and for infrastructure related communications between this pair of peers.

[0530] Another set of calls from the SRV_AP server can establish a path from the SRV_AP 300 to the SRV_GW_EMAIL 57310 via the path from P57208-A to 57208 to P57208-B1 ; and from the SRV_AP 300 to the SRV_GW_FIN 57218 via the path from P57208-A to 57208 to P57208-B2; and from the SRV_AP 300 to the SRV_GW_OTHER 57315 via the path from P57208-A to 57208 to P57208-B3. These can implement API calls for data exchange directly from the SRV_AP 300 to these devices.

[0531] API calls transmitted via P57208-A can also represent relayed API calls by other devices via the SRV_AP 300, such as the call from EPD 100 to SRV_GW_FIN 57318 implemented via the path P57206-B to 57206 to P57206-A to 300 to P57208-A to 57208 to P57208-B2, in which case the API call flow implemented by the SRV_AP 300 is just another hop in the chain, with the client being the EPD 100 at one end, and the server being the SRV_GW_FIN 57318 at the other end.

[0532] API calls and other types of information exchange are critical to the operation of devices in the GVN. There are several types of automated infrastructure operations. These include: keeping device operating system configurations up to date; updating software packages of O / S and modules from reliable repository sources that can accommodate updated software, to easily and predictably implement patches, updates, and fresh installations; deploying new global virtual network software modules and keeping installed modules up to date; controlled replication of GVN databases; keeping API operation libraries up to date; and other operations.

[0533] On individual devices, there are background programs and heartbeat functionality that require automation and inter-device interaction. This includes keeping background programs running, services online, queues online and their equivalents un-jammed, heartbeat functionality, logging functionality.

[0534] Connectivity and fabric structure in the GVN includes virtual interfaces (VIFs), tunnels, multiple tunnels, routing, server availability, geo-destinations, DNS and cache and linked caches.

[0535] Up to date information is required for tunnel establishment and this information needs to be shared between clients and servers, otherwise tunnels cannot be built. Therefore, testing and diagnostics are required, with results data reported for centralized analysis to understand the overall functioning of the GVN. Testing and diagnostic information can include: first tier conditions; connectivity of tunnels; best point-to-point routing on the Internet; advanced smart routing (ASR) for best routing through the GVN; and device operational status.

[0536] APIs can also be used to communicate information about themselves, such as peer-to-peer information, queue information, transaction logs, security / accounting and other logs, and API actions, schemas, data structures, and related scripts that handle actions on the client or server.

[0537] Information about the status and configuration of hosted services can also be transmitted via API calls from devices to SRV_CNTRL or other devices. This information can include service online / offline status, API module online / offline status, and if answerable, site hosting status, database status, secure sockets layer (SSL) certificate status, GVN component status (e.g., whether components such as geo-destinations are running).

[0538] Information exchange via APIs has other uses related to security / FW / monitoring / collaboration / information exchange and other mission critical aspects of the GVN. APIs are a powerful medium for information exchange and a self-healing mechanism for integrity, and therefore can be deployed across devices.

[0539] Figure 58 Steps taken from the client device peer (source) 006 initiating, through the sending of an API call to the server device 007 007B and back to the client 006 006B are described.

[0540] The API transaction is triggered at the API initiation 001. Data is passed to a common class or other type of processor to create an internal payload 002. The internal payload is added to a queue 003 that can be in memory, saved to a database, flat file or other mechanism. The queue step can be bypassed with an API call sent immediately or can be set to send at a certain time. As part of the heartbeat function of the client device 006, and according to the priority flag of the API call in the queue, the payload can be processed immediately, at a certain time or delayed based on factors such as load, queue 003 length, network conditions or other factors. When the item from the queue is processed, the external payload is prepared and the related transaction data 004 is generated for the special, single purpose API call. When the external API REQUEST payload has been prepared to be sent, the external payload is transmitted via the neutral API mechanism 005, in turn sent to the peer target 007 host (server) API through the internet Q01 via the path CP01 to Q01 to CP03 or through the secure tunnel WAN Q02 via the path CP02 to Q02 to CP04.

[0541] After receiving 008 the request payload RP01, the server 007 will then begin to parse and interpret the payload. In processing the request payload RP01, security and data integrity checks will be made and the external payload will be decrypted to discover the contents of the internal payload 009. Further security and data integrity checks will be accomplished by comparing the internal and external payloads. After verification, the payload is passed to the corresponding script to take the prescribed action 010. In completing the request action, an internal payload for the response is created 011. The external payload creation 012 and transaction preparation 013 take the same process as was taken in creating the API request external payload RP01 to create the external API RESPONSE payload RP02. The response is then sent back via the neutral API 014.

[0542] The API RESPONSE RP02 follows the same path back from the API server 007 to the API client 006.

[0543] The API RESPONSE RP02 is received back 015 by the peer source API client device 006. The payload is parsed 016 and processed 017. Depending on the API action type, the data received back will be passed to the API processor script on 006. All transactions are logged 018.

[0544] If 020 callback 019 is specified, a new call is initiated via path P019 and via path P020 in parallel to the original API call, which is completed at API complete 022.

[0545] If 021 callback is not specified in API RESP RP 02, the original call proceeds via P021 to termination point 022 to complete the transaction.

[0546] Figure 59 is a flowchart showing the interaction between the EPD and the SRV_AP for obtaining the geo-destination functionality. Specifically, this figure describes the processing flow of the geo-destination mechanism, which starts at the client 000 and proceeds along sequential and sometimes parallel communication paths from CP0 to the end point device (EPD 100), where the EPD 100 interacts with the access point server (SRV_AP 300).

[0547] When the content within the remote region has been pulled to the SRV_AP 300 and subsequently sent back to the EPD 100 via transport and caching within the geo-destination mechanism, the processing flow ends in step 15 by providing back to the client 000 via path CP203.

[0548] In step 8, content is pulled from the content servers SRV 803, 804, 802 in parallel via CP13, CP14, CP12 and the results are sent back via CP10 for listing and subsequent processing of the data pull.

[0549] Steps 1, 12, 13 and 15 occur in the original region with respect to the client 000 and the EPD 100.

[0550] Steps 2, 10, 11 and 14 are steps that occur when transmitting in either or both directions between the EPD 100 and the SRV_AP 300.

[0551] Steps 5, 6 and 9 occur on the SRV_AP 300.

[0552] Steps 3, 4, 7 and 8 occur from the SRV_AP 300, over the Internet in the remote region in which the SRV_AP 300 resides, via EIP (egress / ingress point).

[0553] Step 3 is for DNS lookup of the initial URL, URI and URN of the content requested by the client 000. Step 7 is for DNS lookup of nested content pulled as part of the initial pull of content.

[0554] Figure 60Device cooperation within a geographic destination is described, the constituent parts generally indicated as modules and their constituent parts referred to on individual devices include information stored in memory and databases and information exchange, and information communicated via communication paths for API traffic and data transfers such as file transfers between devices. The GVN enables control of complex automated structures extending across multiple devices to work together to achieve a common goal.

[0555] This figure illustrates the components of the EPD 100 and shows the geographic destination mechanism on an endpoint device (EPD). This figure also illustrates the components of the SRV_AP 300 and shows the geographic destination mechanism on an access point server (SRV_AP 300) in a remote area from the EPD.

[0556] The content pull agent D 302 is located on the SRV_AP 300. The CPAD 302 receives the target URL / URI from the CDAD 102 located on the EPD. This target address that the client wishes to reach is located in another area from the client and is the location where the client wishes to pull content. The CPAD 302 passes the request address to the remote fetcher BOT (R.F.BOT 301).

[0557] The job of the R.F.BOT D 301 is to do a DNS lookup D 304 and then use that information to pull content via data pull 301. The R.F.BOT D 301 cooperates with the CPAD 302 via CPOl to resolve the fetch results, in turn finding any other addresses of ancillary content that can and should be pulled as constituent parts of that content. The request is stored in the database D 302 for access by the CPAD 302 and the R.F.BOT D 301 and further reference. The list of content files L 301 is passed from the R.F.BOT D 301 to the CPAD 302. The data file content is passed from the data pull 301 to the cache manager D 303 via the R.F.BOT D 301. The pulled files are sent to the cache manager D 303 for aggregation as files or transmission as individual files.

[0558] Depending on the distance from the starting point to the geographic destination area, the file type, and QoS, the pulled files in the cache can be aggregated as a single file transmitted through the chain cache uniformly or can be individual files sent as parallel concurrent streams.

[0559] There are multiple optional paths to the remote region. Data can be transmitted via the path between the API and TP01 to TP02, the path between TP01 and TP03, and the path between TP02 and TP03. Data files can also be transmitted through the GVN via path CP38, CP39, or P06 to CPBB, and so on. CP38 is a path via tunnel from SRV_AP 300 through GVN D888 to SRV_AP D 555. CPBB is a backbone path between SRV_AP D 555 and SRV_AP 300 via the relay SRV_AP D 505 path P06. CP39 is a file transfer path from cache 701 through SRV_AP D 555 to EPD 100 over the GVN. CP02 indicates the possibility of a direct connection path between SRV_AP 300 and EPD 100.

[0560] Based on current conditions, network segment properties, and how those properties contribute to best transmission, data type, and other factors, the optional paths to the remote region provide options for traffic to flow via the best route.

[0561] Figure 61 A global distributed parallel file system (PFS) is shown how it can be connected via the GVN. Specifically, this figure shows how a global distributed parallel file system (PFS) can allow seamless access to one of three 61308, or 61318 or 61328 PFS storage nodes using local RDMA access through the GVN Tapestry framework on top of various non-local network fiber (OTT) to achieve the required quality of service (QoS) and comply with the high performance computing (HPC) principles required for this functionality.

[0562] PFS 61308 is an example of one PFS instance in a client LAN behind an EPD linked to two other PFS instances "in the cloud" where local RDMA over IB between all three PFS storage nodes allows truly parallel access regardless of the network type at the underlying segment. Link 61CP06 is an underlying Internet connection between EPD 100 and SRV_AP 300 and TUN1 runs OTT of 61CP06. 61CP10 is within an IDC or OTT Internet. PFS 61308 connects to PFS 61318 via path 61CP08 -> 8CP02 -> 8CP06 / TUN1 -> 8CP10 -> 8CP12 -> 8CP18, which represents a short distance within a region. These devices are all within the same high performance zone.

[0563] SRV_AP 300 connects to SRV_BBX 61310 via 61CP10 and both are within the same global node.

[0564] PFS 61318 is connected to PFS 61328 via SRV_BBX 61310 connected to SRV_BBX 61320, which represents a long distance communication via global node to global node of the GVN.

[0565] The scope of the application is not limited to the specific embodiments described herein. In fact, apart from what is described herein, other embodiments of and modifications to the application, which are obvious to those of ordinary skill in the art, can be derived from the above description and the figures without departing from the scope of the application. Accordingly, such other embodiments and modifications are intended to be included within the scope of the application. Furthermore, although the application has been described in the context of at least one specific embodiment for at least one particular purpose, those skilled in the art will recognize that the application is not limited to that particular embodiment and that the application can be beneficially implemented in any number of environments for any number of purposes. Thus, the claims are not to be limited to the specific embodiments set forth herein but are to be given the full scope of the application as set forth in the following claims, and any equivalents thereof.

[0566] According to embodiments of the present disclosure, the following notes are also disclosed:

[0567] 1. A network system for connecting devices via a global virtual network, comprising:

[0568] a first device in communication connection with a first endpoint device;

[0569] a second device in communication connection with a second endpoint device; and

[0570] a communication path connecting the first and second endpoint devices, the communication path further comprising one or more intermediate tunnels connecting each endpoint device to one or more intermediate access point servers and one or more control servers.

[0571] 2. The network system of note 1, wherein at least one of the first endpoint device and the intermediate access point servers are configured to perform a domain name system lookup to locate the second device.

[0572] 3. The network system of note 1, wherein at least one of the first endpoint device and the intermediate access point servers are configured to perform a domain name system lookup from a cache to locate the second device.

[0573] 4. The network system of note 1, wherein at least one of the intermediate access point servers are configured to cache content.

[0574] 5. The network system of clause 1, wherein at least one of the endpoint device and the intermediate access point server is configured to perform intelligent routing.

[0575] 6. The network system of clause 5, wherein the intelligent routing is based on at least one of best bandwidth, lowest latency, fewest hops, and no packet loss.

[0576] 7. The network system of clause 5, wherein the intelligent routing is based on at least one of real-time statistics and historical statistics.

[0577] 8. The network system of clause 1, wherein at least one of the endpoint device and the intermediate access point server is configured to perform firewall services.

[0578] 9. The network system of clause 8, wherein the first endpoint device provides firewall services between the first device and the intermediate access point server.

[0579] 10. The network system of clause 8, wherein the intermediate access point server provides firewall services between the first endpoint device and other intermediate access point servers or the second endpoint device.

Claims

1. A network system in a global virtual network, comprising: a first device; a second device; a plurality of intermediate access point servers forming a plurality of end-to-end tunnels connecting the first device and the second device; and a plurality of host devices, wherein a first host device of the plurality of host devices communicates directly with the first device or the second device, and a second host device of the plurality of host devices communicates with the first device or the second device through one or more of the plurality of intermediate access point servers.

2. The network system of claim 1, wherein the first device and at least one of the plurality of intermediate access point servers are configured to perform a domain name system lookup from a cache to locate the second device.

3. The network system of claim 1, wherein at least one of the plurality of intermediate access point servers is configured to cache content.

4. The network system of claim 1, wherein the first device, the second device, and at least one of the plurality of intermediate access point servers are configured to perform intelligent routing. at least one of the plurality of intermediate access point servers connects with an endpoint device at a cloud partner organization location through an end-to-end tunnel.

5. The network system of claim 1, wherein, at least two of the plurality of end-to-end tunnels are bound to transmit traffic as one end-to-end tunnel.

6. The network system according to claim 1 or 2, wherein ​

Images