System for providing a global virtual network (GVN)

Abstract

A system for providing geographic destination service connection to a client based on a global virtual network GVN is disclosed. In one embodiment, the network system may include an endpoint device, an access point server, and a control server. The endpoint device and the access point server may be connected using a first tunnel. The access point server and the control server may be connected using a second tunnel.

Description

[0001] Related Applications

[0002] This application claims priority to U.S. Provisional Application No. 62 / 144,293, filed on April 7, 2015, and U.S. Provisional Application No. 62 / 151,174, filed on April 22, 2015, each of which is incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to networks and, more particularly, to global virtual networks and various associated auxiliary modules. Background Art

[0004] Humans can perceive delays of 200ms or more, as this is typically the average human reaction time to an event. If latency is too high, online systems such as thin clients to cloud-based servers, customer relationship management (CRM), enterprise resource planning (ERP), and other systems will perform poorly or even stop functioning due to timeouts. High latency combined with high packet loss can make a connection unusable. Even if data gets through, sometimes being too slow can result in a poor user experience (UX), and in this case, the result can be that users actually refuse to accept these conditions, rendering the poor service provided useless.

[0005] In order to solve some of these problems, various technologies have been developed. One such technology is WAN optimization, which generally involves a hardware (HW) device located at the edge of a local area network (LAN), which establishes a tunnel to another WAN optimization HW device located at the edge of another LAN, thereby forming a wide area network (WAN) therebetween. This technology assumes a stable connection via which two devices are connected to each other. WAN optimizers strive to compress and protect data streams, which generally result in speed gains. The commercial drive for adopting WAN optimization is to save the amount of data sent to reduce the cost of data transmission. The disadvantage of this WAN optimization is that it is generally point-to-point and may be difficult to perform when the connection between the two devices is poor, because the flow path of the traffic through the Internet can hardly be controlled between the two devices. In order to solve this problem, users of WAN optimizers generally choose to run their WANs through multi-protocol label switching (MPLS) or DDN lines or other dedicated circuits, thereby generating additional costs and generally requiring rigid fixed point-to-point connections again.

[0006] Direct links such as MPLS, DDN, dedicated circuits or other types of fixed point-to-point connections provide guarantees for connection quality and quality of service (QoS). These direct links are very expensive and usually take a long time to install because it is necessary to physically draw lines from the POPs at each connection side. When connected to resources from another LAN via such a directly connected WAN, the point-to-point topology works well. However, when the gateway (GW) of the general Internet is located at the LAN at one end (for example, at the company headquarters), the traffic from the remote LAN in the subsidiary country can be routed to the Internet through the GW. Since the traffic flows back to the server in the same country as the subsidiary through the Internet, slowness occurs. Then the traffic must flow from the LAN to the LAN where the GW is located through the WAN, then flow back to the server in the original country through the Internet, then flow back to the GW through the Internet, and then flow back to the client device in the LAN along the dedicated line. In essence, the global transit time that should account for a small part of the global delay of accessing such a nearby site becomes double or triple (or worse). In order to overcome this situation, the optional connection between another Internet line and the appropriate configuration changes, and the added equipment can provide local traffic to the Internet at each end of such a system.

[0007] Another option for creating a WAN link from one LAN to another involves establishing a tunnel such as an IPSec or other protocol tunnel between two routers, firewalls, or equivalent edge devices. These tunnels are usually encrypted and can provide compression and other logic to try to improve the connection. There is little control over the routing between the two points because they rely on the strategies of various middle players on the Internet, which carry traffic through their networks and peer with other carriers or network operators. Firewalls, routers, switches, and other devices from many equipment vendors typically have tunneling options built into their firmware.

[0008] While last-mile connectivity has improved dramatically in recent years, there are still issues with long-distance connectivity and throughput due to issues related to distance, protocol limitations, peering, interference, and other problems and threats. Because of this, there is a need for secure network optimization services that run over standard Internet connections. Summary of the invention

[0009] The Global Virtual Network (GVN) is a network that provides network optimization over the Internet (OTT). It is a disruptive technology that provides a low-cost alternative to expensive MPLS or dedicated lines. Having a secure tunnel between the endpoint device (EPD) and the access point server (SRV_AP) that links to the wider GVN global network provides many advantages. The core technology of GVN was created to fill the gap where a solution was needed but which technology did not exist.

[0010] In addition to the broader theme of addressing Quality of Service (QoS) issues related to network connectivity, thereby improving general performance and enhancing the user experience, two other key features are: This topology allows the edge of the network to be extended into the cloud. Additionally, the EPD acts as a bridge between the wider network and the Local Area Network (LAN), extending elements of the cloud as local nodes into the edge of the LAN.

[0011] The disclosed subject matter describes various auxiliary modules of the global virtual network, which are promoted by the GVN or assist the GVN in its operation. The requirements of the geographic destination must specifically deal with how the CDA or CPA works, and their interaction and coordination efforts. The geo-multicast element describes how the GeoD mechanism can utilize the topology of the GVN to provide reverse CDN geo-multicast operations. Tunnels describe what operations can be performed using higher-level pipes. The architecture and algorithm / logic invention describe the components. The graphical user interface and related HW and SW frameworks are also outlined for file transfer.

[0012] Disclosed are systems and methods for managing virtual global network connections between endpoint devices and access point servers. In one embodiment, a network system may include an endpoint device, an access point server, and a control server. The endpoint device and the access point server may be connected using a first tunnel. The access point server and the control server may be connected using a second tunnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to facilitate a more complete understanding of the present disclosure, reference is now made to the accompanying drawings, in which like elements are referenced to like reference numerals throughout the accompanying drawings. These drawings should not be construed as limiting the present disclosure, but are merely illustrative.

[0014] Figure 1 Shows packet bloat of an IP transport packet when a header is added to data at each layer.

[0015] Figure 2 The packet expansion of data and headers at each of the seven layers of the OSI model is shown.

[0016] Figure 3A block diagram is shown depicting the resolution of a Uniform Resource Locator (URL) via a lookup through the Internet Domain Name System (DNS) used for routing from a host (client) to the numeric IP address of the host (client).

[0017] Figure 4 Equations for calculating the bandwidth-delay product (BDP) for a connection segment or path taking into account various connection properties are shown in accordance with certain embodiments of the disclosed subject matter.

[0018] Figure 5 Traffic flow paths within an endpoint device (EPD) are shown in accordance with certain embodiments of the disclosed subject matter.

[0019] Figure 6 An over-the-top tunnel created over a conventional Internet connection is shown in accordance with certain embodiments of the disclosed subject matter.

[0020] Figure 7 A virtual interface of an over-the-top tunnel created over a conventional Internet connection is shown in accordance with certain embodiments of the disclosed subject matter.

[0021] Figure 8 A conventional embodiment showing the flow of Internet traffic.

[0022] Fig. 9 A tunnel established between two gateway devices (GWDs) is shown in accordance with certain embodiments of the disclosed subject matter.

[0023] Fig.10 Communications between EPD 100, SRV_CNTRL 200, and SRV_AP 300 via API-2A1-2A2, API-2A3-2A2, and API-2A1-2A3 through the Neutral API Mechanism (NAPIM) of the GVN are shown in accordance with certain embodiments of the disclosed subject matter.

[0024] Fig.11 The information flow required for two peers in a peer pair according to certain embodiments of the disclosed subject matter is shown.

[0025] Fig.12 A process of establishing a tunnel according to certain embodiments of the disclosed subject matter is shown.

[0026] Fig.13 is a flow chart illustrating logic for assigning a port to an IP address used to establish a tunnel, in accordance with certain embodiments of the disclosed subject matter.

[0027] Fig.14 Illustrates the importance of server availability lists and how IP addresses and ranges are assigned to various devices in accordance with certain embodiments of the disclosed subject matter.

[0028] Fig.15 Elements of a GVN tunnel from LAN to EPD to SRV_AP to SRV_AP to EPD to LAN are shown, including peer-to-peer between an ISP and the edge of the network, according to certain embodiments of the disclosed subject matter.

[0029] Fig.16 A multi-perimeter firewall (MPFW) in the cloud supporting personal endpoint devices is shown in accordance with certain embodiments of the disclosed subject matter.

[0030] Fig.17 Shows how tables on the databases of various GVN devices relate to each other and the manner in which these tables interact according to certain embodiments of the disclosed subject matter.

[0031] Fig.18 A block diagram illustrating techniques used and enabled by a global virtual network ("GVN") in accordance with certain embodiments of the disclosed subject matter.

[0032] Fig.19 Three devices are shown working in concert to provide geo-destination services to clients, thereby optimizing the retrieval, transmission, and serving of content from servers located at remote locations, in accordance with certain embodiments of the disclosed subject matter.

[0033] Fig. 20 Further illustrating the specific embodiments according to the disclosed subject matter Fig.19 The operation of the three devices is described in .

[0034] Fig.21 is a typical embodiment that continues to describe the operation of the geographic destination mechanism, which further explains the retrieval of a series of files from various types of servers, the aggregation of the retrieved files, and the subsequent transmission from the access point server (SRV_AP) to the endpoint device (EPD).

[0035] Fig. 22 The prior art is shown regarding how geo-multicast works within a content delivery network (CDN).

[0036] Fig.23 An analysis and interpretation of the period of the time base cycle is shown.

[0037] Fig.24

[0046] Described is how descriptions of two different data sets can provide information when comparing the various different data sets, according to certain embodiments of the disclosed subject matter.

[0038] Fig.25 Integration between a multi-perimeter firewall and other systems within a GVN is shown in accordance with certain embodiments of the disclosed subject matter.

[0039] Fig.26 Both a path through the open Internet and a path that is a combination of network optimized traffic with a mix of global virtual network (GVN) segments on the underlying Internet and open Internet segments are shown in accordance with certain embodiments of the disclosed subject matter.

[0040] Fig. 27 A GVN using a hub-and-spoke topology with backbone and octagonal routing is shown in accordance with certain embodiments of the disclosed subject matter.

[0041] Fig.28 A distributed endpoint management cluster (DEPAC) is shown in accordance with certain embodiments of the disclosed subject matter, wherein the distributed endpoint management cluster (DEPAC) can be virtually constructed in a global virtual network (GVN) or other similar topology.

[0042] Fig.29 Shown is the logical structure of a series of tests to be run on the Internet that test various ports and IP addresses in accordance with certain embodiments of the disclosed subject matter.

[0043] Fig.30 A mechanism for recording connection traffic activity between various devices operating in a global virtual network (GVN) or other similar topology is shown in accordance with certain embodiments of the disclosed subject matter.

[0044] Fig.31 A traditional thick stack 31-20 layered access between an application layer 31-800 and a network interface card (NIC) 31-000 is shown compared to a thin stack approach 31-10 of direct access to the NIC 31-000 in accordance with certain embodiments of the disclosed subject matter.

[0045] Fig.32 A simple exemplary database schema is shown for correlating and storing connection information from both test and production environment usage.

[0046] Fig.33 The use of custom commands that act as a defensive buffer between a trigger at the application layer, such as a button clicked on a graphical user interface, and the deeper logic and commands to be run as a result of the trigger is shown in accordance with certain embodiments of the disclosed subject matter.

[0047] Fig.34 is a simplified description of modules on relevant devices in a geographic destination mechanism within a global virtual network (GVN) according to certain embodiments of the disclosed subject matter.

[0048] Fig.35 Several ways are shown of how a portable endpoint device (PEPD) according to certain embodiments of the disclosed subject matter may be connected to a global virtual network (GVN) or similar topology.

[0049] Fig.36 The relationship between a portable endpoint device (PEPD) and an access point server (SRV_AP) 36-300 is shown in accordance with certain embodiments of the disclosed subject matter.

[0050] Fig.37 Some different types of extensible software that can be run on a portable endpoint device (PEPD) according to a specific embodiment of the disclosed subject matter are shown, where the software is stored on a remote device such as an access point server (SRV_AP) for transmission from the SRV_AP to the PEPD.

[0051] Fig.38 Multiple tunnels between devices within a global virtual network (GVN) across multiple regions are illustrated in accordance with certain embodiments of the disclosed subject matter.

[0052] Fig.39 A framework is shown of a common code base to be utilized by various types of devices, each with its own attributes / type and identity, in accordance with certain embodiments of the disclosed subject matter.

[0053] Fig.40 The relationships, interactions, and correlations of data with various devices within a global virtual network (GVN) are shown in accordance with certain embodiments of the disclosed subject matter.

[0054] Fig.41 A topology of a global virtual network (GVN) on the Internet and various devices operating within the topology are shown in accordance with certain embodiments of the disclosed subject matter.

[0055] Fig.42 Code located above the web root directory and below the web root directory is shown in accordance with certain embodiments of the disclosed subject matter.

[0056] Fig.43 An algorithm for merging information from external sources and internally learned AI knowledge is shown in accordance with certain embodiments of the disclosed subject matter.

[0057] Fig.44 A simplified representation of a method for designing smarter algorithms by considering owned factors and balancing individual needs is shown in accordance with certain embodiments of the disclosed subject matter.

[0058] Fig.45 Some system modules and components of an endpoint device (EPD), a central control server (SRV_CNTRL), and an access point server (SRV_AP) are shown according to certain embodiments of the disclosed subject matter.

[0059] Fig.46 Some system modules and components of an endpoint device (EPD), a central control server (SRV_CNTRL), and an access point server (SRV_AP) are shown according to certain embodiments of the disclosed subject matter.

[0060] Fig.47 Some system modules and components of an endpoint device (EPD), a central control server (SRV_CNTRL), and an access point server are shown according to certain embodiments of the disclosed subject matter.

[0061] Fig.48 A block diagram illustrating a typical computing device in accordance with certain embodiments of the disclosed subject matter.

[0062] Fig.49 The process of downloading files in a LAN and from the Internet is shown according to certain embodiments of the disclosed subject matter.

[0063] Fig.50 The operation of a file transfer manager (FTM) on an endpoint device (EPD) and an FTM on an access point server (SRV_AP) are shown in accordance with certain embodiments of the disclosed subject matter.

[0064] Fig.51 The relationship between the physical storage of files on a secure file storage capacity, the saving of information related to the files into a relational database (DB), and the data relationships between files, file information, and steps of a file transfer process are shown in accordance with certain embodiments of the disclosed subject matter. DETAILED DESCRIPTION

[0065] In the following description, many specific details are set forth about the systems, methods, and media of the disclosed subject matter and the environments in which such systems, methods, and media can work, etc., to provide a comprehensive understanding of the disclosed subject matter. However, it will be apparent to those skilled in the art that the disclosed subject matter can be practiced without such specific details and is not described in detail to avoid complication of the disclosed subject matter. In addition, it should be understood that the examples provided below are typical, and it is contemplated that there are other systems, methods, and media within the scope of the disclosed subject matter.

[0066] The Global Virtual Network (GVN) provides secure network optimization services to clients over their standard Internet connections. This is an overview of the components of the GVN and a description of related technologies that can be used as GVN elements. GVN elements can work independently or for their own purposes within the ecosystem of the GVN, such as using the GVN framework, or can be deployed to enhance the performance and efficiency of the GVN. This overview also describes how other technologies can benefit from the GVN: as an independent deployment using some or all of the components of the GVN, or can be quickly deployed as an independent mechanism on the GVN using the benefits of an existing GVN.

[0067] Software (SW)-based virtual private networks (VPNs) provide privacy via a tunnel between a client device and a VPN server. This has the advantage of encryption, and in some cases also has the advantage of compression. But again, here it is almost impossible to control how traffic flows between a VPN client and a VPN server and between a VPN server and a host server, a host client, or other devices at a destination. This is usually a point-to-point connection, which requires the use of a VPN to install client software for each device, and requires some technical proficiency to maintain the connection for each device. If the VPN server exit point is in close proximity to the destination host server or host client via a quality communication path, performance will be good. If the VPN server exit point is not in close proximity to the destination host server or host client via a quality communication path, then from an availability perspective, performance will be significantly dragged down and dissatisfaction will be generated. It is usually required that a VPN user must disconnect from a VPN server and reconnect to another VPN server to provide high-quality or local access to content from one area and content from another area.

[0068] A Global Virtual Network (GVN) is a computer network on the Internet that provides global secure network optimization services using a network of devices distributed around the world that are securely interconnected through advanced tunnels, communicating collaboratively via application program interfaces (APIs), database (DB) replication, and other methods. Traffic routing in a GVN is always through communication paths managed by Advanced Smart Routing (ASR) powered by an automation system that combines builders, managers, testers, algorithmic analysis, and other methods to configure and reconfigure the system to adapt to conditions and instructions that change over time.

[0069] GVN provides services to provide secure, reliable, fast, stable, accurate and centralized parallel connections over one or more conventional Internet connections. These benefits are achieved by compressing the data streams of multiple connections in a tunnel that is packaged, disguised and encrypted between the EPF and the access point server (SRV_AP) next to the EPD. The quality of the connection between the EPD and SRV_AP' is continuously monitored.

[0070] GVN is a combination of hardware (HW) endpoint devices (EPDs) with installed software (SW), databases (DBs), and other automation modules of the GVN system such as the Neutral Application Programming Interface Mechanism (NAPIM), Postback Channel Manager, Tunnel Manager, and more features for connecting the EPDs to distributed infrastructure devices such as Access Point Servers (SRV_APs) and Central Servers (SRV_CNTRLs) within the GVN.

[0071] Algorithms continuously analyze the current network state, taking into account trailing trends plus long-term historical performance, to determine the best route for traffic to take and the best SRV_AP or set of SRV_AP servers to push traffic to. Configuration, communication paths, and other changes are made automatically and require minimal or no user interaction or intervention in operation.

[0072] Advanced intelligent routing in EPD and SRV_AP ensures that traffic flows through the simplest possible "third layer" of the GVN via the most ideal path from the origin to the destination. This third layer is seen by client devices connected to the GVN as a normal Internet path, but has fewer hops, better security, and in most cases less latency than traffic flowing to the same destination over the regular Internet. Logic and automation run on the "second layer" of the GVN, where the GVN's software automatically monitors and controls the underlying routing and construction of virtual interfaces (VIFs), multiple tunnels, and the binding of communication paths. The third and second layers of the GVN are located above the "first operating layer" of the GVN, where the "first operating layer" interacts with the devices of the underlying Internet network.

[0073] From a technology and network perspective, a cloud refers to a device or grouping or array or cluster of devices that are connected and available to other devices via the open Internet. The physical location of these devices is not important because they often replicate their data to multiple locations and utilize a content delivery network (CDN) or other such technology to distribute data to / from the nearest server of the requesting client to speed up the connection, thereby enhancing the user experience (UX).

[0074] In some embodiments, the disclosed subject matter relates to increasing the utility value of a firewall (FW) by extending the boundary into the cloud. A firewall is a device designed primarily to protect an internal network from external threats from an external network and to protect information data from leaking from the internal network. Traditionally, a firewall is placed at the edge between a network, such as a local area network (LAN), and another network, such as an uplink to a wider network. Network administrators are sensitive to the placement and trustworthiness of FWs because they rely on FWs to protect their networks.

[0075] Figure 1 The packet expansion of an IP transport packet is shown in the case where a header is added to the data at each layer. At the application layer 1-L04, the data complex has an initial size as represented by data 1-D4. The size of the packet is represented by the packet size 1-PBytes. At the next layer "transport layer 1-L03", the packet size 1-PBytes has the original size of data 1-D4 equal to data (UDP) 1-D3. The "transport layer 1-L03" further includes the expansion of (header UDP) 1-H3. At the next layer "internet layer 1-L02", the header payload data 1-D2 is a combination of 1-D3 and 1-H3. The "internet layer 1-L02" increases (header IP) 1-H2 by 1-PBytes. At the link layer 1-L01, the frame data 1-D1 is a combination of 1-H2 and 1-D2. The link layer 1-L01 further increases the header frame 1-H1 and the tail frame 1-F1 by 1-PBytes.

[0076] Figure 2 The packet expansion of data and headers at each of the seven layers of the OSI model is shown. At each level of application OSI layer 7 2-L7, the original data 2-D0 is grown by the addition of headers such as header 2-H7. At each subsequent layer from layer 7 down to layer 1, the data layer is a combination of the data and combined headers of the previous higher-level layer. In the OSI model, the total packet expansion at the physical OSI layer 2-L1 is represented by the packet size 2-PBytes.

[0077] Figure 3A block diagram depicting the resolution of a uniform resource locator (URL) via a lookup through the Internet Domain Name System (DNS) is shown, where the Internet Domain Name System (DNS) is used to route from a host (client) to a numeric IP address of the host (client). A request or push of content from a host (client) source 101 to a host (server) target 301 flows in the direction 001 as a file or stream or data block. A response 002 of a content distribution from the host (server) target 301 to the host (client) source 101 as a file or stream or data block. A host (client) source 101 in a client-server (CS) relationship requests access to content from a remote host (server) or sends data to a remote host (server) via a uniform resource locator (URL) or other network reachable address.

[0078] The connection from the host client to the Internet is labeled P01 - the connection from client 101 to POP 102 may be referred to as the last mile connection, where POP 102 directly faces or may be located in a local area network (LAN) that then connects to the Internet via a point of presence. Point of presence (POP) 102 represents the connection provided by an Internet Service Provider (ISP) from an endpoint to the Internet via its network and connection lines. If the URL is a domain name rather than a numeric address, the URL is sent to a Domain Name System (DNS) server 103, where the domain name is translated into an IPv4 or IPv6 or other address for routing purposes.

[0079] Traffic from client 101 to server 301 is routed through Internet 201 representing transit between POPs (102 and 302), including peering, backhaul, or other transit at the network boundaries.

[0080] The connection P02 from POP 102 to DNS 103 for looking up multiple addresses from a uniform resource locator (URL) to obtain the IPv4 address or other digital address of the target server can be accessed directly from POP 102 or via Internet 201. The connection P03 from the ISP's POP 102 to Internet 201 can be single-angle or multi-angle. There is a connection P04 from Internet 201 to the ISP's or Internet Data Center (IDC)'s Internet-facing POP 302. The connection from the server's POP 302 to the host 301 can be direct or via multiple hops.

[0081] Lookups from names to numeric addresses via the Domain Name System are the standard for the Internet today, and assume that DNS servers are intact and their results are up to date and can be trusted.

[0082] Figure 4An equation for calculating a bandwidth delay product (BDP) 4-080 for a connection segment or path taking into account various connection attributes is shown in accordance with a specific embodiment of the disclosed subject matter. The greater the distance between two points and / or other factors that increase the effect of delay, the greater the amount of data that the line can blindly absorb before the sending device receives a message back from the receiving device regarding whether the receiving device can accept the amount of data.

[0083] In short, the BDP 4-080 calculation represents a measure of how much data can be filled in the pipe before the server knows it is sending too much data at too fast a rate.

[0084] Bandwidth 4-000 may be measured in megabits per second (Mbps) and granularity 4-002 may be a unit of time relative to one second. To accurately reflect BDP 4-080, bytes 4-020 are divided by the number of bits 4-022 of the system. Latency 4-050 is a measure of the round trip time (RTT) between two points in milliseconds (ms).

[0085] Thus, for example, the BDP 4-080 for the following network path with these attributes - a bandwidth 4-000 of 10 GigE using an interval 4-022 of 1 second, on an 8-bit system with a path latency 4-050 of 220 ms - can be calculated as follows:

[0086]

[0087] Therefore, on a 10GigE line, a sending device can theoretically send 33,569.3 megabytes (MB) of information in 220ms before receiving a message back from a receiving client device.

[0088] This calculation can also be the basis for other algorithms, such as algorithms to manage the size of RAM buffers, or algorithms to manage the time and amount of data buffered before realizing a problem such as an attack vector. Throttling by the host server can result in underutilized pipes, but accepting too much data can also cause other problems. The calculation of the BDP 4-080 and the proactive management approach to problems result in efficient use of hardware and network resources.

[0089] Figure 5 Traffic flow paths within an endpoint device (EPD) according to a particular embodiment of the disclosed subject matter are shown. Traffic flows between (LAN) 5-000 and endpoint device (EPD) 5-100 via connection 5-CP00. Endpoint device (EPD) 5-100 flows to point of presence (POP) 5-010 via connection 5-CP06. Point of presence (POP) 5-010 is connected to the Internet 5-020 via connection 5-CP08.

[0090] Figure 6 An over-the-top tunnel created over a conventional Internet connection is shown in accordance with certain embodiments of the disclosed subject matter. Figure 6 and Figure 5 Similarly, and additionally, an access point server (SRV_AP) 6-300 is shown. The access point server (SRV_AP) 6-300 includes a tunnel listener (TNL0) 6-380. The endpoint device (EPD) 6-100 includes a tunnel manager (TMN0) 6-180. A tunnel (TMN0) 6-CP80 is constructed connecting the tunnel manager (TMN0) 6-180 and the tunnel listener (TMN0) 6-380. The tunnel is constructed over conventional Internet connections 6-CP06 and 6-CP08.

[0091] Figure 7 A virtual interface of an over-the-top tunnel created over a conventional Internet connection is shown in accordance with certain embodiments of the disclosed subject matter. Figure 7 and Figure 6 Similarly, and additionally includes a virtual interface (VIF) as a hook point between each device (EPD) 7-100 and (SRV_AP) 7-300 for establishing multiple tunnels between the two. The figure also shows multiple tunnels 7-CP80, 7-CP82 and 7-CP84 between (EPD) 7-100 and (SRV_AP) 7-300. The main advantage of the virtual interface (VIF0) 7-170 and (VIF0) 7-370 on each device is that this method implements clear structural attributes and logical paths for more complex tunnel structures and subsequent routing complexities.

[0092] Certain other advantages related to timing and flow control will be described in subsequent figures below.

[0093] Figure 8 A conventional embodiment showing the flow of Internet traffic. Figure 8 In the figure, traffic over the Internet goes from a host (client) origin 101 connected to a local area network (LAN) 102 to an Internet service provider's (ISP) point of presence (POP) 103 to an Internet data center (IDC) 203's (POP) 204 to a load balancer 202, which routes the traffic to a host (server) target 201.

[0094] Neither the client nor the server can control the route their traffic takes through the Internet and other networks between them.

[0095] Fig. 9 FIG. 2 shows a tunnel established between two gateway devices (GWDs) according to a specific embodiment of the disclosed subject matter. Fig. 9In Figure 1, GWD Al and GWD Bl are each located between their internal networks and the edge of the open Internet, EDGE-1 and EDGE-2. TUN connects the two local area networks (LANs) to the wider wide area network (WAN). GWD Al receives its connection from Internet Service Provider ISP-1, and GWD Bl receives its connection from ISP-3. The key point is that while TUN provides security and other benefits, there are potential negative issues because traffic from ISP-1 to ISP-3 must be transited through ISP-2's network.

[0096] Congestion due to saturation, packet loss or other problems may occur at peering points PP-01 and / or PP-02, or within the network of ISP- 2. Since neither GWD Al nor GWD Bl is a client of ISP-2, they must reach ISP-2 by making complaints to the respective ISPs in ISP-1 or ISP-3.

[0097] Also note that while the Internet path may contain multiple hops, such as, for example, external hop (EH) EH1 through EH 17, within a TUN, there will only be one hop at each endpoint of the tunnel. A tunnel is an encrypted path over the open Internet.

[0098] Fig.10 Communication between (EPD) 10-100, (SRV_CNTRL) 10-200, and (SRV_AP) 10-300 via API-2A1-2A2, API-2A3-2A2, and API-2A1-2A3 through the Neutral API Mechanism (NAPIM) of the GVN is shown in accordance with certain embodiments of the disclosed subject matter.

[0099] For tunnels TUN2-1, TUN2-2 and TUN2-3 to be established between (EPD) 10-100 and (SRV_AP) 10-300 and for tunnels such as TUN2-4 from (EPD) 10-100 to other SRV_AP servers and from other EPDs to SRV_AP 10-300 via TUN2-5, each device in the peer pair requires specific information for each tunnel.

[0100] The NAPIM mechanism stores relevant credentials, coordinates, and other information for the peer pairs on each side to utilize when establishing new tunnels via the tunnel managers 2110 and 2310. The server availability mechanism 2222 on the (SRV_CNTRL) 10-300 evaluates the performance of the various tunnels tested on the EPD side via the tunnel tester 2112 and on the SRV_AP side by the tunnel tester 2312. Information from the tests is relayed to the connectivity analyzer 2288 on the (SRV_CNTRL) 10-200. The test results include the assigned IP address and port combinations, the ports used, results from historical combination usage, results from port spectrum testing, and other relevant information.

[0101] The server availability list presents to the (EPD) 10-100 a list of IP addresses and ports that the tunnel manager can utilize to establish a new tunnel. The (SRV_AP) 10-300 and other SRV_AP servers recorded on the list will be notified to listen for connection attempts to be made by the (EPD) 10-100.

[0102] Server availability prioritizes the list of SRV_AP IP address and port combinations based on expected best performance of the tunnel to be established, while also looking at the current load of available SRV_AP servers, balancing the allocation lists given to other EPDs, and other available information.

[0103] Fig.11 The information flows required for two peers in a peer pair according to a specific embodiment of the disclosed subject matter are shown. The peers can be a client (C) and a server (S), or can be a peer-another peer in a P-2-P topology. To simplify the notation and description within this exemplary embodiment, CS and P-2-P represent two peer relationships of the same type, and CS is described here. GVN primarily uses CS relationships between devices, but its methods and techniques can also be applied to P-2-P peer pairs used for tunnel establishment.

[0104] An encrypted channel is essentially a secure communication path over which data can flow. In situations where the client and server are some distance apart and the connection between the two is through the open, unencrypted internet, an encrypted channel is an ideal channel to exchange data securely. If there is a human network administrator at either end, they can program the equipment. But there are challenges with how to relay secure information such as pass phrases, keys, and other information. Some embodiments may use voice calls for coordination, other embodiments may use a series of postings through a secure website to share information or other methods. Manually setting up a single tunnel can be a task. Managing multiple tunnels can become cumbersome.

[0105] In order to automatically establish a series of encrypted tunnels between two devices in a peer pair, information needs to be shared securely. The tunnel information also needs to be up to date and securely stored on the devices. In addition, during the establishment process, there are threats that must be addressed. In the case where the tunnels are already established, there are other threats that need to be addressed.

[0106] SRV_CNTRL D00 is a central server that contains a repository for managing information in database tables, files stored in a secure file storage system, lists in memory, and other related information. SRV_CNTRL also has algorithms and mechanisms to evaluate certain data to generate information reports.

[0107] The client device D01 represents a device that will initiate the establishment of a tunnel by "dial-up" a connection to a server device via a specific IP address and port. There can be many client devices D01 connected to the GVN at the same time, where these client devices D01 have similar software and configuration, and the distinguishing factor is that the client has a unique device identity UUID, and also has unique information for each tunnel for each client.

[0108] The server device D02 represents a device that will listen for client connection attempts on a specific IP address and port. If the client follows the correct protocol and establishment sequence and provides the correct credentials and other security information, the server will allow the client to establish a tunnel to the server. There can be many server devices D02 connected to the GVN at the same time, where these server devices D02 have similar software and configuration, and their distinguishing factors are the unique device identity UUID and other unique information.

[0109] Tunnel information S1 shows the information stored on client device D01 and server device D02. Each device can establish multiple tunnels, and each tunnel will have its own set of tunnel information and security information. Some tunnel information sets can be used to establish the current active tunnel, while other tunnel information sets can be reserved for future tunnels.

[0110] Some information between C and S is the same, such as a pass phrase that can be presented to others, and other information will differ as appropriate. The information requirements for establishing a tunnel between two points may include: client / server topology and setup; IP and port of each endpoint to be used by the tunnel; tunnel metrics including MTU size, protocols and other information used for operation; keys, pass phrases and other information related to the security protection used by the tunnel; SSL certificates and other information used to protect the tunnel UP before information is exchanged; and other information. The above information is shared between devices using specific API action calls of the neutral API of GVN.

[0111] Before the tunnel S0 describes the process of receiving and sharing information between devices D01 and D02 and repository D00 on SRV_CNTRL and sending information back to devices D01 and D02. API communication paths API-CP0, API-CP1, API-CP2, and API-CP3 represent request-response information exchanges, where arrows represent the direction of information flow from one device to another.

[0112] The server D02 reports the information to the receive information C-0 module of the SRV_CNTRL D00 device via the path API-CP0. SRV_CNTRL D00 receives the information from the server and stores the relevant identity, tunnel, current load and other information in its repository. For example, the algorithm and AI logic on SRV_CNTRL D00 analyzes the server load and updates the server availability C-1 matrix based on the expected demand from the client device D01. The server availability C-1 information can be transmitted by database replication to the client device D01 via the API of the GVN, by direct file sharing via the GVN, or other methods, where the database replication is performed by the shared information C-3 module via the API call path API-CP3.

[0113] The client device D01 reports the information to the receive information C-0 module of the SRV_CNTRL D00 device via the path API-CP1. The information will be stored in the repository of SRV_CNTRL D00. The specific tunnel information from the client device D01 can be shared with the server D02 via the path API-CP2 through the share information C-3 module.

[0114] SRV_CNTRL D00 compiles the current list of clients C-2 that SRV_CNTRL D00 publishes as servers to server D02 via path API-CP2 through shared information C-3.

[0115] If the client device D01 or the server D02 detects a problem in tunnel establishment using the current tunnel information, one device or the other may request a new set of tunnel information to be generated by SRV_CNTRL via API-CP1 or API-CP0, respectively. The new set of tunnel information may be shared with both peers in the peer pair via shared information C-3, where the client device D01 information is sent via API-CP2 and the server D02 information is sent via API-CP3.

[0116] The current status of client list C-2 and server D02 will have a direct impact on server availability C-1.

[0117] Each server D02 needs to organize, protect and coordinate its client list C-2, which will attempt to establish new tunnels to the shared resources of server D02. This information will be fluid and needs to be updated regularly via secure API calls to SRV_CNTRL D00.

[0118] The need to securely coordinate information between devices is critical to protecting the integrity of the tunnels between them.

[0119] The tunnel establishment S3 phase describes the process of tunnel establishment via shared information C-3. Fig.11 Path TP0 represents the path between client device D01 and information exchange C-5, and from information exchange C-5 to server D02 is via path TP2.

[0120] Establishment threats C-4 refer to threats to information exchange C-5 during tunnel establishment. If a tunnel type signature is visible, there may be threats during tunnel establishment C-4, such as a fake Transport Layer Security (TLS) handshake from an illegal actor in the middle, TLS errors related to the handshake, resulting in closed or blocked ports and IP identification, timeouts due to filtering devices, reset packets sent by ISPs or firewalls or intermediate devices, etc.

[0121] If the information exchange C-5 is successful, the step of establishing a tunnel C-6 will be performed with the applied routing and other related operations, so that the tunnel TUN can be securely established between the client device D01 and the server D02.

[0122] Tunnel UP S4 describes the period during which traffic normally flows through the tunnel.Transmitting information between devices is critical, and SRV_CNTRL D00 needs to manage unique information for the various client devices D01 and server devices D02 and for the multiple tunnels to be established between the two.

[0123] The information exchange between the equipment must be a conventional event, because it is often necessary to create an additional, new dynamic tunnel. Some ports on the IP address may be closed or become closed, and simply changing the port of this IP address will allow to set up a tunnel and make data flow. In addition, for each IP address, each tunnel needs one or more unique ports to avoid the conflict between the tunnels. When the client device D01 device request creates new tunnel information, a random port number is generated and the port availability of the specific IP address on the target server D02 is verified by two or more factors including the following: whether the port has been in the use of an existing tunnel (operable tunnel or can become an operable tunnel in standby state), and whether the port has been used and whether the port has been closed by specific client device D01 / server D02 peers in the past. In both cases, a new random number will be generated. For each IP address, there are 65536 available ports, wherein, a specific number of ports are reserved for specific services. For example, a 5500 layer will leave 60036 available ports that a random number generator can use, where the random number generator generates a minimum of 5001 and a maximum of 65536. When the tunnel is torn down and the port is marked as closed for a peer pair, the tunnel becomes available to other peer pairs. This release of ports is necessary to avoid port loss. Therefore, it is critical to use SRV_CNTRL D00 to track IP and port combinations.

[0124] Tunnels can go through some steps to aid their own establishment, but there are limitations. Most tunnels are visible during establishment, if secure. Handshakes and signatures about what the tunnel is can be visible during operation. Manually setting keys is cumbersome, and if they are not changed often, there is an increased risk that these keys may become corrupted if used for too long; therefore, keys should be rekeyed to new keys frequently.

[0125] The automation system needs to ensure that information such as new keys, ports to IP addresses, and other information can be created and made available to both sides of the peer pair so that the tunnel can be established and reestablished. Both sides must be configured and prepared to be able to establish the tunnel. Therefore, the information exchange between the peer pair needs to be secure, otherwise, the integrity of the security of the tunnel itself is compromised.

[0126] When the tunnel is established and traffic is pushed, there is an operational threat C-7. The tunnel signature may be visible (e.g., when the tunnel is perceptible and not obfuscated). If the type of tunnel can be discovered, the structure of the tunnel can be known. This can lead to the risk of packet streams being captured and brute force keys being used to decrypt the contents of the tunnel. Alternatively, if the reset code or other tunnel control code is known, the reset signal may destroy the tunnel. Therefore, in order to maintain the security and integrity of the tunnel between the client device D01 and the server D02 device in the peer pair, the updating and sharing of information needs to be automated and secure.

[0127] The GNV structure allows enabling these devices for automated secure tunnel establishment between pairs of peers based on up-to-date information. The combination of security features and methods provides self-enforcing protection.

[0128] Fig.12 A process of establishing a tunnel according to certain embodiments of the disclosed subject matter is shown. Fig.12 The illustrated example illustrates managers, modules, processes, database libraries (Db), storage devices (HFS), paths, and other features used for communication between three participants in the tunnel establishment and management process.

[0129] In this example, a single tunnel (TUN0) 12-CP80 is established between an endpoint device (EPD) 12-100 and an access point server (SRV_AP) 12-300. The tunnel is initiated by a tunnel manager (establisher) 12-D110 on the (EPD) 12-100. The tunnel manager "dials" a specific point on the IP address where the tunnel manager (listener) 12-D310 on the (SRV_AP) 12-300 is listening.

[0130] The three main steps between the builder and the listener are handshake (12-S1), information exchange (12-S2) and tunnel establishment process (12-S3).

[0131] Other communication paths are also described, such as path 12-CP02, which may be direct SSH or an equivalent type of direct device-to-device communication.

[0132] The information related to the tunnel, the peer (EDP) 12-100 and the (SRV_AP) 300, or other related information may be securely shared via the (SRV_CNTRL) 12-300 using an API path 12-PA1 between the (EDP) 12-100 and the central control server (SRV_CNTRL) 12-200 and another API path 12-PA2 between the (SRV_AP) 12-300 and the (SRV_CNTRL) 12-300.

[0133] Within each device, the tunnel manager collaborates with other modules and managers such as device managers 12-D120 and 12-D320.

[0134] The paths 12-TP01~12-TP02 between the (EPD) 12-100 and the (SRV_AP) 300 may represent another dedicated tunnel for information exchange in addition to the tunnel (TUN0) 12-CP80 for data traffic.

[0135] Fig.13 is a flow chart illustrating logic for assigning a port to an IP address used to establish a tunnel according to certain embodiments of the disclosed subject matter. In selecting the port and IP address to use, the process takes various factors into consideration.

[0136] The first step is to collect parameters 13-010 for port to IP address assignment by checking to see if the desired port and IP_address to be used by a particular device has been specified by device_ID and other factors. These parameters also describe the bottom and top values ​​of the port number, and more management devices.

[0137] The "Logic Gate IP+Port Specified? 13-020" step checks to see if there is a request for a specific port to connect to a specific IP address of a server device by device_IP.

[0138] If a port and IP address have been specified, then the availability of their use is accepted and the logic follows path "yes" 13-P022. If no preferred port and IP have been specified, then the logic follows path "no" 13-P030 to a random number generator to generate a tunnel port 13-030 within a certain range.

[0139] Search in step 13-050, with (via path 13-B102 to Db repository 13-B 100) check the current and historical use of the mapping of the port to the IP address to see if the port is free or whether the port is currently in use. Check the historical use to see if the historical use indicates whether the device or other devices have used the combination of the port and IP in the past, if so, then prove that the use is relatively problematic. Due to some unstable or unreliable ports caused by filtering and congestion or other reasons through the device can be marked as problematic. If there is also a trend of closing the problematic ports of other devices, the combination of the port to the IP address can be marked as unavailable.

[0140] If the port is not available at step 13-060, the process of generating the port to IP address mapping is restarted via node 13-012.

[0141] If the port is available, the port is assigned to an IP address for use at step 13-100. The assignment will be saved in the Db repository 13-B100 via path 13-B112. Next, the assignment of the port to the IP address is published via an API call 13-120 so that the relevant devices know the availability status of the port. The last step is to record the assignment of the port to the IP address 13-130, including the logic used and other factors that may help improve the efficiency of future port assignments.

[0142] Fig.14 The importance of the server availability list and how IP addresses and ranges are assigned to various devices according to a specific embodiment of the disclosed subject matter are shown. While IPv6 provides a large number of possible IP addresses, the IPv4 standard has limited public and private IP addresses. This has an impact on which EPDs can connect to which SRV_APs.

[0143] In this figure, EPD 14-100 establishes tunnels with SRV_APs 14-300, 14-302, and 14-306. EPD 14-102 establishes tunnels with SRV_APs 14-300, 14-304, and 14-308.

[0144] This example illustrates how the internal IP range 10.10.191.0-10.10.191.255 may be used on two SRV_APs 14-302 and 14-304, and the IP range 10.10.192.0-10.10.192.255 may be used on two SRV_APs 14-306 and 14-308.

[0145] Thus, for example, (EPD) 14-100 may use 10.10.191.18 to establish a tunnel to (SVR AP) 14-302, while (EPD) 14-102 may also use 10.10.191.18 to establish a tunnel with (SRV_AP) 14-304.

[0146] (EPD) 14-100 and (EPD) 14-102 do not need to interact directly with each other to avoid conflicts because the server availability list published for each EPD will coordinate with the TUN manager to allocate the (internal and external) IP address combinations for the EPD to connect with the SRV_AP without any conflicts.

[0147] Fig.15Elements of a GVN tunnel from LAN to EPD to SRV_AP to SRV_AP to EPD to LAN are shown, including peer-to-peer between ISPs and network edges, according to a particular embodiment of the disclosed subject matter. The diagram shows an end-to-end GVN tunnel between the edges of two LANs via two SRV_APs, and it also shows more information about the different Internet Service Providers (ISPs) that carry traffic between EH-3 and EH-15 over a particular portion of the Internet.

[0148] EDGE-1 is the demarcation point for the network access connection between LAN-1's devices and ISP-1's POP. PP-01 is the point where peering occurs between ISP-1 and ISP-2's networks. PP-02 is the point where peering occurs between ISP-2 and ISP-3's networks. EDGE-2 is the demarcation point for the network access connection between LAN-2's devices and ISP-3's POP.

[0149] Some advantages can be achieved by placing SRV_AP-1 at PP-1 so that it can directly peer with ISP-1 and ISP-2. Further advantages can be achieved by placing SRV_AP-2 at PP-2 so that it can directly peer with ISP-2 and ISP-3. If ISP-2's network is not ideal, traffic can be optionally routed around ISP-2 through another route or line or ISP or carrier.

[0150] The hop count through the neutral third layer of the GVN is 8. The distances between ISPs are not to scale. Furthermore, there are likely many more hops within the ISP's network, but the number shown is limited for simplicity.

[0151] Fig.16 A multi-perimeter firewall (MPFW) in the cloud supporting personal endpoint devices according to certain embodiments of the disclosed subject matter is shown. The figure shows a portable device hooked into the GVN from a mobile location, and where the 14-M1 boundary is the edge between a personal area network (PAN) 14-010 and the GVN.

[0152] The figure shows the topology of a personal endpoint device (PEPD) 14-130, some of whose connections and other functions are distributed in the cloud. The figure further describes the firewall operations distributed to the cloud, and other operations performed in the cloud on behalf of local devices such as personal endpoint devices (PEPD) 14-130. In the case where the (PEPD) 14-130 is a device that is not as powerful as the endpoint device (EPD) but more portable than the endpoint device (EPD), the (PEPD) 14-130 can still take advantage of the personal area network connectivity provided by the GVN, including features such as advanced smart routing (ASR), multi-border firewalls, etc.

[0153] The key point of the illustration is that the personal device extends its demand for processing power to the cloud. The modules residing on the PEPD include the hardware components of processor (CPU) 106, memory (RAM) 108 and network interface (NIC) 102. The operating system is a minimal O / S that provides a platform for the system software "system SW 112" and "connectivity 172" modules. This basic configuration is sufficient to allow the (PEPD) 14-130 to establish a tunnel 14-TUN between itself and the access point server (SRV_AP) 14-300.

[0154] Components are distributed at the hardware components of processor (CPU) 306, memory (RAM) 308 and network interface (NIC) 302. Operating system O / S 310 is a more extensive installation than O / S 110. O / S 310 provides a platform for the system software "System SW 312" and "Connectivity 372" modules of (SRV_AP) 14-300. Advanced Intelligent Routing (ASR) 350 module and other modules 370 provide functionality to both (SRV_AP) 14-300 and connected (PEPD) 14-130.

[0155] The (PEPD) 14-130 may rely on the tunnel 14-TUN2 to be established and may be capable of carrying traffic to implement cloud-based ASR, FW and other operational functions.

[0156] Fig.17Shows how tables on databases of various GVN devices relate to each other and the manner in which these tables interact according to a specific embodiment of the disclosed subject matter. For example, the repository database DB_2300 on SRV_CNTRL has various tables related to devices and the interaction between devices via the neutral API mechanism (NAPFM) of the GVN. Tables such as the device repository DBT_2310 in the database DB_2300 are designated as REPO_ACTIVE, which means that the table receives information from many sources, is read / written and can be queried as a source of information for selective or complete replication to a table (such as the device identity DBT_2102 as part of the database EPD local Db DB_2100). The table DBT 2101 has a designation SEL_REP+W that allows selective replication from DBT_2310 and reports the relevant identity back to the device registry.

[0157] The control and release of information is managed by the data manager. Database table type specifiers include REGULAR for normal read / write tables, REP INFO for read-only replicated tables, SEL REP for read-only partially replicated tables with only relevant rows, REPOS ACTIVE for tables composed of all sources on the repository of device registry DBT 2310 such as identity, etc. Other possibilities include LOGGING from source tables composed on the database DB2800 of SRV_LOGS. These designations for tables, for example, are unique and may be different in real-world usage, and there are more tables and other types based on usage.

[0158] Fig.18 A block diagram showing technologies used and enabled by a global virtual network ("GVN") according to a particular embodiment of the disclosed subject matter includes a GVN core element G0, a GVN module G100, and technologies G200 enabled by the global virtual network GVN. The GVN core includes an overview of the mechanism G1 and its component topology G2, fabric G3, logic G4, and control G5 layers. The GVN core G0 also includes relationships with GVN elements G6.

[0159] The GVN may include plug-in and / or stand-alone GVN modules G100, including but not limited to: the neutral API mechanism ("NAPIM") G102 described in PCT International Application PCT / US16 / 12178, the entire contents of which are hereby incorporated by reference; the geographic destination ("Geo-D") G104 described in PCT International Application PCT / US15 / 64242, the entire contents of which are hereby incorporated by reference; the advanced intelligent routing ("ASR") G106, connection G108 and other modules G110 described in U.S. Provisional Application US62 / 151,174, the entire contents of which are hereby incorporated by reference.

[0160] GVN also provides a platform that can enable other technologies including but not limited to the following: Network Tapestry G202; (MPFWM) G204; Network Slingshot G206; Network Beacon G208; Time Base Interval G210; and Other Technologies G212. These technologies are described in each of Provisional Application 62 / 174,394 and U.S. Provisional Application 62 / 266,060, which are incorporated herein by reference in their entirety.

[0161] The GVN-enabled GVN modules (G100) and technologies (G200) may work on an existing GVN as an integral part of the GVN, or may be independent and utilize all or some individual parts of the GVN to support its own independent operation.

[0162] Fig.19 Three devices are shown working in concert to provide geo-destination services to clients, thereby optimizing the retrieval, transmission, and serving of content from servers located at remote locations, in accordance with certain embodiments of the disclosed subject matter.

[0163] The devices are an endpoint device (EPD) 19-100, an access point server (SRV_AP) 19-300, and in terms of supporting capabilities, a central control server (SRV_CNTRL) 19-200.

[0164] The content distribution agent (CDA) 19-D120 works on the (EPD) 19-100, and the content extraction agent works on the (SRV_AP) 19-300.

[0165] The (CDA) 19-D120 coordinates with the Advanced Intelligent Routing (ASR) 19-D102 module on the (EPD) 19-100 and with the Cache Manager 19-D130.

[0166] The (CPA) 19-D320 communicates with the URL Lookup and DNS Cache Manager 19-D310, and it directly provides instructions and receives information from the Remote Fetcher BOT (RFBOT) 19-D328, and utilizes the Cache Manager 19-D330 on the (SRV_AP) 19-300 to coordinate its actions.

[0167] The (CPA) 19-D320 receives instructions from the (CDA) 19-D120 to contact the content server to retrieve content from the target server.

[0168] Fig. 20 Further illustrating the specific embodiments according to the disclosed subject matter Fig.19 The three devices constituting the geographic destination mechanism include an endpoint device (EPD) 20-100, an access point server (SRV_AP) 20-300, and a central control server (SRV_CNTRL) 20-200.

[0169] The content extraction agent (CPA) 20-D320 receives an indication from the content distribution agent (CDA) 20-D120 that the client 20-800 connected to the (EPD) 20-100 has requested to obtain a geographic destination and retrieve content from a content server 20-600 located in the area where the (SRV_AP) 20-300 resides.

[0170] (CPA) 20-D320 receives an instruction from (CDA) 20-D120 to obtain content from a URL. (CPA) 20-D320 passes the information to the remote obtainer BOT (RFBOT) 20-D328 to check the URL to see if its DNS information is cached locally at 20-D310. If the DNS information is cached and the information is not outdated, the cached information is used. If the information is not in the cache, or if the information is outdated, a new lookup is performed on the DNS server 20-604. The result will be returned and stored in 20-D310. (RFBOT) 20-D328 notifies (CPA) 20-D320 that the DNS information has been successfully obtained. (CPA) 20-D320 instructs (RFBOT) 20-D328 to contact the content server 20-600, and the (CPA) 20-D320 obtains the content file from the content server. (RFBOT) 20-D328 passes the content to (CPA) 20-D320 for parsing to find more content links embedded in the acquired content. (CPA) 20-D320 then collaborates with URL Lookup and DNS Cache Manager 20-D310 and with (RFBOT) 20-D328 to obtain information about the network coordinates of the servers where other content resides. For example, asset server 20-602 can host images, CSS, JavaScript, and other related asset files. Stream files can be provided by stream specific servers 20-606, files can be provided by file servers 20-610, and included content from third parties will be available from servers 20-608.

[0171] Fig.21 is a typical embodiment that continues to describe the operation of the geographic destination mechanism, which further explains the retrieval of a series of files from various types of servers, the aggregation of the retrieved files, and the subsequent transmission from the access point server (SRV_AP) 21-300 to the endpoint device (EPD) 21-100.

[0172] After the content files are successfully retrieved by (RFBOT) 21-D328, they are passed to the cache manager 21-D330. Information about each file and a list of all retrieved files are analyzed by the content extraction agent (CPA) 21-D320. This information is passed to the cache manager 21-D330 and the cache manager 21-D130 on the (EPD) 21-100.

[0173] The list of content file information is further shared with the content distribution agent (CDA) 21-D120 on the (EPD) 21-100. The (CDA) 21-120 communicates with the cache manager 21-D130 to anticipate and receive file chunks from 21-700 that is separate from and served separately from the local host on 21-100, thereby serving clients connected to and in close proximity to the (EPD) 21-100.

[0174] In many ways, this operates in contrast to a content delivery network.

[0175] Fig. 22 Prior art relates to how geo-multicast works within a content delivery network (CDN), where a server in one zone S22-00 has content, such as streaming video, that a client located in another zone, such as C 22-20, C 22-22, and / or C 22-24, desires to retrieve. There are bandwidth and latency issues that prevent information from being transmitted over long distances from S22-00 located in one zone to a client in another zone.

[0176] To mitigate the negative impact, the owner of S22-00 can set up a regional multicast server (GS) such as GS22-10 to buffer content from S22-00, or act as a stream reflector for real-time streaming from S22-00 to clients C 22-20, C22-22 and / or C 22-24 efficiently served by GS22-10.

[0177] Fig.23 Analysis and interpretation of the period of a time base cycle is shown. Artificial intelligence can assist future calculations to estimate the processing and post-processing time that a time base will require based on the number of items to be processed, taking into account system resources and other factors.

[0178] Metrics for each period calculated using short-term data, short-term and long-term historical data can be plotted on standard deviation curves based on various cyclic trends. The signs of lows, highs, averages, and other analyses can indicate whether current performance is in line with expectations or is better, worse, or otherwise different than previous experience.

[0179] This data collection and contextual analysis assists artificial intelligence (AI) algorithms in making decisions.

[0180] The current cycle 21 -T08 may be compared with the short cycle 21 -T18 and the long cycle 21 -T28.

[0181] Fig.24

[0046] Described is how descriptions of two different data sets can provide information when comparing the various different data sets, according to certain embodiments of the disclosed subject matter.

[0182] The key point is that while Group 1 has data sets A, B, C, D, and F, Group 2 contains data sets A, C, D, and E. Information from a direct comparison where sets A, C, and D overlap can provide contrast or corroboration of the characteristics of the compared sets.

[0183] And comparison of the results of the B set with the E set can also provide insight into differences and alternatives that can be considered.

[0184] Intelligent algorithms that will be presented with different data sets at different times need to be able to compare sets where overlap occurs between groups, as well as be able to consider how sets in one group but not another are analyzed and weigh that into the results.

[0185] Fig.25 The integration between the multi-border firewall and other systems in the GVN according to a specific embodiment of the disclosed subject matter is shown. The behavior of information warfare always occurs. Whether it is an enterprise, a hacker or other role that launches information warfare, these attacks are cruel, and according to the trend, the threat is increasing. Using the topology described here, there is information related to real-time attacks detected by passive firewalls or other such monitoring middleware integrated on the Internet aggregated and reported by security provider companies or organizations. Regardless of the nature of the attack is an intrusion, a phishing attack, an attempt to steal intellectual property, a DDoDS attack or other known or located threats, the key point is to protect the network.

[0186] Fig.25 Request / response (REQ / RESP) API loops between various devices are shown. These information loops can share information learned by a cloud firewall such as (CFW-DPI) 25-142LB or (CFW-SPI) 25-144LB about the flow through the cloud firewall, which is reported to (SRV_CNTRL) 25-200. The information loop can share information related to attacks in other locations by passing information from (SRV_CNTRL) 25-200 to a cloud firewall such as (CFW-DPI) 25-142LB or (CFW-SPI) 25-144LB. In addition, the information stored in (database Db) 25-B200 on (SRV_CNTRL) 25-200 can also include heuristic patterns, signatures of known threats, and information from global Internet monitoring sources to be shared. Visibility of human administrators can also be obtained from the hosted example via a graphical user interface (GUI) on the client 25-018 via path 25-GUI-AJAX.

[0187] This flexibility in information exchange topology also allows for performance monitoring, cloud-based billing modules for scalable usage of firewall resources, system management, and other purposes.

[0188] A graphical user interface (GUI) hosted on an endpoint device (EPD) 25-100 or a central control server (SRV_CNTRL) 25-200 may provide information regarding the operation of the firewall and also provide point-of-control and click-control surfaces for a user to utilize.

[0189] Fig.26 Two paths are shown in accordance with certain embodiments of the disclosed subject matter: one path is through the open Internet, and the other path is a combination of network optimized traffic and open Internet segments via a global virtual network (GVN) on the underlying Internet.

[0190] One path is a path from P0 - 1 through the open Internet via the local Internet 26 - CPT140 , to the inter-regional Internet 26 - CPT340 , to the local Internet 26 - CPT240 and then hopping to P0 - 14 .

[0191] The other path also jumps from P0-1 to P0-14, but the difference from the previous path is that this path is from the last mile of the over-the-top tunnel to the GVN's (SRV_AP) 26-CPT100, to the long-distance connection via the GVN tunnel 26-CPT300, to the target EIP via the Internet in the remote area 26-CPT200.

[0192] Metrics related to various pieces of the network segment, such as the time measurement Δt=26-TZ ms for a long distance connection via a GVN tunnel 26-CPT300, provide quantifiable values ​​that can be compared. Algorithmic analysis can compare paths such as starting at 26-D00. One analysis of the measured Internet (QoS) 26-D10 can look at the end-to-end open Internet path, while another analysis can look at the measured (GVN QoS) 26-D20. Compare and contrast evaluations are made at connecting the Internet to the (GVN) 26-D30.

[0193] If the open Internet is better (26-DP40), use that path. If the GVN optimized path is better (26-DP50), use that path. This is just one example for illustration purposes and is not limiting.

[0194] Fig. 27 A GVN using a hub-and-spoke topology with backbone and octagonal routing is shown in accordance with certain embodiments of the disclosed subject matter. Fig. 27The network topology of the GVN in two different regions 27-RGN-A and 27-RGN-B is shown, and how these regions are connected by the global connection 27-RGN-ALL via paths 27-POA and 27-POB. In addition, Fig. 27 The hub-and-spoke connections in each of the two regions are shown.

[0195] SRV_BBX 27-280 and SRV_BBX 27-282 are backbone switching servers and provide global connectivity. SRV_BBX can be one or more load balancing servers in a region used as a global link. Access point servers (SRV_AP) 27-302, 27-304, and 27-306 in 27-17-RGN-A are connected to SRV_BBX 27-280. Central control server (SRV_CNTRL) 27-200 provides services for all devices in the region, and it can be one or more master SRV_CNTRL servers. End point devices (EPD) 27-100~27-110 will be connected to one or more SRV_AP servers through one or more concurrent tunnels.

[0196] The figure also shows multiple egress entry points (EIPs) 27-EIP420, 27-EIP400, 27-EIP430, and 27-EIP410 in each region as added spokes to the hub-and-spoke model with paths to and from the open Internet. The topology can provide EPD connections to EIPs in remote regions routed through the GVN. In an alternative, the topology also supports EPD connections to EIPs in the same region, to EPDs in the same region, or to EPDs in remote regions. These connections are security optimized through the GVN.

[0197] Fig.28 A distributed endpoint management cluster (DEPAC) 28-800 is shown in accordance with certain embodiments of the disclosed subject matter, wherein the distributed endpoint management cluster (DEPAC) 28-800 can be virtually implemented in a global virtual network (GVN) 28-000 or other similar topology.

[0198] Traffic between devices is carried within encrypted tunnels, and these tunnels connect devices together into a wider network on the underlying internet.

[0199] Two types of endpoints are described here: an endpoint device (EPD) such as 28-120 behind a local area network (LAN), or a portable endpoint device (PEPD) such as 28-150 behind a personal area network (PAN).

[0200] Other devices described here are access point servers (SRV_AP) such as 28-308 and the central control server 28-200.

[0201] This topology provides security for traffic passing through the tunnel and by hiding the IP address of the device as its traffic goes out to the Internet via a remote egress entry point (EIP).

[0202] The best and most efficient routing for traffic within the GVN 28-100 may be routed through an Advanced Smart Routing (ASR) mechanism.

[0203] Fig.29 The logical structure of a series of tests to be run on the Internet, testing various ports and IP addresses according to a specific embodiment of the disclosed subject matter is shown. It allows a series of ports run on a specific IP address to be specified via data array 29-BT030.

[0204] A counter tracks the tests 29-030. The results of each test are stored in an array 29-090 and then returned to and recorded at step 29-800.

[0205] The algorithm can be used to test the inventory of available IPs and ports allocated to production to test and analyze IP and port performance against the same access point server (SRV_AP) for comparison. It is useful to review the results of the current test with the performance log of the production connection to establish a baseline and evaluate whether the test results are within the expected performance range. In addition, monitor the network connection during the test to ensure that the test does not affect the performance of other connections.

[0206] Fig.30 A mechanism for recording connection traffic activity between various devices operating in a global virtual network (GVN) or other similar topology according to a particular embodiment of the disclosed subject matter is shown. Such devices may include an endpoint device (EPD) 30-100 and access point servers (SRV_AP) 30-300 and 30-320.

[0207] Metered Point of Logging (MPoL) on network interfaces and virtual interfaces as well as tunnel interfaces records inbound and outbound traffic for each device.

[0208] Thus, for example, the tunnel (TUN0) 30-T00 between the (EPD) 30-100 and the (SRV_AP) 30-300 has four such logging points: MPoL (inbound) 30-108 and MPoL (outbound) 30-106 on the (EPD) 30-100 and MPoL (inbound) 30-308 and MPoL (outbound) 30-306 on the (SRV_AP) 30-300.

[0209] There is a direct relationship between the outbound MPoL on a device and the inbound MPoL on the devices to which it is connected.

[0210] Thus, for example, the outbound (MPoL) 30-306 on the (SRV_AP) 30-300 measures the traffic sent to the (EPD) 30-100 and the direct association with the inbound (MPoL) 30-108 via the path 30-P108. The total of the traffic sent can be compared to the traffic received at the other end. These quantities should be equal to each other. If the amount received is less than the amount sent, it represents a loss. From the perspective of fairness of the consumption billing mechanism, if traffic is charged on the (EPD) 30-100, such charging should be based on the traffic received on the inbound 30-108. Conversely, traffic arriving at the (SRV_AP) 30-300 from the (EPD) 30-100 should be charged based on the amount received by the inbound (MPoL) 30-308 on the (SRV_AP) 30-300.

[0211] If there is a situation where the traffic received on an inbound MPoL is greater than the traffic sent to its corresponding outbound MPoL, this may indicate a bug or malfunction or data injection by a malicious player in the middle.

[0212] The difference between the bandwidth (BW) charging model and the consumption (CONS) charging model is that in the bandwidth model 30-1008, traffic can be shaped to an amount less than the carrying BW capacity of the line 30-1002. In contrast, in the consumption-based model, the BW is not shaped, and the maximum bandwidth 30-1086 may actually exceed the rated BW carrying capacity of the line 30-1002.

[0213] The advantage of BW Model 30-1008 is unlimited data flow up to the amount in Mbps of the paid BW tunnel. The cost of doing so is relatively high, and actual utilization is usually only at certain times, while capacity is wasted at other times.

[0214] The advantage of CONS model 30-1088 is that the tunnel is not throttled, so the line BW carrying capacity can be reached or even exceeded. This model is advantageous because data will be transmitted as quickly as possible and billing is based on pay-as-you-go. And paying by the gigabyte GB transmitted, this model can provide consumers with a low entry barrier.

[0215] Fig.31A traditional thick stack 31-20 layered access between an application layer 31-800 and a network interface card (NIC) 31-000 is shown compared to a thin stack approach 31-10 of direct access to the NIC 31-000 in accordance with certain embodiments of the disclosed subject matter.

[0216] The advantage of a thin stack over a thick stack is that a thin stack is faster, consumes fewer resources, and can therefore handle relatively more network connection interactions.

[0217] Fig.32 A simple exemplary database schema is shown for correlating and storing connection information from both test and production environment usage.

[0218] Those skilled in the art will see the advantages of this. Tables, fields, labels, and other information are provided for reference only and may differ in various production deployments.

[0219] The server availability 32-280 module examines the current allocation of IP address and port combinations, taking into account actual usage, load and other factors of the live production environment. The server availability 32-280 module further analyzes the potential impact of allocated but standby configurations to predict the impact of these real-time resource requirements.

[0220] When a device sends a request to the server availability module, what it is really requesting is a list of available servers, specifically the IPs and ports that the device can use to establish tunnels currently or in the future. Each tunnel has specific information that each pair in the tunnel establishment process needs to know, as defined in Table 32-220 based on tunnel information.

[0221] The server availability list is associated with the connection information used to create the tunnel 32-210 through the device_ID 32-P288 and the device_type_ID 32-P220.

[0222] Peer_source_ID refers to the ID of the originating device that creates the tunnel.

[0223] Peer_Target_ID refers to the ID of the listening device for which the tunnel will be created. In most cases, the Device_ID field in array 32-BT020 is equivalent to the Peer_Source_ID field in 32-BT020.

[0224] Therefore, the server availability column is a list that all devices can utilize, but for all devices, the context list for that device is easily retrieved using the context record associated with one of the specific devices. The field "Flag Status" in array 32-BT020 is important because it can indicate the status of this information, if not used, then in production, the information may be returned, blocked, or otherwise marked.

[0225] The server availability mechanism 32-280 takes into account many factors to balance the client's demand for resources, such as the EPD's desire to establish a tunnel to obtain available resources of the SRV_AP to which it will connect.

[0226] This management mechanism ensures that automatic jumping of tunnels from many EPDs to a specific device does not occur. For example, if all EPDs share the same server availability list without affecting how the EPDs choose which servers to connect to, there are scenarios where there are too many EPDs connected to some SRV_APs and too few EPDs connected to other SRV_APs. And when the EPDs connected to the oversaturated SRV_AP realize that the connection they are receiving is less than ideal, if these EPDs share the same list, they will completely cause unstable changes by jumping to the next server on the list.

[0227] Server Availability addresses such scenarios by contextually ranking the best SRV_AP for that particular EPD, taking a holistic view when generating the list and predictively spreading the load in anticipation of demand pressure on the production environment.

[0228] Fig.33 The use of custom commands that act as a defensive buffer between a trigger at the application layer, such as a button clicked on a graphical user interface, and the deeper logic and commands to be run as a result of the trigger, are shown in accordance with certain embodiments of the disclosed subject matter.

[0229] The deeper a trigger's code execution can penetrate, the greater the risk of damage to the system. This is a difficult problem because, on the one hand, unprivileged clients should not have deep access. However, some scripts that a user wants to trigger will require deep research or will be considered invalid and will not take effect when executed.

[0230] The diagram has sufficient labeling so that a reader with reasonable skill in the art will be able to understand the flow of information.

[0231] Step "Execute custom command" 33-200 assumes that the triggering user has limited permissions only for custom commands 33-L20 execution level and not below. These limited permissions exclude the ability to run sensitive commands, access deep folders or files, or otherwise compromise or penetrate deeper into the system. The path 33-P150 to "Custom command has permission to call deeper scripts and commands 33-D 150" defines a series of steps that will allow a specific custom command with an intended specific function to be run in a safe and secure manner.

[0232] At "O / S Install Package + Configuration" 33-12, embedded commands within custom commands may be able to trigger control of deep O / S level functions, but in a controlled manner. The lowest possible level of commands may also be triggered, but in a controlled manner and based only on the content of the untampered custom command.

[0233] Thus, the triggering user cannot dig deeper than 33-L20 and does not have permission to call sensitive commands, however, by calling a custom command that the triggering user is allowed to access, the contents of the custom command file can be run in an elevated privilege state without compromising the security of the system, and only the results of the call and potentially the custom command are available to the triggering client 33-800.

[0234] Fig.34 is a simplified description of modules on relevant devices in a geographic destination mechanism within a global virtual network (GVN) according to certain embodiments of the disclosed subject matter. The figure depicts the interaction between a content distribution agent (CDA) and a content extraction agent (CPA).

[0235] Although CDA and CPA and associated modules are designed to operate independently, there is a flow of information between them that triggers their respective operations.

[0236] For example, the ASR module on the (EPD) 34-D120 will be able to suggest to the (CDA) 34-D110 that a URL will be obtained from a remote region where the (SRV_AP) 34-300 is located. The (CDA) 34-D110 will communicate to the (CPA) 34-D310 via the path 34-CP310 that the (CDA) 34-D110 expects the geographic destination mechanism to obtain the content and all associated content at the URL from the remote region.

[0237] (CPA) 34-D310 instructs remote fetcher BOT (RFBOT) 34-D312 to do a lookup at 34-D350 to find the exact target server and have (RFBOT) 34-D312 fetch the content. (CPA) 34-D310 resolves the content of the embedded link to include the (CPA) 34-D310 and then requests (RFBOT) 34-D312 to fetch more content.

[0238] The (CPA) 34-D310 lists the subsequently extracted content for two purposes. The first purpose is to send the list and the retrieved data to the cache manager 34-D330, and send a copy of the list back to the (CDA) 34-D110.

[0239] This collaboration is useful because the (CDA) 34-D110 is able to coordinate with the cache manager 34-D130 to expect receipt of acquired data and can compare the received data to the manifest created by the (CPA) 34-D312. There are efficiency gains achieved through this controlled approach as well as security advantages because the received data must match the manifest or a red flag may trigger a review for possible malfunction or a malicious man-in-the-middle type data modification attack.

[0240] (SRV_CNTRL) 34-200 may be used as a neutral channel for information transfer between devices, and may also be used for overall management of chained caches on various devices.

[0241] Fig.35 Several ways are shown of how a portable endpoint device (PEPD) such as 35-150 or 35-152 may be connected to a global virtual network (GVN) 35-000 or similar topology in accordance with certain embodiments of the disclosed subject matter. In this example, (PEPD) 35-150 has multiple connection options to three different devices. The first device reaches a second different access point server (SRV_AP) 35-304 via paths 35-T10B to 35-T10A in an area close to it, and reaches (SRV_AP) 35-300 in another area via paths 35-T12A to 35-T12B. The third option for (PEDP) 35-150 is to reach (EPD) 35-100 via 35-T12A to 35-T12C.

[0242] The PEPD may connect to as many SRV_AP servers as it has resources and permissions as permitted by the server availability mechanism. However, for (PEPD) 35-150 to connect to (EPD) 35-100, there must be a relationship between them such as would exist if both belonged to the same Distributed Endpoint Area Cluster (DEPAC), and the PEPD is granted the right to connect to that EPD.

[0243] Fig.36 The relationship between a portable endpoint device (PEPD) 36-150 and an access point server (SRV_AP) 36-300 is shown in accordance with a particular embodiment of the disclosed subject matter. The figure also describes some of the modules on each of the above devices. The amount of physical resources such as processing power 36-106, RAM 36-102, storage 36-H100, and other system attributes of the (PEPD) 36-150 will typically be relatively small. The (SRV_AP) 36-300 can be a cloud-based dynamically virtualized instance that provides a large amount of (RAM) 36-302, processing power 36-306, storage 36-H300, and other resources.

[0244] The (PEPD) 36-150 will typically have a minimal operating system 36-110, a simple security suite 36-114, minimized system software 36-120, and typically have a small footprint. This barebones approach ensures that there is a balance between the required functionality and other relevant factors such as cost, battery consumption, physical size, etc.

[0245] One solution is for a virtual software container 36-160 to be available on a (PEPD) 36-150 that can download and run software locally, or can serve as a hook point for symbolic connections between a local system and software running at a remote location such as a (SRV_AP) 36-300.

[0246] This arrangement is possible and protected by a tunnel TUN0 between the two devices, through which not only data traffic can flow, but also software commands, queries, results and other functions that can be run remotely.

[0247] There are several advantages to this approach as described above, but there may also be several advantages to other approaches.

[0248] Fig.37 Some different types of extensible software that may be run on a portable endpoint device (PEPD) 37-150 in accordance with a particular embodiment of the disclosed subject matter are shown, wherein the software is stored on a remote device such as an access point server (SRV_AP) 37-300 for transmission from 37-158 to the (PEPD) 37-150.

[0249] The first type is the core software stored and run on the (PEPD) 37-150, such as software 37-1520 downloaded and executed locally. The other type is the distributed software 37-1300 running remotely, where the distributed software 37-1300 ties the (PEPD) 37-150 to the (SRV_AP) 37-300 running the software.

[0250] This mechanism has many advantages, including efficient use of PEPD resources and ensuring that the latest version of the software is running.

[0251] Another element is protecting the intellectual property of software producers, since some software is physically in one location and runs with results used by software hook points in another location. In this topology, both devices will be accessed to have visibility into the full framework, code base, and runtime properties.

[0252] Fig.38 Multiple tunnels between devices within a global virtual network (GVN) across multiple regions according to a specific embodiment of the disclosed subject matter are shown. (EPD) 38-100 is located at a location 38-MO. SRV_APs in region 38-M2 include (SRV_AP) 38-300, (SRV_AP) 38-302, and (SRV_AP) 38-304. SRV_APs in region 38-M3 include (SRV_AP) 38-310, (SRV_AP) 38-312, and (SRV_AP) 38-314. Advanced Intelligent Routing (ASR) is used to manage routing through multiple tunnels and paths between (EPD) 38-100 and various (SRV_AP) devices. ASR can mitigate the risk of loops, incorrect geographic destination routing, ASR remote redirect backtracking, SRV_AP, link disconnection between regions, etc.

[0253] Fig.39 A framework is shown of a common code base to be utilized by various types of devices, each with its own attributes / type and identity, in accordance with certain embodiments of the disclosed subject matter.

[0254] The hardware UUID 39-110 utilized by the device identity module 39-100, associated with the device related record stored in the database 39-B100, and calling the identity class 39-120, sets a number of different contextual relationships for that particular device.

[0255] Within the common code framework, by knowing its own identity from several sources via 39-100, a device can gather many things, such as its role 39-310, its type 39-300, which elements of the base code 39-200 are relevant to the device, and which applications 39-800 the device should load and execute.

[0256] The logical next step is that with the device type 39-300 known, device type specific modules 39-320 may be loaded, and these device type specific modules 39-320 along with the device role 39-310 may trigger O / S specific modules 39-500 to be loaded to maintain operation for that type of device.

[0257] The device type 39-300 and role 39-310 also have an impact on peer-to-peer relationships 39-520 and operations and integration 39-600.

[0258] Once the environment specific to the device is running, the application 39-800 may run in context for that particular device.

[0259] This should be apparent to a reader with sufficient skill in this area.

[0260] Fig.40 The relationships, interactions, and correlations of data with various devices within a global virtual network (GVN) are shown in accordance with certain embodiments of the disclosed subject matter.

[0261] The port and IP address manager 40-200 will use various data regarding the tunnels 40-270, servers 40-220, and other factors to allocate resources for the server availability 40-260 to then utilize and issue to various devices.

[0262] There is a logical relationship between data, such as a list of devices 40-110 and servers 40-220, and how the modules interact with each other to use this information.

[0263] The internal subnet manager [EPD_subnets] + [SRV_AP subnets] + [Infrastructure subnets] 40-300 uses this information to calculate subnet ranges that can be used while avoiding conflicts, and can also calculate "reserved" ranges to ensure there are no future conflicts either in automatic setup or in sending messages to administrators regarding usage.

[0264] The Global Virtual Network (GVN) relies on various devices that work together to weave together the types of networks on the Internet. The registry 40-B100 tracks the devices, their relationships, their specifications, their types, and other factors. The core elements of the system include modules and managers that work together to construct and efficiently manage the GVN structure.

[0265] The Advanced Intelligent Routing (ASR) manager 40-500 looks at the underlying Internet connections and the networks within the GVN layer on the Internet. In order for the automation system to run efficiently and effectively, the port and IP address manager 40-200 and the internal subnet manager 40-300 interact with various other modules.

[0266] Server availability 40-260 looks at the tunnel 40-270, its IP and port usage 40-276, checks connection tests 40-158 (current and historical), and calculates port availability based on the usage of each IP address assigned to the server 40-122, the device 40-210, and other factors to manage how server resources are utilized and allocated.

[0267] The main benefit of the server availability module is to catalog the resource usage, IP addresses and ports of the access point server (SRV_AP) and see which devices 40-210 (such as endpoint devices (EPDs)) want to connect to the SRV_AP to establish a tunnel (TUN) 40-270 between them. The server availability module calculates and compiles a list of customized IP addresses and ports that are designed in context to provide the best options for the EPD to utilize. The tunnel registry 40-150 allows the server availability check Db: connection test 40-158 and weighs and ranks the best connection options for establishing tunnels 40-270 and routing via the ASR 40-500 based on current and historical conditions.

[0268] The Distributed Endpoint Administration Cluster (DEPAC) 40-250 is an extension of the GVN fabric. The DEPAC allows the Local Area Network (LAN) and Personal Area Network (PAN) subnets behind the EPD and Portable Endpoint Devices (PEPD) 40-210 to be linked into a wider network of connections by connecting them together. The internal subnet manager needs to actively understand the EPD subnets, the tunneled SRV_AP subnets, and other related infrastructure subnets 40-300 so that when the internal subnet manager allocates new subnets and interweaves the edges of one of the subnets with another as part of a wider fabric, the internal subnet manager ensures that there are no conflicts in overlapping allocations of subnets by understanding those subnets that are currently in use.

[0269] The Db 40-120 for IP address allocation provides information to be used by the internal subnet manager 40-300. The DEPAC cluster 40-250 has member devices 40-210, and each device has its own unique identity 40-212.

[0270] There is also a relationship between the user 40-280 and the device 40-210 based on the device identity 40-212 managed by the identity class or similar logic 40-282. Thus, the user can manage the DEPAC cluster.

[0271] The described topology allows for both automatic and manually assisted construction of tunnels 40-270, with efficient routing due to the ASR manager 40-500 and other factors.

[0272] If a device fails, requires an update, is oversubscribed, or is in a suboptimal state, the device flag 40-112 of the device may be set to a device flag other than "available" so that the server availability module 40-260 knows that the device is not available for assignment to new server availability lists, where these new server availability lists will be published for other devices to utilize.

[0273] Another benefit is that analysis of historical usage can be compared with current conditions and AI algorithms used to make decisions related to routing, resource allocation, etc.

[0274] By accessing data in the database registry 40-B100 and displaying it in the GUI, centralized real-time monitoring of connections, resource usage, and other factors can be achieved.

[0275] Administration is facilitated by using the information provided by GVN modules and managers to establish and manage automated systems for construction elements.

[0276] Deep analysis can be performed on traffic, paths set by ASR, server availability 40-260, and other modules not shown here. This not only helps with visibility into past and current usage, but can also be used as the basis for predictive cognitive analysis of how future connections will perform.

[0277] Fig.41 The topology of a global virtual network (GVN) on the Internet and various devices operating within the topology are shown according to certain embodiments of the disclosed subject matter. Some examples of devices include an endpoint device (EPD) such as (EPD) 41-110, a portable endpoint device (PEPD) such as 41-152, an access point server (SRV_AP) such as 41-308, and a central control server (SRV_CNTRL) such as 41-200.

[0278] There are also egress entry points (EIPs) such as 41-318 between GVN devices such as (SRV_AP) 41-308 and the Internet such as 41-018.

[0279] exist Fig.41 In the , there is another layer representing two independent distributed endpoint management clusters DEPAC0 and DEPAC2.

[0280] The members of DEPAC0 are (PEPD)41-150, (EPD)41-102 and (EPD)41-106.

[0281] The members of DEPAC2 are (EPD)41-100, (PEPD)41-152, (EPD)41-110 and (EPD)41-108.

[0282] DEPAC members are connected to each other through existing GVN paths and only connect subnets together, as well as internal tunnels connecting local area networks (LANs) and personal area networks (PANs) into the wider wide area network (WAN).

[0283] Fig.42 Code located above web root directory 42-Bound2 and below web root directory 42-BoundO is shown in accordance with certain embodiments of the disclosed subject matter.

[0284] In files within folders that are directly accessible from the open Internet below the web root 42-Bound0, as little code as possible should be stored.

[0285] Sensitive files including configuration files, classes, functions, credentials, sensitive information and other files are stored above the web root directory 42-Bound2.

[0286] While one might think that the server-side script package (parser) 42-D110 should process sensitive code within files when retrieving them, there are situations where a malfunction or configuration error or hacker attack could allow a file to be downloaded, thereby directly viewing embedded code that should have been executed.

[0287] The only server-side code included in the files stored below should be relative links to files above the web root directory 42-Bound4.

[0288] The terms above and below refer to file paths such as:

[0289] / volume / folder / instance / code / secure / ***secure folders and paths here

[0290] / volume / folder / instance / code / public / http / ALL_here_is_on_open_internet_HTTP

[0291] / volume / folder / instance / code / public / https / ALL_here_is_on_open_internet_HTTPS

[0292] The web root directory in the above example is:

[0293] / volume / folder / instance / code / public / http /

[0294] / volume / folder / instance / code / public / https /

[0295] Any content on these web directories can only be accessed from within the server and not from the internet. All files and folders within http and https folders are accessible.

[0296] Fig.43 An algorithm for merging information from external sources and internally learned AI knowledge is shown in accordance with certain embodiments of the disclosed subject matter.

[0297] Step "Begin checking information accuracy" 43-000 checks the information by source and type.

[0298] The defined patterns 43-102 may be validated by 43-P108 to become rules. Warnings 43-122 based on information from other sources may be ignored, or these warnings may also become rules 43-112.

[0299] Fig.44 A simplified representation of a method for designing smarter algorithms by considering owned factors and balancing individual needs in accordance with a particular embodiment of the disclosed subject matter is shown. In this "juice" example, the step of "mixing together 44-130" can be described as a process that is of great concern to achieve a desired result. "Analysis of satisfied needs 44-140" determines if the best result is achieved if the process is run again with slightly different inputs and in a way that is closer to the desired result.

[0300] In some embodiments, based on the need and need for optimal revenue, elements that are not incorporated into the final processed product but still have an impact on the outcome may be utilized during processing. Expanding on the example of juice, citrus peels may perk up a person (e.g., the maker of the juice) when the fruit is peeled, but are ultimately discarded and not used. Another example is that both apples and oranges are great for a person's health, but citrus may not be ideal for a person with an acidic condition.

[0301] The key point is to make the algorithm flexible and take into account various conditions, processing, inputs and outputs, thereby taking a comprehensive evaluation approach.

[0302] Fig.45Some system modules and components of an endpoint device (EPD) 45-100, a central control server (SRV_CNTRL) 45-200, and an access point server (SRV_AP) 45-300 are shown in accordance with a particular embodiment of the disclosed subject matter. The figure also shows a database 5100 on the (EPD) 45-100, a database 5200 on the (SRV_CNTRL) 45-200, a repository database 5202 on the (SRV_CNTRL) 45-200, and a database 5300 on the (SRV_AP) 45-300. The figure is hierarchical, with the lowest level hardware devices at the bottom, and subsequent systems, components, modules, and managers built on the lower levels. Files and data are stored on Hierarchical File System (HFS) attached storage devices 45-H100 on EPD 45-100, 45-H200 on (SRV_CNTRL) 45-200, and 45-H300 on (SRV_AP) 45-300. The components shown in these system diagrams all operate independently, but may also rely on information about other devices with which they interact.

[0303] RAM stands for Random Access Memory, CPU stands for Central Processing Unit (which may also include subprocessors), NIC stands for Network Interface Card, Db stands for Database Software, DNS stands for Domain Name System, HOST stands for Hosting Software, API stands for Application Programming Interface, ASR stands for Advanced Smart Routing, GeoD stands for Geolocation, GVN stands for Global Virtual Network, CDA stands for Content Distribution Agent, CPA stands for Content Extraction Agent, and RF BOT stands for Remote Fetcher Bot. There may also be other modules, managers, systems, or software components.

[0304] Fig.46 Some system modules and components of an endpoint device (EPD) 46-100, a central control server (SRV_CNTRL) 46-200, and an access point server (SRV_AP) 46-300 are shown in accordance with a particular embodiment of the disclosed subject matter. The figure further identifies subsystems of each device, such as the EPD subsystem 46-1000 of the (EPD) 46-100, the SRV_CNTRL subsystem 46-2000 of the (CNTRL) 46-200, and the SRV_AP subsystem 46-3000 of the (SRV_AP) 46-300. The subsystems are identified by function and represented by prefixes, including: FW represents firewall-related subsystems, TUN represents tunnel-related subsystems, VIF represents virtual interface-related subsystems, SRV_Avail represents server availability list and related subsystems, BUFF Mgr represents buffer management and related subsystems, LOG represents logging module and related subsystems, and CONNECTIVITY represents general connection operations.

[0305] Fig.47 Some system modules and components of an endpoint device (EPD) 47-100, a central control server (SRV_CNTRL) 47-200, and an access point server (SRV_AP) 47-300 according to a specific embodiment of the disclosed subject matter are shown. Subsystems are identified by function and represented by prefixes, including: Connectivity represents general connection operations, ASR represents advanced intelligent routing, API represents application programming interface, LOG represents logging module and related subsystems, GeoD represents geographic destination module and related subsystems, SRV_Avail represents server availability list and related modules, Buffer represents buffer management and related modules.

[0306] Fig.48 A block diagram of a typical computing device 4800 according to a specific embodiment of the disclosed subject matter is shown. In some embodiments, the computing device 4800 can be any suitable device disclosed in the present invention. The computing device 4800 may include at least one processor 4802 and at least one memory 4804. The processor 4802 can be hardware configured to execute computer-readable instructions such as software. The processor 4802 can be a general-purpose processor or dedicated hardware (e.g., an application-specific integrated circuit (ASIC), a programmable logic array (PLA), a field programmable gate array (FPGA), or any other integrated circuit). The processor 4802 can execute computer instructions or computer codes to perform the required tasks. The memory 4804 can be a temporary computer-readable medium or a non-temporary computer-readable medium, such as a flash memory, a disk drive, an optical drive, a programmable read-only memory (PROM), a read-only memory (ROM), a random access memory (RAM), or any other memory or a combination of memories.

[0307] In some embodiments, memory 4804 includes module 4806. Processor 4802 may be configured to execute module 4806 stored in memory 4804, which module 4806 is configured to cause processor 4802 to perform various steps discussed throughout the disclosed subject matter.

[0308] Fig.49 The process of downloading files in a LAN and from the Internet is shown in accordance with certain embodiments of the disclosed subject matter.The figure compares and contrasts the files served by the EPD at the LAN edge 49-880 with the files served by the remote server 49-840.

[0309] Downloading files within the LAN 49-880 provides comprehensive control and filtering of the firewall as well as relatively high bandwidth (BW), ultra-low latency, and few hops.

[0310] Downloading from a remote server on the open Internet provides little control over the intermediate paths, and there may also be filtering from firewalls or compression due to intermediary devices.

[0311] By optimizing the traffic in the middle, many issues are solved. However, download BW is still limited by the last mile connection or the smallest peer or network segment.

[0312] Therefore, in most cases, downloading file 49-810 from a remote server from the remote Internet 49-050 will be slower than downloading file 49-800 from the (LAN) 49-020.

[0313] Fig.50 Operation of a file transfer manager (FTM) 50-360 on an endpoint device (EPD) 50-100 and an access point server (SRV_AP) 50-300 are shown in accordance with certain embodiments of the disclosed subject matter.

[0314] The client 50-000 initiates an upload transmission of a file 50-800 to the server 50-350, wherein the file 50-800 is transmitted via path 50-P000 to the LAN 50-020, to the EPD 50-100, which routes the traffic over an optimized long distance 50-980 through the global virtual network (GVN) 50-020 to the SRV_AP 50-300, to 50-P050, to the Internet 50-050 in the remote region, and to the server 50-350 via path 50-P350. The server 50-350 accepts the connection from the client 50-000 and begins processing the file received in the multi-part attachment sent within the packetized payload stream.

[0315] For files larger than a certain size set by the threshold parameter variable, the original upload stream of the file 50-800 from the client 50-000 is detected at step 150-10 by (FTM) 50-160, and then the file stream is captured and extracted into the file cache 50-180 on the EPD 50-100. Since the network path is an extremely short distance with ultra-low latency 50-920, the remaining data in the file upload is uploaded very quickly, thereby reducing the upload time from the client's perspective. This has the advantage of freeing up the client. (FTM) 50-160 maintains the upload stream to the server 50-350 and sends the data from the file cache 50-180 as it is received, and after the complete file has been received, (FTM) 50-160 continues to upload on behalf of the client until the upload is complete.

[0316] At step 1 50-10, the original stream is detected and the stream is considered valid only after the server accepts the connection and opens a valid socket.

[0317] At step 2 50-12, the upload stream is redirected to the local file cache 50-180, thereby accumulating at the local file cache 50-180 at the high rate of the (LAN) 50-020. The (FTM) 50-160 will also continue to send the upload stream to the server 50-350 via the optimized GVN topology, including relaying to the file cache 50-380 on the (SRV_AP) 50-300 in the remote area in close proximity to the target server 50-350 at step 350-28.

[0318] The (FTM) 50-360 on the (SRV_AP) 50-300 remains connected and sends the file 50-810 to the server 50-350 as quickly as possible via the path 50-P350. Step 5 50-32 notes that this is where the connection remains active and healthy, and where the file 50-810 will come out of the GVN with a short distance and low latency 50-950 between the (SRV_AP) 50-300 and the server 50-350.

[0319] Fig.51 The relationship between the physical storage of files on the secure file storage capacity 51-H750 according to a specific embodiment of the disclosed subject matter, the storage of information related to the files into a relational database (DB) such as 51-B700, and the data relationship between the files, file information, and the steps of the file transfer process are shown. Files are sent from a client 51-000 to a server 51-350 by storing the file in a file cache 51-180 close to the source file, pushing the file to the next file cache 51-380, and then saving the file to a server 51-380 managed by a file transfer manager (FTM) 51-360 via upload stream capture and chain caching of files. Multidimensional array 51-BT030 describes information related to the file (File_lnfo), the path (File_Path) through the GVN that the file should take, as well as metadata (File_Meta) and destination information (Dest_lnfo). The destination information includes the target device type, credentials (if necessary), and other information.

[0320] The sub-arrays of the file path multidimensional array 51-BT050 include information related to the source, the destination, and the intermediate hops.

[0321] The sub-array describes the characteristics of each hop 51BT-070, including information about the device, its type, code or its host name, and other information. This sub-array is also associated with Table: Device 51-730.

[0322] Table: File 51-720 contains information related to the file, including a universally unique identifier of file 51-722. The file name on the physical storage may be UUID 51-722, symbolically linked to the file information record containing the file name through Table: UUID file 51-720.

[0323] It should be understood that in this application, the disclosed subject matter is not limited to the details of the structure, but is limited to the arrangement of the components set forth in the following description or shown in the accompanying drawings. The disclosed subject matter can have other embodiments and can be practiced and executed in various ways. In addition, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered as limiting.

[0324] As such, those skilled in the art will appreciate that the concepts upon which the present disclosure is based may readily be utilized as a basis for designing other structures, systems, methods, and media for carrying out the several purposes of the disclosed subject matter. It is important, therefore, that the claims be regarded as including such equivalent structures as long as they do not depart from the spirit and scope of the disclosed subject matter.

[0325] While the disclosed subject matter has been described and illustrated in the foregoing exemplary embodiments, it should be understood that the present disclosure is made by way of example only and that many changes may be made to the implementation details of the disclosed subject matter without departing from the spirit and scope of the disclosed subject matter, which is limited only by the following claims.

Claims

1. A system for providing geographic destination services to clients based on a global virtual network GVN, comprising: an endpoint device, the endpoint device being connected to the client, the endpoint device comprising a content distribution agent, the content distribution agent receiving a request for obtaining a geographic destination service from the client; an access point server comprising a content extraction proxy, a uniform resource locator lookup and domain name system cache manager, a remote fetcher robot, and a cache manager, wherein the content extraction proxy communicates with the uniform resource locator lookup and domain name system cache manager and coordinates the operation of the content extraction proxy with the cache manager; and a central control server for a neutral channel for information transmission between the endpoint device and the access point server; The content extraction agent receives instructions from the content distribution agent to instruct the remote acquirer robot to acquire content files from the content server, and the acquired content files are provided to the client, and the remote acquirer robot passes the acquired content files to the content extraction agent for parsing, so as to find content file links embedded in the acquired content files.

2. The system according to claim 1, wherein: The content extraction agent analyzes information related to content files and creates a manifest of all content files retrieved by the content extraction agent, and shares the manifest with the content distribution agent so that the content distribution agent compares the received data with the manifest.

3. The system according to claim 1, wherein: The content extraction agent receives the instruction to obtain the content file from the uniform resource locator from the content distribution agent, and transmits the instruction to the remote obtainer robot to refer to the uniform resource locator to determine whether the domain name system information of the uniform resource locator is cached in the uniform resource locator lookup and domain name system cache manager.

4. The system according to claim 3, wherein: If the uniform resource locator lookup and domain name system cache manager has cached the domain name system information of the uniform resource locator and the domain name system information is not outdated, the cached domain name system information is used, otherwise, additional domain name system information is looked up from a domain name system server.

5. The system according to claim 1, wherein: The content extraction agent cooperates with the URL lookup and DNS cache manager and the remote fetcher robot to obtain information about the network coordinates of the server where the content file corresponding to the content file link resides.

Images