Hybrid Software-Defined Networking and Mobile Ad-hoc Networking Routing in Mesh Networks

Abstract

A network controller (300) is configured to enable the network (100) to implement a primary network configuration (700) of the network and a secondary network configuration (800) as a backup for the primary network configuration. The network controller may be configured to receive information from a plurality of nodes (107, 110) of the network and information related to client data to be transmitted through the network. Based on the node information, the network controller is configured to determine available nodes and possible links within the network, and then determine the topology (500) of the network. The primary network configuration is determined based on the topology. Then, the network controller implements the primary network configuration and sends commands to a plurality of nodes of the network to switch to the secondary network configuration in the event of a failure of the primary network configuration, wherein the secondary network configuration implements mobile ad-hoc networking in the determined topology.

Description

Technology Field

[0001] Cross-reference regarding related applications

[0002] This application claims priority and interest to U.S. Application No. 17 / 138,019 filed December 30, 2020, the entire disclosure of which is incorporated herein by reference. Background Technology

[0003] Information can be transmitted via directional point-to-point networks, such as aerospace and other mobile networks. In these networks, links can be formed between pairs of nodes by aiming the transceivers of each pair of nodes toward each other. In some implementations, nodes may include non-geostationary satellite orbit (NGSO) satellites or other high-altitude platforms (HAPs) moving relative to the Earth.

[0004] Aspects of the present disclosure provide systems and methods for planning a primary network configuration and a secondary network configuration for a network, wherein the secondary network configuration serves as a backup for the primary network configuration.

[0005] Some embodiments of the present disclosure provide a system having a network controller comprising one or more processors. The one or more processors receive node information from a plurality of nodes of a network, wherein the plurality of nodes includes a first node that moves toward a second node; determine available nodes and possible links within the network based on the received node information; determine a topology of the network based on the available nodes and possible links; receive client data information from one or more client devices regarding client data to be transmitted over the network; determine a primary network configuration for the determined topology, wherein the primary network configuration includes one or more first routing paths for the client data; and transmit implementation instructions to the plurality of nodes that cause the plurality of nodes of the network to implement the primary network configuration and, in the event of a failure of the primary network configuration, to switch to a secondary network configuration, wherein the secondary network configuration includes one or more second routing paths for the client data through the determined topology that implements mobile ad-hoc networking.

[0006] In one example, implementation instructions also cause each node of the network to determine the next hop in the secondary network configuration using mobile ad-hoc networking in the determined topology. In another example, one or more processors are also configured to determine a first forwarding rule for each node of the network based on one or more first routing paths; and implementation instructions include the first forwarding rule. In this example, the first forwarding rule optionally includes instructions regarding forming a link in the primary network configuration.

[0007] Additionally, optionally, in this example, implementation commands for each node of the network include switching to a second forwarding rule for a second network configuration when the first forwarding rule cannot be implemented. In one example, the second forwarding rule is for proactive routing. In another example, the second forwarding rule is for reactive routing. In an additional example, the second forwarding rule is for hybrid routing. In yet another example, the implementation commands further include detecting a link failure for the next hop before switching to the second forwarding rule.

[0008] Other aspects of the present disclosure provide a computer-implemented method comprising: receiving node information from a plurality of nodes of a network by one or more processors, wherein the plurality of nodes includes a first node that moves toward a second node; determining available nodes and possible links within the network by one or more processors based on the received information; determining a topology of the network by one or more processors based on the available nodes and possible links; receiving client data information related to client data to be transmitted through the network from one or more client devices by one or more processors; determining a primary network configuration for the determined topology by one or more processors, wherein the primary network configuration includes one or more first routing paths for the client data; and transmitting implementation instructions to a plurality of nodes of a network by one or more processors, wherein the primary network configuration includes one or more second routing paths for the client data through a determined topology that implements mobile ad-hoc networking.

[0009] In one example, the implementation instructions also cause each node of the network to determine the next hop in the secondary network configuration using mobile ad-hoc networking in the determined topology. In another example, the step of determining the primary network configuration further includes the step of determining a first forwarding rule for each node of the network based on one or more first routing paths; the implementation instructions include the first forwarding rule. In this example, the first forwarding rule includes instructions regarding forming a link in the primary network configuration.

[0010] Additionally, optionally, in this example, implementation commands for each node of the network include switching to a second forwarding rule for a second network configuration when the first forwarding rule cannot be implemented. In one example, the second forwarding rule is for proactive routing. In another example, the second forwarding rule is for reactive routing. In an additional example, the second forwarding rule is for hybrid routing. In yet another example, the implementation commands further include detecting a link failure for the next hop before switching to the second forwarding rule.

[0011] Further embodiments of the present disclosure provide a non-transient, tangible computer-readable storage medium in which computer-readable instructions of a program are stored. When the instructions are executed by one or more processors, one or more processors perform a method. The method comprises the steps of: receiving node information from a plurality of nodes of a network, wherein the plurality of nodes includes a first node moving toward a second node; determining available nodes and possible links within the network based on the received information; determining a topology of the network based on the available nodes and possible links; receiving client data information from one or more client devices regarding client data to be transmitted over the network; determining a primary network configuration for the determined topology, wherein the primary network configuration includes one or more first routing paths for the client data; and the method includes the step of sending implementation commands to multiple nodes of a network to enable them to implement a primary network configuration and to switch to a secondary network configuration in the event of a failure of the primary network configuration - the secondary network configuration includes one or more second routing paths for client data through a determined topology that implements mobile ad-hoc networking -

[0012] In one example, the step of determining the primary network configuration also includes the step of determining a first forwarding rule for each node of the network based on one or more first routing paths; the implementation instructions include the first forwarding rule and cause each node of the network to determine the next hop in the secondary network configuration using mobile ad-hoc networking in the determined topology. Brief explanation of the drawing

[0013] FIG. 1 is a pictorial drawing of an exemplary directional point-to-point network (100) according to embodiments of the present disclosure. FIG. 2 is a functional diagram of a part (200) of the network (100) illustrated in FIG. 1 according to embodiments of the present disclosure. FIG. 3 is a functional diagram of a network controller (300) according to embodiments of the present disclosure. FIG. 4 is a flowchart (400) of a method according to embodiments of the present disclosure. FIG. 5 is a functional diagram of the topology (500) of the network (100) illustrated in FIG. 1 according to embodiments of the present disclosure. FIG. 6 is another functional diagram of the topology (500) of the network (100) illustrated in FIG. 1 according to embodiments of the present disclosure. FIG. 7 is a functional diagram of a primary network configuration (700) of a network (100) according to embodiments of the present disclosure. FIG. 8 is a functional diagram of a secondary network configuration (800) of a network (100) according to embodiments of the present disclosure. FIG. 9 is a flowchart (900) of a method according to embodiments of the present disclosure. Specific details for implementing the invention

[0014] survey

[0015] The present invention relates to planning and implementing network configurations for a temporal communication network, comprising a primary configuration and a secondary configuration. The primary configuration may be a software-defined configuration based on the locations and trajectories of nodes within the network. The secondary configuration may be a distributed routing protocol. When a local failure occurs at a given node, the forwarding rules of the secondary configuration may be implemented to route data affected by the local failure in order to reduce latency within the network.

[0016] The network controller can be configured to generate primary and secondary configurations for the network based on the network topology. The primary and secondary configurations can be generated to be implemented in the network at any given time. The primary configuration can be a whole configuration generated using software-defined networking. The secondary configuration can be generated using distributed routing protocols, such as Mobile Ad-hoc Network (MANET) routing protocols.

[0017] A network controller can transmit forwarding rules for a primary configuration and a secondary configuration to one or more nodes of a network. For each node, the network controller can transmit one or more first forwarding rules for implementing the primary configuration and one or more second forwarding rules for implementing the secondary configuration.

[0018] A first node receiving forwarding rules may store one or more first forwarding rules for a primary configuration and one or more second forwarding rules for a secondary configuration. At any given time, the first node may implement one or more first forwarding rules, including directing a signal to a second node. The first node may detect a local failure at the second node. After a local failure at the second node is detected, the first node may implement at least one of one or more second forwarding rules, including directing a signal to a third node.

[0019] The network controller may generate an updated primary configuration to manage local failures of a second node. The updated primary configuration may be an overall configuration generated using software-defined networking in the same or similar manner as the first primary configuration. Once generated, updated forwarding rules for the updated primary configuration may be transmitted to the nodes of the network to be implemented. The updated primary configuration may replace any one or more first forwarding rules and one or more second forwarding rules implemented by the nodes of the network.

[0020] This technology enables more reliable software-defined networks. Communications can be transmitted with lower latency and less packet loss during events such as local failures, while maintaining the global optimality of the software-defined network. For example, when the control plane of a primary network is passed to a secondary network following a local failure in the primary network, the primary network can have a shorter healing time. Consequently, client data can be transmitted and received faster or in greater quantities according to a schedule than in other networks. The network as a whole can support the transmission of a larger volume of data.

[0021] Exemplary systems

[0022] FIG. 1 is a diagram of an exemplary directional point-to-point network (100). The network (100) is a directional point-to-point computer network composed of nodes mounted on various ground and air-based devices, some of which may change their location relative to other nodes within the network (100) over time. For example, the network (100) includes nodes associated with each of two ground-based data centers (105a and 105b) (generally referred to as data centers (105)), nodes associated with each of three ground stations (107a, 107b and 107c) (generally referred to as ground stations (107)), and nodes associated with each of six air high-altitude platforms (HAPs) (110a-110f) (generally referred to as HAPs (110)). As illustrated, HAP (110a) is an airship, HAP (110b) is an airplane, HAP (110c) is a balloon, and HAPs (110d-110f) are satellites. In some implementations, a client device may be a node of the network (100) or may be connected directly or indirectly to an HAP of the network (100). For example, a client device may form a communication link with HAP (110d) or, like a data center (105a), may form a communication link with a ground station (107a).

[0023] In some embodiments, nodes within the network (100) may be equipped to perform free-space optical communication (FSOC), thereby making the network (100) an FSOC network. Additionally or alternatively, nodes within the network (100) may be equipped to communicate via radio frequency signals or other communication signals that can travel through free space. Arrows drawn between a pair of nodes indicate possible communication links (120, 122, 130-137, 140-145) between the nodes. Link (120) may be between a data center (105a) and a ground station (107a), and link (122) may be between a data center (105b) and a ground station (107b). HAP (110a) can be linked to ground station (107a), HAP (110b), and HAP (110c), respectively, via links (130, 131, and 132). HAP (110b) can be linked to HAP (110c), ground station (107b), HAP (110d), and HAP (110e), respectively, via links (133, 135, 136, and 137). HAP (110c) can be linked to ground station (107b) via link (134). HAP (110d) can be linked to ground station (107a), ground station (107b), HAP (110e), and HAP (110f), respectively, via links (140, 141, 142, and 142). HAP (110f) can be linked to HAP (110e) and ground station (107c) respectively through links (144 and 145).

[0024] The network (100) as illustrated in FIG. 1 is merely exemplary, and in some implementations, the network (100) may include additional or different nodes. For example, in some implementations, the network (100) may include additional HAPs which may be balloons, airships, airplanes, unmanned aerial vehicles (UAVs), satellites, or any other type of high-altitude platform.

[0025] In some implementations, the network (100) may serve as an access network for client devices such as cellular phones, laptop computers, desktop computers, wearable devices, or tablet computers. The network (100) may also be connected to a larger network such as the Internet and may be configured to provide client devices with access to resources stored on or provided through the larger computer network. In some implementations, the HAPs (110) may include radio transceivers associated with cellular or other mobile networks, such as eNodeB base stations, or other wireless access points such as WiMAX or UMTS access points. The HAPs (110) may together form all or part of a wireless access network. The HAPs (110) may be connected to data centers (105) via, for example, backbone network links or transit networks operated by third parties. Data centers (105) may include systems that monitor or control components of the network (100), as well as servers that host applications accessed by remote users. HAPs (110) may provide wireless access to users, forward user requests to data centers (105), and return responses to users via backbone network links.

[0026] As illustrated in FIG. 2, each node, such as ground stations (107) and HAPs (110), may include one or more transceivers configured to create one or more links, such as links (120, 122, 130-137, 140-145), between a given HAP (110) and other nodes in the network. Referring to HAP (110a), each node, such as ground stations (107) and HAPs (110) of the network (100), may include one or more processors (210), memory (212), and one or more transceivers (220). For clarity and simplification, only ground stations (107a) and HAPs (110a, 110b) are illustrated in FIG. 2. However, other ground stations and HAPs in the network may have the same or similar configuration as ground station (107b) or HAPs (110a, 110b).

[0027] One or more processors (210) may be any conventional processors, such as commercially available CPUs. Alternatively, one or more processors may be dedicated devices, such as application-specific integrated circuits (ASICs), or other hardware-based processors, such as field-programmable gate arrays (FPGAs). Although FIG. 2 functionally depicts one or more processors (210) and memory (212) as being within the same block, it will be understood that one or more processors (210) and memory (212) may actually comprise a number of processors and memory that may or may not be housed within the same physical housing. Accordingly, references to processors or computers will be understood to include references to sets of processors or computers or memories that may or may not operate in parallel.

[0028] Memory (212) stores information accessible by one or more processors (210), including instructions (216) and data (214) that can be executed by one or more processors (210). Memory may be any type capable of storing information accessible by a processor, including non-transient and tangible computer-readable media containing computer-readable instructions, such as hard drives, memory cards, ROM, RAM, DVDs, or other optical discs, as well as other writable and read-only memory. Systems and methods may include different combinations of the foregoing, thereby storing different parts of data (214) and instructions (216) on media of different types. In the memory of each node, such as the memory (212) of the HAP (110a), a forwarding information base or forwarding data structure, such as a database or table, indicating how signals received at each node should be forwarded or transmitted, may be stored. That is, the memory of each node may store one or more forwarding rules for the corresponding node. For example, a forwarding table stored in memory (212) may include a forwarding rule that a signal received from the ground station (107a) must be forwarded to the HAP (110d).

[0029] Data (214) may be retrieved, stored, or modified by one or more processors (210) according to instructions (216). For example, although the system and method are not limited by any specific data structure, data (214) may be stored in relational databases as tables, XML documents, or flat files having multiple different fields and records within computer registers. Data (214) may also be formatted in any computer-readable format, such as but not limited to binary values ​​or Unicode. As just an additional example, image data may be stored as computer instructions for drawing graphics, as well as bitmaps consisting of grids of pixels stored according to compressed or uncompressed, lossless (e.g., BMP) or lossy (e.g., JPEG), and bitmap or vector-based (e.g., SVG) formats. The data (214) may include any information sufficient to identify relevant information, such as numbers, descriptive text, dedicated codes, references to data stored in other areas of the same memory or different memories (including other network locations), or information used by a function to calculate relevant data.

[0030] Instructions (216) may be any set of instructions to be executed directly (e.g., machine code) or indirectly (e.g., scripts) by one or more processors (210). For example, instructions (216) may be stored as computer code on a computer-readable medium. In this regard, the terms “instructions” and “programs” may be used interchangeably herein. Instructions (216) may be stored in an object code format for direct processing by one or more processors (210), or in any other computer language including sets of scripts or independent source code modules that are interpreted or pre-compiled as required. Functions, methods, and routines of instructions (216) are described in more detail below.

[0031] One or more transceivers (220) may be mounted on actuators that can be controlled or steered to point in a desired direction. To form a link between two nodes, such as a node associated with HAP (110a) and a node associated with HAP (110d), the transceivers of each node may be controlled to point in each other's directions, thereby allowing data to be transmitted and received between the nodes. In some implementations, the power of the signals transmitted by each transceiver may also be controlled by one or more processors of each node to facilitate the formation of links (120, 122, 130-137, 140-145) within the network (100) (see, for example, FIG. 1). For example, nodes separated by a relatively large distance may be configured to operate at a higher power to compensate for the reduction in the signal-to-noise ratio that occurs over the distance separating the two nodes. Nodes spaced closer together can be controlled to operate at a relatively lower power to save power.

[0032] In some implementations, the network (100) may be a software-defined network (SDN) controlled by an SDN controller, such as the network controller (300) shown in FIG. 3. The network controller (300) may be located at one of the network nodes or on a separate platform, such as one of the data centers (105), for example. The nodes of the network (100) may be configured to communicate with each other using controllable transceivers, such as one or more transceivers (220). As the HAPs (110) move toward each other and toward the data centers (105), ground stations (107), and other ground locations over time, some of the links shown in the block diagram of FIG. 1 may become unfeasible. For example, if the path of the HAP (110a) brings the HAP (110a) to a location outside the range of the ground station (107a), or to a location where the earth is located between it and the ground station (107a), the link (130) between the ground station (107a) and the HAP (110a) may not be feasible. In other examples, weather events between the HAPs (110) and the data centers (105), ground stations (107), and other ground locations may also make certain links unfeasible. Thus, due to the continuous movement of the HAPs (110), the topology of the network (100) may require regular (i.e., periodic) or irregular reconfiguration to maintain connectivity and satisfy the determined network flows. The network controller (300) may be configured to determine configurations to efficiently satisfy the determined network flows, as further described below, so as to handle the requirements of big data-driven networking.

[0033] FIG. 3 is a functional diagram of a network controller (300). The network controller (300) may be configured to configure the topology of the network (100), transmit routing information to the nodes (107, 110) of the network (100), and transmit control messages to the network (100) to schedule changes to the topology of the network (100) for transmitting client data. As illustrated in FIG. 3, the network controller (300) may include one or more processors (310), memory (320), and a communication system (340). One or more processors (310) may be similar to one or more processors (210) described above.

[0034] Memory (320) may store information accessible by one or more processors (310), including instructions (324) and data (322) that can be executed by the processor (310). Memory (320), data (322), and instructions (324) may be configured similarly to the memory (212), data (214), and instructions (216) described above. Data (322) may include a database, table, or other storage structure representing all available nodes and possible links within the network (100) at a given time or time frame, such as a table. The table may have columns for all nodes and links within the network (100), and rows for the time or time frame. In some cases, the columns and rows may be reversed. The table may also store scheduled times or time frames available to the node or link for each node and each link. Alternatively, a graph or other form of information organization may be used. The commands (324) may include a topology and routing manager module (326), a topology determination module (328), a flow determination module (330), a solver module (332), a flight control module (334), and a scheduling module (336).

[0035] Returning to FIG. 3, the communication system (340) may be configured to communicate with nodes (107, 110) of the network (100) as well as with one or more client devices (350). In some embodiments, the communication system (340) includes a Control to Data-Plane Interface (CDPI) driver configured to communicate with a CDPI agent at each of the nodes (107, 110). Additionally, the communication system (340) of the network controller (300) may include one or more northbound interface (NBI) agents configured to communicate with an NBI driver at each of the client devices (350) associated with one or more SDN applications. The communication system (340) may optionally or alternatively be configured to transmit and receive signals to and from nodes (107, 110) and one or more client devices (350) within the network (100) via radio frequency, optical frequency, optical fiber, cable, or other communication means.

[0036] Each client device (350) may be a personal computing device or server having one or more processors (360), memory (370), data (372), and instructions (374) similar to those described above in relation to one or more processors (210 and 310), memories (212 and 320), data (214 and 322), and instructions (216 and 324). Personal computing devices may include a personal computer having all components typically available in relation to a personal computer, such as a central processing unit (CPU), memory for storing data and instructions (e.g., RAM and internal hard drives), an electronic display (e.g., a monitor with a screen, a small LCD touchscreen, a projector, a television, or any other electrical device operable to display information), a user input unit (e.g., a mouse, keyboard, touchscreen, or microphone), a camera, speakers, a network interface device, and all components used to connect these elements to one another. Personal computing devices may also include mobile devices such as PDAs, cellular phones, and similar devices. In practice, client devices (350) may include any device capable of processing instructions and transmitting data to and from humans and other computers, including general-purpose computers, network computers lacking local storage capabilities, and television set-top boxes. In some embodiments, client devices may be associated with one or more SDN applications and may have one or more NBI drivers.

[0037] Returning to the modules of the instructions (324) of FIG. 3, the topology and routing manager module (326) can enable one or more processors (310) to interface between the network controller (300) and the network (100). Using the topology and routing manager module (326), one or more processors (310) can receive information from each of the nodes within the network (100). For example, in some implementations, the topology and routing manager module (326) can enable one or more processors (310) to receive from each node (107, 110) within the network (100) information corresponding to the current location of each node, the predicted path of each node, the current links associated with each node, the routing information stored by each node, and the current storage capacity, for example, how much of the spare or available bits are available at each node. Information received from each node may also include other reports regarding weather conditions, turbulence, radiation, or other conditions that may affect the FSOC between nodes. Each node may also transmit information corresponding to any faulty link to one or more processors (310), which may be caused by unexpected obstacles between nodes, turbulence at a node, or failure of one or more transceivers.

[0038] The topology and routing manager module (326) may also enable one or more processors (310) to receive predicted link metrics and conditions. For example, predicted link metrics may include predicted values ​​of network performance metrics for virtual links that may be formed now or in the future based on information received from nodes (107, 110). Network performance metrics may include bandwidth capacity, latency, or link lifetime and may be based on the predicted relative movement or trajectory of nodes (107, 110) within the network (100). Link lifetime may indicate the period during which a link is feasible in the network (100). Weather forecasts for node locations, predicted node locations, or predicted links may also be received by one or more processors (310) from nodes (107, 110) or optionally from a remote system.

[0039] Using the topology and routing manager module (326), one or more processors (310) can store information received from the network (100) in memory (320). For example, a table representing all available nodes and available links within the network (100) may be updated or annotated with information related to a specific node or link within the table. The annotations in the table may indicate the availability of each node within the network (100), the current and future location of each node, current and future expected weather conditions, as well as the current amount of storage available at each node and the future (estimated) amount of storage available at each node within the network. Additionally, the annotations in the table may indicate the current and future availability of specific links, as well as the current and future expected bandwidth for such links. Faulty links and predicted conditions may also be recorded and stored in the table.

[0040] The topology determination module (328) may enable one or more processors (310) to determine the current or future topology of the network (100). The determination of the current topology of the network (100) may be based on information received and stored by one or more processors using the topology and routing manager module (326). For example, the topology determination module (328) may enable one or more processors (310) to aggregate information regarding the current location of each node (107, 110), the links (130-137, 140-145) formed between each pair of nodes, and any faulty links that may exist within the network (100). One or more processors (310) may receive this information through the use of the topology and routing manager module (326) or retrieve this information from memory (320).

[0041] Additional information may also be used by one or more processors (310) using a topology determination module (328) to determine the current topology of the network (100). Predicted link metrics are received by one or more processors (310) using a topology and routing manager module (326) and may also be used to determine bandwidth, quality of service, and other characteristics of the links available in the current topology. In some implementations, using the topology determination module (328), one or more processors (310) may also receive information corresponding to the flight paths of airborne network nodes, such as HAPs (110), over a specific time frame at or near the current time or at a specific time by using a flight control module (334), and the determination of the current topology may also be made based on the received flight information.

[0042] To determine the future topology of the network (100), one or more processors (310) may use a topology determination module (328) to aggregate location information, predicted link conditions, flight information, available storage, and / or weather forecasts related to future times. One or more processors (310) may access information regarding nodes and links available at future times, location information, predicted link conditions, flight information, and / or weather forecasts, which is stored elsewhere in a table or memory (320). Information regarding future times may be used by one or more processors (310) to determine where nodes are predicted to be, and what the availability of nodes and links predicted to be at future times and the storage capabilities at each node are.

[0043] The topology determination module (328) can cause one or more processors (310) to store current or future topology or other topology information in memory (320) by creating and / or updating a table representing, for example, all available nodes and available links within the network (100) and scheduled times or time frames associated with each node or link.

[0044] The flow determination module (330) may enable one or more processors (310) to determine all flows determined in the network (100) at a given time or time frame. A given flow may be one or more requirements for a routing path through the network (100). For example, each flow may include a start station, an end station, a time frame, a minimum bandwidth, or other requirements for transmission. One or more processors (310) may determine flows based on topology information determined using a topology determination module (328) and / or information regarding the characteristics of client data of one or more client devices (350). Client data information may be received by one or more processors (310) from one or more client devices (350) or a remote system using a scheduling module (336) as described below. Client data information may include sources and destinations for client data, the amount of client data to be transmitted, and / or timing for the transmission of client data. Additionally or alternatively, the amount of data may be the estimated average amount of data to be transmitted from or to a specific ground station or ground station within a geographical area over a certain period.

[0045] The minimum bandwidth of a flow may be pre-set or predetermined by one or more processors (310) in consideration of available system resources and link capabilities, or alternatively, determined based on requirements included in the client data. Larger bandwidths may be set for flows that transmit a larger amount of data. One or more processors (310) may determine a flow between a starting station and a destination station over a network capable of transmitting such an amount of client data at a requested time. In some embodiments, one or more processors (310) may also determine other information related to the determined flows, such as a service class or quality of service for each determined flow. Other information may be based on requirements received from the client device.

[0046] In some implementations, the flow determination module (330) may cause one or more processors (310) to aggregate client data from one or more client devices (350) to determine the total amount of bandwidth required between each pair of nodes within the network (100). The aggregated client data may be stored, for example, in memory (320). Additionally, the client data may be aggregated at a granular level. For example, network data for each pair of nodes may be aggregated by a service class, quality of service, or any other relevant network traffic discriminator. Flows may be further determined based on any relevant network traffic discriminator.

[0047] In other cases, past client data trends may be used to predict the volume, sources, and destinations of client data at a future point in time. The flow determination module (330) may allow one or more processors (310) to determine multiple available flows between all nodes that can be directly connected to the client device at a future point in time. Directly connectable nodes, such as ground stations (107), may communicate with the client device without using the network (100). The predicted volume of client data between each pair of nodes may be used to determine the bandwidth requirements between each pair of nodes.

[0048] Alternatively, if client data information is unavailable, one or more processors (310) may determine multiple available flows among all nodes that can be directly connected to the client device at present or future time. The determination of multiple available flows may be based on a current or future topology. Additionally, the determination may be based on minimum system requirements.

[0049] The flow determination module (330) can cause one or more processors (310) to store the determined flows in memory (320). In some examples, one or more processors (310) can attach annotations to a table with the flows.

[0050] The solver module (332) may enable one or more processors (310) to generate a network configuration or a schedule of network configurations based on the topology of the network, for example, based on a topology represented in a table stored in memory. The network configuration(s) generated using the first solver module may be a primary configuration to be implemented across other possible configurations. A network configuration may represent a feasible network topology capable of satisfying all determined network flows and may include a list of nodes and links to be used in the feasible network topology, and a schedule when the nodes and links are used. A schedule of network configurations may represent a feasible series of network topologies capable of satisfying all determined network flows. A feasible series of network topologies may include a list of nodes and links, and a schedule when the nodes and links are used for each network configuration within the schedule of network configurations. In some examples, a feasible series of network topologies includes a network topology in which data can be stored at a node having available storage, and a network topology in which a node forms a new connection or link with another node and transmits data through the newly established link.

[0051] Network configuration(s) may be generated by one or more processors (310) based on the topology for a given point in time within a table and network performance metrics of the topology at a given point in time. For example, various network performance metrics such as link bandwidth, link latency, flow bandwidth, flow priority, link switching time (i.e., the time required to implement a new topology in the network (100)), link duration, and / or topology duration may be modeled as weighted constraints for the topology at a given point in time. In some embodiments, one or more network performance metrics may not be included in a table stored in memory and may be received from another module, another node, or a remote system.

[0052] One or more processors (310) may also calculate routing paths for flows determined across a topology represented by a network configuration. A given routing path may be one method of implementing a given flow that satisfies determined flow requirements and may include a list of hops between specific nodes and links of the network, or a series of nodes. In some examples, a given routing path may include a node having an available storage that satisfies determined flow requirements regarding the amount of data to be transmitted through the network. Data following a given routing path may be stored at the node for a certain period before moving to the next hop.

[0053] Additionally, information corresponding to the previous state of the network and the previous network topology may also be used to determine the network configuration, or the schedule of network configurations. For example, one or more processors (310) may generate a network configuration based at least partially on a number of changes from the previous network topology required for the network to implement the network configuration, and the amount of time required for the network to perform the number of changes. Alternatively, one or more processors (310) may generate a schedule of network configurations based at least partially on a number of changes between network topologies of the network configurations within the schedule of network configurations, and the amount of time between changes utilizing information from routing tables. For example, changes may include manipulating a transceiver to point in a new direction, or changing a forwarding rule in a forwarding table stored in the node's memory. Manipulating a transceiver may take more takes than changing a forwarding table stored in the node's memory. The generated network configuration may require fewer than a threshold number of changes and / or less than a threshold amount of time.

[0054] For some pairs of subsequent network configurations within a schedule of network configurations, the difference between the earlier network configuration and the later network configuration may be a single change that does not entail a change in direction of the transmitters and receivers, such as a routing change at a single node.

[0055] After one or more processors (310) generate network configurations and routing paths using a solver module (332), one or more processors (310) can implement the topology represented by the generated primary configuration by controlling the nodes of the network (100) according to the topology and routing manager module (326) and sending implementation commands to the nodes to cause the nodes to form links included in the generated primary configuration (e.g., by controlling their respective transceivers, adjusting their respective transmission power levels, setting their transmission and reception frequency bands, etc.), and can update the forwarding rules in the forwarding tables stored in memory at each node according to the routing paths calculated for the primary configuration and the secondary configuration. For example, for each node, the network controller can send one or more first forwarding rules to implement the primary configuration. Some forwarding tables may be updated with a schedule of changes based on the schedule of network configurations, and may also include instructions to save data at the node before the next hop.

[0056] Additionally, the network controller may also transmit commands to switch to a secondary network configuration when the primary network configuration fails. In this implementation, one or more first forwarding rules may be transmitted with a priority indication that stores the first forwarding rules in a manner such that they are implemented before any other forwarding rules at each node. Other forwarding rules may include one or more second forwarding rules for the secondary network configuration. The secondary configuration may include one or more routing paths between nodes of the network to be used as a backup in case the primary configuration is not allowed to be used at one or more locations within the network. In particular, the secondary configuration may implement a distributed routing protocol, such as a mobile ad-hoc routing protocol. For example, the secondary configuration may include one or more routing paths between a given node and one or more other nodes within the network through available nodes and links, such as those within the topology generated for the primary configuration.

[0057] The flight control module (334) may cause one or more processors (310) to generate flight commands regarding the flight paths of air nodes, such as HAPs (110). For example, one or more processors (310) may not be able to determine a network configuration representing a network topology capable of satisfying all determined network flows using the solver module (332). One or more processors may determine, using the solver module (332), that the reason for this failure is that one or more of the air network nodes within the network (100) have moved too far from other network nodes to form a link. In response, one or more processors (310) may use the flight control module (334) to generate and transmit flight commands for the air nodes of the network (100), which cause the air nodes to change their flight paths so that additional links can be formed. For example, the flight commands may cause the air nodes to move closer to each other or avoid obstacles. After the nodes are relocated according to flight commands generated by one or more processors using the flight control module (334), an updated table may be generated using the topology and routing manager module (326) or the topology determination module (328) based on the new locations of the network nodes. Next, the updated table may be processed by one or more processors (310) using the solver module (332) to determine the network configuration.

[0058] The scheduling module (336) may enable one or more processors (310) in the network controller (300) to interface with one or more client devices (350). One or more processors (310) may use the scheduling module (336) to receive client data information to be transmitted through the network (100) from the client device (350), such as sources and destinations for client data, for example. Other information received from the client device (350) may include data related to client requests, such as the amount of client data to be transmitted and the timing for transmission. The information may be stored in memory (320) and / or may be used according to the flow determination module (330) to determine determined flows through the network (100). In some implementations, one or more processors (310) of the network controller (300) may use the scheduling module (336) to request client data information from one or more client devices (350) based on the bandwidth available at a given ground station or a given geographic location.

[0059] As described above, after the determined flows are determined using the flow determination module (330) and the network configuration is generated using the solver module (332), one or more processors (310) may generate routing instructions for transmitting client data through the network (100) based on the table and the generated network configuration. These routing instructions may include the source location of the client data, the destination location of the client data, and the timing for transmitting the client data. In some embodiments, the routing instructions may include storage instructions for a node to temporarily store data from a previous node to be transmitted to the next node. The routing instructions may include a schedule that can be stored in a node of the network that is directly connected to the client device (350) transmitting the client data. Next, one or more processors (310) may transmit the routing instructions to a node that is directly connected to the client device (350), so that the node may receive the client data over the flow determined according to the schedule and begin its transmission.

[0060] In some embodiments where flows are determined without client data information, the scheduling module (336) may cause one or more processors (310) to send a message to one or more client devices (350) regarding the availability of flows through the network based on the determined flows determined using the flow determination module (330) and the network configuration generated using the solver module (332). The message may also include the time or time frame at which the flows are available and / or the price for the transmission of data associated with each flow. Using the scheduling module (336), one or more processors (310) may receive a response from one of one or more client devices (350) that includes a request to use one of the determined flows to transmit client data. Next, one or more processors (310) may send routing commands to one or more nodes to initiate the transmission of client data over the determined flows.

[0061] Exemplary methods

[0062] In addition to the systems described above and illustrated in the drawings, various operations will now be described. One or more processors (310) can generate a primary network configuration to be implemented in the network, along with instructions for implementing a secondary network configuration when the primary network configuration experiences a failure, as described below. In FIG. 4, the flowchart (400) is illustrated according to some of the embodiments described above that can be performed by one or more processors (310) of the network controller (300). Additionally, or alternatively, one or more processors of the node may perform one or more of the operations. FIG. 4 shows the blocks in a specific order, but the order may be changed, and multiple operations may be performed simultaneously. Also, operations may be added or omitted.

[0063] In block (402), one or more processors (310) of the network controller (300) may receive information from each of the nodes in the network (100) using the topology and routing manager module (326). The information may relate to the current or predicted conditions, weather, or links of the nodes at the current time or at a future time. In an exemplary scenario, location A may be received as the current location of the HAP (110a) at the current time; location B may be received as the current location of the HAP (110b) at the current time; location C may be received as the current location of the HAP (110c) at the current time; location D may be received as the current location of the HAP (110d) at the current time; location E may be received as the current location of the HAP (110e) at the current time; and location F may be received as the current location of the HAP (110f) at the current time. Weather condition reports from HAPs (110a, 110c, 110d, 110e, and 110f) may indicate that current weather conditions are clear at locations A, C, D, E, and F, and weather condition reports from HAP (110b) may indicate that current weather conditions include thunderstorms near location B in the direction of HAP (110e). HAP (110a) may also transmit an indication to one or more processors (310) that HAP (110c) has not responded to requests for the link (131). Additionally, HAP (110a) may be predicted to move from location A to location G within one hour from the current time, and HAP (110d) may be predicted to move from location D to location H within one hour from the current time. Weather forecasts for one hour from the current time may include thunderstorms between location H and the location of the ground station (107a).For links within a network (100) or each subnet of the network (100), information regarding bandwidth, latency, or link lifetime duration may be received by one or more processors (310). For example, information regarding links within the network (100) may include how much bandwidth is available.

[0064] In block (404), available nodes and available links within the network (100) can be determined based on information received from the nodes of the network using the topology and routing manager module (326). In some implementations, the determined available nodes, available links, and corresponding data may be stored in memory (320), for example, in a table. In an exemplary scenario, based on the received information, HAP (110a) is determined to be available at location A where the current weather is clear, and HAP (110a) is predicted to be at location G within one hour. For HAP (110b), it is determined to be available at location B where the weather at location B is generally clear, except for thunderstorms in one direction toward HAP (110e). For HAPs (110c, 110e, and 110f), it is determined to be available at their respective locations C, E, and F where the current weather is also clear. For HAP (110d), it is determined that it is currently located at position D, and is predicted to be at position H within one hour, where the weather will include thunderstorms at that time between position H and the location of the ground station (107a).

[0065] Additionally, links (130, 132-136 and 140-145) are determined to be available at the current time (indicated by solid arrows in FIG. 5), and links (131 and 137) are not available (indicated by dashed lines). For link (131), it may be determined that link (131) is currently out of service based on the indication of no response from HAP (110c) to requests from HAP (110a). For link (137), it may be determined that link (137) is not available at the current time due to thunderstorms between HAP (110b) and HAP (110e). For link (140), it may be determined that link (140) is not available at a future time due to thunderstorms between HAP (110d) and ground station (107a). Available nodes and possible links can also be determined for a time other than the current time based on the predicted future locations of the nodes, such as weather forecasts, or the predicted conditions at each node.

[0066] In block (406), one or more processors (310) may use a topology determination module (328) to determine the topology of the network at a given point in time or during a given time frame based on available nodes and available links. In some implementations, one or more processors (310) may access a table or other data structure representing available nodes and available links in memory (320). For an exemplary scenario, to determine the current topology, one or more processors (310) may determine which nodes and links are available at the current time from the table and the scheduled times associated with each node and link. According to the received information regarding the HAPs (110a-110f), the current topology (500) may be determined as illustrated in FIG. 5. The current topology may be determined to include nodes (107, 110). Specifically, HAP (110a) may be included in the current topology at location A, HAP (110b) at location B, HAP (110c) at location C, HAP (110d) at location D, HAP (110e) at location E, and HAP (110f) at location F, as these are the locations associated with each HAP for the current time as determined in block (404). As indicated by arrows in the current topology (500), links (130, 132-137 and 140-145) are included in the current topology, whereas links (131 and 137) marked as having failures for the current time according to block (404) are not included (shown as dashed lines without arrows in FIG. 5).In another example, if the HAP (110c) has not reported its location at location C to one or more processors (310) but has previously reported a flight path or trajectory using the flight control module (334), one or more processors (310) may determine that the current HAP (110c) is at location C based on the flight path or trajectory and include the HAP (110c) in the current topology. In some implementations, each possible link (130, 132-136, and 140-145) within the current topology may also be labeled with link metrics, such as bandwidth, which are determined based on the received information.

[0067] In block (408), information regarding client data to be transmitted over the network (100) may be received by one or more processors (310) using a scheduling module (336). Client data information may be received from one or more client devices located in a geographic area that is close to a given ground station or communicates with the network controller (300). Client data information may include the amount of data, the source location and destination location of the client data, and the requested transmission time. For example, in an exemplary scenario, the received information may indicate that, for the first client data, the amount of client data is 10 Gb, the source location of the first client data is ground station (107b), the destination location is ground station (107a), and the requested transmission time is the current time. Additionally or alternatively, the amount of data may be the estimated average amount of data to be transmitted from or to a given ground station or client devices or ground stations within a given geographic area. In some cases, the information also includes transmission requirements such as bandwidth, service class, quality of service, etc. In some embodiments, information related to client data may be predicted by one or more processors (310) of the network controller (300) or by a remote system.

[0068] In block (410), the primary network configuration (700) for the determined topology (500) can be determined by one or more processors (310) using a solver module (332). The primary network configuration (700) can be determined by determining flows through the topology for client data according to received client data information, such as information received in block (408), and by selecting one or more flows through the topology as routing paths for the client data. For example, the primary network configuration (700) includes all nodes and links available at the current time, and specifically omits links (131, 137) and node (110e). As illustrated by the brightly colored arrows in FIG. 6, the first routing path within the topology (500) for the first client data may include portions of the routing path between ground station (107b) and ground station (107a). The first routing path section is from the ground station (107b) to the HAP (110b) via the link (135), the second routing path section is from the HAP (110b) to the HAP (110a) via the link (132), and the third routing path section is from the HAP (110a) to the ground station (107a) via the link (130).

[0069] Other client data to be transmitted from ground station (107b) to ground station (107a) may be transmitted along the same routing path as for the first client data, or due to constraints such as bandwidth constraints, it may be determined to be transmitted at least partially along a second routing path from ground station (107b) to ground station (107a). As illustrated by the hash-filled arrows in FIG. 6, the second routing path may include a first routing path portion from ground station (107b) to HAP (110c) via link (134), a second routing path portion from HAP (110c) to HAP (110b) via link (133), a third routing path portion from HAP (110b) to HAP (110a) via link (132), and a fourth routing path portion from HAP (110a) to ground station (107a) via link (130).

[0070] As illustrated by the gray arrows in FIG. 6, a third routing path for additional client data may include a first routing path portion from the ground station (107a) to the HAP (110d) via the link (140), a second routing path portion from the HAP (110d) to the HAP (110f) via the link (143), and a third routing path portion from the HAP (110f) to the ground station (107c) via the link (145).

[0071] Based on routing paths for client data to be transmitted over the network at the current time, one or more processors (310) may determine a primary network configuration (700) to include links (130, 132-136, 140, 143, and 145) as illustrated in FIG. 7. Other links do not need to be formed at the current time. Determining the primary network configuration (700) may also include determining first forwarding rules for each node based on the routing paths included in the primary network configuration. The first forwarding rules may be determined for each node along the path or may be determined for each client data. When determined for specific client data, the first forwarding rules may be transmitted to the source node and encapsulated into packets carrying the specific client data. The first forwarding rules may also include instructions regarding forming links in the primary network configuration, such as pointing directions or beam configurations.

[0072] In block (412), a secondary network configuration (800) for a network (100) can be determined at each available node of the network. In particular, the secondary network configuration can be determined by one or more processors at a given node using a distributed routing protocol, such as a mobile ad-hoc network routing protocol. For example, one or more next hops from a given node can be determined based on nodes and links from the primary network configuration (700). One or more processors at a given node can determine secondary routing paths from the given node to a network controller (300), one or more destination nodes, and / or intermediate upper-level nodes. The secondary routing paths within the secondary network configuration may differ from the routing paths within the primary network configuration. There may be more or fewer possible secondary routing paths in the secondary network configuration than there are routing paths in the primary network configuration. One or more processors at a given node can determine the secondary network configuration periodically or continuously based on the available nodes and links within the network. Additionally or alternatively, one or more processors at a given node may determine a secondary network configuration after receiving available nodes or links from a network controller, for example, based on nodes and links within a primary network configuration.

[0073] Next, one or more processors of a given node may determine second forwarding rules from the given node based on secondary routing paths. The second forwarding rules may be for proactive (table-based) routing, reactive (on-demand) routing, hybrid (proactive and reactive) routing, or hierarchical routing. In some implementations, the second forwarding rules for a given node include next-hop instructions for the secondary routing path between the given node and the destination node for client data. In other implementations, the second forwarding rules for a given node include next-hop instructions to the next node associated with the desired destination node, rather than all nodes within the routing path to the desired destination node. In yet other implementations, the second forwarding rules may include a routing protocol and a table of nodes and links of the primary configuration. Next, one or more processors of a given node may determine the routing path to take based on the table and the routing protocol after the table has been updated to reflect any link failure or other changes in the topology.

[0074] In the example illustrated in FIG. 8, the secondary network configuration (800) may include a first secondary routing path (filled with hashes) and second forwarding rules for a second secondary routing path (greyed out). Other secondary routing paths may be included in the secondary network configuration but are not illustrated for clarity. The first secondary routing path may be determined as a backup path for the first client data. The first secondary routing path is from the ground station (107b) to the HAP (110c) via link (134), to the HAP (110b) via link (133), to the HAP (110a) via link (132), and to the ground station (107a) via link (130). These secondary routing paths differ from the routing paths within the primary network configuration in that the link (135) between the ground station (107b) and the HAP (110b) is replaced by links (134 and 133) connected to the HAP (110c) that can be used in the event of a failure of the link (135). The second secondary routing path may be determined as a backup path for additional client data to be transmitted along the routing path in the primary network configuration between the ground stations (107a and 107c). In particular, the second secondary routing path is from the ground station (107a) to the HAP (110a) via link (130), to the HAP (110b) via link (132), to the HAP (110d) via link (136), to the HAP (110f) via link (143), and to the ground station (107c) via link (145). These secondary routing paths differ from routing paths within the primary network configuration in that the link (140) between the ground station (107a) and the HAP (110d) is replaced by links (130, 132, and 136) connected to HAPs (110a and 110b) that can be used in the event of a failure of the link (140).

[0075] As further illustrated in FIG. 4, in block (414), one or more processors (310) of the network controller (300) may transmit implementation commands to nodes (107, 110) of the network (100) to cause the nodes of the network to implement a primary network configuration and to switch to a secondary network configuration for backup at a given point in time or during a given time frame. In an exemplary scenario, for the primary network configuration, the implementation commands for the nodes (107, 110) may include commands for forming links and implementing at least the routing path portions illustrated in FIG. 6.Accordingly, the implementation commands are commands for a ground station (107a) to form links (130 and 132) respectively by directing one or more transceivers of the ground station (107a) toward the HAP (110a and 110d); commands for a ground station (107b) to form links (135 and 134) respectively by directing one or more transceivers of the ground station (107b) toward the HAP (110b and 110b); commands for a ground station (107c) to form a link (145) by directing one or more transceivers of the ground station (107c) toward the HAP (110f); and commands for a HAP (110a) to form links (130 and 132) respectively by directing the transceivers of the HAP (110a) toward the ground station (107a) and the HAP (110b). Commands for HAP (110b) to form links (135, 132, 133, and 136) respectively by directing the transceivers of HAP (110b) toward the ground station (107b), HAP (110a), HAP (110b), and HAP (110d); commands for HAP (110c) to form links (134 and 133) respectively by directing the transceivers of HAP (110c) toward the ground station (107b) and HAP (110b); commands for HAP (110d) to form links (140, 136, and 143) respectively by directing the transceivers of HAP (110d) toward the ground station (107a), HAP (110b), and HAP (110f); The HAP (110f) may include commands for the HAP (110f) to form links (143 and 145) respectively by directing the transceivers of the HAP (110f) toward the HAP (110d) and the ground station (107c).

[0076] In some cases, one or more of the links within the primary network configuration may already be formed, in which case no change to the direction of the transceivers is required. Additionally, implementation commands for the ground station (107a or 107b), i.e., the start station, may include first forwarding rules for receiving and transmitting client data to be transmitted through routing paths through the given network configuration. The first forwarding rules may include the source location of the client data, the destination location of the client data, the timing for transmission, and / or the rate for transmission. For example, when the first forwarding rules for the first client data are received at the ground station (107b), the ground station (107b) may transmit the first client data at the current time through the primary network configuration.

[0077] For a primary network configuration created for a future time, a future time frame, or another point in time, implementation commands may include storing scheduled changes within the network (100) at each node that may occur before transmitting client data at a future time, such as controlling transceivers to implement new routing paths. Accordingly, implementation commands may include updating forwarding rules in the forwarding tables at each node with new routing paths and a time or time frame to implement new routing paths according to the future network configuration. When the time or time frame arrives, the nodes (107, 110) of the network (100) may automatically implement the future network configuration according to the implementation commands.

[0078] The implementation commands may also include commands for preparing the network to implement a secondary network configuration at the same time or time frame as the primary network configuration when there is a failure in the primary network configuration. In this case, the implementation commands may include storing second forwarding rules for implementing secondary routing paths, such as those illustrated in FIG. 8, as a backup for the first forwarding rules of the primary network configuration. For example, the implementation commands may include next-hop commands for a ground station (107a) for routing first client data to HAP (110c) via link (134), next-hop commands for HAP (110c) for routing first client data to HAP (110b) via link (133), next-hop commands for a ground station (107b) for routing additional client data to HAP (110a) via link (130), next-hop commands for HAP (110a) for routing additional client data to HAP (110b), and next-hop commands for HAP (110b) for routing additional client data to HAP (110d).

[0079] The implementation instructions may also include instructions for switching from the implementation of a first forwarding rule to the implementation of a second forwarding rule under specific circumstances. For example, the instructions for switching may include detecting when the first forwarding rule cannot be completed, such as when there is a link failure between the first node and the second node, when transmitted client data is not received from the first node to the second node, or when transmitted client data is not received from the destination node within an expected amount of time. Next, the instructions may include implementing the second forwarding rule from the first node or another node in the network to bypass the failed first forwarding rule. In some examples, the instructions may include determining whether the second forwarding rule can be implemented or which second forwarding rule to implement, based on the primary network configuration and the failed first forwarding rule, before implementing the second forwarding rule.

[0080] In FIG. 9, the flowchart (900) is illustrated according to some of the embodiments described above that can be performed by one or more processors of a given node of a network, such as one or more processors (210) of the HAP (110a). Additionally or alternatively, one or more processors (310) of the network controller (300) may perform one or more of the operations. FIG. 9 illustrates the blocks in a specific order, but the order may be changed, and multiple operations may be performed simultaneously. Also, operations may be added or omitted.

[0081] In block (902), one or more processors of the first node receiving implementation instructions, such as one or more processors of the ground station (107a), may store one or more of the first forwarding rules for the primary network configuration in relation to one or more of the second forwarding rules for the secondary network configuration. The first forwarding rules and the second forwarding rules may be determined as described above with respect to the flowchart (400) of FIG. 4. In some implementations, one or more second forwarding rules may be determined or updated by one or more processors of the first node after receiving the first forwarding rules. Based on the implementation instructions, one or more first forwarding rules may be stored with a higher priority than one or more second forwarding rules. The higher priority of one or more first forwarding rules may cause one or more first forwarding rules to be implemented at a given time, unless one of the first forwarding rules can no longer be implemented. For example, at the ground station (107a), the first forwarding rule for additional client data may be the next hop to the HAP (110d) heading toward the destination of the ground station (107c) via the link (140), and the second forwarding rule for additional client data may be the next hop to the HAP (110a) heading toward the destination of the ground station (107c) via the link (130).

[0082] In block (904), one or more processors of the first node may implement one or more first forwarding rules, including directing a signal to the second node at a given time. Implementing one or more first forwarding rules may include establishing a link with the second node. For example, to implement the first forwarding rule in one or more first forwarding rules, the first node may physically or electronically align an antenna with the second node, which is the next hop in the network, according to a routing path. The first node may also adjust one or more signal characteristics for the link with the second node in the network. For the ground station (107a), one or more processors of the ground station (107a) may form a link (140) with the HAP (110d) by directing at least one transceiver to position D according to the first forwarding rule. HAP (110d) may likewise have at least one transceiver directed toward the ground station (107a) so as to be positioned to receive a signal from the ground station (107a). At least one transceiver of the ground station (107a) may be configured with a specific frequency band or other signal characteristics for the link. Additional client data may be transmitted from the ground station (107a) to HAP (110d) over the link (140).

[0083] In block (906), the first node can detect a local failure at the second node. A local failure at the second node can be detected when communication with the second node cannot be established. Alternatively, a local failure can be detected when the first node receives a communication from a network controller or a third party in the network indicating that the second node is offline or otherwise unreachable. For example, one or more processors of the ground station (107a) may determine that the HAP (110d) has experienced a local failure when the link (140) cannot be established, or when a weather communication is received indicating that a storm is located between the ground station (107a) and the HAP (110d) and makes the HAP (110d) unreachable for a certain period of time.

[0084] In block (908), after a local failure at the second node is detected, the first node may implement at least one of one or more second forwarding rules, including directing a signal to the third node. One or more second forwarding rules may be stored in the memory of the first node or determined after the local failure is detected. In some implementations, at least one second forwarding rule may be selected from one or more second forwarding rules based on the destination for the data or based on an available intermediate node between the first node and the destination. Implementing at least one second forwarding rule may include establishing a link with the third node. Alternatively, the link with the third node may already exist from the primary configuration, and implementing at least one second forwarding rule may include redirecting a signal from a first port directed to the second node to a second port directed to the third node. For example, one or more processors of the ground station (107a) may switch to transmitting additional client data to the HAP (110a) on the link (130). Since the link (130) has already been established to route the first client data, the HAP (110a) is already positioned to receive communication from the ground station (107a).

[0085] In some examples, the first node may transmit a notification of a local failure of the second node to the network controller, as illustrated in block (910). In other examples, the network controller may detect a local failure of the second node using other nodes or other sensors that communicate with the network controller.

[0086] The network controller may generate an updated primary configuration to manage local failures of a second node. The updated primary configuration may be a complete configuration generated using software-defined networking in the same or similar manner as the first primary configuration. Once generated, updated forwarding rules for the updated primary configuration may be transmitted to the nodes of the network to be implemented. The updated primary configuration may replace any one or more first forwarding rules and one or more second forwarding rules implemented by the nodes of the network.

[0087] In some alternative implementations, the network controller may determine a secondary network configuration to back up a primary network configuration that includes one or more second forwarding rules, and may transmit one or more second forwarding rules to their respective nodes in the same or similar manner as one or more first forwarding rules are transmitted.

[0088] This technology enables more reliable software-defined networks. Communications can be transmitted with lower latency and less packet loss during events such as local failures, while maintaining the global optimality of the software-defined network. For example, when the control plane of a primary network is passed to a secondary network following a local failure in the primary network, the primary network can have a shorter healing time. Consequently, client data can be transmitted and received faster or in greater quantities according to a schedule than in other networks. The network as a whole can support the transmission of a larger volume of data.

[0089] Unless otherwise noted, the alternative examples described above are not mutually exclusive and may be implemented in various combinations to achieve unique advantages. Since these and other variations and combinations of the features discussed above may be utilized without departing from the subject matter defined by the claims, the foregoing description of the embodiments should be taken as an example rather than as a limitation of the subject matter defined by the claims. Additionally, the examples described herein, as well as the provision of phrases expressed as “similar to,” “comprising,” and similar, should not be interpreted as limiting the subject matter of the claims to specific examples; rather, the examples are intended to illustrate only one of many possible embodiments. Furthermore, the same reference numerals in different drawings may identify identical or similar elements.

Claims

Claim 1 As a system, the system comprises a network controller including one or more processors, wherein the one or more processors: receive node information from a plurality of nodes of a network, wherein the plurality of nodes include a first node moving toward a second node, and the node information includes location information and flight information for the first node; determine a current and predicted path for the first node based on the location information and the flight information; determine available nodes and possible links within the network based on the received node information; determine a current topology and a future topology of the network based on the available nodes and possible links; receive client data information from one or more client devices regarding client data to be transmitted through the network, wherein the client data includes at least a destination node; determine a primary network configuration for the current topology, wherein the primary network configuration includes one or more first routing paths for the client data; Determining a first forwarding rule for each node of the network based on one or more first routing paths; determining a secondary network configuration for the future topology, wherein the secondary network configuration includes one or more second routing paths for the client data; determining a second forwarding rule for each node of the network based on the secondary network configuration;A system configured to transmit implementation commands to the plurality of nodes of the network, wherein the implementation commands cause the storage of the first forwarding rule and the second forwarding rule, wherein the first forwarding rule has a higher implementation priority than the second forwarding rule in each of the plurality of nodes, and wherein the implementation commands cause the plurality of nodes to implement the primary network configuration and to switch to the secondary network configuration under circumstances including that transmitted client data is not received at the destination node.; Claim 2 A system according to claim 1, wherein the implementation commands cause each node of the network to additionally determine the next hop in the secondary network configuration using mobile ad-hoc networking in the determined topology. Claim 3 In claim 1, the above implementation commands include the above first forwarding rule, in a system. Claim 4 In paragraph 3, the system, wherein the first forwarding rule comprises instructions for forming a link in the primary network configuration. Claim 5 delete Claim 6 A system according to claim 1, wherein the second forwarding rule is for proactive routing. Claim 7 A system according to claim 1, wherein the second forwarding rule is for reactive routing. Claim 8 In claim 1, the system wherein the second forwarding rule is for hybrid routing. Claim 9 delete Claim 10 A computer implementation method comprising: receiving node information from a plurality of nodes of a network by one or more processors communicating with a plurality of nodes of a network, wherein the plurality of nodes include a first node moving relative to a second node, and the node information includes location information and flight information for the first node; determining a current and predicted path for the first node based on the location information and flight information by the one or more processors; determining available nodes and possible links within the network based on the received node information by the one or more processors; determining a current topology and a future topology of the network based on the available nodes and possible links by the one or more processors; receiving client data information related to client data to be transmitted through the network from one or more client devices by the one or more processors, wherein the client data includes at least a destination node; determining a primary network configuration for the current topology by the one or more processors, wherein the primary network configuration includes one or more first routing paths for the client data; and, by the one or more processors, the one or more first A step of determining a first forwarding rule for each node of the network based on a routing path; a step of determining a secondary network configuration for the future topology by the one or more processors, wherein the secondary network configuration includes one or more second routing paths for the client data; a step of determining a second forwarding rule for each node of the network based on the secondary network configuration by the one or more processors;A method comprising the step of transmitting implementation instructions to the plurality of nodes of the network by the one or more processors, wherein the implementation instructions cause the storage of the first forwarding rule and the second forwarding rule, wherein the first forwarding rule has a higher priority of implementation than the second forwarding rule at each of the plurality of nodes, and wherein the implementation instructions cause the plurality of nodes to implement the primary network configuration and to switch to the secondary network configuration under circumstances including that transmitted client data is not received at the destination node. Claim 11 In claim 10, the above implementation commands cause each node of the network to additionally determine the next hop in the secondary network configuration using mobile ad-hoc networking in the determined topology. Claim 12 In paragraph 10, the above implementation instructions include the above first forwarding rule, a method. Claim 13 In paragraph 12, the method wherein the first forwarding rule comprises instructions for forming a link in the primary network configuration. Claim 14 delete Claim 15 In paragraph 10, the above-mentioned second forwarding rule is for prior routing, a method. Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete

Images