Method and system for fault-tolerant dynamic routing

Abstract

A method for optimizing data transmission in a network. This method involves generating multiple copies of a data packet intended for transmission, and then selecting and using specific data paths between a source and a destination device for each copy to ensure fault-tolerant and efficient data delivery. The method may be embodied on a computer-readable medium and be performed by a system, such as using a fault tolerant appliance or by suitably configuring the source and destination devices themselves.

Description

METHOD AND SYSTEM FOR FAULT-TOLERANT DYNAMIC ROUTING

[0001] This application claims the benefit of priority to United States Provisional Application No. 63 / 606,480, filed on December 5, 2023, the entirety of which is hereby incorporated by reference herein.TECHNICAL FIELD

[0002] The present disclosure relates to methods, systems and techniques for fault- tolerant dynamic routing. Specifically, the present disclosure relates to selecting and utilizing multiple data paths between source and destination devices, generating and managing multiple copies of data packets, and improving fault tolerance and overall network security in data communications.BACKGROUND

[0003] In today's interconnected digital landscape, networks and communications systems serve as the backbone for businesses, governments, and individuals. This digital infrastructure enables the seamless exchange, processing, and storage of vast amounts of information, fostering global collaboration and economic growth. However, as the digital realm expands, so does the risk associated with potential cyber threats. Cyber-attacks, ranging from data breaches to distributed denial-of-service (DDoS) attacks, threaten the integrity, confidentiality, and availability of data. The impact of such breaches can be immense, resulting in financial loss, reputational damage, and operational disruption.

[0004] As cyber threats continue to evolve in complexity and scale, the methods and tools used to prevent, detect, and mitigate these threats must keep pace. Traditional cybersecurity strategies, which often rely heavily on perimeter defenses and signature-based detection mechanisms, are challenged in the face of modern adversary tactics. The digital world demands innovative and robust cybersecurity approaches that go beyond traditional defenses.SUMMARY

[0005] According to a first aspect, there is provided a method for data transmission, which comprises generating a plurality of copies of a data packet to be sent from a source device to a destination device; selecting a plurality of different data paths for the plurality of copies of the data packet from a plurality of different data paths in a network connecting the source device to the destination device, respectively; and transmitting the plurality of copies of the data packet along the plurality of different data paths, respectively, from the source device toward the destination device.

[0006] In one embodiment, the method may further comprise labeling the different data paths by assigning an identifier to each of the plurality of different data paths using at least one controller, wherein the identifiers are indicative of routing priorities of the different data paths, and wherein the obtaining further comprises obtaining the assigned identifiers. In one embodiment, the labeling may comprise applying virtual local area network (VLAN) or multiprotocol label switching (MPLS) labeling.

[0007] In one embodiment, the at least one controller may comprise at least two controllers, and the method may further comprise monitoring an inconsistency of the identifiers assigned by the at least two controllers; and in response to the identifiers being inconsistent, identifying a faulty controller.

[0008] In one embodiment, the plurality of copies of the data packet may comprise at least three copies of the data packet.

[0009] In one embodiment, the method may further comprise: in response to the plurality of copies of the data packet having been transmitted through the plurality of data paths in the network, identifying one of the plurality of data paths as an anomalous data path by determining that a deviation exists between one of the plurality of copies of the data packet transmitted through the one of the plurality of data paths and another one of the plurality of copies of the data packet.

[0010] In one embodiment, the method may further comprise counting instances where the deviation exists to generate a count, wherein the identifying may comprise, in response to the count exceeding a threshold, identifying the one of the plurality of data paths as the anomalous data path.

[0011] In one embodiment, the method may further comprise: obtaining the deviation by comparing the one of the plurality of copies of the data packet transmitted through the one of the plurality of data paths with the others of the plurality of copies of the data packet, with information on routing removed from the plurality of copies of the data packet.

[0012] In one embodiment, the method may further comprise: in response to one byte of the one of the plurality of copies of the data packet transmitted through the one of the plurality of data paths being different from a corresponding byte of the others of the plurality of copies of the data packet, determining that the deviation exists for the one copy.

[0013] In one embodiment, the method may further comprise: in response to identifying the anomalous data path, removing the anomalous data path from the plurality of different data paths.

[0014] In one embodiment, the method may further comprise: in response to identifying the anomalous data path, selecting a new data path in place of the anomalous data path from the plurality of different data paths.

[0015] In one embodiment, the plurality of copies of the data packet may comprise at least three copies of the data packet, and at least one of the plurality of copies of the data packet may be precluded from the determination of the deviation.

[0016] In one embodiment, the method may further comprise: in response to identifying that the one of the plurality of data paths as an anomalous data path, changing the plurality of data paths to avoid the anomalous data path for a subsequent data packet.

[0017] In one embodiment, the plurality of different data paths may have no common network nodes with each other.

[0018] In one embodiment, the method may be performed by one of the source device or the destination device.

[0019] In one embodiment, the method may be performed by an inline appliance that is connected to one of the source device or the destination device.

[0020] In one embodiment, the inline appliance may further comprise a network interface, the network interface comprising a first Ethernet port connecting to one of the source device or the destination device and a second Ethernet port connecting to the network.

[0021] According to a second aspect, there is provided a system for data transmission, which comprises a processor and a non-transitory memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the processor to: obtain a plurality of different data paths in a network connecting a source device to a destination device; generate a plurality of copies of a data packet to be sent from a source device to a destination device; select a plurality of different data paths for the plurality of copies of the data packet from a plurality of different data paths in a network connecting the source device to the destination device, respectively; and transmit the plurality of copies of the data packet along the plurality of different data paths, respectively, from the source device toward the destination device.

[0022] In one embodiment, the processor and the non-transitory memory may comprise part of an inline appliance, and wherein the system may further comprise at least one controller configured to: label the different data paths by assigning an identifier to each of the plurality of different data paths, wherein the identifiers are indicative of routing priorities of the different data paths, and wherein the instructions stored in the non-transitory memory further cause the processor to obtain the assigned identifiers. In one embodiment, the at least one controller may be configured to apply virtual local area network (VLAN) or multiprotocol label switching (MPLS) labeling.

[0023] In one embodiment, the at least one controller may comprise at least two controllers, and wherein the system may further comprise an operation, administration, and maintenance (0AM) device configured to: monitor an inconsistency of the identifiers assignedby the at least two controllers; and in response to the identifiers being inconsistent, identify a faulty controller.

[0024] In one embodiment, the plurality of copies of the data packet may comprise at least three copies of the data packet.

[0025] In one embodiment, the processor and the non-transitory memory may comprise part of an inline appliance, and wherein the system may further comprise at least one controller configured to: in response to the plurality of copies of the data packet having been transmitted through the plurality of data paths in the network and a deviation having been determined to exist between one of the plurality of copies of the data packet transmitted through one of the plurality of data paths and another one of the plurality of copies of the data packet, identify the one of the plurality of data paths as an anomalous data path.

[0026] In one embodiment, the at least one controller may be configured to count instances where the deviation exists to generate a count; and in response to the count exceeding a threshold, identify the one of the plurality of data paths as the anomalous data path.

[0027] In one embodiment, the determination of the deviation may be performed by the destination device or a second inline appliance connected inline with the destination device.

[0028] In one embodiment, the at least one controller may be configured to: obtain the deviation by comparing the one of the plurality of copies of the data packet transmitted through the one of the plurality of data paths with the others of the plurality of copies of the data packet, with information on routing removed from the plurality of copies of the data packet.

[0029] In one embodiment, the at least one controller may be configured to: in response to one byte of the one of the plurality of copies of the data packet transmitted through the one of the plurality of data paths being different from a corresponding byte of the others of the plurality of copies of the data packet, determine that the deviation exists for the one copy.

[0030] In one embodiment, the at least one controller may be configured to, in response to identifying the anomalous data path, remove the anomalous data path from the plurality of different data paths.

[0031] In one embodiment, the at least one controller may be configured to, in response to identifying the anomalous data path, select a new data path in place of the anomalous data path from the plurality of different data paths.

[0032] In one embodiment, the plurality of copies of the data packet may comprise at least three copies of the data packet, and at least one of the plurality of copies of the data packet may be precluded from the determination of the deviation.

[0033] In one embodiment, the at least one controller may be configured to, in response to identifying that the one of the plurality of data paths as an anomalous data path, change the plurality of data paths to avoid the anomalous data path for a subsequent data packet.

[0034] In one embodiment, the plurality of different data paths may have no common network nodes with each other.

[0035] In one embodiment, the processor and the non-transitory memory may comprise part of one of the source device or the destination device.

[0036] In one embodiment, the processor and the non-transitory memory may comprise part of an inline appliance that is connected to one of the source device or the destination device.

[0037] In one embodiment, the inline appliance may further comprise a network interface, the network interface comprising a first Ethernet port connecting to one of the source device or the destination device and a second Ethernet port connecting to the network.

[0038] According to a third aspect, there is provided a non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to: generate a plurality of copies of a data packet to be sent from a source device to a destination device; select a plurality of different data paths for the plurality of copies of the data packet from a plurality of different data paths in a network connecting the source device to the destination device,respectively; and transmit the plurality of copies of the data packet along the plurality of different data paths, respectively, from the source device toward the destination device.

[0039] This summary does not necessarily describe the entire scope of all aspects. Other aspects, features and advantages will be apparent to those of ordinary skill in the art upon review of the following description of specific embodiments.BRIEF DESCRIPTION OF THE FIGURES

[0040] In the accompanying drawings, which illustrate one or more example embodiments:

[0041] FIG. 1 depicts a flowchart illustrating a method designed to enhance network cybersecurity, according to an example embodiment.

[0042] FIGS. 2 and 3 depict network architectures within a Software-Defined Networking (SDN) framework that may be used to implement the method of FIG. 1, according to example embodiments.

[0043] FIG. 4 depicts a schematic block diagram of a fault-tolerant controller, according to an example embodiment.

[0044] FIG. 5 depicts a hierarchical model of a network that spans the application layer to the physical layer, according to an example embodiment.

[0045] FIG. 6 depicts an enhanced network layer configuration that integrates a fault- tolerant protocol within the datalink layer to bolster network reliability and security, according to an example embodiment.

[0046] FIG. 7 depicts a schematic representation of data flow through a fault-tolerant inline appliance, according to an example embodiment.

[0047] FIGS. 8A-8C depict a data transmission process within a network using fault- tolerant appliances to maintain high reliability and security against potential cyber threats by employing disjointed data paths, according to an example embodiment.

[0048] FIG. 9 depicts a network environment where a fault-tolerant communication system is subject to real-time packet manipulation by an adversarial entity, according to an example embodiment.

[0049] FIG. 10 depicts a flowchart detailing the operations for processing and dispatching data packets within a network system, according to an example embodiment.

[0050] FIG. 11 depicts a decision-making flowchart used by a fault-tolerant network system to handle incoming data packets, according to an example embodiment.

[0051] FIG. 12 depicts a schematic of the structure of a data packet as it is prepared for transmission within a fault-tolerant network, according to an example embodiment.

[0052] FIG. 13 depicts an example computer system that may be used to implement the method for data transmission, according to an example embodiment.DETAILED DESCRIPTION

[0053] In today's digital era, networks serve a vital role in connecting devices, systems, and individuals, facilitating smooth communication and data exchange. As networks have evolved, becoming larger and more intricate, the challenge of efficiently managing and transmitting data has grown. At its core, a network involves routing data packets from a source to a destination, ensuring their prompt and precise delivery.

[0054] Software-Defined Networking (SDN) emerges as a solution to these challenges. SDN differentiates itself by decoupling the control plane, responsible for decision-making regarding packet routing, from the data plane, which performs packet forwarding based on these decisions. This separation not only enhances programmability but also creates a more adaptable and manageable network.

[0055] Central to SDN is the role of the SDN controller. Unlike traditional networks that decide routing in a distributed fashion, SDN places the controller in charge. This controller maintains a "network information base" and, with a comprehensive view of the network topology, makes optimized routing decisions. Further, switches within the SDN frameworkpossess flow tables that determine the processing and forwarding of incoming packets. Through its centralized nature, the SDN controller populates these tables, ensuring uniform packet handling throughout the network.

[0056] SDN may offer multiple advantages. It promotes policy-based routing, allowing network administrators to set guidelines for traffic management, be it prioritization, blocking, or rerouting — a level of control earlier network models found challenging. Additionally, SDN's programmability stands out, particularly benefiting developers. The SDN controller provides Application Programming Interfaces (APIs), with the Northbound and Southbound APIs being pivotal in mediating communications between applications, the controller, and network devices. Northbound APIs in the SDN connect the SDN controller with high-level applications, allowing these applications to communicate their network needs. This facilitates network automation and abstraction. Southbound APIs, such as OpenFlow™, link the SDN controller to network devices like switches and routers, translating high-level network policies into specific device configurations. This capability, combined with SDN's advanced traffic engineering, ensures optimal network performance.

[0057] An intersection of SDN with Network Functions Virtualization (NFV) further amplifies its potential. While NFV is dedicated to virtualizing network services, its amalgamation with SDN facilitates dynamic routing through these virtualized functions, bringing forth both flexibility and cost benefits.

[0058] It should be appreciated that, while the primary focus of various embodiments of the present disclosure is on SDN networks, its scope is not limited to them. The various embodiments, and in particular the anti-cyber attack measures thereof, are versatile and suitable for a variety of network configurations, underscoring their broad relevance and applicability.

[0059] Referring now to FIG. 1, there is shown a flowchart illustrating a method 100 for enhancing network cybersecurity in diverse network environments. This method, shown in four distinct blocks 101-104, is based on the principle of using multiple data paths and redundancy to help ensure data delivery and resilience to network anomalies.

[0060] The first block, identified by reference number 101, represents the initial phase of obtaining multiple data paths, which may be optional. In this operation, the method 100 obtains a plurality of different data paths that span a network and establishes a connection between a source device and a destination device (i.e., different or distinct data paths are not identical but could still have overlapping nodes or links). The different data paths may be disjointed from each other. The term “disjoint” indicates that, aside from the source and destination devices, data paths do not share any common network nodes or links. These paths are extracted from at least one controller. Depending on the network environment, whether it's a local area network (LAN), wide area network (WAN), or even a cloud-based environment, the complexity and number of available paths can vary. For example, in densely connected LANs, there may be numerous alternative routes between devices, while in WANs, the paths may be more limited or involve various intermediate devices and subnetworks. Data paths may be obtained at block 101 in a number of different ways, as described further below. They may, for example, be downloaded to a device (e.g., to an appliance or to the source / destination device, as discussed further below) from the at least one controller as described above. They may alternatively be predetermined and manually entered to the device prior to their actual use in data transmission.

[0061] Moving to the second block represented by reference number 102, the method 100 focuses on providing redundancy. In different network conditions, the susceptibility to data loss or corruption may vary. Therefore, in this operation, multiple copies of a given data packet are generated. These copies, which are set to be sent from the source device to the destination device, ensure that even if a particular route or network segment experiences problems or one copy has been compromised by a cyberattack, other copies can still reach their destination to preserve the authenticity of data.

[0062] The third block, labeled 103, deals with path selection for each packet copy. After the data paths are determined and the packets are replicated, the routes are strategically assigned to each packet. This operation takes into account the dynamic conditions of different network environments. For example, in congested networks, the method may prioritize less congested paths or even alternate between wired and wireless routes, if available, to ensure smooth delivery.

[0063] In the context of the present disclosure, data path generation can be achieved through two approaches. First, automated discovery is employed, where data paths are automatically generated by the system. The system may, for example, automatically re-generate the data paths from time to time (e.g., periodically, such as every ten seconds, or in response to a certain event such as a detected change in network topology). Second, for certain scenarios, particularly in smaller network environments, data paths may be manually pre-calculated. This manual calculation can be facilitated either by hand or through auxiliary tools, and is primarily managed by a controller or an equivalent mechanism when pre-set paths are involved.

[0064] It should be appreciated that pre-calculated data paths represent a static view of network topology, based on the assumption of a stable and unchanging network environment. This approach requires manual configuration of network switches to recognize and process specific labels. These predetermined labels are then distributed throughout the network to specific devices (such as fault-tolerant modules or appliances, which will be described below) during the initialization phase. In one embodiment, Virtual Local Area Network (VLAN) configurations may be established to ensure compliance with a specific data path. Each VLAN defines a unique path through the network. As a result, the fault-tolerant modules or appliances transmit messages on designated VLANs, using VLAN labels embedded in packet headers to guide the flow of data. However, it should be understood that the static routing is not limited to VLAN configurations.

[0065] On the other hand, in the automated discovery scenario, the controller adheres to predefined rules or criteria for sequencing the calculated data paths, which are then communicated to each fault-tolerant module or appliance. The rules or criteria for sequencing these paths may include network congestion levels, the number of hops, or a combination of several factors. The controller then generates and prioritizes a list of potential paths according to these rules or criteria. The fault-tolerant modules or appliances are provided with this prioritized path list, with the most optimal or otherwise preferred route at the top. If anomalies are detected along a particular path, the system can flag the compromised path and switch to an alternate path (such as the next available path on the list, such as a fourth path if three paths are originally used).

[0066] Then, block 104 embodies the final transmission phase of the method 100. Here, each replicated data packet embarks on its designated route to be routed from the source device to the end recipient or destination device. The transmission process may involve various protocols and mechanisms tailored to the specifics of the network environment. For example, in cellular networks, the transmission may hop through various base stations, while in mesh networks, data packets may be relayed through multiple nodes before reaching their final destination.

[0067] As illustrated in FIG. 1 and the associated blocks 101-104, the described embodiment not only provides a dynamic method for ensuring data integrity across varying network landscapes, but also represents a way to anticipate and mitigate potential network challenges, thereby increasing data transmission capability and robustness.

[0068] Turning now to FIG. 2, which illustrates a more specific embodiment within an SDN framework, the dynamics of packet routing and fault-tolerant communication are explored. One or more fault-tolerant controllers integrate with an SDN controller 210 to ensure efficient and secure data transmission in challenging network environments. It should be appreciated that while the deployment of a single fault-tolerant controller is possible, it might not fully capitalize on the advantages of fault-tolerant functionality, primarily due to the absence of redundancy. Advantageously, the implementation of two or more fault-tolerant controllers may further enhance system reliability: in scenarios where one controller encounters a failure or security compromise, another controller can seamlessly assume control. In certain configurations, multiple fault-tolerant controllers may operate independently, each performing validation checks on the operations executed by its other controllers. Additionally, there is flexibility in the deployment of these controllers; they can be configured to run as virtual machines or in containers, which allows them to operate either on the same hardware device or across multiple distinct hardware platforms.

[0069] In the network topology 200 depicted in FIG. 2, two applications 220, 230 are observed communicating with each other. The underlying mechanism that orchestrates the routing of packets between these applications 220, 230 is the flow table present within each of switches 241, 242, 243, 244, which are distributed in a data plane, such as an SDN data plane240 in this example. Primarily, this routing process is influenced by several parameters, including aspects such as bandwidth, latency, load, and the inherent needs of the applications. Traditional Ethernet networks tend to base their routing strategies on such parameters, often resulting in subsequent packets favouring similar, if not identical, paths.

[0070] In scenarios that require high fault tolerance and cybersecurity, such a routing technique could potentially become a vulnerability. If a particular path is compromised due to a cyber threat or network anomaly, all packets traversing that same path are at risk. To avoid this, the present disclosure introduces a fault-tolerant module. The fault-tolerant module may comprise a standalone hardware appliance that is communicatively coupled to the source or destination device, computer program code that is resident on and executed by at least one processor comprising part of the source or destination device (e.g., as part of or integrated with the applications 220, 230), or a combination of both. The applications 220, 230 themselves may also be in the form of computer program code stored in at least one computer memory or storage and be executable by at least one processor. The processor and computer memory / storage may comprise part of a computer, a server, an loT device, a network router, or any other electronic device that demands secure and efficient communication, such as is described in respect of FIG. 13 below. In this context, the device on which the application 220 for transmitting the data packet is located may be referred to as the source device, while the device on which the application 230 for receiving the data packet is located may be referred to as the destination device. However, it should be appreciated that the source device may also serve as the destination device and the destination device may also serve as the source device, because each of the applications 220, 230 may simultaneously be both the source and destination devices (e.g., different packets may be sent bi-directionally between the two devices). This flexibility ensures that diverse hardware architectures and software environments can benefit from enhanced security protocols, ultimately fortifying the entire network ecosystem against potential threats.

[0071] The fault-tolerant module shown in FIG. 2, either in software form or hardware form, is embedded in the source and destination devices, such as in the form of computer program code running on the source device and destination devices. In this embodiment, originating packets from the source device are replicated, such as in triplicate, and then sent across thenetwork to reach their intended destination device. At the receiving end, the destination device performs a comparison among these packet copies. Using a principle of majority consensus, at least two of the three packets shall match. If this criterion is met, the packet moves forward in the processing pipeline. It should be appreciated that two copies of the data packet or more than three copies of the data packet can also be produced by the source device or application.

[0072] The fault-tolerant controller associated with the SDN controller 210 recognizes possible routes in this architecture. Its primary task is to discover and optionally label data paths between different applications. After identifying these data paths, it passes this information to the fault-tolerant module at the source device and optionally to the fault-tolerant module(s) at additional device(s). Armed with the path information (such as all discovered routes and labels for these routes), the fault-tolerant module associated with the source device (such as the application 220 in this example) can route data packets along different or disjointed data paths, thereby increasing fault tolerance. Alternatively, in certain configurations, the fault-tolerant module associated with the source device may actively query the fault-tolerant controller to obtain information about available data paths.

[0073] A benefit of this configuration is its adaptability to existing network infrastructures. There is no need to overhaul or replace current electronics or cabling. By integrating the fault-tolerant module, the network immediately strengthens its defenses against cyber threats. Should a module experience problems, it can be quickly located and addressed, ensuring minimal downtime and service disruption.

[0074] The potential applications of this fault-tolerant dynamic routing (FTDR) mechanism span a wide range. From commercial and military shipping, to loT configurations, to critical infrastructure components such as hydroelectric plants, water treatment facilities, and automated ports, the use of such a system can significantly strengthen their cyber resilience. This not only minimizes potential disruptions, but also ensures consistent and secure operational efficiency across multiple industry sectors.

[0075] FIG. 3 illustrates another network architecture 300 according to another embodiment of the present disclosure. This robust architecture 300 is designed to fortify datapacket transmission over a network, such as an Ethernet network. The primary objective is to thwart potential cyber threats by dynamically routing data packets between source and destination applications through multiple disjointed paths. The term “fault-tolerant” is abbreviated as “FT” in FIG. 3.

[0076] In this context, "dynamic" signifies the capability to change the routing of different data packets sent as part of transmission of a series of packets. This allows for the selection and modification of the paths for data transmission during the actual time of communication, adapting to various factors such as network traffic congestion, potential security threats, or hardware failures. In the dynamic scenario, there are multiple potential paths, more than the number of copies of the packet. For example, before packets are sent, three routes may be selected based on current network conditions. If one of the three selected paths is compromised during transmission, the system can dynamically switch to an alternate route, such as a fourth path. This flexibility ensures more reliable data transmission because the system can quickly adapt to any disruptions or security risks in the network. Notably, the dynamic switching of data paths for data transmission in response to detecting a compromised data path occurs for a subsequent data packet instead of the current data packet that was transported on the compromised data path. It should be understood, however, that static routing, in which a number of available routes (more than the intended number of copies of data packet) are pre-calculated for each fault-tolerant module, is still feasible in the present disclosure.

[0077] In a traditional network scenario, particularly one in which switches are not SDN- enabled but can route VLANs, each data path between a source device and a destination device is precomputed and predetermined. In these networks, separate data paths are created by configuring separate VLANs across the network infrastructure. Each VLAN acts as a unique path for data transmission. As a result, a table is created containing the source device, destination device, and VLAN identifier for each path. It should be noted that there can be multiple distinct data paths for each source-destination pair. This table, specific to the network configuration, is then stored in the fault-tolerant appliance to manage data routing. Any changes to the network would require this process to be repeated. This approach applies the principles of the presentdisclosure to traditional networks by leveraging VLAN capabilities for routing, even in the absence of SDN-enabled switches.

[0078] Alternatively, the implementation allows for automatic discovery of the network topology. This enables automatic calculation of distinct paths and configuration of SDN switches for packet forwarding. This can be accomplished using Multiprotocol Label Switching (MPLS) labels or VLAN identifiers. A table correlating source device, destination device, and MPLS labels (or VLAN identifiers) is created and stored in the fault-tolerant appliance. Another possible alternative is to keep the table in the fault-tolerant controller and have the fault-tolerant appliance retrieve the distinct routes for a given destination from the controller.

[0079] In this particular embodiment with respect to FIG. 3, SDN switches and MPLS labels are used. This design is characterized by its distributed fault-tolerant structure, designed specifically for SDN networks. The SDN network is equipped with the ability to auto-discover the network topology almost instantaneously. For example, the discovery performed by the SDN controller may be done every 10 seconds by re-calculating the network topology. It can also compute and optimize all potential data paths between any source and destination pair, given constraints. For example, when the network changes or an anomaly occurs, a list of possible data paths may be renewed based on shortest distance as measured by number of hops. However, it should be understood that the present disclosure is not intended to be limited to these specific types of switches, labels, and networks, and other available options may be substituted as appropriate.

[0080] In this example, the architecture is designed to allow SDN switches to perform data packet forwarding based on observed labels for path switching. This aspect of the architecture is complemented by the capacity of the fault-tolerant appliance to dynamically select data paths for each newly generated data packet. The architecture is versatile enough to accommodate both scenarios: one where data paths are pre-calculated and established, and the other where data paths are dynamically altered in response to detected faults for subsequent packets. This adaptability is further enhanced by the integration of one or more fault-tolerant controllers within the SDN network, reinforcing the network's ability to swiftly adapt to changing conditions.

[0081] FIG. 3 depicts another example of a network architecture 300. Turning to FIG. 3, an SDN data plane 330 includes multiple switches 331, 332, 333, 334 interconnected to each other, like the switches in the data plane described in respect of FIG. 2. These switches 331, 332, 333, 334, designed for multidirectional data transmission as depicted by interconnected lines, ensure an intricate arrangement for redundancy and flexibility in data path selection. Such a configuration improves, and ideally optimizes, data flow while enhancing the system's fault tolerance. In this example, the switches 331, 332, 333, 334 may be SDN switches.

[0082] A control plane 320 is provided independently from the data plane 330. While the data plane 330 is responsible for directing data packets from their origin to their destination, the control plane 320 sets the rules for routing. It is worth noting that these SDN switches 331, 332, 333, 334 are in sync with one or more controllers in the control plane, ensuring that every switch in the network is consistently, and ideally continuously, updated and monitored.

[0083] The control plane 320 may include at least one controller. Each of these controllers may include both a fault-tolerant controller and an SDN controller, or it may be an integrated entity that consolidates the functionality of both types of controllers. In the given configuration, there are three different fault-tolerant controllers: 321, 323, and 325. Each of these is tightly linked to its own corresponding SDN controller, labeled 322, 324, and 326, respectively. Typically, the fault-tolerant controller ensures uninterrupted and coherent communication with its associated SDN controller. The main functions of the fault-tolerant controller include autodiscovery of the network topology and formulation of label-based routing policies derived from observations and data from the SDN switches. In this setup, the SDN controllers are primarily tasked with obtaining the network topology. They set up VLANs, MPLS policies and other network configurations in response to direct instructions from their corresponding fault-tolerant controllers. This hierarchical relationship ensures that the SDN controller acts upon the instructions provided by the fault-tolerant controller. The SDN switch is configured by the SDN controller to process traffic according to forwarding policies. In essence, the SDN controller facilitates automated network management, and may be referred to as an operating system for the network. It can propagate any changes or updates back to the fault-tolerant controller, ensuring that the network remains resilient, efficient, and adaptable to evolving requirements.The overall topology underscores the importance of availability, reliability, and rapid recovery from cyber breaches.

[0084] It should be understood that the specific configuration depicted in FIG. 3 serves merely as a representative example, and is by no means restrictive. Depending on the complexity and requirements of the network, the number of controllers employed may vary. A combination of multiple fault-tolerant controllers could be associated with a single SDN controller, or vice versa. Furthermore, while the scenario described reflects a typical SDN network, the architecture and principles can be applied to a variety of network types, including but not limited to hybrid networks, legacy infrastructures, or cloud-based systems. In addition, the components presented and their functionalities may be modular, allowing for customization and adaptability to different use cases.

[0085] Looking at the details of these fault-tolerant controllers, each one is a combination of several components. For example, a fault-tolerant manager component acts as the glue that holds all the other components together. A network discovery component leads the way in understanding the overall network structure. A controller discovery component specializes in discovering other controllers. Another notable component is the fault-tolerant broker component, which is designed to manage new devices being joined to the network. An SDN manager component oversees all SDN-related functions and operations, while a communication manager component ensures communication between the different modules of the fault-tolerant network (where the fault-tolerant network may include different modules, such as inline appliance(s); gateway device(s); operation, administration, and maintenance module(s); other controller(s); and SDN module(s)). These components will be described in more detail in respect of FIG. 4.

[0086] An operation, administration, and maintenance (0AM) unit 310, tied to each controller (whether an integrated controller or a pair of the fault-tolerant and SDN controllers), may act as an overarching management tool, overseeing operations, administration, and management of the network architecture 300. Its main roles include monitoring system health, fault detection and ensuring a high-level of, and ideally optimal, performance.

[0087] In this example, a first application 340 is connected to the data plane 330 via a first fault-tolerant appliance 341, and a second application 350 is connected to the data plane 330 via a second fault-tolerant appliance 351. The first and second applications 340, 350 may represent a variety of digital devices, such as computers, mobile devices, servers, loT gadgets, or any other network-enabled entities. When data packets need to be relayed from one application to another, the initiating application assumes the role of the source device, while the recipient becomes the destination device. The function of the fault-tolerant appliance can vary in its implementation. While in some embodiments it is a standalone hardware entity or an inline appliance, featuring a primary Ethernet port connected to the respective application and a secondary Ethernet port connected to the data plane 330, other configurations are feasible in other embodiments, such as wireless transceivers in place of the Ethernet ports. It may also be fully integrated with its associated application, eliminating the need for separate hardware. Additionally, in certain scenarios, the fault-tolerant functionality can manifest as a software component, seamlessly embedded within the application's ecosystem, which offers flexibility and scalability. However, the choice between hardware and software, or a hybrid of both, will largely hinge on the specific requirements of the network, the resources at hand, and the desired level of fault tolerance and performance efficiency.

[0088] Moving from applications and their connectivity to the broader network infrastructure, fault-tolerant controllers play an advantageous role. An SDN controller initiates a network discovery process via Link Layer Discovery Protocol (LLDP) at regular intervals. For example, the SDN controller(s) may map out the network topology with the LLDP protocol every 10 seconds. As a result, it creates the network topology which is then used by the Fault-tolerant controller. When a new device joins the network, a fault-tolerant controller determines all accessible distinct or disjointed data paths. These controllers are not isolated entities; they may maintain awareness of their peers operating within the same subnet. This networked knowledge ensures that all identified data paths are consistently shared and updated across some or all of the fault-tolerant appliances. Then, the fault-tolerant controller may label the discovered data paths and send the labeled route information to one or more, if not all, fault-tolerant appliances in the network architecture including all other controllers. However, if discrepancies occur in the labeled paths or the consistency among the controllers drop below a predetermined threshold,this anomaly is communicated to other fault-tolerant controllers and the 0AM unit. This rapid identification helps pinpoint the problematic controller, which is then logged, alerted, and taken offline to prevent further inconsistencies. In such scenarios, redundancy comes to the fore, as another fault-tolerant controller from the same subnet steps up to take over the responsibilities of its compromised counterpart.

[0089] The fault-tolerant appliances may receive labeled data paths from the fault- tolerant controller(s), and this process may be initiated by either the fault-tolerant controller or the fault-tolerant appliance. For example, after discovering and labeling the available distinct or disjointed data paths between applications on the network, the fault-tolerant controller(s) can pass this information to the fault-tolerant appliance(s). Alternatively, when a fault-tolerant appliance receives data packets from its corresponding application, it can query the fault-tolerant controller(s) for available data paths.

[0090] Upon receiving a data packet to be sent to a destination device, the fault-tolerant appliance replicates it into a number of identical copies, such that there are a total of two or more identical data packets, and forwards each copy via a corresponding distinct or disjoint path to its intended destination. At the receiving end, another fault-tolerant appliance uses a voting mechanism to detect any discrepancies and records any anomalies with the fault-tolerant controller(s). In one example, the voting mechanism may check the data packet, excluding the routing information in the header, against its counterparts in the other copies in a byte-by-byte manner. If there is a discrepancy at the byte level for one packet compared with the other packets, the mechanism identifies the packet with the discrepancy as a different data packet. For instance, if two packets are identical while the third differs, the voting mechanism considers the contents of the two packets in the majority to be accurate and the contents of the differing data packet to be anomalous; consequently, the data path taken by the anomalous data packet may be identified as having been compromised. In addition, a fault-tolerant controller anomaly counter may be involved to register the total number of anomalies. If these anomalies exceed a predefined threshold, the data path used to transport the anomalous data packet(s) can be blacklisted, either due to component failures or cyber-attacks. The fault-tolerant controller(s) can then update all fault-tolerant appliances, pruning the failed path from the available data paths. It should beappreciated that other methods for the comparison of the copies of data packet are possible, such as checksum-based verification, cryptographic hash functions, and the like, which can be used independently or in combination.

[0091] For the transmission of the data packet, the fault-tolerant appliances may optionally assign different labels to the multiple copies of the data packet so that each copy can be routed independently over distinct paths in accordance with the labeling of the data paths. This labeling of data packet ensures that even if one data path becomes compromised or encounters a cyber attack, the other copies may still maintain data integrity and continuous service.

[0092] The integration of fault-tolerant appliances ensures the redundancy of transmitted data. These appliances, equipped with the voting mechanism, dynamically adapt to anomalies and continuously update the fault-tolerant controller with observed anomalies.

[0093] The network architectures described herein leverage distributed fault-tolerant computing to detect, mitigate, and rectify cyberattacks in real-time, achieving this accurately and without necessitating a connection to cloud-based services. An advantage of at least some of the disclosed embodiments is their non-disruptive integration into existing infrastructures. There is no prerequisite to displace or overhaul current electronic systems or wiring frameworks. Compatibility with existing applications is maintained, as no modifications or upgrades are necessary to harness the cybersecurity enhancements provided by this technology.

[0094] Moreover, the operational maintenance of this system is streamlined and cost- efficient, characterized by the simple identification and replacement of any impacted fault- tolerant appliance. This feature renders the disclosed solution both economically viable and highly adaptable, particularly suitable for networks that operate under rigorous conditions, such as industrial settings or wartime environments.

[0095] The utility of this solution extends across diverse domains, offering significant protective benefits to various forms of shipping, encompassing both commercial and defense- related shipping, as well as a multitude of Internet of Things (loT) applications. This aids in mitigating the repercussions of cyber threats. Additionally, the proposed technology underscores the importance of FTDR, which presents substantial benefits to several critical market sectors.Notably, this includes but is not limited to, industries such as hydroelectric power distribution (inclusive of microgrid systems), water treatment facilities, and automated material handling within container ports, which are pivotal in supply chain management.

[0096] FIG. 4 illustrates a schematic block diagram of the fault-tolerant controller that comprises part of the network architecture described herein. The term “fault-tolerant” is abbreviated as “FT” in FIG. 4.

[0097] A fault-tolerant manager block, shown as 440, serves as the central coordination hub, interfacing with various components, including network discovery block 410, controller discovery block 420, fault-tolerant broker block 430, SDN manager block 460, databases 450, and communication manager block 470. It orchestrates operations and ensures harmonious cooperation among all components within the fault-tolerant controller. In addition, the fault- tolerant manager block 440 may analyze all incoming anomaly reports. Upon identifying an active threat, it promptly selects and executes an appropriate response action to mitigate the threat. These actions include, but are not limited to, calculating new data paths, isolating compromised network segments, or optionally reporting to the 0AM unit.

[0098] The network discovery block, shown as 410, is responsible for acquiring near realtime network topology from the SDN controller(s). This block analyzes the network to create a comprehensive map detailing all potential data paths between any given source-destination pairs. It computes these paths, taking into account specific constraints, to prioritize network routes. If available, disjointed paths may be calculated and prioritized first.

[0099] Adjacent to the network discovery block 410 is the controller discovery block, shown as 420. The function of this block is to identify other fault-tolerant controllers on the same subnet or control plane. In doing so, it facilitates seamless and integrated operation among controllers, promoting a robust and unified network defense mechanism against potential cyber threats.

[0100] The fault-tolerant broker block, shown as 430, performs the tasks of discovery and registration of all fault-tolerant appliances deployed in the network architecture. This ensures that each fault-tolerant appliance is recognized and able to communicate within the network.

[0101] In the software-defined networking domain, the SDN manager block, shown as 460, is responsible for bi-directional communication between the fault-tolerant controller(s) and the SDN controller(s). It translates and routes messages and commands, enabling dynamic network configuration and immediate response to changing network conditions or threats. However, it should be understood that, in instances where the SDN controller(s) functionalities are combined within the fault-tolerant control! erf s), the SDN manager block 460 would not operate in isolation. Instead, it would interface with an additional control block that embodies the functionalities of the SDN controller, ensuring cohesive management of both fault-tolerance and SDN-related tasks.

[0102] Last, the communication manager block, shown as 470, is responsible for managing the exchange of information between the 0AM unit, the fault-tolerant controller(s) and appliances, and gateway appliances. This includes both the coordination and transmission of data to ensure that all components within the network are updated and working together.

[0103] Expanding the scope of the architecture to include multiple subnets, the system also permits the integration of a fault-tolerant gateway (not shown in FIG. 4). The purpose of this gateway is to facilitate the sharing of discovered fault-tolerant appliances by fault-tolerant controllers across different subnets. The deployment of multiple fault-tolerant gateways per subnet is intended to ensure that the resilience and self-healing capabilities of the network are comprehensively extended to the entire interconnected environment, thereby achieving the goals of cyber-attack survivability and system recovery across the entire network footprint.

[0104] Referring now to FIG. 5, an example illustration of the hierarchical network model extending from the application layer to the physical layer is shown. This model represents a framework within the art of computer networking and is used to categorize the communication layers in a network stack.

[0105] At the top of the hierarchy is the application layer 510. This layer includes the high-level interfaces and protocols that facilitate end-user services and applications. It provides protocols that software applications use to communicate over the network, and ensures that data is properly packaged for further processing down the stack.

[0106] Below the application layer is the transport layer 520, which includes both the User Datagram Protocol (UDP) 521 and the Transmission Control Protocol (TCP) 522. These protocols are useful for providing communication services directly to application processes running on different hosts. UDP allows applications to send messages, known as datagrams, without the need for prior communication to establish special transmission channels or data paths. Conversely, TCP provides reliable, ordered, and error-tested delivery of a stream of octets between applications running on hosts communicating over an IP network.

[0107] Following the transport layer is the Internet Protocol (IP) layer 530, also referred to as the network layer. The core function of the IP layer is to deliver packets from the source host to the destination host based on their addresses. This layer defines packet structures that encapsulate the data to be delivered. It also handles the routing of these packets across complex networks through routers and other devices.

[0108] After the IP layer, the datalink layer 540, or simply the data layer, is responsible for node-to-node data transfer - a link between two directly connected nodes. It also handles error correction from the underlying physical layer. In this context, the data layer may be implemented as two sub-layers: the Logical Link Control (LLC) layer, which handles frame synchronization, flow control, and error checking; and the Media Access Control (MAC) layer, which handles protocol access to the physical network medium.

[0109] The fundamental layer of this model is the physical layer 550, shown in FIG. 5 as the Ethernet interface. This layer transports the bit stream through the network at the electrical and mechanical level. It provides the hardware means to transmit raw, unstructured data streams over a physical medium.

[0110] It should be noted that the fault-tolerant protocol, as mentioned earlier, can be used in two different ways within this layered architecture. One implementation is as an inline device that is physically integrated into the network to transparently process and manage data. Alternatively, the fault-tolerant protocol can be executed within the network stack itself, potentially eliminating the need for a separate inline device and allowing for a more streamlined and integrated approach to error management and network data processing.

[0111] FIG. 6 illustrates the enhanced network layer architecture that incorporates the fault-tolerant protocol 641 within the datalink layer 640, which is integral to the overall functionality and security of the network. The application layer 610, transport layer 620, IP protocol layer 630, and physical layer 650 are identical to those described in FIG. 5 and provide the same basic network functionality as previously described. The term “fault-tolerant” is abbreviated as “FT” in FIG. 6.

[0112] The fault-tolerant protocol 641 is inserted at the datalink layer 640 and interfaces directly with the IP protocol. By adding FTDR capabilities at this junction, immediately above the physical layer, the protocol provides a higher level of data integrity and security without requiring changes to the higher layers.

[0113] Specifically, the fault-tolerant protocol 641 captures and processes IP protocol packets, enforcing a layer of fault tolerance and redundancy. It seamlessly adds robustness to network communications, operating transparently to applications and end users, while significantly increasing the resiliency of the network infrastructure.

[0114] In essence, the fault-tolerant protocol 641 acts as a strategic extension within the network stack, contributing to a resilient network design that can withstand a range of operational challenges and security threats. The inclusion of this protocol in the standard model underscores a forward-thinking approach to network architecture aimed at ensuring continuous and secure data exchange.

[0115] FIG. 7 is a schematic representation of the data flow through a fault-tolerant inline appliance 700, illustrating its operational integration into a network environment to enhance fault tolerance by utilizing FTDR. The term “fault-tolerant” is abbreviated as “FT” in FIG. 7. The fault-tolerant inline appliance 700 is a particular example of the first fault-tolerant appliance 341 or the second fault-tolerant appliance 351 as discussed above in respect of FIG. 3, and it is designed to seamlessly integrate into network infrastructures where application source code is not accessible, such as proprietary third-party systems. This fault-tolerant inline appliance 700 enables the adoption of FTDR benefits without requiring changes to existing application software.

[0116] At the heart of the fault-tolerant inline appliance 700 is the fault-tolerant protocol component 710. This component intercepts network traffic at the datalink layer and applies FTDR methods to facilitate the transmission of data packets with enhanced security and redundancy. The fault-tolerant protocol component 710 operates transparently, implementing fault tolerant mechanisms that are imperceptible to the end user.

[0117] The fault-tolerant inline appliance 700 is equipped with a minimum of two Ethernet interfaces 720, 730 that serve as a conduit for data packets between the application network and the fault-tolerant network infrastructure. The first Ethernet interface 720 receives incoming network traffic from the application's device network, while the second Ethernet interface 730 dispatches the processed traffic to the fault-tolerant network infrastructure.

[0118] The data layer 740, 750 is an integral part of the fault-tolerant inline appliance 700, located both upstream and downstream of the fault-tolerant protocol component 710. Its function is to maintain the structural and operational integrity of the data packets as they pass through the appliance 700, ensuring that the encapsulated data remains compliant with network standards and protocols.

[0119] In operation, data flow through the fault-tolerant inline appliance 700 begins with the first Ethernet interface 720 receiving packets from a remote application. These packets then proceed to the first data layer 740, where initial processing occurs before they are passed to the fault-tolerant protocol component 710. Here, the data packets are replicated into multiple copies (and optionally assigned with labels) and then passed back through the second data layer 750, exiting the fault-tolerant inline appliance 700 via the second Ethernet interface 730 to reach the fault-tolerant network, thus completing the transmission cycle.

[0120] This hardware configuration illustrates the utility of the fault-tolerant inline appliance 700 as a key element in establishing a resilient and secure network framework capable of supporting data communications such as the software configuration discussed with respect to FIG. 6.

[0121] FIGS. 8 A to 8C depict an example schematic representation of a data transmission mechanism employed by fault-tolerant appliances within a network to ensure high reliability andsecurity against potential attacks, such as a man-in-the-middle attack, by utilizing path diversity. It should be understood that inline appliances 820, 840 are merely used as an example for this representation, while software fault-tolerant functionality can also be embedded with the end applications 810, 830, as discussed above.

[0122] The inline appliance's protocol incorporates a routing strategy that enhances data integrity and reliability. Upon initiation of a data transmission, the fault-tolerant protocol dynamically generates multiple redundant copies of the data packet, hereafter referred to as "replicated packets", which are transmitted across the network over multiple, different data paths.

[0123] As shown in FIG. 8, the inline appliances 820, 840 are positioned at both the sending and receiving ends of the transmission process. The role of each inline appliance is to facilitate the multiplication of the data packets and to orchestrate the selection of divergent routes across the network topology for each replicated packet to traverse, thereby mitigating the risk of complete data loss or corruption.

[0124] The network switches, shown as 850, 860, 870, 880, represent the intermediate network devices that direct the flow of data packets through the network. Each switch operates to forward incoming data packets to the next designated switch in the path, or ultimately to the receiving inline appliance 840.

[0125] The scenario shown illustrates three different paths. The first path for the replicated packet includes a route through switch 850 to switch 860 and then to switch 880. The second path diverges and routes the packet from switch 850 directly to switch 880, bypassing switches 860 and 870. The third path illustrates yet another alternative routing in which the data packet proceeds from switch 850 to switch 870, then to switch 880. Each path represents a unique sequence of switches, thus providing path redundancy. It should be understood that two or more than three different paths is also feasible.

[0126] Path differentiation is an advantageous feature of the fault-tolerant protocol, ensuring that even if one or more paths are compromised, either by malicious activity such as eavesdropping or tampering, or due to network failures, at least one of the replicated packets has a significantly higher chance of arriving at its destination unaltered. Alternatively, usingdisjointed paths may be further advantageous, because it ensures that there is no single path that an attacker can select to compromise multiple paths.

[0127] The process concludes with the receiving inline appliance 840, which is responsible for reconstructing the original data from the replicated packets. Utilizing a voting mechanism, for example, the receiving inline appliance 840 scrutinizes each data packet's integrity, comparing replicas to detect variances indicative of tampering or corruption. This mechanism will be further explained in the following with respect to FIG. 9.

[0128] Referring now to FIG. 9, a network scenario embodying the principles of a fault- tolerant communication system of FIG. 8 is depicted in which an adversarial entity 910 engages in real-time packet manipulation, typically referred to as a "man-in-the-middle" attack, on a first data transmission path. The figure explicitly shows the adversary's interception of data packets between Switch 850 and Switch 860, using dashed lines to symbolize the unauthorized access and potential modification of the packet's contents.

[0129] However, the fault-tolerant functionality in the form of inline appliance 820, 840, represented at each end of the communication, is designed to negate the effectiveness of such an attack by utilizing a redundancy protocol in which each data packet is triplicated or further multiplied, with each copy traversing a separate path within the network infrastructure. The receiving inline appliance 840, which incorporates previously disclosed data replication and routing mechanisms (as introduced with respect to FIG. 8), performs an analysis upon receipt of the data packets.

[0130] The fault-tolerant protocol inherent in the receiving inline appliance invokes a "voting" mechanism to determine the integrity of the received data packets. This mechanism identifies discrepancies between replicas, effectively isolating the compromised data packet as an outlier. When an anomaly is detected, the exact path of the outlier, as well as the anomaly itself, is systematically reported to one or more centralized fault-tolerant controller. In at least some example embodiments, a simple majority voting mechanism may be used (e.g., if three replicated data packets are received and only two match, the two that match are deemed to be theauthentic data packet and the one that doesn’t is deemed inauthentic and its data path compromised).

[0131] The fault-tolerant controller, a central component of the network's defense, maintains a meticulous log of all activities and anomalies related to each inline appliance. It may be equipped with a counter that triggers a network response when a predefined threshold - a quantitative measure of persistent adversarial activity - is exceeded. In other words, the counter tracks the number of times a particular data path is considered compromised, and when that number reaches a specified threshold, it activates pre-defined protection responses. These responses may include rerouting data through alternative paths, isolating the compromised path to prevent further unauthorized access, initiating automated diagnostic procedures to identify and address the underlying security breach, or disabling the compromised route and then distributing an updated routing table to the affected inline appliances, thereby preserving the integrity of the network's data transmission and maintaining operational continuity.

[0132] FIG. 10 is a flowchart illustrating a method for processing and forwarding data packets within a network employing any of the embodiments described herein. The method shown enhances the security and reliability of data transmission by employing replication and optional encryption techniques.

[0133] The method begins when the system enters a state of readiness to receive data packets (operation 1001). Upon receipt of a data packet designated for protection, the method proceeds to replicate the original packet to create multiple identical copies (three copies in this example), thereby initiating the fault tolerant mechanism (operation 1002).

[0134] The packet's index is initially set to zero (operation 1003), which serves as a reference count for the replication process. The method then enters a loop in which the index is incremented by one for each copy of the packet (operation 1004). The incremented index uniquely identifies each replicated packet.

[0135] At this point, the method provides an option to encrypt the packet (operation 1005), which increases the security of the data during transmission. If encryption is selected, anencryption identifier is assigned to the packet (operation 1006), which is used to decrypt the packet upon arrival at the destination.

[0136] Next, the method includes adding a routing label to each packet (operation 1007). This label may be derived from a lookup operation that determines the destination based on the current index. This operation ensures that each replicated packet is routed through a different data path within the network.

[0137] Once the data packet is prepared with the routing label (and encrypted, if applicable), it is packaged and placed in a queue for transmission (operation 1008). This process is repeated for each replicated packet until the index indicates that all copies have been processed and queued.

[0138] Finally, the method concludes by sending the queued packets out into the network, where they traverse the designated paths to their intended destination (operation 1008). By using multiple different or disjointed data paths, the method mitigates the risks associated with single points of failure and potential security breaches, as discussed with respect to the embodiments described herein.

[0139] FIG. 11 depicts a decision flow implemented by a fault-tolerant network system according to an embodiment described herein when handling received data packets. This flow may be a component of the fault-tolerant protocol that facilitates data integrity and security.

[0140] The process begins with the system in a standby mode, waiting for incoming data packets from the fault-tolerant network (operation 1101). Upon receipt, the system first determines if the data packet is marked for fault-tolerant protection. If the data packet does not require protection, it is sent directly to the target device (operation 1102). Conversely, if the data packet is marked for protection, it means that there should be multiple (specifically, in this example, three) replicated instances of that data packet generated by the fault-tolerant protocol to protect against data corruption or cyber attack. The system, if applicable, may decrypt the data packet (operation 1103) and then waits until receiving all three replicated packets for comparison.

[0141] If three packets are received or if a timeout since the first packet received has been triggered, the system proceeds to a comparison algorithm (operation 1104). For example, this algorithm may perform a "voting" process to determine if all of the three packets are in concurrence - that is, they are identical and therefore presumably unaltered during transmission. Then, the data packet is sent to the destination device (operation 1102). It is important to note that the system is capable of receiving one, two, three, or even more packets. In cases where only two packets are received, the system will compare these two packets. If they are found to match, one of them is forwarded to the destination application, with any routing labels or tags removed. If the third packet is received after a timeout has occurred, it may not be used for comparison, but may be reported to the fault-tolerant controller for further processing. This process, although not shown in the diagrams or explicitly described in the text, is an integral part of the system's functionality. It should be understood that there may be various edge cases in this process, and it is expected that anyone skilled in the art would comprehend the operations necessary to address these scenarios.

[0142] If not all of the three copies of the data packet agree (meaning that the difference is larger than a predetermined threshold) with each other, the system may check whether two of the three copies agree. If so, the receiving fault-tolerant appliance may log the culprit data path (used to transport the culprit data packet) with the fault-tolerant controller (operation 1105), as discussed above with respect to various embodiments described herein. This majority rule ensures that even if one packet is compromised, the integrity of the data is maintained by the agreement of the other two.

[0143] In the event of a discrepancy where no two packets agree, the system identifies this as a critical anomaly and raises a critical alarm with the fault-tolerant controller (operation 1106). This alert indicates the need for immediate attention and possible reconfiguration of the network to mitigate any ongoing security risks.

[0144] This flowchart illustrates the robust error checking and security mechanisms inherent in the fault-tolerant network design that help to ensure data integrity and network resilience to cyber threats.

[0145] FIG. 12 shows an example schematic of the structure of a data packet as it is prepared for transmission within a fault-tolerant network, with an expanded view of the protocol header 1220 associated with the fault-tolerant appliance, according to an embodiment described herein. The term “fault-tolerant” is abbreviated as “FT” in FIG. 12. This expanded header view shows the inclusion of an MPLS label 1221 (or alternatively a VLAN label) and an encryption identifier 1222. The encryption identifier 1222 may alternatively be incorporated directly into the packet 1230 itself. An alternative embodiment may utilize MPLS label stacking, whereby the encryption ID is integrated in the form of a special MPLS label positioned at the bottom of the stack. These features are advantageous for securing and routing the data packet through the network.

[0146] The datalink header 1210, located at the top of the data packet 1230, facilitates the transfer of data between directly connected network nodes. Next to the datalink header 1210 is the fault-tolerant protocol header 1220, which encapsulates routing and / or security information provided by the fault-tolerant protocol.

[0147] Within the fault-tolerant protocol header 1220, the MPLS label 1221 (or VLAN label in alternative implementations) is positioned according to a relevant standard. These labels are for the network's switches and routers to make forwarding decisions, allowing the data packet to traverse the precomputed paths established by the network's control layer. MPLS labels are particularly advantageous in facilitating efficient and flexible data forwarding, while VLAN labels are typically used to segment network traffic into distinct, isolated groups to enhance security and traffic management.

[0148] The encryption identifier 1222 in the protocol header 1220 serves as an indicator of the packet's security status. In an example embodiment, a non-zero identifier may be used to indicate that the data within the packet has been encrypted, providing an additional layer of security by obfuscating the contents from potential eavesdroppers. Conversely, a zero identifier may be used to indicate that the packet's contents are unencrypted, possibly because the data does not require encryption or is being transmitted within a secure or trusted network segment. It should be noted that variations in the configuration of the encryption identifier 1222 and theprotocol header 1220 are possible, accommodating different system requirements and specifications.

[0149] In light of the foregoing detailed description of the various embodiments, it should be recognized that the process of discovering and establishing disjoint paths between nodes in a network is not limited to any particular pattern of data flow. It can be symmetric, where the paths for transmitting (TX) and receiving (RX) data are identical, thereby facilitating a straightforward and uniform flow of information. Alternatively, the paths can be asymmetrical, providing different paths for the TX and RX directions. This asymmetric approach may enhance privacy and security because a monitoring entity on one path would only capture a unidirectional stream of data, potentially missing information flowing in the opposite direction. Such a configuration could be particularly advantageous in scenarios where increased confidentiality is required and network traffic monitoring is a concern.

[0150] An example computer system in respect of which the one or more of the methods described above may be implemented is presented as a block diagram in FIG. 13. The example computer system is denoted generally by reference numeral 1300 and includes a display 1302, input devices in the form of keyboard 1304a and pointing device 1304b, computer 1306 and external devices 1308. While pointing device 1304b is depicted as a mouse, it will be appreciated that other types of pointing device, or a touch screen, may also be used.

[0151] The computer 1306 may contain one or more processors or microprocessors, such as a central processing unit (CPU) 1310. The CPU 1310 performs arithmetic calculations and control functions to execute software stored in a non-transitory internal memory 1312, preferably random access memory (RAM) and / or read only memory (ROM), and possibly storage 1314. The storage 1314 is non-transitory may include, for example, mass memory storage, hard disk drives, optical disk drives (including CD and DVD drives), magnetic disk drives, magnetic tape drives (including LTO, DLT, DAT and DCC), flash drives, program cartridges and cartridge interfaces such as those found in video game devices, removable memory chips such as EPROM or PROM, emerging storage media, such as holographic storage, or similar storage media as known in the art. This storage 1314 may be physically internal to the computer 1306, or externalas shown in FIG. 13, or both. The storage 1314 may also comprise a database for storing a set of images as described above.

[0152] The one or more processors or microprocessors may comprise any suitable processing unit such as an artificial intelligence accelerator, programmable logic controller, a microcontroller (which comprises both a processing unit and a non-transitory computer readable medium), Al accelerator, system-on-a-chip (SoC). As an alternative to an implementation that relies on processor-executed computer program code, a hardware-based implementation may be used. For example, an application-specific integrated circuit (ASIC), field programmable gate array (FPGA), or other suitable type of hardware implementation may be used as an alternative to or to supplement an implementation that relies primarily on a processor executing computer program code stored on a computer medium.

[0153] Any one or more of the methods described above may be implemented as computer program code and stored in the internal memory 1312 and / or storage 1314 for execution by the one or more processors or microprocessors.

[0154] The computer system 1300 may also include other similar means for allowing computer programs or other instructions to be loaded. Such means can include, for example, a communications interface 1316 which allows software and data to be transferred between the computer system 1300 and external systems and networks. Examples of communications interface 1316 can include a modem, a network interface such as an Ethernet card, a wireless communication interface, or a serial or parallel communications port. Software and data transferred via communications interface 1316 are in the form of signals which can be electronic, acoustic, electromagnetic, optical or other signals capable of being received by communications interface 1316. Multiple interfaces, of course, can be provided on a single computer system 1300.

[0155] Input and output to and from the computer 1306 is administered by the input / output (VO) interface 1318. This VO interface 1318 administers control of the display 1302, keyboard 1304a, external devices 1308 and other such components of the computer system 1300. The computer 1306 also includes a graphical processing unit (GPU) 1320. The latter may alsobe used for computational purposes as an adjunct to, or instead of, the CPU 1310, for mathematical calculations.

[0156] The external devices 1308 include a microphone 1326, a speaker 1328 and a camera 1330. Although shown as external devices, they may alternatively be built in as part of the hardware of the computer system 1300.

[0157] The various components of the computer system 1300 are coupled to one another either directly or by coupling to suitable buses.

[0158] In the context of the various controllers, appliances, and source / destination devices described herein, it should be noted that these components may be implemented using the system architecture as illustrated in FIG. 13. Specifically, the appliances, in certain embodiments, may be configured without the display and input / output devices illustrated in FIG. 13. Instead, these appliances may primarily include the processor(s), such as CPU 1310 and / or GPU 1320, non-transitory internal memory 1312, and storage 1314, supplemented by necessary network interfaces, such as communications interface 1316. This streamlined configuration is intended to facilitate efficient pass-through of communications to and from the source and destination devices. It is also worth noting that while the full suite of input and output devices, including microphone 1326, speaker 1328, and camera 1330, may be integral to the computer system 1300, they may be optional or entirely absent in the specialized implementations of the appliances, underscoring their focused role in network communications.

[0159] The term “computer system”, “data processing system” and related terms, as used herein, is not limited to any particular type of computer system and encompasses servers, desktop computers, laptop computers, networked mobile wireless telecommunication computing devices such as smartphones, tablet computers, as well as other types of computer systems.

[0160] It is imperative to note that the foregoing description is merely an illustrative embodiment and the actual implementation may vary in myriad ways. While the example encapsulates a particular set of operations and features, various elements within this configuration may be substituted, modified, or omitted altogether without detracting from theessence of the disclosure. For example, the methods of data collection could vary - alternative devices or sensors could be used, or different data parameters could be collected.

[0161] The embodiments have been described above with reference to flow, sequence, and block diagrams of methods, apparatuses, systems, and computer program products. In this regard, the depicted flow, sequence, and block diagrams illustrate the architecture, functionality, and operation of implementations of various embodiments. For instance, each block of the flow and block diagrams and operation in the sequence diagrams may represent a module, segment, or part of code, which comprises one or more executable instructions for implementing the specified action(s). In some alternative embodiments, the action(s) noted in that block or operation may occur out of the order noted in those figures. For example, two blocks or operations shown in succession may, in some embodiments, be executed substantially concurrently, or the blocks or operations may sometimes be executed in the reverse order, depending upon the functionality involved. Some specific examples of the foregoing have been noted above but those noted examples are not necessarily the only examples. Each block of the flow and block diagrams and operation of the sequence diagrams, and combinations of those blocks and operations, may be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0162] The terminology used herein is only for the purpose of describing particular embodiments and is not intended to be limiting. Accordingly, as used herein, the singular forms “a”, “an”, and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and “comprising”, when used in this specification, specify the presence of one or more stated features, integers, steps, operations, elements, and components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and groups. Directional terms such as “top”, “bottom”, “upwards”, “downwards”, “vertically”, and “laterally” are used in the following description for the purpose of providing relative reference only, and are not intended to suggest any limitations on how any article is to be positioned during use, or to be mounted in an assembly or relative to an environment. Additionally, the term “connect” andvariants of it such as “connected”, “connects”, and “connecting” as used in this description are intended to include indirect and direct connections unless otherwise indicated. For example, if a first device is connected to a second device, that coupling may be through a direct connection or through an indirect connection via other devices and connections. Similarly, if the first device is communicatively connected to the second device, communication may be through a direct connection or through an indirect connection via other devices and connections.

[0163] Herein, use of language such as “at least one of X, Y, and Z,” “at least one of X, Y, or Z,” “at least one or more of X, Y, and Z,” “at least one or more of X, Y, and / or Z,” or “at least one of X, Y, and / or Z,” is intended to be inclusive of both a single item (e.g., just X, or just Y, or just Z) and multiple items (e.g., {X and Y}, {X and Z}, { Y and Z}, or {X, Y, and Z } ). The phrase “at least one of’ and similar phrases are not intended to convey a requirement that each possible item must be present, although each possible item may be present.

[0164] It is contemplated that any part of any aspect or embodiment discussed in this specification can be implemented or combined with any part of any other aspect or embodiment discussed in this specification, so long as such those parts are not mutually exclusive with each other.

[0165] While every effort has been made to provide a detailed and accurate description of the disclosure herein, it should be noted that the scope of the disclosure is not limited to the exact configurations and embodiments described. The description provided is intended to illustrate the principles of the disclosure and not to limit the disclosure to the specific embodiments illustrated. It is intended that the scope of the disclosure be defined by the appended claims, their equivalents, and their potential applications in other fields.

Claims

What is claimed is:

1. A method for data transmission, comprising: generating a plurality of copies of a data packet to be sent from a source device to a destination device; selecting a plurality of different data paths for the plurality of copies of the data packet, respectively, wherein the plurality of different data paths are paths in a network connecting the source device to the destination device, respectively; and transmitting the plurality of copies of the data packet along the plurality of different data paths, respectively, from the source device toward the destination device.

2. The method of claim 1, further comprising: labeling the different data paths by assigning an identifier to each of the plurality of different data paths using at least one controller, wherein the identifiers are indicative of routing priorities of the different data paths, and wherein the obtaining further comprises obtaining the assigned identifiers.

3. The method of claim 2, wherein the labeling comprises applying virtual local area network (VLAN) or multiprotocol label switching (MPLS) labeling.

4. The method of claim 2 or 3, wherein the at least one controller comprises at least two controllers, and the method further comprises: monitoring an inconsistency of the identifiers assigned by the at least two controllers; andin response to the identifiers being inconsistent, identifying a faulty controller.

5. The method of any one of claims 1 to 4, wherein the plurality of copies of the data packet comprises at least three copies of the data packet.

6. The method of any one of claims 1 to 5, further comprising: in response to the plurality of copies of the data packet having been transmitted through the plurality of data paths in the network, identifying one of the plurality of data paths as an anomalous data path by determining that a deviation exists between one of the plurality of copies of the data packet transmitted through the one of the plurality of data paths and another one of the plurality of copies of the data packet.

7. The method of claim 6, further comprising: counting instances where the deviation exists to generate a count, wherein the identifying comprises, in response to the count exceeding a threshold, identifying the one of the plurality of data paths as the anomalous data path.

8. The method of claim 6 or 7, further comprising: obtaining the deviation by comparing the one of the plurality of copies of the data packet transmitted through the one of the plurality of data paths with the others of the plurality of copies of the data packet, with information on routing removed from the plurality of copies of the data packet.

9. The method of claim 8, further comprising: in response to one byte of the one of the plurality of copies of the data packet transmitted through the one of the plurality of data paths being different from a corresponding byte of the others of the plurality of copies of the data packet, determining that the deviation exists for the one copy.

10. The method of any one of claims 6 to 9, further comprising: in response to identifying the anomalous data path, removing the anomalous data path from the plurality of different data paths.

11. The method of any one of claims 6 to 10, further comprising: in response to identifying the anomalous data path, selecting a new data path in place of the anomalous data path from the plurality of different data paths.

12. The method of any one of claims 6 to 11, wherein the plurality of copies of the data packet comprises at least three copies of the data packet, and wherein at least one of the plurality of copies of the data packet is precluded from the determination of the deviation.

13. The method of any one of claims 6 to 12, further comprising: in response to identifying that the one of the plurality of data paths as an anomalous data path, changing the plurality of data paths to avoid the anomalous data path for a subsequently transmitted data packet.

14. The method of any one of claims 1 to 13, wherein the plurality of different data paths have no common network nodes with each other.

15. The method of any one of claims 1 to 14, wherein the method is performed by one of the source device or the destination device.

16. The method of any one of claims 1 to 14, wherein the method is performed by an inline appliance that is connected to one of the source device or the destination device.

17. The method of claim 16, wherein the inline appliance further comprises a network interface, the network interface comprising a first Ethernet port connecting to one of the source device or the destination device and a second Ethernet port connecting to the network.

18. A system for data transmission, comprising: a processor; and a non-transitory memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the processor to: generate a plurality of copies of a data packet to be sent from a source device to a destination device; select a plurality of different data paths for the plurality of copies of the data packet from a plurality of different data paths in a network connecting the source device to the destination device, respectively; andtransmit the plurality of copies of the data packet along the plurality of different data paths, respectively, from the source device toward the destination device.

19. The system of claim 18, wherein the processor and the non-transitory memory comprise part of an inline appliance, and wherein the system further comprises at least one controller configured to: label the different data paths by assigning an identifier to each of the plurality of different data paths, wherein the identifiers are indicative of routing priorities of the different data paths, and wherein the instructions stored in the non-transitory memory further cause the processor to obtain the assigned identifiers.

20. The system of claim 19, wherein the at least one controller is configured to apply virtual local area network (VLAN) or multiprotocol label switching (MPLS) type labeling.

21. The system of claim 19 or 20, wherein the at least one controller comprises at least two controllers, and wherein the system further comprises an operation, administration, and maintenance (0AM) device configured to: monitor an inconsistency of the identifiers assigned by the at least two controllers; and in response to the identifiers being inconsistent, identify a faulty controller.

22. The system of any one of claims 18 to 21, wherein the plurality of copies of the data packet comprises at least three copies of the data packet.

23. The system of any one of claims 18 to 22, wherein the processor and the non-transitory memory comprise part of the source device or a first inline appliance communicatively coupled inline with the source device, and wherein the system further comprises at least one controller configured to: in response to the plurality of copies of the data packet having been transmitted through the plurality of data paths in the network and a deviation having been determined to exist between one of the plurality of copies of the data packet transmitted through one of the plurality of data paths and another one of the plurality of copies of the data packet, identify the one of the plurality of data paths as an anomalous data path.

24. The system of claim 23, wherein the at least one controller is configured to: count instances where the deviation exists to generate a count; and in response to the count exceeding a threshold, identify the one of the plurality of data paths as the anomalous data path.

25. The system of claim 23 or 24, wherein the determination of the deviation is performed by the destination device or a second inline appliance connected inline with the destination device.

26. The system of any one of claims 22 to 24, wherein the at least one controller is configured to: obtain the deviation by comparing the one of the plurality of copies of the data packet transmitted through the one of the plurality of data paths with the others of the plurality of copies of the data packet, with information on routing removed from the plurality of copies of the data packet.

27. The system of claim 26, wherein the at least one controller is configured to:in response to one byte of the one of the plurality of copies of the data packet transmitted through the one of the plurality of data paths being different from a corresponding byte of the others of the plurality of copies of the data packet, determine that the deviation exists for the one copy.

28. The system of any one of claims 23 to 27, wherein the at least one controller is configured to, in response to identifying the anomalous data path, remove the anomalous data path from the plurality of different data paths.

29. The system of any one of claims 23 to 28, wherein the at least one controller is configured to, in response to identifying the anomalous data path, select a new data path in place of the anomalous data path from the plurality of different data paths.

30. The system of any one of claims 23 to 29, wherein the plurality of copies of the data packet comprises at least three copies of the data packet, and wherein at least one of the plurality of copies of the data packet is precluded from the determination of the deviation.

31. The system of any one of claims 23 to 30, wherein the at least one controller is configured to, in response to identifying that the one of the plurality of data paths as an anomalous data path, change the plurality of data paths to avoid the anomalous data path for a subsequent data packet.

32. The system of any one of claims 18 to 31, wherein the plurality of different data paths have no common network nodes with each other.

33. The system of any one of claims 18 to 32, wherein the processor and the non-transitory memory comprise one of the following: part of one of the source device or the destination device; or part of an inline appliance that is connected to one of the source device or the destination device.

34. The system of claim 33, wherein the inline appliance further comprises a network interface, the network interface comprising a first Ethernet port connecting to one of the source device or the destination device and a second Ethernet port connecting to the network.

35. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 17.

Images