Adaptive Packet Routing

Abstract

The computer implementation method involves encoding a packet at the source endpoint of a multipath communication network, the packet having a hash seed for routers to use to route the packet to the destination endpoint across the multipath communication network. Network performance is tracked with respect to the packet at the source endpoint. The hash seed is modified according to network performance. The packet is retransmitted so that the modified hash seed is used to route the packet to the destination endpoint.

Description

Background Art

[0001] Background

[0001] Many different strategies can be used to route data packets within a network. Equal Cost Multi-Path Routing (ECMP) routes packets across multiple network paths between endpoints with equal routing priority. Multi-path routing can be used in conjunction with most routing protocols since it is a per-hop local decision made independently at each router. The use of ECMP can substantially increase bandwidth by load-balancing traffic across multiple paths.

[0002]

[0002] Serious problems can exist when actually deploying ECMP when a link from a router goes down. The downed link will be removed from the list of available paths, but it might have been the only link available for the router to send packets in the direction of their destinations. Packets can still reach the router, but will be ignored since only one path from the link is still down.

Summary of the Invention

[0003] Summary

[0003] A computer-implemented method includes encoding a packet at a source endpoint of a multi-path communication network, the packet having a hash seed for use by a router to route the packet via the multi-path communication network to a destination endpoint. Network performance is tracked with respect to the packet at the source endpoint. The hash seed is modified according to the network performance. The packet is resent such that the modified hash seed is used to route the packet to the destination endpoint.

Brief Description of the Drawings

[0004] Brief explanation of the drawing [Figure 1]

[0004] This is a block diagram illustrating route selection using modified hashing in a simple exemplary two-level network according to an exemplary embodiment. [Figure 2]

[0005] This is a circular buffer 200 for use when tracking hash seeds corresponding to broken paths in an exemplary embodiment. [Figure 3]

[0006] This is a block diagram illustrating the generation of hash seed modification according to an exemplary embodiment. [Figure 4]

[0007] This example demonstrates the use of performance-based hash seed allocation for enhanced load balancing. [Figure 5]

[0008] This flowchart shows a computer implementation method for modifying packet routing based on network performance according to an exemplary embodiment. [Figure 6]

[0009] This flowchart shows a method for filtering hash seeds to optimize hash seed modification according to an exemplary embodiment. [Figure 7]

[0010] This flowchart shows a method 700 for selecting a hash seed based on the measured performance of previously used hash seeds according to an exemplary embodiment. [Figure 8]

[0011] This is a block schematic diagram of a computer system implementing one or more exemplary embodiments. [Modes for carrying out the invention]

[0005] Detailed explanation

[0012] The following detailed description refers to accompanying drawings that constitute part of the embodiments and illustrate specific embodiments in which the present invention can be carried out. These implementations are described in sufficient detail to enable those skilled in the art to carry out the present invention, and it should be understood that other embodiments may be used without departing from the scope of the invention and that logical or electrical modifications may be made. Accordingly, the following detailed description of the exemplary embodiments should not be taken as restrictive, and the scope of the invention is defined by the appended claims.

[0006]

[0013] Equal-cost multipath routing (ECMP) enables forgotten route selection through routers in a multipath data network based on a hash function H(x), where x is a hash seed contained in one or more arbitrary fields in each packet. In ECMP, when each router receives a packet, it examines a set of possible candidate output ports for a given destination address and then uses H(x) to select one possible candidate output port. H(x) is deterministic such that the hash function always selects the same output port for the same x.

[0007]

[0014] If a router determines that an adjacent link is down, it will remove the adjacent link from possible ECMP destinations and forward packets only on the remaining working links. This will alter existing mappings that routed to the broken link, and is likely to alter all other mappings passing through the same switch. Since mapping decisions are made at each router, a network problem arises where a link from a given router is broken and therefore the router has no reasonable alternative route.

[0008]

[0015] In a Clos network topology, there is only one down path from each top router (also called a top switch) to a specific destination (endpoint). When a packet reaches a top switch, the link from the top switch to the down path fails, and if some packets are still routed correctly to the top switch, those packets will not reach their destination and will be ignored. ECMP cannot handle this failure mode, and therefore this failure mode must be handled at the endpoint (source and destination network interface card (NIC)) by bypassing the specific top switch when sending packets to several destinations with broken downstream paths.

[0009]

[0016] An enhanced ECMP method for selecting a new route in response to poor network performance modifies the hash seed found in one or more fields of the packet so that x' = x + e. x can be the User Datagram Protocol (UDP) source port number in the packet, and e (also called the entropy enhancer) can be a random number from a random number generator, or even an increment, where e adds some entropy to the routing decision at each switch, resulting in a different route selection than the one selected using x (because it has a seed with a higher probability).

[0010]

[0017] Predicting the exact mapping of H(x') (or even H(x)) to a given path is extremely difficult. This difficulty in predicting accurate mapping largely stems from the challenges associated with hard link failures, which can alter the network topology and result in different plausible sets of paths.

[0011]

[0018] Figure 1 is a block diagram illustrating route selection using modified hashes in a simple, exemplary two-level Clos network 100 with four endpoints (0, 1, 2, and 3). Network 100 includes four nodes, including routers 110, 115, 120, and 125. Routers 115 and 120 are top-level routers. Endpoints 0 and 1 are part of router 110, and endpoints 2 and 3 are part of router 125. Router 110 has a connection 130 to router 115 and a connection 135 to router 120. Connection 140 connects router 115 to router 125, and connection 145 connects router 120 to router 124.

[0012]

[0019] Network 100 has a logical connection 150 from router 110 endpoint 0 to router 125 endpoint 2, which generates a packet header with hash seed x. If connection 140 breaks, packets on logical connection 150 will travel from endpoint 0 to router 115 but will not be able to respond to endpoint 2 in router 125.

[0013]

[0020] Source endpoint 0 would determine that the network is not working properly. This determination may be based on information about one or more packets, using the current seed x. Such information may include tracking one or more of the lack of received acknowledgments resulting in timeouts and the number of packet retransmission thresholds, or receiving data through the network management infrastructure that packets are no longer successfully routed to endpoint 2. In response, source endpoint 0 selects a new hash seed by modifying the packet header. The packet header may be modified, for example, by randomly changing the UDP source port to generate a new seed: x'=x+e.

[0014]

[0021] When a packet with a new hash seed is received at router 110, different logical paths 155 using connections 135 and 145 may arise, depending on the type of hash used and the length of the seed. If a logical connection 150 (broken) arises as a result of using the new hash seed, the seed can be repeatedly modified again to converge onto the operational logical path 155.

[0015]

[0022] In one example, node 0 tracks that hash seed x and other hash seeds (which are broken) that would result in logical connection 150 should not be used, or that x+e (the hash seed that will eventually work) should be used instead. A table-based mechanism may be used to track non-working or working hash seeds.

[0016]

[0023] Larger networks with greater path diversity (where the number of layers in Clos is exponential, and the exponential number is large (e.g., 32) 3 In a network with 64 or more ports, the desired number of entropy is very large. In one example, the entire 16-bit space of a UDP port could be used to modify the hash seed. Assuming a 3-layer open Clos network with 64 port switches, each switch could potentially use any of its 32 (encoded as 5 bits) output ports. For all three decisions (one per layer), 15 bits (bits of entropy) could be used to modify the hash seed. Different numbers of bits could be used in another example to ensure that traffic is well balanced by a number determined based on simulation, analysis, or heuristically determined by actual network use. Preliminary results suggest that the number of entropy for 2 or 3-level Clos networks and networks with 64 or more port switches is probably several thousand per connection.

[0017]

[0024] Even if a single link fails within a network or the like, a very large (and exponential in terms of the depth of the Clos) number of hash seeds can be affected. For example, assuming the same 3-level Clos as above, there are 32,768 (2 15 ) possible paths per source-destination pair (a fat tree would have 1 / 2 of that due to connecting top-level switches). If any link fails, only one path per source-destination pair is affected by using a pseudorandom hash seed. Remembering all hash seeds for all N-point-to-point connections per endpoint is resource-intensive. Limiting the hash seeds can lead to a degradation of path utilization and potentially undesirable / unpredictable polarization effects (since each connection is biased).

[0018]

[0025] FIG. 2 is a circular buffer 200 for use in tracking the hash seeds corresponding to broken paths. To handle failures and avoid sending packets into a broken downstream path, a history of the most recently seen hash seeds corresponding to operational paths is tracked. The history can be tracked by simply recording the hash seeds corresponding to the last k ACK packets within buffer 200. Circular buffer 200 can be implemented in hardware or software. The hash seeds of each incoming positive acknowledgment: seed A at 210, seed N at 220,... seed P at 230 (which the packet carries in its header) are added to circular buffer 200 or cache (potentially overwriting the oldest hash seed currently shown as SEED P). Note that the letters associated with the seeds are not necessarily consecutive in this example (since the seeds need not be consecutive or sequential).

[0019]

[0026] Packets may have a retransmission counter shown for each packet at 240, 245, 250 to track how many times a packet has been retransmitted due to transmission failure. If the retransmission counter of a packet ends, this end is probably due to a link failure. Next, the packet is transmitted along the next operating path defined by a new hash seed, and the (circular) operating path counter is incremented by 1. The operating path counter initially points to the first entry as shown at 255.

[0020]

[0027] One benefit of having multiple cache entries is to avoid transient imbalances that may be caused by buffer delays in the network. If k is too small, the delay in receiving either a new ACK or a large number of packet timeouts within a short time frame may cause many packets to be sent along the same path, leading to further imbalances (which may be amplified randomly). If those packets are sent round-robin via k paths, this effect is mitigated, k can be 2 or more, and can vary in various examples. A k of 1 may be effective for some multi-path networks, but can cause transient imbalances as described above.

[0021]

[0028] The use of a circular cache works well for timeout packets and for large flow packets because the hash seeds of each packet in the flow are the same. Some flows will converge to a good distribution within a fairly short time among the active state paths of the multi-path network, achieving better throughput.

[0022]

[0029] Figure 3 is a block diagram illustrating the generation of hash seed modifications in 300. New packets in various packet flows may still be transmitted along dead paths with some probability because the hash seeds of the new packets may be different. With respect to new packet flows, data structures 310 (described in more detail below), such as Bloom filters, may be used to efficiently filter out hash seeds corresponding to bad paths or network performance degradation at the origin endpoint (referred to as the source).

[0023]

[0030] For example, whenever j packets (where j can be an integer greater than or equal to 1) with the same hash seed time out and are retransmitted, the corresponding hash seed is added to data structure 310. Generating new seeds, such as by using a random number generator (also called a random entropy generator) 320, is now a bit more complex in that it maintains a pool 330 of hash seeds that, when added in 335, do not conform to the Bloom filter.

[0024]

[0031] Pool 330 allows the generation rate of the filtered random number generator 320 to be isolated from the packet consumption rate of the source endpoint's NIC packet pipeline. The NIC packet pipeline should run at line speed (processing packets as they are received) but cannot tolerate occasional "hiccups" caused by filtering. The size of the Bloom filter when used as data structure 310 can be selected via simulation or heuristically to avoid buffering.

[0025]

[0032] The Bloom filter can be cleared from time to time via reset 340 to allow the restored connection to be utilized by entropy again. Clearing can be done at fixed time intervals or after the Bloom filter has reached a certain fill-in (e.g., 15% of the set bits).

[0026]

[0033] A Bloom filter is a data structure for performing approximate set membership that has false positives but no false negatives. Using a Bloom filter not only allows for very efficient filtering of all broken paths, but also potentially reduces the number of reasonable entropies. This may introduce a small bias in path selection and utilization.

[0027]

[0034] A Bloom filter is characterized by two parameters: the number of bits m and the number of hash functions k. Each incoming set membership query x is hashed by all hash functions H_l(x), ..., H_k(x) (possibly in parallel). The Bloom filter then tests whether all bits B[H_l(x)], ..., B[H_k(x)] are set. If all bits are set, x is assumed to be in a set state. An insert performs the same k hash function but sets those bits instead of testing which k bits are set. Thus it is clear that if an element has been inserted, the element will always be found, but if m and / or k are too small or the hash functions collide, a false element y that has not been inserted may also be found in the set.

[0028]

[0035] Hash functions are typically expensive. The only requirement that needs to be met is that the hash function covers m slots as expected. Therefore, if the input is a number uniformly sampled from 16 bits, any log2(m) subset of the bits can be taken as the hash function. Naturally, each bit should often be represented equally by k hash functions.

[0029]

[0036] To cover up to 128 true failure entropies, m = 1024. Ten bits can be used as the output of each hash function that can be simply selected from a 16-bit input. Given k = 8 hash functions, 80 bits are selected in a way that each bit of x is represented by exactly five hash functions. The hash function implementation is a bounded-order selection circuit, and querying a Bloom filter is trivial, so the bounded-order selection circuit can have very little overhead and very little time, and as a result can be done in a single processor. The bounded-order selection circuit may be reset when filled up to 128, or we may maintain multiple circuits in a round-robin window configuration to maintain the most recently discovered failure entropies. The configuration, m, k, and false-positive result bias can be evaluated in simulation.

[0030]

[0037] Within an exemplary multipath network, there exists a route selection data structure 310 for each connection. If entropy (hash seed) is shared across all connections, one data structure per destination may be used. This can be easily achieved by configuring ECMP to simply assign source ports as x instead of all source port identifiers or tuples. Memory requirements per connection are reduced by reducing the memory / state requirements per connection.

[0031]

[0038] Figure 4 generally illustrates the use of performance-based hash seed allocation for enhanced load balancing in 400. For load balancing, instead of maintaining a cache of hash seeds corresponding to active routes, multiple caches of hash seeds corresponding to active routes may be maintained along with various network congestion features. In its simplest form, two caches may be used. Cache 410 may be used for slower routes, and cache 420 may be used for faster routes. For each incoming packet ACK 430, the corresponding hash seed may be added to either one (based on either explicit congestion notification (ECN) marking or directly on its round-trip time (RTT)) or to a mixture of both. Multiple grading of caches may be used in some examples. In any case, selection and insertion are far less expensive than table-based schemes with ternary content addressable memory (TCAM).

[0032]

[0039] In one example, the selection of fast and slow routes can be based on RTT statistics. The fastest or top 10% of packets are added to the fast route cache 420, and the slowest 10% or bottom 10% of packets are added to the slow route cache 410. All others are intermediate performers 435 and therefore do not need to be cached. Parameters α and β correspond to a 10% threshold or other threshold for determining performance and can be estimated for simple classification in hardware. Updating these parameters can be done in software by using a small control processor.

[0033]

[0040] Rerouting can be initiated if a route becomes too slow. For example, connections in the cache of a slow route can be rerouted by simply selecting the slow route hash seed at 440 and replacing it with the hash seed from the fast route cache at 450. To avoid route flapping, the standard deviation of the RTT distribution may be observed and used to simply reroute outlier routes. This can also be done at the cache selection stage (using the distance from the mean / standard deviation metric rather than the percentage of caches allowed). If the bottom 10% are rerouted to the top 10%, the deviation (width) of the distribution will decrease over time.

[0034]

[0041] The use of such rerouting should ultimately converge to an optimal set of paths with far less memory than would be needed to store all paths. Rerouting can be stopped when β-α falls below a rerouting threshold based on a sufficiently narrow distribution. Dynamic parameters can be adjusted to achieve the desired convergence and memory usage. Simulation can also be used to adjust the parameters.

[0035]

[0042] Maintaining α and β in hardware is straightforward. Assume that N "fastest" entropies will be stored (and the slowest entropy will be stored as well, and the same hardware can be reused). An array A of size N+1 can be used and initialized with zeros. The array can be maintained in a sorted order. The RTT (or other congestion metric with a defined order) of newly incoming ACKs is compared to A[N-2] (the second to last entry in A). If the RTT is larger, it is added to A[N-1] and the array is resorted. Resoring can be efficiently performed by a sorting network that can be reused across the entire connection. Since the array is almost sorted, a single-cycle sort can be used for a small / reasonable N of each incoming ACK.

[0036]

[0043] Figure 5 is a flowchart of a computer implementation method 500 for modifying packet routing based on network performance. Method 500 begins in operation 510 by encoding a packet at the source endpoint of a multipath communication network. The packet has a hash seed for use by a router to route the packet to the destination endpoint through the multipath communication network. In one example, the multipath communication network is a Clos network utilizing Equal-Cost Multipath Routing (ECMP).

[0037]

[0044] In operation 520, the network performance of a packet is tracked at the source endpoint. Tracking network performance may include tracking at least one of the following: the number of packet acknowledgment timeouts, the number of retransmissions of packets with the same hash seed, or receiving an explicit congestion notice (ECN) or round-trip time (RTT), or a combination thereof.

[0038]

[0045] The source endpoint modifies the hash seed in operation 530 depending on network performance, and then retransmits the packet with the modified hash seed in operation 540 so that the modified hash seed is used to deliver the packet to the destination endpoint. Modifying the hash seed makes it likely that the packet will use a different route to improve packet routing performance. In one example, the hash seed is modified based on a random number generator.

[0039]

[0046] Method 500 may be repeated by modifying the packet hash seed in accordance with the tracked network performance indicating a poor or failed connection, as shown in 550.

[0040]

[0047] Figure 6 is a flowchart of method 600 for filtering hash seeds to optimize hash seed correction. Method 600 begins in operation 610 by tracking hash seeds that have resulted in tracked network performance indicating a poor or failed connection. The tracked hash seeds are filtered in operation 620 because they are used to send packets. In one example, the hash seeds are tracked in a k-entry circular buffer, where k is an integer greater than or equal to 2.

[0041]

[0048] Method 600 may continue in operation 630 by sharing a data structure that identifies tracked hash seeds across multiple endpoints. In operation 640, the source endpoint is configured to use the same base hash seed corresponding to the source port identifier to facilitate the use of the data structure when selecting a new hash seed. For example, a Bloom filter may be used as a data structure for tracking hash seeds corresponding to tracked network performance indicating a poor or failed connection.

[0042]

[0049] Figure 7 is a flowchart of method 700 for selecting a hash seed based on the measured performance of previously used hash seeds. Method 700 begins in operation 710 by tracking hash seeds associated with high-performance packet transmissions in a high-performance list. Operation 720 tracks hash seeds associated with low-performance packet transmissions in a low-performance list. The high-performance and low-performance lists may reside in separate caches that logically identify the high-performance and low-performance hash seeds, or in a single memory space or cache. Operation 730 exchanges hash seeds from the tracked low-performance packet transmissions with hash seeds from the high-performance packet transmissions for use in further packet transmissions.

[0043]

[0050] For example, a high-performance packet transmission hash seed is associated with packets having round-trip times in the top 10 percent, while a low-performance packet transmission hash seed is associated with packets having round-trip times in the bottom 10 percent.

[0044]

[0051] Figure 8 is a schematic block diagram of a computer system 800 for generating a modified hash seed based on network performance and for performing methods and algorithms according to exemplary embodiments. Not all components are required to be used in all embodiments.

[0045]

[0052] One exemplary computing device of the form of computer 800 may include a processing unit 802, memory 803, removable storage 810, and non-removable storage 812. Although the exemplary computing device is shown and described as computer 800, the computing device can take various forms in various embodiments. For example, the computing device could instead be a smartphone, tablet, smartwatch, smart storage device (SSD), or other computing device (such as a circuit configuration containing the same or similar elements shown and described with reference to Figure 8). Devices such as smartphones, tablets, and smartwatches are usually collectively referred to as mobile devices or user devices.

[0046]

[0053] While various data storage elements are shown as part of the computer 800, storage may also alternatively include cloud-based storage accessible over a network, such as the internet or server-based storage. It should also be noted that the SSD may include a processor on which a parser may run, enabling the transfer of parsed and filtered data over an I / O channel between the SSD and main memory.

[0047]

[0054] Memory 803 may include volatile memory 814 and non-volatile memory 808. Computer 800 may include, or can access, a wide variety of computer-readable media (such as volatile memory 814 and non-volatile memory 808, removable storage 810 and non-removable storage 812). Computer storage includes random access memory (RAM), read-only memory (ROM), erasable PROM (EPROM) or electrically erasable PROM (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital general-purpose disc (DVD) or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other media capable of storing computer-readable commands.

[0048]

[0055] Computer 800 may include or be able to access a computing environment including an input interface 806, an output interface 804, and a communication interface 816. The output interface 804 may include a display device such as a touchscreen that can also function as an input device. The input interface 806 may include one or more of the following: a touchscreen, a touchpad, a mouse, a keyboard, a camera, one or more device-specific buttons, one or more sensors integrated within Computer 800 or one or more sensors connected to Computer 800 via a wired or wireless data connection, and other input devices. The computer may operate in a network environment by using a communication connection to connect to one or more remote computers, such as a database server. Remote computers may include personal computers (PCs), servers, routers, network PCs, or other common data flow network switches, etc. The communication connection may include a local area network (LAN), a wide area network (WAN), cellular, Wi-Fi, Bluetooth, or other networks. According to one embodiment, various components of Computer 800 are connected to a system bus 820.

[0049]

[0056] Computer-readable instructions stored on a computer-readable medium are executable by a processing unit 802 of computer 800 (such as program 818). In some embodiments, program 818 includes software for performing one or more methods described herein. Hard drives, CD-ROMs, and RAMs are some examples of articles containing non-temporary computer-readable medium, such as storage devices. Terminology: Computer-readable medium, machine-readable medium, and storage devices do not contain carrier waves or signals (to the extent that carrier waves and signals are considered too temporary). Storage may also include network storage, such as a storage area network (SAN). Computer program 818 may be used together with a workspace manager 822 to cause the processing unit 802 to perform one or more methods or algorithms described herein. [Examples]

[0050]

[0057] Examples:

[0058] 1. The computer implementation method includes encoding a packet at the source endpoint of a multipath communication network, the packet having a hash seed for the router to use to route the packet to the destination endpoint through the multipath communication network. Network performance is tracked with respect to the packet at the source endpoint. The hash seed is modified according to the network performance. The packet is retransmitted so that the modified hash seed is used to route the packet to the destination endpoint.

[0051]

[0059] 2. The multipath communication network is a network that utilizes Equal-Cost Multipath Routing (ECMP), as described in Example 1.

[0052]

[0060] 3. The method according to any one of Examples 1 to 2, wherein tracking the network performance of a network includes tracking at least one of the number of packet acknowledgment timeouts, the number of retransmissions of packets having the same hash seed, or receiving explicit congestion notices (ECN) or round-trip time (RTT) or a combination thereof.

[0053]

[0061] 4. The method according to any one of Examples 1 to 3, wherein the hash seed is modified based on a random number generator.

[0054]

[0062] 5. A method according to any one of Examples 1 to 4, further comprising repeatedly modifying the packet hash seed in accordance with traced network performance indicating a poor or failed connection.

[0055]

[0063] 6. A method according to Example 5, further comprising tracking hash seeds that have produced tracked network performance indicating a poor or failed connection, and filtering out the tracked hash seeds so that they are not used in transmitted packets.

[0056]

[0064] 7. The method according to Example 6, wherein the hash seed is tracked in a k-entry circular buffer, and k is an integer greater than or equal to 2.

[0057]

[0065] 8. A method according to any one of Examples 6 to 7, further comprising sharing a data structure that identifies tracked hash seeds among multiple endpoints, and configuring source endpoints to use the same base hash seed corresponding to a source port identifier.

[0058]

[0066] 9. A method according to any one of Examples 6 to 8, further comprising using a Bloom filter to track a hash seed corresponding to the performance of a tracked network indicating a poor or failed connection.

[0059]

[0067] 10. A method according to any of Examples 5 to 9, further comprising tracking hash seeds associated with high-performance packet transmissions in a high-performance list, tracking hash seeds associated with low-performance packet transmissions in a low-performance list, and exchanging hash seeds from the tracked low-performance packet transmissions with hash seeds from high-performance packet transmissions for use in further packet transmissions.

[0060]

[0068] 11. The method according to Example 10, wherein a high-performance packet transmission hash seed is associated with packets having round-trip times in the top 10 percent, and a low-performance packet transmission hash seed is associated with packets having round-trip times in the bottom 10 percent.

[0061]

[0069] 12. The machine-readable storage device has machine processor execution instructions for causing the processor to perform an operation to perform any of the methods described in Examples 1 to 11.

[0062]

[0070] 13. The device includes a processor and a memory device coupled to the processor that stores a program for execution by the processor to perform an operation in any of the methods of Examples 1 to 11.

[0063]

[0071] The functions or algorithms described herein may be implemented in software in one embodiment. The software may consist of computer-executable instructions stored on a computer-readable medium or computer-readable storage device (such as one or more non-temporary memories or other types of hardware-based storage devices), whether local or networked. Furthermore, such functions may correspond to modules that may be software, hardware, firmware, or any combination thereof. Since multiple functions may be performed in one or more modules as desired, the embodiments described are merely examples. The software may run on a digital signal processor, ASIC, microprocessor, or other type of processor operating on a computer system (such as a personal computer, server, or other computer system), making such a computer system a machine specifically programmed for it.

[0064]

[0072] Functionality may be configured to be operated by using, for example, software, hardware, firmware, or other similar entities. For example, the phrase “configured” may refer to the logic circuit structure of a hardware element that will implement the relevant functionality. The phrase “configured” may also refer to the logic circuit structure of a hardware element that will implement the coding design of the relevant functionality in firmware or software. The term “module” refers to a structural element that can be implemented by using any suitable hardware (e.g., a processor in particular), software (e.g., an application in particular), firmware, or any combination of hardware, software, and firmware. The term “logic” encompasses any functionality for performing a task. For example, each operation shown in a flowchart corresponds to the logic for performing that operation. Operations may be performed by using software, hardware, firmware, or other similar entities. The terms “component,” “system,” etc., may refer to computer-related entities, hardware, software, firmware, or combinations thereof. A component may be a processor, object, executable file, program, function, subroutine, computer, or process running on a combination of software and hardware. The term “processor” may refer to a hardware component such as a processing unit of a computer system.

[0065]

[0073] Furthermore, the claimed subject matter may be implemented as a method, apparatus, or product using standard programming and engineering techniques for generating software, firmware, hardware, or any combination thereof for controlling a computing device to implement the subject matter of this disclosure. The term “product” as used herein is intended to encompass computer programs accessible from any computer-readable storage device or medium. Computer-readable storage mediums may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, optical disks, compact disks (CDs), digital general-purpose disks (DVDs), smart cards, and flash memory devices). In contrast, computer-readable mediums (i.e., non-storage mediums) may additionally include communication mediums, such as transmission mediums for radio signals, etc.

[0066]

[0074] While several implementations have been described in detail above, other modifications are possible. For example, the logic flow depicted in the accompanying drawings does not require a specific sequence or sequence to achieve the desired result. Other steps may be provided, or steps may be removed from the described flow, and other components may be added to or removed from the described system. Other embodiments may fall within the scope of the following claims.

Claims

1. Encoding a packet at a source endpoint (0) of a multipath communication network (100) (510), wherein the packet has a hash seed (210) that a router uses to route the packet to a destination endpoint (2) via the multipath communication network (510), Tracking the network performance of the packet at the source endpoint (520), Modifying the hash seed (220) according to the network performance (530), (540) Retransmitting the packet so that the modified hash seed is used to route the packet to the destination endpoint. A computer implementation method (500) including the above.

2. The method according to claim 1, wherein the multipath communication network is a network that utilizes equal-cost multipath routing (ECMP).

3. The method according to claim 1, wherein tracking the network performance of the network includes tracking at least one or a combination of the following: the number of packet acknowledgment timeouts, the number of retransmissions of packets having the same hash seed, or the receipt of explicit congestion notices (ECNs), or the round-trip time (RTT).

4. The method according to claim 1, wherein the hash seed is modified based on a random number generator (320).

5. A method according to any one of claims 1 to 4, further comprising repeatedly modifying the packet hash seed (550) in accordance with the performance of the tracked network indicating a poor or failed connection.

6. The method according to claim 5, Tracking the hash seed that produced the traced network performance indicating a poor or failed connection (610), Filtering so that the tracked hash seed is not used in the transmitted packets (620) A method that further includes this.

7. The method according to claim 6, wherein the hash seed is tracked in a k-entry circular buffer (200), where k is an integer greater than or equal to 2.

8. The method according to claim 6, Sharing a data structure that identifies the tracked hash seed among multiple endpoints (630), Configure the source endpoint to use the same base hash seed corresponding to the source port identifier (640) A method that further includes this.

9. A method according to claim 6, further comprising using a Bloom filter to track the hash seed corresponding to the performance of a tracked network indicating a poor or failed connection.

10. The method according to claim 5, Tracking the hash seeds associated with high-performance packet transmission in the high-performance list (710), Tracking hash seeds associated with low-performance packet transmissions in the low-performance list (720), (730) Exchanging the hash seed from the tracked low-performance packet transmission with the hash seed from the high-performance packet transmission for further use of packet transmission. A method that further includes this.

11. The method according to claim 10, wherein a high-performance packet transmission hash seed is associated with packets having round-trip times in the top 10 percent, and a low-performance packet transmission hash seed is associated with packets having round-trip times in the bottom 10 percent.

12. A machine-readable storage device (803) having a machine (800) processor (802) command (818) for execution to cause the processor to perform an operation to perform a method, wherein the operation is Encoding a packet at the source endpoint of a multipath communication network, wherein the packet has a hash seed that a router uses to route the packet to the destination endpoint via the multipath communication network, Tracking the network performance of the packet at the source endpoint, The hash seed is modified according to the network performance, The modified hash seed is used to retransmit the packet to the destination endpoint. A machine-readable storage device (803) including the following.

13. The device according to claim 12, wherein the multipath communication network is a network utilizing equal-cost multipath routing (ECMP).

14. The device according to claim 12, wherein tracking the network performance of the network includes tracking at least one or a combination of the following: the number of packet acknowledgment timeouts, the number of retransmissions of packets having the same hash seed, or the receipt of explicit congestion notices (ECNs), or the round-trip time (RTT).

15. The device according to any one of claims 12 to 13, further comprising repeatedly modifying the packet hash seed in accordance with traced network performance indicating a poor or failed connection.

16. The aforementioned operation is, Tracking the hash seed that produced the traced network performance indicating a poor or failed connection, The tracked hash seed is filtered so that it is not used in the transmitted packets. The device according to claim 15, further comprising:

17. The aforementioned operation is, Sharing a data structure that identifies tracked hash seeds across multiple endpoints, Configure the source endpoint to use the same base hash seed corresponding to the source port identifier. The device according to claim 15, further comprising:

18. The aforementioned operation is, Tracking hash seeds associated with high-performance packet transmission in the high-performance list, Tracking hash seeds associated with low-performance packet transmissions in the low-performance list, To exchange the hash seed from the tracked low-performance packet transmission with the hash seed from the high-performance packet transmission for further use of packet transmission. The device according to claim 15, further comprising:

19. Processor (802), and A memory device (803) coupled to the processor, Encoding a packet at the source endpoint of a multipath communication network, wherein the packet has a hash seed that the router uses to route the packet to the destination endpoint via the multipath communication network, Tracking the network performance of the packet at the source endpoint, The hash seed is modified according to the network performance, The modified hash seed is used to retransmit the packet to the destination endpoint. A device (800) including a memory device (803) having a program (818) stored thereon for execution by the processor in order to perform an operation including the above.

20. The device according to claim 19, wherein the multipath communication network is a network utilizing equal-cost multipath routing (ECMP), and the operation further includes repeatedly modifying the packet hash seed in accordance with traced network performance indicating a poor or failed connection.

Images