Providing symmetric coherency hashing in forwarding plane of network device

Abstract

The invention relates to providing symmetric coherency hashing in a forwarding plane of a network device. A network device may receive hash keys from a control plane of the network device, and may sort the hash keys based on a next hop address field and to generate sorted hash keys. The network device may locally store the ranked hash keys in a data plane of the network device, and may generate a ranked hash table based on the ranked hash keys. The network device may utilize the sorted hash table to route traffic symmetrically with another network device.

Description

[0001] Cross Reference to Related Applications

[0002] This patent application claims priority to U.S. Provisional Patent Application No. 63 / 662,741, filed June 21, 2024, and entitled “PROVIDING SYMMETRIC CONSISTENT HASHING IN A FORWARDING PLANE OF A NETWORK DEVICE.” The disclosure of the prior application is considered part of this patent application and is incorporated by reference into this patent application. BACKGROUND

[0003] A network device can utilize routing tables to route data throughout a network. Each routing table can include a hash table, which is a data structure that implements an associative array (e.g., an abstract data type that maps keys to values). A hash table uses a hash function to compute an index (also called a hash code) into an array of buckets or slots, according to which the desired value can be found. During a lookup, a key is hashed and the resulting hash indicates where the corresponding value is stored. SUMMARY

[0004] Some implementations described herein relate to a method. The method can include receiving a hash key from a control plane of a network device and ordering the hash key based on a next hop address field and generating an ordered hash key. The method can include storing the ordered hash key locally in a data plane of the network device and generating an ordered hash table based on the ordered hash key. The method can include utilizing the ordered hash table to symmetrically route traffic with another network device.

[0005] Some implementations described herein relate to a network device. The network device can include one or more memories, one or more processors, a control plane, and a data plane. The one or more processors can be configured to receive a hash key from the control plane, order the hash key based on a next hop address field and generate an ordered hash key. The one or more processors can be configured to store the ordered hash key locally in the data plane and generate an ordered hash table based on the ordered hash key. The one or more processors can be configured to receive first traffic provided from a customer and destined for a core network and route the first traffic to the core network through a firewall network device and another network device based on the ordered hash table.

[0006] Some implementations described herein relate to a non-transient computer-readable medium storing a set of instructions. When executed by one or more processors of a network device, this set of instructions enables the network device to receive hash keys from its control plane, where the hash keys are load-balancing hash key functions. When executed by one or more processors of the network device, this set of instructions enables the network device to sort the hash keys based on a next-hop address field and generate sorted hash keys, and to locally store the sorted hash keys in the network device's data plane. When executed by one or more processors of the network device, this set of instructions enables the network device to generate a sorted hash table based on the sorted hash keys and to symmetrically route traffic with another network device using the sorted hash table. Attached Figure Description

[0007] Figures 1A-1L This is a diagram illustrating an example of symmetric consistent hashing in the forwarding plane of a network device.

[0008] Figure 2 This is a diagram of an example environment in which the systems and / or methods described in this paper can be implemented.

[0009] Figure 3 and Figure 4 yes Figure 2 A diagram of example components of one or more devices.

[0010] Figure 5 This is a diagram of an example process for providing symmetric consistent hashing in the forwarding plane of network devices. Detailed Implementation

[0011] The following detailed description of the example implementation is with reference to the accompanying drawings. The same reference numerals in different drawings may identify the same or similar elements.

[0012] Hash tables in network devices need to be symmetric and consistent to ensure data can be correctly load-balanced to the appropriate destinations on the network. For example, in scenarios where service providers offer security services, they want to maintain symmetric traffic flows (e.g., firewalls handling traffic should see traffic coming from customers to the internet and from the internet to customers). Symmetry is important because it is inherently stateful, and firewalls can create session table entries accordingly in both directions. If network devices fail to route traffic symmetrically, customers may experience service and session disruptions. However, current technologies for managing hash tables fail to provide consistency and symmetry in traffic flows between upstream and downstream traffic handled by network devices.

[0013] Therefore, the current technology used to manage hash tables consumes computing resources (e.g., processing resources, memory resources, communication resources, and / or the like) and network resources associated with the failure to provide consistency and symmetry of traffic flows for upstream and downstream services handled by network devices, service interruptions associated with the failure to provide consistency and symmetry of traffic flows, session interruptions associated with the failure to provide consistency and symmetry of traffic flows, and / or the like.

[0014] Some implementations described in this paper involve providing symmetric consistent hashing in the forwarding plane of a network device. For example, a network device can receive hash keys from its control plane and sort those hash keys based on a next-hop address field to generate sorted hash keys. The network device can locally store the sorted hash keys in its data plane and generate a sorted hash table based on the sorted hash keys. The network device can use the sorted hash table to symmetrically route traffic with another network device. The network device can receive a first service from a customer destined for the core network and route it to the core network via firewall network devices and other network devices based on the sorted hash table. The network device can receive a second service from the core network destined for a customer, where the second service is received from another network device via a firewall network device and based on the sorted hash table. The network device can then provide the second service to the customer.

[0015] In this way, symmetric consistent hashing is provided to network devices in the forwarding plane. For example, network devices can utilize parameters that symmetrically select the same hash table or selector table index in both the upstream and downstream directions. The hash tables of active network devices across the upstream and downstream directions can be symmetric, such that the selected hash table index includes the same next hop in both directions. Network devices can provide forwarding plane-driven solutions that operate independently of routing protocols. Therefore, network devices save computational, network, and / or similar resources that would otherwise be consumed by failing to provide consistency and symmetry in traffic flow for upstream and downstream traffic handled by the network device, handling service interruptions associated with the failure to provide consistency and symmetry in traffic flow, handling session interruptions associated with the failure to provide consistency and symmetry in traffic flow, and / or the like.

[0016] Figures 1A-1L This is a schematic diagram of example 100 associated with symmetric consistent hashing in the forwarding plane of a network device. (Example 100) Figure 1AAs shown, Example 100 includes a customer or access network, the Internet or core network, user equipment, server equipment, and multiple network devices. Further details regarding the customer or access network, the Internet or core network, user equipment, server equipment, and multiple network devices are provided elsewhere in this document. A first network device (VRF-1) may be associated with a customer or access network, and a second network device (VRF-2) may be associated with the Internet or core network. The first and second network devices may be connected to a firewall group comprising a first firewall network device (vSRX1), a second firewall network device (vSRX2), and a third firewall network device (vSRX3).

[0017] like Figure 1B As shown by reference numeral 105 in the accompanying drawing, the first network device can receive a first service provided by a customer and destined for the Internet. For example, a customer associated with a customer or access network can generate a first service destined for the Internet or the core network. In some implementations, the first service can be associated with a service provided by a service provider, and the service provider wants to maintain service flow symmetry (e.g., a firewall processing a service should see services provided from the customer to the Internet and services provided from the Internet to the customer). The customer or access network can provide the first service to the first network device, and the first network device can receive the first service.

[0018] like Figure 1B As further illustrated by reference numeral 110 in the accompanying drawing, the first network device can route the first service to the Internet via a third firewall network device. For example, the first network device may include a control plane, a data plane, and a hash table or selector table. The control plane and data plane may include a hash key comprising a next-hop (Nh) order field and a next-hop Internet Protocol (IP) address field. The next-hop order field may include values ​​of 0, 1, 2, and / or similar, and the next-hop IP address field may include IP address entries such as 10.1.1.1, 10.1.1.5, 10.1.1.2, and / or similar. In some implementations, the first network device may utilize the hash key to generate a hash table and may utilize the hash table to generate a forwarding path for routing the service from the first network device to the Internet. Based on the hash table, the first network device may generate a forwarding path that routes the first service to the third firewall network device, then to the second network device, and finally to the Internet.

[0019] like Figure 1BAs further illustrated by reference numeral 115 in the accompanying drawings, the second network device can receive a second service provided from the Internet and destined for a customer. For example, a service provider's service can receive the first service and can generate a second service in response to the first service. The second service can be destined for a customer or access a network. The Internet or core network can provide the second service to the second network device, and the second network device can receive the second service.

[0020] like Figure 1B As further illustrated by reference numeral 120 in the accompanying drawing, the second network device can route the second service to the customer via a second firewall network device. For example, the second network device may include a control plane, a data plane, and a hash table or selector table that is asymmetric to the hash table of the first network device. The control plane and data plane may include a hash key comprising a next-hop order field and a next-hop IP address field. The next-hop order field may include values ​​of 0, 1, 2, and / or similar, and the next-hop IP address field may include IP address entries such as 20.1.1.5, 10.1.1.2, 10.1.1.1, and / or similar. In some implementations, the second network device may utilize the hash key to generate a hash table and may utilize the hash table to generate a reverse path for routing services from the second network device to the customer or access network. Based on the hash table, the second network device may generate a reverse path that routes the second service to the second firewall network device, then to the first network device, and finally to the customer. However, the firewall network device (vSRX3) handling the first service differs from the firewall network device (vSRX2) handling the second service. Therefore, the traffic is asymmetrical. Symmetry is inherently stateful, and firewalls can create session table entries accordingly in both traffic flow directions. If network devices fail to route traffic symmetrically, customers may experience service and session interruptions.

[0021] Figure 1C The control plane and data plane of the first network device are depicted. For example... Figure 1C As shown, the control plane and data plane may include a hash key comprising a next-hop order field and a next-hop IP address field. For example, the next-hop order field may include values ​​of 0, 1, 2, and / or similar, and the next-hop IP address field may include IP address entries such as 10.1.1.1, 10.1.1.5, 10.1.1.2, and / or similar. In some implementations, the control plane of the first network device may communicate with the firewall network device to determine the hash key and provide that hash key to the network device's data plane.

[0022] like Figure 1CAs further illustrated by reference numeral 125 in the accompanying drawing, the data plane of the first network device can sort the hash keys based on the next-hop IP address field to generate sorted hash keys, and store the sorted hash keys locally in the data plane. For example, the data plane can use the next-hop IP address field to sort the hash keys in a specific order (e.g., ascending order) and generate sorted hash keys. As shown, the IP address entries for 10.1.1.1, 10.1.1.5, and 10.1.1.2 can be sorted in the order of 10.1.1.1, 10.1.1.2, and 10.1.1.5. The first network device can store the sorted hash keys in the data plane of the first network device.

[0023] In some implementations, to achieve symmetry and consistency between the first and second network devices (e.g., in the upstream (forward) and downstream (reverse) directions), the first and second network devices can ensure that consistent hash events are symmetrical. If an equal-cost multipath routing (ECMP) path is disabled in the upstream direction, that ECMP path can be symmetrically disabled in the downstream direction to maintain the symmetry of the hash table. For example, if the path between the first network device (VRF-1) and the first firewall network device (vSRX1) is disabled, the path between the second network device (VRF-2) and the first firewall network device (vSRX1) can also be disabled. The first and second network devices can assign next-hop addresses (e.g., IP addresses) in both directions (e.g., downstream: 10.1.1.11, 10.1.1.12, 10.1.1.13 and upstream: 20.1.1.11, 20.1.1.12, 20.1.1.13) such that the index in the sorted list of ECMP devices (e.g., vSRX1, vSRX2, and vSRX3) (e.g., based on the next-hop IP as a hash key) points to the same ECMP device in both directions.

[0024] like Figure 1CAs further illustrated by reference numeral 130 in the accompanying drawing, the data plane of the first network device can generate a sorted hash table based on a sorted hash key. For example, based on the sorted hash key, the data plane of the first network device can generate a sorted hash table including an index field and a selector table field. The index field may include values ​​of 0, 1, 2, 3, 4, and / or similar, and the selector table field may include entries identifying the order of the next-hop IP addresses (e.g., 0→10.1.1.1; 1→10.1.1.2; 2→10.1.1.5; 3→10.1.1.1; and / or similar). In some implementations, the first network device may store the sorted hash table in the data plane of the first network device. In some implementations, the second network device may perform the functions of the first network device described above, such that the second network device generates the same sorted hash table as the first network device. In this way, the first network device and the second network device can route traffic symmetrically to each other via a firewall network device.

[0025] like Figure 1C As further illustrated by reference numeral 135 in the accompanying drawing, the first network device can utilize a sorted hash table to symmetrically route traffic. For example, when the first network device receives traffic destined for the Internet or the core network, it can use the sorted hash table to generate a forwarding path to route the traffic from the first network device to the Internet or the core network. In some implementations, the first network device can select one of the firewall network devices identified in the sorted hash table for the forwarding path.

[0026] like Figure 1D As further illustrated by reference numeral 140 in the accompanying drawing, the first network device can receive a first service provided by a customer and destined for the Internet. For example, a customer associated with a customer or access network can generate a first service destined for the Internet or the core network. In some implementations, the first service can be associated with a service provided by a service provider, and the service provider wants to maintain symmetrical service flow (e.g., a firewall processing a service should see services provided from the customer to the Internet and services provided from the Internet to the customer). The customer or access network can provide the first service to the first network device, and the first network device can receive the first service.

[0027] like Figure 1DAs further illustrated by reference numeral 145 in the accompanying drawing, the first network device can route the first service to the Internet via the second firewall network device based on a sorted hash table. For example, the first network device can utilize the sorted hash table to generate a forwarding path for routing the first service from the first network device to the Internet. Based on the sorted hash table, the first network device can generate a forwarding path that routes the first service to the second firewall network device, then to the second network device, and finally to the Internet.

[0028] like Figure 1D As further shown by reference numeral 150 in the accompanying drawings, the second network device can receive a second service provided from the Internet and destined for a customer. For example, a service provider's service can receive the first service and can generate a second service in response to the first service. The second service can be destined for a customer or access a network. The Internet or core network can provide the second service to the second network device, and the second network device can receive the second service.

[0029] like Figure 1D As further illustrated by reference numeral 155 in the accompanying drawing, the second network device can route the second service to the customer via the second firewall network device based on a sorted hash table. For example, the second network device can utilize the sorted hash table to generate a reverse path for routing the second service from the second network device to the customer or access network. Based on the sorted hash table, the second network device can generate a reverse path that routes the second service to the second firewall network device, then to the first network device, and finally to the customer. In this implementation, the firewall network device (vSRX2) handling the first service is the same as the firewall network device (vSRX2) handling the second service. Therefore, the services are symmetrical.

[0030] like Figure 1E As further illustrated by reference numeral 160 in the accompanying drawing, the data plane of the first network device can receive a new hash key that includes a new next-hop IP address field entry. For example, the control plane of the first network device can communicate with a firewall network device to determine the new hash key and provide it to the network device's data plane. As shown, the new hash key may include... Figure 1C The new next-hop IP address field entry (e.g., 10.1.1.3) is not included in the hash key shown.

[0031] like Figure 1EAs further illustrated by reference numeral 165 in the accompanying drawing, the data plane of the first network device can sort the new hash keys based on the next-hop IP address field and generate sorted new hash keys. For example, the data plane can use the next-hop IP address field to sort the new hash keys in a specific order (e.g., ascending order) and generate sorted new hash keys. As shown, the IP address entries for 10.1.1.1, 10.1.1.5, 10.1.1.2, and 10.1.1.3 can be sorted in the order of 10.1.1.1, 10.1.1.2, 10.1.1.3, and 10.1.1.5. The first network device can store the sorted new hash keys in the data plane of the first network device.

[0032] like Figure 1E As further illustrated by reference numeral 170 in the accompanying drawing, the data plane of the first network device can compare the sorted new hash key with the sorted hash key to identify new next-hop IP address field entries. For example, the data plane can compare the entries of the sorted new hash key with the entries of the sorted hash key to determine whether the sorted new hash key includes entries not included in the sorted hash key, and / or to determine whether the sorted new hash key includes entries included in the sorted hash key. In some implementations, the data plane can identify new next-hop IP address field entries based on a comparison of the sorted new hash key and the sorted hash key.

[0033] like Figure 1E As further illustrated by reference numeral 175 in the accompanying drawing, the data plane of the first network device can update the sorted hash table and generate an updated hash table based on new next-hop IP address field entries. For example, based on new next-hop IP address field entries, the data plane of the first network device can generate an updated hash table including an index field and a selector table field. The index field may include values ​​of 0, 1, 2, 3, 4, and / or similar, and the selector table field may include entries identifying the order of the next-hop IP addresses (e.g., 0→10.1.1.1; 1→10.1.1.2; 3→10.1.1.5; 2→10.1.1.3; and / or similar). In some implementations, the first network device may store the updated hash table in its data plane. In some implementations, the second network device may perform the same functionality described above for the first network device, such that the second network device generates the same updated hash table as the first network device. In this way, the first and second network devices can route traffic symmetrically to each other via a firewall network device.

[0034] like Figure 1EAs further illustrated by reference numeral 180 in the accompanying drawing, the first network device can utilize the updated hash table to symmetrically route traffic. For example, when the first network device receives traffic destined for the Internet or the core network, it can use the updated hash table to generate a forwarding path to route the traffic from the first network device to the Internet or the core network. In some implementations, the first network device can select one of the firewall network devices identified in the updated hash table for the forwarding path.

[0035] In some implementations, the first and second network devices can receive new hash keys from missing next-hop IP address field entries provided in the hash keys, and can sort the new hash keys based on the next-hop IP address field to generate sorted new hash keys. The first and second network devices can compare the sorted new hash keys with the sorted hash keys to identify missing next-hop address field entries, and can update the sorted hash table based on the missing next-hop address field entries to generate an updated hash table. The first and second network devices can utilize the updated hash table to symmetrically route services between the customer or access network and the Internet or core network.

[0036] In some implementations, for symmetric consistent hashing, the first and second network devices can handle ECMP device / next-hop events in the data plane as follows: When a device (e.g., a firewall network device) fails, the first and second network devices can update the ECMP device / next-hop in the data plane to have an inactive state and invalid weights. The failed device can be disregarded during selector recalculation.

[0037] When a device / next hop with the exact same IP address as the failed device appears in a new ECMP next-hop list from the control plane, both the first and second network devices can treat this as a device addition and can again mark the device as active with valid weights. If a next hop is not resolved during the initial ECMP next-hop list sent from the control plane, both the first and second network devices can retain the unresolved next hops in the data plane ECMP next-hop list and can also treat these next hops as inactive with invalid weights. When updating the new ECMP next-hop list from the control plane, both the first and second network devices can update these unresolved next hops to resolved next hops, where the new device is set at its position in the data plane ECMP next-hop list.

[0038] The first and second network devices can treat any new ECMP next hop (e.g., one not found in the existing data plane) as an unseen device until an unresolved next hop exists in the data plane's ECMP next hop list. The first and second network devices can provide hash functions or hash key symmetry based on pre-existing load balancing hashes (e.g., hash key functions), where only the source IP is used in the forward direction and only the destination IP is used in the reverse direction to maintain hash key symmetry.

[0039] The first and second network devices can provide deterministic ECMP path ordering by ensuring the same order / index position of ECMP devices by maintaining the same next-hop IP to active next-hop index mapping across the first and second network devices. This ensures that the hash table index points to the same ECMP next hop / device in both upstream and downstream directions. For this implementation, the first and second network devices can maintain ECMP elements in the data plane in sorted order based on the next-hop IP as the key. The first and second network devices can maintain a local copy of the next-hop IP to active index mapping in the data plane to determine if there is any movement / shift of the next hop due to any new device and its index position dependency in the sorted ECMP list.

[0040] The first and second network devices can provide control plane out-of-order handling in the data plane. When multiple ECMP devices fail or are added back, the data plane may need to provide ECMP link / node events from the control plane in a specific order to ensure hash table symmetry. To handle such out-of-order events, the first and second network devices can maintain two states in the data plane hash table: the original selector state and the original selector state from the last failure. Once all unseen device additions are completed in both directions, the first and second network devices can retain this state as the original selector state. During multiple device failures, the first and second network devices can build the selector state from this state in a specific order (e.g., ascending during device failures and descending during seen device additions). This can maintain the failure / addition order and can provide a reference original state from which the hash table is built across the two independent network devices in a similar manner. During multiple device failures, the first and second network devices can retain the last device failure state as the original last failed selector state. During the observed device addition process, this state can be used as a reference, and each time a device is added, the network device can generate a hash table from this selector state in descending order (e.g., the opposite of device failure). This helps maintain the symmetry of the hash table and keep it balanced.

[0041] Figure 1F An example of hash table sorting without symmetric consistent hashing in a forwarding plane is depicted. For example... Figure 1F As shown, the hash table state 1-1 of the first network device can initially be matched with the hash table state 2-1 of the second network device. If the first firewall network device (server 1) fails, the first network device can update hash table state 1-1 to hash table state 1-2. If the second firewall network device (server 2) fails, the second network device can update hash table state 2-1 to hash table state 2-2, and the first network device can update hash table state 1-2 to hash table state 1-3. If the first firewall network device (server 1) fails, the second network device can update hash table state 2-2 to hash table state 2-3. If the first firewall network device (server 1) is added back, the first network device can update hash table state 1-3 to hash table state 1-4, and the second network device can update hash table state 2-3 to hash table state 2-4. If the second firewall network device (server 2) is added back, the first network device can update hash table state 1-4 to hash table state 1-5, and the second network device can update hash table state 2-4 to hash table state 2-5. As shown in the diagram, hash table states 1-2, 1-3, 1-4, and 1-5 do not match the corresponding hash table states 2-2, 2-3, 2-4, and 2-5. Therefore, the hash table states in the first and second network devices are asymmetric.

[0042] Figures 1G-1K An example of hash table sorting with symmetric consistent hashing in the forwarding plane is depicted. For example... Figure 1G As shown, the hash table state 1-1 of the first network device can initially match the hash table state 2-1 of the second network device. If the first firewall network device (server 1) fails, it can update hash table state 1-1 to hash table state 1-2. If the second firewall network device (server 2) fails, it can update hash table state 2-1 to hash table state 2-2. At this point, because not all hash events are received from the control plane, there may be transient asymmetric states.

[0043] like Figure 1H As shown, if the second firewall network device (server 2) fails, the first network device can update hash table state 1-2 to hash table state 1-3, and the second network device can update hash table state 2-2 to hash table state 2-3. Hash table state 1-3 can match hash table state 2-3.

[0044] like Figure 1IAs shown, if the first firewall network device (server 1) is added back, the first network device can update hash table state 1-3 to hash table state 1-4, and the second network device can update hash table state 2-3 to hash table state 2-4. Hash table state 1-4 can match hash table state 2-4.

[0045] like Figure 1J As shown, the hash table states in the first and second network devices are symmetrical. In some implementations, server failures can be handled in ascending order, starting from the original selector state on each failure. In some implementations, server additions can be handled in descending order, starting from the original state where the last failure occurred on each addition.

[0046] like Figure 1K As shown, if the second firewall network device (server 2) is added back, the first network device can update hash table states 1-3 to 1-4, and the second network device can update hash table states 2-3 to 2-4. Hash table states 1-4 can match hash table states 2-4. Figure 1K As further illustrated, if the first firewall network device (server 1) is added back, the first network device can update hash table state 1-4 to hash table state 1-5, and the second network device can update hash table state 2-4 to hash table state 2-5. Hash table state 1-5 can match hash table state 2-5.

[0047] Figure 1L Another example of hash table ordering with symmetric consistent hashing in the forwarding plane is depicted. For example... Figure 1LAs shown, hash table state 1-1 of the first network device can initially match hash table state 2-1 of the second network device. If the first firewall network device (server 1) is added, the first network device can update hash table state 1-1 to hash table state 1-2, and the second network device can update hash table state 2-1 to hash table state 2-2. Hash table state 1-2 can match hash table state 2-2. If the second firewall network device (server 2) is added, the first network device can update hash table state 1-1 to hash table state 1-3, and the second network device can update hash table state 2-1 to hash table state 2-3. Hash table state 1-3 can match hash table state 2-3. If the first firewall network device (server 1) is added after the second firewall network device (server 2) is added, the first network device can update hash table state 1-3 to hash table state 1-4, and the second network device can update hash table state 2-3 to hash table state 2-4. Hash table state 1-4 can match hash table state 2-4. Therefore, the hash table states in the first network device and the second network device are symmetrical.

[0048] In this way, symmetric consistent hashing is provided to network devices in the forwarding plane. For example, network devices can utilize parameters that symmetrically select the same hash table or selector table index in both the upstream and downstream directions. The hash tables of active network devices across the upstream and downstream directions can be symmetric, such that the selected hash table index includes the same next hop in both directions. Network devices can provide forwarding plane-driven solutions that operate independently of routing protocols. Therefore, network devices save computational, network, and / or similar resources that would otherwise be consumed by failing to provide consistency and symmetry in traffic flow for upstream and downstream traffic handled by the network device, handling service interruptions associated with the failure to provide consistency and symmetry in traffic flow, handling session interruptions associated with the failure to provide consistency and symmetry in traffic flow, and / or the like.

[0049] As indicated above, Figures 1A-1L Provided as an example. Other examples may be provided with reference to [the relevant information]. Figures 1A-1L The examples described in the text are different. Figures 1A-1L The number and layout of devices shown are provided as an example. In practice, they can exist in combination with... Figures 1A-1L The equipment shown is compared to additional equipment, fewer equipment, different equipment, or equipment arranged differently. Furthermore, Figures 1A-1L The two or more devices shown can be implemented within a single device, or Figures 1A-1L The single device shown can be implemented as multiple distributed devices. Additionally, or alternatively, Figures 1A-1LThe set of devices shown (e.g., one or more devices) can perform actions described as being performed by Figure 2 One or more functions performed by another set of devices shown.

[0050] Figure 2 This is a diagram of an example environment 200 in which the systems and / or methods described in this paper can be implemented. (See diagram 200 for example environments 200.) Figure 2 As shown, environment 200 may include user equipment 210, server equipment 220, a group of network devices 230 (shown as network devices 230-1 to network devices 230-N), and network 240. The devices in environment 200 may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0051] User equipment 210 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information (such as the information described herein). For example, user equipment 210 may include mobile phones (e.g., smartphones or wireless phones), laptop computers, tablet computers, desktop computers, handheld computers, gaming devices, wearable communication devices (e.g., smartwatches, smart glasses, heart rate monitors, fitness trackers, smart clothing, smart jewelry, or head-mounted displays), network devices, server devices, a group of server devices, or similar types of devices. In some implementations, user equipment 210 may receive network services from other user equipment 210 and / or server devices 220 and / or provide network services to other user equipment 210 and / or server devices 220 via network 240 (e.g., by using network device 230 as an intermediary for packet routing).

[0052] Server device 220 may include one or more devices capable of receiving, generating, storing, processing, providing, and / or routing information, as described elsewhere herein. Server device 220 may include communication devices and / or computing devices. For example, server device 220 may include servers such as application servers, client servers, web servers, database servers, host servers, proxy servers, virtual servers (e.g., executing on computing hardware), or servers in cloud computing systems. In some implementations, server device 220 may include computing hardware used in a cloud computing environment.

[0053] Network device 230 includes one or more devices capable of receiving, processing, storing, routing, and / or providing services (e.g., packets or other information or metadata) in the manner described herein. For example, network device 230 may include routers such as label switching routers (LSRs), label edge routers (LERs), ingress routers, egress routers, provider routers (e.g., provider edge routers or provider core routers), virtual routers, route reflectors, area border routers, or other types of routers. Additionally, or alternatively, network device 230 may include gateways, switches, firewalls, hubs, bridges, reverse proxies, servers (e.g., proxy servers, cloud servers, or data center servers), load balancers, and / or similar devices. In some implementations, network device 230 may be a physical device implemented within a enclosure such as a chassis. In some implementations, network device 230 may be a virtual device implemented by one or more computer devices in a cloud computing environment or data center. In some implementations, a group of network devices 230 may be a group of data center nodes used to route service flows through network 240.

[0054] Network 240 includes one or more wired and / or wireless networks. For example, network 240 may include packet-switched networks, cellular networks (e.g., fifth-generation (5G) networks, fourth-generation (4G) networks, such as Long Term Evolution (LTE) networks, third-generation (3G) networks, Code Division Multiple Access (CDMA) networks, Public Land Mobile Networks (PLMN)), local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), telephone networks (e.g., Public Switched Telephone Networks (PSTN)), private networks, self-organizing networks, intranets, the Internet, fiber-optic-based networks, cloud computing networks, or similar networks, and / or combinations of these or other types of networks.

[0055] Figure 2 The number and arrangement of devices and networks shown are provided as examples. In practice, similar arrangements may exist. Figure 2 The devices and / or networks shown are compared to additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or devices and / or networks arranged differently. Furthermore, Figure 2 The two or more devices shown can be implemented within a single device, or Figure 3 The single device shown can be implemented as multiple distributed devices. Additionally, or alternatively, the set of devices in environment 200 (e.g., one or more devices) can perform one or more functions described as being performed by another set of devices in environment 200.

[0056] Figure 2 yes Figure 3A diagram illustrating example components of one or more devices. Example components may be included in device 300, which may correspond to user device 210, server device 220, and / or network device 230. In some implementations, user device 210, server device 220, and / or network device 230 may include one or more devices 300 and / or one or more components of device 300. Figure 3 As shown, device 300 may include bus 310, processor 320, memory 330, input component 340, output component 350 and communication component 360.

[0057] Bus 310 includes one or more components that enable wired and / or wireless communication between components of device 300. Bus 310 can... Figure 3 Two or more components are coupled together, such as via operational coupling, communication coupling, electronic coupling, and / or electrical coupling. Processor 320 includes a central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), microprocessor, controller, microcontroller, digital signal processor (DSP), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), and / or another type of processing component. Processor 320 is implemented in hardware, firmware, or a combination of hardware and software. In some implementations, processor 320 includes one or more processors capable of being programmed to perform one or more operations or processes described elsewhere herein.

[0058] Memory 330 includes volatile memory and / or non-volatile memory. For example, memory 330 may include random access memory (RAM), read-only memory (ROM), hard disk drive, and / or another type of memory (e.g., flash memory, magnetic memory, and / or optical memory). Memory 330 may include internal memory (e.g., RAM, ROM, or hard disk drive) and / or removable memory (e.g., removable via a universal serial bus connection). Memory 330 may be a non-transient computer-readable medium. Memory 330 stores information, instructions, and / or software (e.g., one or more software applications) related to the operation of device 300. In some implementations, memory 330 includes one or more memories, such as those coupled to one or more processors (e.g., processor 320) via bus 310.

[0059] Input component 340 enables device 300 to receive input, such as user input and / or sensed input. For example, input component 340 may include a touchscreen, keyboard, keypad, mouse, button, microphone, switch, sensor, GPS sensor, accelerometer, gyroscope, and / or actuator. Output component 350 enables device 300 to provide output, such as via a display, speaker, and / or light-emitting diode. Communication component 360 enables device 300 to communicate with other devices via wired and / or wireless connections. For example, communication component 360 may include a receiver, transmitter, transceiver, modem, network interface card, and / or antenna.

[0060] Device 300 may perform one or more operations or procedures described herein. For example, a non-transient computer-readable medium (e.g., memory 330) may store a set of instructions (e.g., one or more instructions or code) for execution by processor 320. Processor 320 may execute the set of instructions to perform one or more operations or procedures described herein. In some implementations, execution of the set of instructions by one or more processors 320 causes one or more processors 320 and / or device 300 to perform one or more operations or procedures described herein. In some implementations, hardwired circuitry may be used instead of or in combination with instructions to perform one or more operations or procedures described herein. Additionally, or alternatively, processor 320 may be configured to perform one or more operations or procedures described herein. Therefore, the implementations described herein are not limited to any particular combination of hardware circuitry and software.

[0061] Figure 3 The number and arrangement of components shown are provided as an example. Device 300 may include components with... Figure 4 The components shown are compared to additional components, fewer components, different components, or components arranged differently. Additionally, or alternatively, the set of components of device 300 (e.g., one or more components) can perform one or more functions described as being performed by another set of components of device 300.

[0062] Figure 2 yes Figure 4 A diagram illustrating example components of one or more devices. Example components may be included in device 400. Device 400 may correspond to network device 230. In some implementations, network device 230 may include one or more devices 400 and / or one or more components of device 400. Figure 4As shown, device 400 may include one or more input components 410-1 to 410-B (B≥1) (hereinafter collectively referred to as input components 410, and individually referred to as input components 410), switching component 420, one or more output components 430-1 to 430-C (C≥1) (hereinafter collectively referred to as output components 430, and individually referred to as output components 430), and controller 440.

[0063] Input component 410 may be one or more attachment points of a physical link and may be one or more entry points for incoming traffic (such as packets). Input component 410 may process incoming traffic, such as by performing data link layer encapsulation or decapsulation. In some implementations, input component 410 may transmit and / or receive packets. In some implementations, input component 410 may include an input line card that includes one or more packet processing components (e.g., in the form of integrated circuits), such as one or more interface cards (IFCs), packet forwarding components, line card controller components, input ports, processors, memory, and / or input queues. In some implementations, device 400 may include one or more input components 410.

[0064] Switching component 420 interconnects input component 410 with output component 430. In some implementations, switching component 420 may be implemented via one or more crossbar switches, via a bus, and / or utilizing shared memory. Shared memory may act as a temporary buffer to store packets from input component 410, which are then eventually scheduled for delivery to output component 430. In some implementations, switching component 420 may enable communication between input component 410, output component 430, and / or controller 440.

[0065] Output component 430 can store packets and schedule packets for transmission on the output physical link. Output component 430 can support data link layer encapsulation or decapsulation, and / or various higher-level protocols. In some implementations, output component 430 can send and / or receive packets. In some implementations, output component 430 may include an output line card that includes one or more packet processing components (e.g., in the form of integrated circuits), such as one or more IFCs, packet forwarding components, line card controller components, output ports, processors, memory, and / or output queues. In some implementations, device 400 may include one or more output components 430. In some implementations, input component 410 and output component 430 may be implemented from the same set of components (e.g., the input / output component may be a combination of input component 410 and output component 430).

[0066] Controller 440 includes a processor, which may take the form of, for example, a CPU, GPU, APU, microprocessor, microcontroller, DSP, FPGA, ASIC, and / or another type of processor. The processor may be implemented in hardware, firmware, or a combination of hardware and software. In some implementations, controller 440 may include one or more processors that can be programmed to perform functions.

[0067] In some implementations, controller 440 may include RAM, ROM, and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) for storing information and / or instructions for use by controller 440.

[0068] In some implementations, controller 440 can communicate with other devices, networks, and / or systems connected to device 400 to exchange information about the network topology. Controller 440 can create routing tables based on the network topology information, create forwarding tables based on the routing tables, and forward the forwarding tables to input component 410 and / or output component 430. Input component 410 and / or output component 430 can use the forwarding tables to perform route looks for incoming and / or outgoing packets.

[0069] Controller 440 may execute one or more of the processes described herein. Controller 440 may execute these processes in response to the execution of software instructions stored in a non-transitory computer-readable medium. Computer-readable medium is defined herein as a non-transitory storage device. Memory devices include memory space within a single physical storage device or memory space distributed across multiple physical storage devices.

[0070] Software instructions may be read from another computer-readable medium or from another device via a communication component into the memory and / or storage components associated with controller 440. When executed, the software instructions stored in the memory and / or storage components associated with controller 440 may cause controller 440 to perform one or more processes described herein. Additionally, or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Therefore, the implementation described herein is not limited to any particular combination of hardware circuitry and software.

[0071] Figure 4 The number and arrangement of components shown are provided as an example. In practice, device 400 may include components with... Figure 5 The components shown are compared to additional components, fewer components, different components, or components arranged differently. Additionally, or alternatively, the component set of device 400 (e.g., one or more components) can perform one or more functions performed by another component set of device 400.

[0072] Figure 5 This is a flowchart of an example process 500 for providing symmetric consistent hashing in the forwarding plane of network devices. In some implementations, Figure 5 One or more process blocks can be executed by a network device (e.g., network device 230). In some implementations, Figure 5 One or more process blocks may be executed by another device or group of devices (such as another network device (e.g., network device 230)) that is separate from or includes the network device. Additionally or alternatively, Figure 5 One or more processing blocks can be executed by one or more components of device 300, such as processor 320, memory 330, input component 340, output component 350, and / or communication component 360. Additionally or alternatively, Figure 5 One or more processing blocks can be executed by one or more components of the device 400, such as input component 410, switching component 420, output component 430 and / or controller 440.

[0073] like Figure 5 As shown, process 500 may include receiving a hash key from the control plane of the network device (block 510). For example, as described above, the network device may receive the hash key from the control plane of the network device. In some implementations, the hash key is a load-balanced hash key function.

[0074] like Figure 5 As further shown, process 500 may include sorting the hash keys based on the next-hop address field and generating sorted hash keys (block 520). For example, as described above, the network device may sort the hash keys based on the next-hop address field and generate sorted hash keys. In some implementations, sorting the hash keys based on the next-hop address field and generating sorted hash keys includes using the network device's data plane to sort the hash keys based on the next-hop address field and generate sorted hash keys.

[0075] like Figure 5 As further shown, process 500 may include storing the sorted hash keys locally in the data plane of the network device (block 530). For example, as described above, the network device may store the sorted hash keys locally in the data plane of the network device.

[0076] like Figure 5 As further shown, process 500 may include generating a sorted hash table based on the sorted hash keys (block 540). For example, as described above, a network device may generate a sorted hash table based on the sorted hash keys.

[0077] like Figure 5As further shown, process 500 may include using a sorted hash table to symmetrically route traffic with another network device (block 550). For example, as described above, a network device may use a sorted hash table to symmetrically route traffic with another network device.

[0078] In some implementations, process 500 includes receiving a first service provided by a customer and destined for the core network, and routing the first service to the core network through a firewall network device and another network device based on a sorted hash table. In some implementations, process 500 includes receiving a second service provided by the core network and destined for a customer, wherein the second service is received from another network device through a firewall network device and based on a sorted hash table, and providing the second service to the customer. In some implementations, the core network is an Internet Protocol (IP) based network. In some implementations, the first service is received from an access network.

[0079] In some implementations, process 500 includes: receiving a new hash key including a new next-hop address field entry; sorting the new hash key based on the next-hop address field to generate a sorted new hash key; and comparing the sorted new hash key with the sorted hash key to identify the new next-hop address field entry. In some implementations, process 500 includes updating the sorted hash table based on the new next-hop address field entry and generating an updated hash table; and symmetrically routing traffic using the updated hash table.

[0080] In some implementations, process 500 includes receiving a new hash key from a missing next-hop address field entry provided in the hash key, sorting the new hash key based on the next-hop address field to generate a sorted new hash key, and comparing the sorted new hash key with the sorted hash key to identify the missing next-hop address field entry. In some implementations, process 500 includes updating the sorted hash table based on the missing next-hop address field entry and generating an updated hash table, and using the updated hash table to symmetrically route traffic.

[0081] In some implementations, process 500 includes receiving an indication from another network device that an equal-cost multipath routing (ECMP) path is disabled in the upstream direction, and disabling the ECMP path in the downstream direction based on the indication to maintain hash table symmetry. In some implementations, process 500 includes receiving an indication that a path between another network device and a firewall network device is disabled, and disabling the path between that network device and the firewall network device based on the indication.

[0082] Although Figure 5 An example block of process 500 is shown, but in some implementations, process 500 may include... ​The blocks shown are compared to additional blocks, fewer blocks, different blocks, or blocks arranged differently. Additionally, or alternatively, two or more blocks of process 500 can be executed in parallel.

[0083] The foregoing disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit the implementation to the precise form disclosed. Modifications may be made in light of the foregoing disclosure or may be derived from the practice of implementation.

[0084] As used herein, the term "component" is intended to be broadly understood as hardware, firmware, or a combination of hardware and software. It will be apparent that the systems and / or methods described herein can be implemented in various forms of hardware, firmware, and / or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit these implementations. Therefore, this document does not refer to specific software code in describing the operation and behavior of the systems and / or methods—it should be understood that software and hardware can be used to implement the systems and / or methods based on the description herein.

[0085] Although specific combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of various implementations. In fact, many of these features can be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may be directly dependent on only one claim, the disclosure of various implementations includes combinations of each dependent claim with every other claim in the claim set.

[0086] Unless explicitly stated otherwise, no element, action, or instruction used herein should be construed as critical or necessary. Furthermore, as used herein, the words “a” and “an” are intended to include one or more items and are interchangeable with “one or more.” Furthermore, as used herein, the word “the” is intended to include one or more items mentioned in connection with the word “the” and is interchangeable with “the one or more.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, and / or similar) and is interchangeable with “one or more.” When only one item is intended, the phrase “only one” or similar language is used. Furthermore, as used herein, the terms “have,” “possess,” “have,” or similar are intended to be open-ended terms. Furthermore, unless explicitly stated otherwise, the phrase “based on” is intended to mean “at least partially based on.” Furthermore, as used herein, unless otherwise expressly stated (e.g., if used in combination with “any one” or “only one of”), the term “or” is intended to be inclusive when used in a series and can be used interchangeably with “and / or”.

[0087] Various exemplary embodiments have been described in the foregoing description with reference to the accompanying drawings. However, it will be apparent that various modifications and alterations can be made thereto, and additional embodiments can be implemented, without departing from the broader scope of the invention as set forth in the appended claims. Therefore, the description and drawings should be considered illustrative rather than restrictive.

Claims

1. A method comprising: The hash key is received by the network device from its control plane. The network device sorts the hash keys based on the next-hop address field and generates sorted hash keys. The network device locally stores the sorted hash keys in the data plane of the network device; The network device generates a sorted hash table based on the sorted hash keys; as well as The network device uses the sorted hash table to route services symmetrically with another network device.

2. The method according to claim 1, further comprising: Receive the first service provided by the customer and destined for the core network; as well as Based on the sorted hash table, the first service is routed to the core network through the firewall network device and the other network device.

3. The method according to claim 2, further comprising: Receive a second service provided from the core network and destined for the customer. The second service is received from the other network device via the firewall network device and based on the sorted hash table; and The second service is provided to the customer.

4. The method according to claim 3, wherein the core network is a network based on the Internet Protocol.

5. The method of claim 2, wherein the first service is received from the access network.

6. The method according to claim 1, further comprising: Receive a new hash key that includes an entry for the new next-hop address field; The new hash keys are sorted based on the next-hop address field, and sorted new hash keys are generated. as well as The sorted new hash key and the sorted hash key are compared to identify the new next-hop address field entry.

7. The method according to claim 6, further comprising: The sorted hash table is updated based on the new next-hop address field entries, and an updated hash table is generated. as well as The updated hash table is used to symmetrically route services.

8. A network device, comprising: One or more memory units; as well as One or more processors, for: Receive hash keys from the control plane of the network device; The hash keys are sorted based on the next-hop address field to generate sorted hash keys; The sorted hash keys are stored locally in the data plane of the network device; A sorted hash table is generated based on the sorted hash keys; Receive the first service provided by the customer and destined for the core network; as well as Based on the sorted hash table, the first service is routed to the core network through a firewall network device and another network device.

9. The network device of claim 8, wherein the one or more processors are further configured to: Receive a new hash key that lacks the next-hop address field entry provided in the hash key; The new hash keys are sorted based on the next-hop address field, and sorted new hash keys are generated. as well as The sorted new hash key and the sorted hash key are compared to identify the missing next-hop address field entry.

10. The network device of claim 9, wherein the one or more processors are further configured to: The sorted hash table is updated based on the missing next-hop address field entries, and an updated hash table is generated; and The updated hash table is used to symmetrically route services.

11. The network device of claim 8, wherein, in order to sort the hash keys based on the next-hop address field and generate the sorted hash keys, the one or more processors are configured to: The data plane of the network device is used to sort the hash keys based on the next-hop address field and generate the sorted hash keys.

12. The network device of claim 8, wherein the one or more processors are further configured to: Receive an indication from the other network device that the equal-cost multipath (ECMP) route is disabled in the upstream direction; and The ECMP path is disabled downstream based on the instruction to maintain hash table symmetry.

13. The network device of claim 8, wherein the one or more processors are further configured to: Receive an indication that the path between the other network device and the firewall network device is disabled; and The path between the network device and the firewall network device is disabled based on the instruction.

14. The network device according to claim 8, wherein the hash key is a load balancing hash key function.

15. A non-transient computer-readable medium storing a set of instructions, the set of instructions comprising: One or more instructions, which, when executed by one or more processors of the network device, cause the network device to: Receive hash keys from the control plane of the network device. The hash key mentioned above is a load-balanced hash key function; The hash keys are sorted based on the next-hop address field to generate sorted hash keys; The sorted hash keys are stored locally in the data plane of the network device; A sorted hash table is generated based on the sorted hash keys; as well as The sorted hash table is used to route services symmetrically with another network device.

16. The non-transient computer-readable medium of claim 15, wherein one or more instructions further cause the network device to: Receive the first service provided by the customer and destined for the core network; Based on the sorted hash table, the first service is routed to the core network via a firewall network device and the other network device; and Receive a second service provided from the core network and destined for the customer. The second service is received from the other network device via the firewall network device and based on the sorted hash table; and The second service is provided to the customer.

17. The non-transient computer-readable medium of claim 15, wherein one or more instructions further cause the network device to: Receive a new hash key that includes an entry for the new next-hop address field; The new hash keys are sorted based on the next-hop address field, and sorted new hash keys are generated. as well as The sorted new hash key and the sorted hash key are compared to identify the new next-hop address field entry.

18. The non-transient computer-readable medium of claim 17, wherein one or more instructions further cause the network device to: The sorted hash table is updated based on the new next-hop address field entries, and an updated hash table is generated; and The updated hash table is used to symmetrically route services.

19. The non-transient computer-readable medium of claim 15, wherein one or more instructions further cause the network device to: Receive a new hash key that lacks the next-hop address field entry provided in the hash key; The new hash keys are sorted based on the next-hop address field, and sorted new hash keys are generated. as well as The sorted new hash key and the sorted hash key are compared to identify the missing next-hop address field entry.

20. The non-transient computer-readable medium of claim 19, wherein one or more instructions further cause the network device to: The sorted hash table is updated based on the missing next-hop address field entries, and an updated hash table is generated; and The updated hash table is used to symmetrically route services.

Images