PACKAGE INTERNAL VERSION TAGGING USING A PERIMETER NAT

Abstract

A computer-implemented method that features: Receiving (702) a packet from a node in a first network at a trace unit (430); Determining (704) a version identifier (81) for the packet by the link tracing unit, wherein the version identifier (81) includes information that identifies the version of a state of a second network; Encoding (706) the version identifier (81) into the packet by the link tracking unit, wherein encoding the version identifier (81) into the packet includes encoding the version identifier (81) into a destination terminal field of the packet; Transfer (708) of the packet containing the encoded version identifier (81) by the connection tracking unit (430) to a forwarding unit (450), wherein the forwarding unit (450) selects a back-end unit based on a client connection affinity, and wherein the client connection affinity is partly based on the encoded version identifier (81) in the packet; Restore, through the forwarding unit (450), the destination terminal field of the packet; and The forwarding unit (450) transmits the packet to the back-end unit, wherein the back-end unit is located in a second network, and wherein the forwarding unit (450) is either in the first network or in the second network.

Description

BACKGROUND

[0001] Embodiments of the present invention relate to sending data between networks, and more particularly to maintaining flow affinity independent of version or state changes in a network receiving data.

[0002] Maintaining flow affinity ensures that related data packets, or "flows," are forwarded to the same destination, e.g., data transfers between a client in a given network and a back-end server in another network. Maintaining flow affinity across a network is a desirable network attribute of a network forwarding function that supports multiple destinations and / or multiple paths (e.g., a load balancing system). For example, data packets from the same Transmission Control Protocol (TCP) connection require affinity to avoid connection loss, data packets from the same source IP address require affinity to enable abbreviated Transport Layer Security (TLS) handshakes, and data packets from the same client compute unit require affinity to support application-specific optimizations and session state.

[0003] US 2016 / 0127224 A1 describes a method in which a routing bridge sends an election packet to an STP device to effect spanning tree propagation and then performs a DRB election based on a second election packet to minimize VLAN traffic disruption and ensure network stability. SUMMARY

[0004] The invention is described by the features of the independent claims. Embodiments are specified in the dependent claims. It is an object of the invention to provide an improved computer-implemented method according to claim 1 and an improved computer system according to claim 10.

[0005] Embodiments of the present invention utilize the connection tracking functionality of a connection tracking unit for making affinity-preserving decisions by a forwarding unit. Relevant information, such as the configuration version at the initiation of each connection, can be passed by the connection tracking unit to the forwarding unit. The connection tracking unit can be located anywhere along the path between a source (e.g., the client) and the forwarding unit. The connection tracking unit can be located at the perimeter of a source network or at the perimeter of the forwarding unit's network.The connection tracking and forwarding units can be units that also perform network address translation (NAT) or load balancing, but can be any other network unit where the connection tracking unit performs connection tracking and the forwarding unit makes affinity-based decisions.

[0006] The data transfer between the connection tracking unit and the forwarding unit for passing the version information can be performed in any manner that does not impact the connection. One embodiment includes encoding a version identifier in the destination port of each packet, which is a method that conveniently does not change packet size, is a standard Network Address and Port Translation (NAPT) operation, and can be easily reversed by the forwarding unit, making the operation transparent to clients and back-end servers. In another embodiment, an overlay tunnel, such as a virtual private network (VPN), may exist between the connection tracking unit and the forwarding unit.In this embodiment, the version information can be encoded into the overlay rather than into each individual packet, where a tunnel can be created for a specific version and the version identifier can be appended to the tunnel label in the header.

[0007] Various embodiments relate to a method implemented on a computer. In one embodiment, the method may include receiving a packet from a node in a first network at a connection tracking entity, determining a version identifier for the packet, encoding the version identifier into the packet, and transmitting the packet containing the encoded version identifier to a forwarding entity. These operations may be performed by the connection tracking entity. In one embodiment, the forwarding entity selects a backend entity in a second network based on client connection affinity and transmits the packet to the selected backend entity. The version identifier may be encoded into a destination port field of the first packet. The connection tracking entity may be a perimeter network address translation (NAT) entity.The forwarder can be a load balancer. The connection tracking unit can specify its own timestamp, and the load balancer can track the correlation between timestamp and version. This may not be fully synchronized, but this is acceptable if the timestamp is updated much faster than the version. The load balancer only needs a way to repeat the decision it made for the first packet of a connection for all other packets on that connection. The connection tracking unit can ensure that the same timestamp is used for all packets on a single connection. The timestamp must be updated quickly enough to allow new connections to be mapped differently if the load balancer's version changes.If this happens quickly enough, the number of packages allocated using the old version will be small.

[0008] In one embodiment, the version identifier corresponds to a set of backend devices currently operational in the second network. In various embodiments, a current version is based on time, and determining the version identifier may include determining a period of time during which a current version identifier has been used and updating the version identifier if the determined period exceeds a period specified for updating the version identifier.

[0009] In various embodiments, a current version is based on a set of packets, and determining the version identifier may include determining a set of packets received at the connection tracking entity for which a particular version identifier was used, and updating the version identifier if the determined set of packets exceeds a certain set of packets specified for updating the version identifier. As with the timestamp-version correlation above, the load balancer only needs one way to repeat the decision it made for the first packet of a connection for all other packets of that connection. The connection tracking entity may ensure that the same version is used for all packets of a single connection.The version must be updated quickly enough to allow new connections to be mapped differently when the load balancer version changes. If this happens quickly enough, the number of packets mapped using the old version will be small.

[0010] In various embodiments, determining the version identifier includes receiving, by the connection tracking unit, a control message from another node and updating the version identifier in response to the control message.

[0011] In various embodiments, determining the version identifier includes receiving a return packet by the connection tracking unit, wherein, in response to transmitting the packet containing the encoded version identifier to the forwarding unit, the return packet is transmitted to the connection tracking unit, and the return packet is encoded with a version identifier. A current version is updated based on the version identifier encoded in the return packet.

[0012] In various embodiments, the version identifier corresponds to a hash function used by the load balancer in the second network or a policy deployed by the load balancer.

[0013] In another embodiment, a method implemented on a computer includes receiving a packet from a network device of a first network by a network device of a second network. The forwarding device and at least one back-end device may be included in the second network. The method may include extracting, by the forwarding device, a version identifier for the packet from the packet. Furthermore, the method may include extracting, by the forwarding device, a source address and a source port from the packet and determining a hash value using a particular hash function based on the extracted source address and source port.Furthermore, the method may include: using the hash value, by the forwarding unit, to select a network address of a back-end unit that corresponds to the remainder generated by dividing the integer hash value by the number of back-end units; encoding, by the forwarding unit, the selected network address into the packet as a destination address; and transmitting, by the forwarding unit, the packet containing the selected network address to a back-end unit having the selected network address.

[0014] In some embodiments, the particular hash function used to determine the hash value is selected from two or more hash functions based on the version identifier for the package.

[0015] In some embodiments, the particular set of backend devices currently operational in the second network is selected from two or more sets of backend devices based on the version identifier included in the packet.

[0016] Other embodiments relate to a system, device, or computer program product for performing the methods disclosed herein.

[0017] This Summary is provided to introduce, in a simplified form, a selection of concepts that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential characteristics of the claimed subject matter, nor is it intended to be used as an aid to determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 illustrates a cloud computing environment according to an embodiment of the present invention. Fig. 2 illustrates abstraction model layers according to an embodiment of the present invention. Fig. 3 is a block diagram showing a data processing unit used as network units of Fig. 4 can be used, according to an embodiment of the invention. Fig. 4 is a functional block diagram illustrating an exemplary computing environment according to an embodiment of the present invention. Fig. 5 is a functional block diagram illustrating data flows between networks according to an embodiment of the present invention. Fig. Figure 6A is a block diagram of an example packet flow between networks. Fig. 6B is a block diagram of packet flow between networks according to various embodiments of the present invention. Fig. 7A is a flowchart of a process for processing packets by a network gateway (referred to herein as a "perimeter NAT" or a "connection trace" entity) according to various embodiments of the present invention. Fig. 7B is a flowchart of a process for processing packets by a load balancer (referred to herein as a "forwarding" entity) according to various embodiments of the present invention. Fig. 8 is a block diagram of packet forwarding in a load balancer according to various embodiments of the present invention. DETAILED DESCRIPTION

[0018] In distributed utility computing environments such as clouds and other networks, it is beneficial to ensure that resources providing computing services can be elastically expanded and contracted according to the service load imposed by client computing devices. As a client service load increases, a network is expected to adapt by allocating more resources and increasing service capacity so that client computing devices are not subject to network disruptions resulting from the increased service load. As the client service load decreases, the network is expected to adapt by releasing unused resources in a timely manner to reduce total operating costs and improve overall resource utilization by reallocating resources elsewhere.Resource elasticity is typically implemented through a scalable pool of back-end network service nodes, such as a back-end network server. Backend network service nodes are added and removed to align allocated resource capacity with the overall service load imposed by the client. An additional entity, a load balancer, can be added to the distributed computing environment to receive client requests and multiplex them among the back-end network service nodes in the pool.

[0019] Typically, the load balancer is designed to provide balanced distribution and forwarding of client requests for data processing services across the network to back-end network service nodes. The load balancing algorithm that determines the forwarding path for forwarding network client data packets must consider several dynamic parameters, such as the size of a back-end network service node pool, resource utilization, resource capacity, and characteristics of a network client request. To ensure that the load balancer is not a single point of failure or a service bottleneck, the load balancer can also be implemented as an elastic service that uses a pool of load balancer nodes.The pool of load balancer nodes can also be dynamic in that load balancers can be added to or removed from the pool in response to load balancing failures or changes in network load. In this case, data packets from the same data flow (i.e., from the same network client computing device) can be load balanced by different load balancer nodes. In an elastic pool of load balancers, an attempt is usually made to maintain affinity between clients and load balancers.

[0020] When implementing network load balancing, it is important and often required that related data packets ("flows") be forwarded from the same network client computing device to the same back-end network service node destination. The practice of forwarding flows from a client computing device to the same back-end network service node destination is referred to herein as "client connection affinity." Maintaining affinity in a large network and in large cloud environments presents significant scalability challenges and can lead to network bottlenecks. These problems are further complicated in dynamic networks where network components may be added, modified, removed, or become unavailable. The load balancing forwarding decision must change the affinity while still maintaining it.When adding a new back-end network service node (e.g., a back-end network server), new flows can be routed to the new back-end network service node. However, existing flows must continue to be routed to their existing back-end network service node destination, where possible. As another example, in a network environment that utilizes a pool of load balancer nodes, there may be multiple load balancer nodes that are dynamically added or removed, and the resulting per-flow forwarding decision must be consistent across all load balancer nodes.

[0021] Connection tracking can be used to maintain affinity in large-scale dynamic networks by tracking active connections. A connection trace remembers the forwarding decision for each data flow throughout its lifetime. For example, one can track each 5-tuple of a Transmission Control Protocol (TCP) throughout the lifetime of the TCP flow and log the forwarding decision entry as a forwarding rule in a forwarding table. The term "5-tuple" refers to a set of five different values ​​required to initiate a connection over the Internet Protocol (IP): the source IP address / port number, the destination IP address / port number, and the network protocol in use.It should be noted that "IP address" refers to an address that may be used at the network layer by protocols such as Internet Protocol (IP) to identify specific clients, and that "port" refers to the port number that may be assigned to a specific service at the transport layer, e.g., the Telnet protocol uses port 23. Furthermore, it should be noted that embodiments of the invention do not require a specific protocol; any protocol may be used, such as TCP or User Datagram Protocol (UDP).

[0022] Connection tracking can be implemented in a load balancer. With connection tracking, each newly arriving IP packet is first checked to see if it is part of an existing flow by referencing its 5-tuple, and if found, the existing forwarding rule is applied. However, such connection tracking is not scalable, as both the system memory requirements and the time required to search for forwarding decision entries increase with the number of flows. Another disadvantage of connection tracking is that it requires synchronization of the connection tracking state across all load balancer nodes.It may be possible to capture a minimal state such as a "version" of the load balancer's configuration, but there still needs to be a mechanism to consistently associate traffic flow with a specific version.

[0023] Another way to maintain affinity is hash-based forwarding. With this approach, a load balancer calculates a hash of flow attributes based on the number of healthy backend servers. Using hash-based forwarding maintains affinity as long as the number of backend servers remains static. When backend servers are added or removed, the affinity is lost.

[0024] Hash-based forwarding has been improved through the use of consistent hashing, a distributed hashing scheme that operates independently of the number of back-end servers. In this approach, a load balancer computes a consistent hash of flow attributes and uses the hash result for forwarding decisions. Consistent hashing ensures that for most flows, the hash value remains mostly unchanged, even when service instances are added or removed. Some disadvantages of consistent hashing are that its high processing requirements result in prohibitively large memory requirements and that it is a solution for most flows, but not all flows.

[0025] Other approaches to maintaining affinity are known, such as the use of cookies, but these approaches have several disadvantages. First, the use of cookies requires the client to participate in maintaining affinity by inserting cookies into messages, which in turn exposes the forwarding decision and therefore the IP address of the back-end server to the client. This could have a detrimental impact on network security. Another disadvantage is the performance penalty imposed by the load balancer, as it must parse and process protocol messages, such as the cookie being inserted by the client, in a protocol-appropriate manner.

[0026] Accordingly, a need arises for a technique that provides fast and reliable transition between different versions of the multi-node service deployed in the utility computing system, including, in a broader sense of version, changes such as configuration changes, hardware or software upgrades of the nodes, and additions or removals of nodes (i.e., elastic changes in the size of the service pool), etc.

[0027] Advantages of the novel embodiments described herein include the use of a network device at the perimeter of a first network containing multiple clients, for example, a private network. Using a network device at the perimeter of a first network reduces the need for connection tracking in a load balancer in a second network, such as a network hosting multiple back-end servers. Other advantages include reducing processing and storage requirements in the load balancer in a second network. Another advantage is that the novel embodiments described herein are transparent to clients and back-end servers.

[0028] Embodiments of the present data transmission systems and methods may provide this functionality by offloading the connection tracking function to another network entity. Many network environments include perimeter gateway entities, such as routers or switches, to isolate various attributes between separate network domains (such as private versus public network addresses). A common function of the perimeter gateway entity is network address translation (NAT), which replaces internal IP addresses with external IP addresses (and vice versa) in each packet. NAT requires connection tracking to ensure that all packets for the same connection undergo the same address translation and to be able to reverse-translate return packets, meaning that NAT inherently preserves flow affinity.Since these perimeter gateway units typically perform NAT, they are also referred to herein as "perimeter NATs." In addition to translating IP addresses, perimeter NATs can also translate or modify port numbers. The perimeter NAT can maintain a "version" of the distributed, load-balanced network and insert the current version into a field, such as the destination port field, within each packet. The load balancer can then be configured to record the current version each time its state has changed. The load balancer can also examine the allocated field within the packet to determine the version that was in use when the connection was initiated (i.e.,, the version associated with the first packet in the data flow), and use this information with the other fields within the packet's 5-tuple to determine the back-end server to which the packet should be forwarded.

[0029] It should be understood that, although this disclosure includes a detailed description of cloud computing, implementation of the teachings herein is not limited to a cloud computing environment. Rather, embodiments of the present invention may be implemented in conjunction with any type of computing environment now known or later developed.

[0030] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management overhead or interaction with a service provider. This cloud model may include at least five characteristics, at least three service models, and at least four implementation models.

[0031] The properties are as follows: On-demand self-service: A cloud user can unilaterally and automatically provision data processing functions such as server time and network storage as needed, without the need for human interaction with the service provider. Broad Network Access: Functions are available over a network and accessed through standard mechanisms that support use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs). Resource pooling: The provider's computing resources are pooled to serve multiple users using a multi-tenant model, with various physical and virtual resources dynamically allocated and reassigned as needed. There is a perceived location independence, as the user generally has no control or knowledge over the exact location of the provided resources, but may be able to specify a location at a higher level of abstraction (e.g., country, state, or data center). Rapid Elasticity: Features can be deployed quickly and elastically for rapid horizontal scaling out, in some cases automatically, and released quickly for rapid scale-in. To the user, the features available for deployment often appear unlimited, and they can be purchased in any quantity at any time. Measured Service: Cloud systems automatically control and optimize resource usage by leveraging a measurement function at a certain level of abstraction appropriate for the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, creating transparency for both the provider and the user of the service.

[0032] The service models are as follows: Software as a Service (SaaS): The functionality provided to the user is to use the provider's applications running on a cloud infrastructure. The applications can be accessed from various client devices through a thin client interface such as a web browser (e.g., web-based email). The user does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, with the possible exception of limited user-specific application configuration settings. Platform as a Service (PaaS): The functionality provided to the user is to deploy applications created or obtained by the user, using programming languages ​​and tools supported by the provider, into the cloud infrastructure. The user does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but has control over the deployed applications and possibly over configurations of the application hosting environment. Infrastructure as a Service (IaaS): The functionality provided to the user consists of providing processing, storage, networking, and other basic computing resources, allowing the user to deploy and run any software, including operating systems and applications. The user does not manage or control the underlying cloud infrastructure, but has control over operating systems, storage, deployed applications, and possibly limited control over selected network components (e.g., host firewalls).

[0033] The implementation models are as follows: Private Cloud: The cloud infrastructure is operated solely for an organization. It can be managed by the organization or a third party and can be located on or off-premises. Community Cloud: The cloud infrastructure is shared by multiple organizations and supports a specific user community with common concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by the organizations or a third party and can be located on or off-premises. Public Cloud: The cloud infrastructure is made available to the general public or a large industry group and is owned by an organization that sells cloud services. Hybrid Cloud: Cloud infrastructure is a composition of two or more clouds (private, community, or public) that remain separate entities but are interconnected by a standardized or proprietary technology that enables data and application portability (e.g., cloud targeting for load balancing between clouds). A cloud computing environment is service-oriented, focusing on state independence, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure containing a network of interconnected nodes.

[0034] It will now Fig. 1, which depicts an illustrative cloud computing environment 50. As depicted, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud customers, such as a personal digital assistant (PDA) or mobile phone 54A, a desktop computer 54B, a laptop computer 54C, and / or an automotive computer system 54N, can communicate with each other. The nodes 10 can communicate with each other. They can be physically or virtually aggregated into one or more networks (not shown), for example, into private, community, public, or hybrid clouds as described hereinabove, or a combination thereof.This enables the cloud computing environment 50 to offer infrastructure, platforms, and / or software as services for which a cloud customer does not need to maintain resources on a local computing device. It is understood that the services described in . Fig. 1 are for illustrative purposes only, and that the computing nodes 10 and the cloud computing environment 50 may communicate with any type of computerized device over any type of network and / or network-addressable connection (e.g., using a web browser).

[0035] With reference to Fig. 2 shows a set of functional abstraction layers used by the cloud computing environment 50 ( Fig. 1) are provided. It should be clear from the outset that the Fig. The components, layers, and functions shown in Figure 2 are intended to be illustrative only, and embodiments of the invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0036] A hardware and software layer 60 includes hardware and software components. Examples of hardware components include: mainframe computers 61; servers 62 based on a RISC (Reduced Instruction Set Computer) architecture; servers 63; blade servers 64; storage devices 65; and networks and network components 66, such as a load balancer. In some embodiments, software components include network application server software 67 and database software 68.

[0037] A virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities may be provided: virtual servers 71; a virtual storage 72; virtual networks 73, including virtual private networks; virtual applications and operating systems 74; and virtual clients 75.

[0038] In one example, the management layer 80 may provide the functions described below. Resource provisioning 81 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing 82 provides cost monitoring during the use of resources within the cloud computing environment, as well as billing or invoicing for the consumption of those resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud users and tasks, as well as protection for data and other resources. A user portal 83 provides users and system administrators with access to the cloud computing environment.Service level management 84 provides allocation and management of cloud computing resources so that required service levels are achieved. Service level agreement (SLA) planning and fulfillment 85 provides preparation for and provision of cloud computing resources for which a future requirement is expected according to an SLA.

[0039] A processing layer 90 provides examples of functionality for which the cloud computing environment can be used. Examples of processing and functions that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; provision of virtual training 93; data analysis processing 94; transaction processing 95; and other applications 96 such as the programs located in the memory 308 in Fig. 3 are located.

[0040] With reference to Fig. 3 illustrates a block diagram of a computing device 300 in which processes involved in the embodiments described herein may be implemented. Computing device 300 is typically a programmed general-purpose computer system, such as an embedded processor, a system-on-chip, a personal computer, a workstation, a server system, a minicomputer, or a mainframe computer. Likewise, computing device 300 may be implemented in a communications device, such as a network switch, a router, etc. Computing device 300 may include one or more processors (CPUs) 302A-B, an input / output circuit 304, a network adapter 306, and memory 308. CPUs 302A-B execute program instructions to perform the functions of the present communications systems and methods. Fig. 3 illustrates an embodiment in which the computing device 300 is implemented as a single multi-processor computer system in which multiple processors 302A-B share system resources, such as memory 308, input / output circuitry 304, and network adapter 306. However, the present data transmission systems and methods also include embodiments in which the computing device 300 is implemented as a plurality of networked computer systems, which may be single-processor computer systems, multi-processor computer systems, or a mixture thereof.

[0041] Input / output circuitry 304 provides the functionality to input data to or output data from computing device 300. Input / output circuitry may include, for example, input devices such as keyboards, mice, touchpads, trackballs, scanners, analog-to-digital converters, etc.; output devices such as video adapters, displays, printers, etc.; and input / output devices such as modems, etc. Network adapter 306 establishes a connection between computing device 300 and a network 310, which may be any public or proprietary LAN or WAN, including, but not limited to, the Internet.

[0042] The memory 308 stores program instructions executed by the CPU 302A-B and data used and processed by them to perform the functions of the data processing unit 300. The memory 308 may include, for example, electronic storage devices such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), flash memory, etc., as well as electromechanical storage devices such as magnetic disk drives, tape drives, optical disk drives, etc.which may use an Integrated Drive Electronics (IDE) interface or a variant or evolution thereof such as Enhanced IDE (EIDE) or Ultra-Direct Memory Access (UDMA), or a Small Computer System Interface (SCSI) interface or a variant or evolution thereof such as Fast SCSI, Wide SCSI, Fast and Wide SCSI, etc., or Serial Advanced Technology Attachment (SATA) or a variant or evolution thereof, or a Fibre Channel Arbitrated Loop (FC-AL) interface.

[0043] The contents of memory 308 may vary depending on the function that data processing unit 300 is programmed to perform. Fig. In the example illustrated in Figure 3, exemplary memory contents representing routines and data for embodiments of the processes described herein are illustrated. However, it should be appreciated that these routines, along with the memory contents associated with these routines, may not be contained within any one system or device, but rather may be distributed among a plurality of systems or devices based on well-known technical considerations. The present data transmission systems and methods may include any such arrangements.

[0044] In the Fig. 3, in examples where computing device 300 implements a perimeter NAT 430, memory 308 may include a perimeter NAT program 324. In examples where computing device 300 implements a load balancer node 450, memory 308 may include a load balancer program 326. In further examples, if computing device 300 implements a client 410, memory 308 may include a client program 320, or if computing device 300 implements a back-end node 470, memory 308 may include a back-end program 322. These and other examples may include an operating system 328. For example, the perimeter NAT program 324 may include routines and data that provide the functionality to process packets by substituting destination IP addresses and ports or any other packet field.The load balancer 322 may include routines and data that provide the functionality to distribute traffic to other devices, such as the back-end nodes 470. The operating system 328 may provide the overall system functionality.

[0045] As in Fig. 3, the present data transfer systems and methods may include implementation in a system or systems providing multiprocessor, multitasking, multi-process, and / or multi-threaded data processing, as well as implementation in systems providing only uniprocessor and single-threaded data processing. Multiprocessor data processing involves performing data processing using more than one processor. Multitasking data processing involves performing data processing using more than one operating system task. A task is an operating system concept that refers to the combination of a program being executed and bookkeeping information used by the operating system. Each time a program is executed, the operating system creates a new task for it.The task acts like an envelope for the program, assigning it a task number and associating other bookkeeping information with it. Many operating systems are capable of executing many tasks simultaneously and are known as multitasking operating systems. Multitasking is the ability of an operating system to run more than one executable program at a time. Each executable program runs in its own address space, meaning that the executable programs have no way of sharing their memory. This has advantages because it is impossible for one program to interfere with the execution of any other program running on the system. However, the programs have no way of exchanging information except through the operating system (or by reading files stored in the file system).Multi-process computing is similar to multitasking computing in that the terms task and process are often used interchangeably, although some operating systems distinguish between the two.

[0046] With reference to Fig. 4 shows a functional block diagram illustrating a system 400 according to an embodiment of the present invention. The system 400 may include multiple clients, e.g., clients 410A, 410B, 410C, multiple perimeter NATs, e.g., perimeter NAT 430, multiple load balancers, e.g., load balancer 450, and multiple backends, e.g., backend nodes 470A, 470B, all interconnected via the network 310. In the Fig. 4, the clients 410 may belong to a particular network and the back-end nodes 470 may belong to another network. For clarity, the two networks may also be referred to as "source" and "destination" networks. Although in the example of Fig. 4 illustrates two networks, it should also be noted that in other embodiments, more than two networks may be involved. In particular, it should be understood that the perimeter NAT 430 and the load balancer 450 may, but do not have to, be located in either the source or destination network; that is, the perimeter NAT 430 does not have to be located on the same network as the clients 410, and the load balancer 450 does not have to be located on the same network as the back-end nodes 470. A connection tracer, e.g., the perimeter NAT 430, may be located anywhere on the path between a source (the client 410) and a back-end entity. In various embodiments, the connection tracer may be located at the perimeter of the source network or at the perimeter of a network that includes back-end entities.

[0047] In various embodiments, network 310 may, for example, represent an intranet, a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination of the two. Network 310 may, for example, include wired, wireless, or fiber optic connections. In general, network 310 may be any combination of connections and protocols that supports data transfers between clients 410A-C, perimeter NAT 430, load balancer 450, and backend nodes 470A-B according to an embodiment of the invention.

[0048] The clients 410A-C may represent network computing devices capable of providing hosting for client applications. For example, the clients 410A-C and the perimeter NAT 430 may be included in a private network. The clients 410A-C may include the client program 320. In an exemplary embodiment, the client 410 may be a desktop computer, a notebook, a laptop computer, a tablet computer, a handheld device, a smartphone, a thin client, or any other electronic device or computing system capable of sending and receiving data over a network 110 through the perimeter NAT 430 to and from other computing devices such as the load balancer 450 or the backend 470. As shown in the exemplary illustration in Fig. 5, the client 410 may be implemented using multiple units referred to as client 410A, client 410B, and client 410C, etc.

[0049] In various embodiments, client program 320 may be a program or a subroutine contained within a program that may enable a user of client 410 to request and consume services from a back-end network server node 470A-B via perimeter NAT 430, network 310, and load balancer 450. Client program 320 may include components used to transfer data to an application located in perimeter NAT 430. In an exemplary embodiment, client program 320 may send a TCP data packet to perimeter NAT 430 as part of a data flow. The data packet that client 410A-C transfers to perimeter NAT 430 includes a destination address, which may be an internal address.

[0050] The perimeter NAT 430 may represent a network component or entity capable of hosting an address translation program. Furthermore, the perimeter NAT 430 may represent a network component or entity capable of tracking a connection and, as such, may be referred to herein as a "connection tracking" entity. The perimeter NAT program 324 may perform network address translation (NAT) and connection tracking functions. The perimeter NAT 430 enables data to flow from one network to another, e.g., from a network containing multiple clients 410A-C to the network 110. The perimeter NAT 430 isolates attributes between domains, e.g., between an internal network containing the clients 410A-C and an external network.The perimeter NAT 430 tracks the connection for packets in a data flow between a particular network client and a particular back-end network service node. The perimeter NAT 430 can receive data packets from the clients 410A through C, the load balancers 450, the back-end nodes 470A through B, or other sources. Each packet that the perimeter NAT 430 receives contains a destination address. The perimeter NAT 430 can replace the destination address with another address. For example, in a data packet sent by a client 410A through C to the perimeter NAT 430, the perimeter NAT 430 can replace an internal address with an external address. In addition to modifying network addresses, the perimeter NAT 430 can modify the port number.The packet sent by a client 410A-C to the perimeter NAT 430 may be one of several related data packets that form a data flow from the client 410A-C.

[0051] The load balancer 450 may represent a network component or device capable of making packet forwarding decisions that preserve client connection affinity. The load balancer 450 may provide hosting for a load balancer 326. In an exemplary embodiment, the load balancer 450 may be a desktop computer, a networked computer system, a server blade, a specialized computer server, or any other electronic device or data processing system capable of providing hosting for a load balancer 326. As shown in the exemplary illustration in Fig. As shown in Figure 5, the load balancing function 450 may be implemented using multiple units (a distributed load balancer), referred to as load balancing function 450A and load balancing function 450B. The load balancing function 450 is an example of a packet forwarding unit.

[0052] In various embodiments, load balancer 326 may be a program or a subroutine included within a program that implements the functionality to provide balanced distribution and forwarding of network client requests across the network to back-end network service nodes. Load balancer 326 may make packet forwarding decisions according to algorithms having goals of preserving client connection affinity and balancing processing efforts. Load balancer 326 may preserve client connection affinity as much as possible. In one exemplary embodiment, load balancer 326 may receive a data packet from client program 320 over network 310 and forward the received data packet to back-end program 322.The received data packet may be one of several related data packets that form a data flow from the client 410A to C.

[0053] The back-end nodes 470A-B may represent a computing platform capable of hosting one or more resources for use by another computing device, such as the client 410A-C. The back-end nodes 470A-B may include the back-end program 322. In an exemplary embodiment, the back-end node 470A-B may be a desktop computer, a networked computer system, a server blade, a specialized computer server, or any other electronic device or computing system capable of sending and receiving data over the network 310 to and from other computing devices, such as the client 410A-C, the perimeter NAT 430, and the load balancer 450.The back-end node 470A-B may include a collection of servers executing the same software to collectively process and monitor requests from other computing devices, such as clients 410A-C. The back-end node 470A-B may be implemented using multiple devices, referred to as back-end node 470A, back-end node 470B, back-end node 470C, and back-end node 470D.

[0054] In various embodiments, back-end program 322 may be a program or a subroutine contained within a program that may facilitate data requests received from client program 320 via load balancer 326. In an exemplary embodiment, back-end program 322 may receive a data packet from load balancer 326. The received data packet may originate from client program 320 and represent part of a data flow from client 410A-C. Back-end program 326 may process the received data packet.

[0055] With reference to Fig. 5, a functional block diagram is shown illustrating an exemplary network environment 500 having a plurality of clients 410A to F as previously described with respect to Fig. 4, a plurality of perimeter NATs 430A to B, as previously described with respect to Fig. 4, a plurality of load balancing functions 450A to N, as previously described with respect to Fig. 4, and may include a plurality of back-end nodes 470A to N, as previously described with respect to Fig. 4. The network environment 500 is also illustrated with exemplary data flows transmitted by the clients 410A-F to the perimeter NAT 430A for the private network 420, which may include the clients 410A-C and the perimeter NAT 430A, or to the perimeter NAT 430B for the private network 440, which may include the clients 410D-F and the perimeter NAT 430B. The data flows are forwarded by the perimeter NATs 430A-B to the load balancers 450A-N, according to various embodiments of the present invention, which then forward the data flow to the backend nodes 470A-N. The forwarding of data flows described herein may be performed by the system 400 of Fig. 4 must be implemented.

[0056] Load balancing nodes 450A through N may each distribute traffic to other devices, such as backend nodes 470A through N, which may typically include backend web servers or other servers, storage, or computing devices. The number of load balancing nodes may change. In this example, there are initially two load balancing nodes 450A through B, but the number of load balancing nodes may change to N, resulting in load balancing nodes 450A through N. Likewise, the number of backend nodes may change. In this example, there are initially four backend nodes 470A through D, but the number of backend nodes may change to N, resulting in backend pool nodes 470A through N.

[0057] In one embodiment, load balancer 450A may forward data sent from client 410A through perimeter NAT 430A (flow 502) to backend node 470A. Furthermore, flow 504 transmitted by client 410C and forwarded through perimeter NAT 430A may be forwarded to backend 470D by both load balancer 450A and load balancer 450B. Furthermore, in one embodiment, load balancer 450A may forward data sent from client 410D through perimeter NAT 430B (flow 506) to backend 470B. In addition, the flow 508 transmitted by the client 410F and forwarded by the perimeter NAT 430B may be forwarded to the backend 470C by both the load balancer 450A and the load balancer 450B.

[0058] It should also be noted that network switches, routers and other units that are used in Fig. 5, may be present in the network environment 500. For example, network switches may be located between the load balancing nodes 450A through B and the back-end nodes 470A through D to provide the desired network connectivity.

[0059] With reference to Fig. Figure 6A shows a functional block diagram of a known TCP / IP network process from the perspective of a network packet. The client 610 may be located on a private network in a home or on a virtual private network (VPN) connection to a corporate network and wishes to exchange data over the public Internet with a web service located at one or more backends 640. The Fig. The example illustrated in Figure 6A corresponds to a virtual private endpoint (VPE) or a similar service that can establish a VPN tunnel connection between a source and a destination by adding the appropriate header(s) to the network packet. A network packet 612 that the client 610 sends to the perimeter NAT 620 includes the IP address of the client 610 as the source address, a selected source port, the IP address of the private network interface of the perimeter NAT 620 as the destination address, and the destination port associated with the selected Internet protocol, e.g., Hypertext Transfer Protocol (HTTP), which corresponds to port 80, as the destination port.In this example, the perimeter NAT 620 receives this packet 612 and performs a network address translation (NAT) process to allow the data flow to continue across another network. However, it should be understood that the present invention does not require a connection tracer to perform a NAT process. In this example, the NAT process converts the packet 612 into a modified packet 622, and, as discussed above, this requires the perimeter NAT to trace the connection for each packet it forwards. In other examples, the packet 612 is converted into a modified packet 622 in a process separate and distinct from any NAT process.The modified packet 622 has the IP address of the perimeter NAT 620 as its source IP address, a newly selected source port, the IP address of the load balancer 630 as the destination IP address, and the same destination port as packet 612. This packet 622 is forwarded to the load balancer 630, which determines the backend 640 with the service that the packet 622 is attempting to request. In this example, the load balancer 630 may modify the packet 622 by changing the destination IP address to the address of the backend 640, resulting in a packet 632. The load balancer 630 forwards the packet 632 to the backend 640. The back-end 640 receives the packet 632 and can respond directly to the client 610 via the perimeter NAT 620, now using its own address and port for the source information and the perimeter NAT information as the destination address and port.

[0060] With reference to Fig. 6B, a functional block diagram of an embodiment of the present invention is also shown from the perspective of a network packet. Packets 652, 662, and 672 may contain data and header information (not shown) including a source network address (“SRC”) and a destination network address (“DST”). Packets 652, 662, and 672 may contain a source and destination port number field. Packets 652, 662, and 672 may contain the same data, but different source and destination network address information and different data in the source and destination port fields. For example, packet 652 contains a destination network address of 10.240.10.10 and a port number of 80. The network addresses of the client 410, the perimeter NAT 430, the load balancer 450, and the backend 470 are shown in Fig. 6B. For example, the address of the load balancer 450 is 161.26.0.30. It can be seen that the destination network address of the packet 652 matches the network address of the perimeter NAT 430. As previously discussed with regard to Fig. 4, the client 410 can forward the packet 652 to the perimeter NAT 430.

[0061] In response to receiving packet 652, perimeter NAT 430 may, in some embodiments, perform NAT. Furthermore, according to various embodiments, perimeter NAT 430 may modify the destination port to a version identifier. In this example, a version identifier 81 is encoded in packet 662. Version identifier 81 replaces destination port number 80 in packet 652. Perimeter NAT 430 forwards modified packet 662 to load balancer 450, which determines a version from packet 662. Load balancer 450 performs the appropriate hash function according to the version, as described in more detail below. Load balancer 450 uses the extracted version identifier and hash function to select a destination backend 470, as described in more detail below.The load balancer 450 restores the destination port 80, which is the port number before modification by the perimeter NAT 430. The load balancer 450 forwards the packet 672 to the selected backend 470. The backend 470 can receive the packet 672 and respond directly to the client 410 via the perimeter NAT 430, now using its own address and port for the source information and the perimeter NAT information as the destination address and port.

[0062] With reference to Fig. Figure 7A illustrates a flowchart of a process 700 for tagging network packets with a version number. Process 700 may be performed by a connection-tracking network device such as a perimeter NAT, a gateway, a virtual private endpoint (VPE), a virtual private network (VPN) ingress server, a software-defined network (SDN) egress device, or an SDN node-based NAT.

[0063] At 702, a data packet is received from a client computing device, such as client 410, by a connection tracer. The packet may be received over a TCP connection. In one embodiment, the connection tracer is configured to perform NAT. In one embodiment, the source address and port in the packet identify the device in the internal network to which they belong, e.g., a private network. The device may use NAT to change the private source address and port to an address in the public address space, e.g., another network.

[0064] At 704, a version identifier for the packet may be determined. In one embodiment, the connection tracking entity, e.g., perimeter NAT 430, maintains versions and corresponding version identifiers in memory. How the version identifier is determined at 704 depends on whether the received packet is a first packet or a subsequent packet in a sequence of packets for which connection affinity is maintained. If the received packet is the first packet of the sequence, the version identifier is created or set. If the received packet is a subsequent packet of the sequence, the version identifier is determined by identifying the packet as belonging to an existing (trace) connection and retrieving the version identifier for the sequence from memory. According to various embodiments, version information may change.In some embodiments, a current version may be updated based on a time or quantity condition. For example, the current version may be incremented after a certain number of packets are received or transmitted by a network device, e.g., every 10,000 packets. As another example, the current version may be incremented after a period of time has elapsed, e.g., every 5 seconds. In other embodiments, a current version is updated in response to receiving a message indicating a change in the number of operational devices in a network. Networks are dynamic. For example, the number of gateways, load balancers, endpoints, or other network devices may change. In some embodiments, any particular network configuration may correspond to a version.In other embodiments, a policy change may correspond to a version rather than a change in backend capacity. Accordingly, a version identifier may be changed in response to receiving a message. The message may be a control message from a software-defined network (SDN). The message may originate from a load balancer 450, such as an Internet Control Message Protocol (ICMP) message or another external message. In some embodiments, the current state or version may be retrieved via a DNS lookup for a service or using a mock SYN. ​​In yet another example, the load balancer 450 may encode the current version as a source port in a return packet.

[0065] At 706, the version identifier may be encoded into the packet. The current version number may be inserted into any suitable field in the received network packet. In some embodiments, the version identifier is encoded into the destination port field. At this stage, the version identifier encoded into the packet is stored in a memory of the perimeter NAT to be used with subsequent packets within the sequence of packets, i.e., in the same data flow.

[0066] In 708, the packet containing the encoded version identifier is transmitted to a forwarding network device, e.g., a load balancer.

[0067] Fig. 7B is a flowchart of a process 720 for processing packets by a load balancer according to various embodiments of the present invention. The process 720 may be performed by any suitable network device having forwarding functionality, e.g., the load balancer 450. At 722, a data packet is received by a forwarder. The packet may be received, for example, from the perimeter NAT 430. The packet may be received through a TCP connection. The receiving device and at least one backend device may be included in the same network.

[0068] At 724, a version identifier may be extracted from the received packet. In one embodiment, the version identifier may be encoded into a destination port field. In other embodiments, the version identifier may be encoded in any other suitable part of the packet. Furthermore, the source IP address / port number, destination IP address / port number, and the network protocol in use may be extracted from the packet.

[0069] At 726, the destination (i.e., the backend server) to which the packet should be forwarded is selected. For example, a specific hash function can be used to determine a hash value using the extracted source address and source port number as inputs to the hash function. At the same time, a table of backend server addresses can be selected based on the version identifier extracted from the packet. The table stores a list of available backends for a specific version. The hash value is divided by a specific number of backend units currently operational (determined by the number of rows in the selected table) in the network of backend units. A remainder produced by the division operation is determined and corresponds to the row number of the selected table that is being used.A network address of a backend unit in this table row is selected as the destination for the packet. A more specific example of this technique is shown below in . Fig. 8. When the load balancer 450 detects state changes, it can note the version located in the specified field for each of the flows it is processing and update its forwarding rules accordingly.

[0070] At 728, the selected network address is encoded into the first packet as the destination address. Furthermore, a source port of an originating node may be encoded into the first packet as the source port to replace the version identifier with the source port of the node from which the packet originated. The first packet containing the selected network address is transmitted to a backend device having the selected network address.

[0071] Fig. Figure 8 is a block diagram of packet forwarding in a load balancer (or forwarding unit) according to various embodiments. Load balancer 450 receives a packet 862 from NAT unit 430. In one embodiment, client 410 and NAT unit 430 may be part of a first network. In another embodiment, NAT unit 430, load balancer 450, and backends 470A and 470B may be part of a second network.

[0072] The load balancer 450 extracts a version identifier for the package 862 from the package 862. In this example, the load balancer 450 extracts the version identifier 81 from the destination port field of the package 862.

[0073] Additionally, load balancer 450 extracts a source address and source port from packet 862. In this example, load balancer 450 extracts 165.20.10.10 as the source address and 35017 as the source port. Load balancer 450 determines a hash value using a specific hash function chosen based on the version identifier, with the extracted source address and source port as inputs. The specific hash function can be any hash function known today or developed later.

[0074] The hash function can be a hash function that produces a remainder or a hash function that does not produce a remainder. In these two examples, Fig. 8 assume that the load balancer 450 determines a hash value 71. The load balancer 450 determines a remainder after dividing the hash value 71 by a certain number of functional backend units (determined from the version identifier as current at the time the connection was initiated). Although in Fig. 8 only two backend units are shown (to simplify the drawing), it should be noted that any number of backend units may be operational even though they are not shown in the figure. In a first example, using a hash function that produces a remainder and the version identifier is 81, assume that three (3) backend units are currently operational. Dividing 71 by 3 produces a remainder of 2 (71 = 23*3 + 2). The load balancer 450 uses the remainder of 2 as an index to a row in a table 804. In this example, the remainder of 2 points to row 2. Row 2 lists the address 10.131.10.3 for the backend server.

[0075] Load balancer 450 selects the network address of a backend device from the table row corresponding to the remainder for the applicable version if the hash function used produces a remainder. If the hash function does not produce a remainder, a result generated by the hash function can be used to select a row from the table. Continuing the example in which the hash function produced a hash value of 71, load balancer 450 selects backend device 470A with address 10.131.10.3 for version identifier 81 from row 2.

[0076] In a second example, which is Fig.8 with a version identifier 82, assume that four (4) backend units are currently operational. Furthermore, assume that load balancer 450 extracts version identifier 82 from packet 862. Dividing hash value 71 by 4 produces a remainder of 3 (71 = 17*4 + 3). Load balancer 450 uses the remainder of 3 as an index to a row in table 804. In the second example, the remainder of 3 points to row 3. Row 3 lists the address 10.131.10.4 for the backend server. In the second example, load balancer 450 selects backend unit 470B with address 10.131.10.4 for version identifier 82 from row 3.

[0077] Load balancer 450 encodes the selected network address as the destination address in the packet and transmits the packet containing the selected network address to a backend device having that address. Continuing the examples, if the selected address is 10.131.10.3, packet 872 is transmitted to backend device 470A. If the selected address is 10.131.10.4, packet 874 is transmitted to backend device 470B.

[0078] The load balancer 450 may encode a source port number of an originating node (the client 410) as the source port in the packet 872, 874. The NAT unit replaces the source port number of the originating node in the packet 862 with a version identifier. By encoding a source port number into the packet 872, 874, the packet is restored to a format expected by the back-end units 470A, 470B. The back-end units 470A, 470B may use the source port number of the originating node in the packet 862 when processing the packet, e.g., when generating a response.

[0079] In some embodiments, the particular hash function used to determine the hash value is selected from two or more hash functions based on the version identifier for the first packet, and in some embodiments, a consistent hash method may be used.

[0080] In some embodiments, the particular number of backend devices currently operational in the second network is selected from two or more numbers of backend devices based on the version identifier for the first packet.

[0081] If the load balancer employs more than a single load balancer 450, each of the load balancer instances must know the set of backend entities corresponding to each version. Accordingly, various embodiments provide for distributing the set of backend entities corresponding to a particular version among each of the load balancer instances.

[0082] The present invention may be a system, a method, and / or a computer program product with any degree of integration of technical details. The computer program product may include computer-readable storage medium(s) storing computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0083] The computer-readable storage medium may be any physical device that can retain and store instructions for use by an instruction-executing device. The computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of the computer-readable storage medium includes the following: a portable computer diskette, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM).Flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device such as punched cards or raised structures in a groove on which instructions are stored, and any suitable combination thereof. A computer-readable storage medium, as used herein, shall not be construed as containing transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., pulses of light carried through a fiber optic cable), or electrical signals carried through a wire.

[0084] Computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to respective computing / processing units or to an external computer or storage unit via a network such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, fiber optic transmission lines, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing unit receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing unit.

[0085] Computer-readable program instructions for performing operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, integrated circuit configuration data, or either source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, or the like, and procedural programming languages ​​such as the C programming language or similar programming languages.The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In the latter case, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, over the Internet using an Internet service provider).In some embodiments, electronic circuits, including, for example, programmable logic circuits, field programmable gate arrays (FPGA), or programmable logic arrays (PLA), may execute the computer-readable program instructions by utilizing state data of the computer-readable program instructions to personalize the electronic circuits to perform aspects of the present invention.

[0086] Aspects of the present invention are described herein with reference to flowchart and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart and / or block diagrams, as well as combinations of blocks in the flowchart and / or block diagrams, may be implemented using computer-readable program instructions.

[0087] These computer readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions executing via the processor of the computer or other programmable data processing apparatus produce means for implementing the functions / steps specified in the flowchart and / or block diagram block(s). These computer readable program instructions may also be stored on a computer readable storage medium capable of directing a computer, programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored thereon comprises an article of manufacture, including instructions implementing aspects of the function / steps specified in the flowchart and / or block diagram block(s).implement the function / step specified in the blocks of the flowchart and / or block diagrams.

[0088] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of process steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-executing process such that the instructions executing on the computer, other programmable apparatus, or other device implement the functions / steps specified in the flowchart and / or block diagram block(s).

[0089] The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing the particular logical function(s). In some alternative implementations, the functions specified in the blocks may occur in a different order than shown in the figures.For example, two blocks shown in sequence may actually execute as a single step, concurrently, substantially concurrently, partially or completely overlapping, or the blocks may sometimes execute in reverse order depending on the respective functionality. It should also be noted that each block of the block diagrams and / or flowchart, as well as combinations of blocks in the block diagrams and / or flowchart, may be implemented by specialized hardware-based systems that perform the specified functions or steps, or by combinations of specialized hardware and computer instructions.

[0090] The descriptions of the various embodiments of the present invention have been presented for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, practical application, or technical improvement over commercially available technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

[1] A method implemented on a computer, comprising: Receiving (702) a packet from a node in a first network at a connection tracking unit (430); Determining (704) a version identifier (81) for the packet by the connection tracking unit, the version identifier (81) comprising information identifying the version of a state of a second network; encoding (706) the version identifier (81) into the packet by the connection tracking unit, wherein encoding the version identifier (81) into the packet includes encoding the version identifier (81) into a destination port field of the packet; transmitting (708) the packet containing the encoded version identifier (81) by the connection tracking unit (430) to a forwarding unit (450), wherein the forwarding unit (450) selects a back-end unit based on a client connection affinity, and wherein the client connection affinity is based in part on the encoded version identifier (81) in the packet; Restoring, by the forwarding unit (450), the destination port field of the packet; and Transmitting the packet by the forwarding unit (450) to the back-end unit, wherein the back-end unit is located in a second network, and wherein the forwarding unit (450) is either in the first network or in the second network. [2] The computer-implemented method of claim 1, wherein the connection tracking unit is a perimeter network address translation, NAT, unit. [3] The computer-implemented method of claim 1, wherein the version identifier (81) corresponds to a set of back-end units in the second network. [4] A computer-implemented method according to claim 1, wherein a current version is based on time and determining the version identifier (81) includes: Determining a period in which a current version identifier (81) has been used; and Updating the version identifier (81) if the determined period exceeds a period specified for updating the version identifier (81). [5] A computer-implemented method according to claim 1, wherein a current version is based on a set of packages and determining the version identifier (81) includes: Determining a set of packets received at the connection tracking unit for which a particular version identifier (81) has been used; and Updating the version identifier (81) if the determined set of packages exceeds a certain set of packages specified for updating the version identifier (81). [6] A computer-implemented method according to claim 1, wherein a current version is based on a control message received by the connection tracking unit. [7] A computer-implemented method according to claim 1, further comprising: Receiving a return packet by the connection tracking unit, wherein in response to the transmission of the packet containing the encoded version identifier (81) to the forwarding unit, the return packet is transmitted to the connection tracking unit and the return packet is encoded with a version identifier (81); and Update the version identifier (81) in response to the return packet. [8] The computer-implemented method of claim 1, wherein the version identifier (81) corresponds to a hash function used by the forwarding unit or a policy employed by the forwarding unit. [9] A computer program product comprising computer-readable program instructions suitable, when the computer program is executed on a computer, for carrying out the method according to any one of claims 1-8. [10] A computer system (400) comprising: A connection tracking unit (430); and A forwarding unit (450); Wherein the connection tracking unit (430) is configured to: Receiving a packet from a node in a first network; Determining a version identifier (81) for the packet, the version identifier (81) comprising information identifying the version of a state of a second network; encoding the version identifier (81) into the packet, wherein encoding the version identifier (81) into the packet includes encoding the version identifier (81) into a destination port field of the packet; transmitting the packet containing the encoded version identifier (81) to a forwarding unit (450); and Wherein the forwarding unit (450) is configured to: Selecting a backend device based on a client connection affinity, and wherein the client connection affinity is based in part on the encoded version identifier (81) in the packet; Restoring the package's destination port field; and Transmitting the packet to the back-end unit, wherein the back-end unit is located in a second network, and wherein the forwarding unit (450) is either in the first network or in the second network. [11] The computer system of claim 10, wherein the version identifier (81) corresponds to a set of back-end units in the second network. [12] The computer system of claim 10, wherein a current version is based on time and determining the version identifier (81) includes: Determining a period in which a current version identifier (81) has been used; and Updating the version identifier (81) if the determined period exceeds a period specified for updating the version identifier (81). [13] A computer system according to claim 10, wherein a current version is based on a set of packages and determining the version identifier (81) includes: Determining a set of packets received at the connection tracking unit for which a particular version identifier (81) has been used; and Updating the version identifier (81) if the determined set of packages exceeds a certain set of packages specified for updating the version identifier (81). [14] The computer system of claim 10, wherein a current version is based on a control message received by the connection tracking unit.

Images