Travel time estimation for transmission control protocol

Abstract

A method (300) includes sparsely sampling a plurality of packets (170) at a middlebox (160) from a transmission control protocol (TCP) connection (180) between a source endpoint (190) and a destination endpoint passing through the middlebox. For each packet in the sampled plurality of packets, the method includes generating a timestamp (212) when the packet was sampled, and recording a sequence number (172) and an acknowledgment number (174) of the corresponding packet. When an acknowledgment number exists for the corresponding packet, the acknowledgment number is recorded. The method also includes generating an estimated performance characteristic (202) of the TCP connection between the source endpoint and the destination endpoint passing through the middlebox during the period of time based on at least two of the corresponding timestamp, the corresponding sequence number, or the corresponding acknowledgment number of one or more packets in the sampled plurality of packets.

Description

Technical Field

[0001] The present disclosure relates to trip time estimation for a transmission control protocol. Background Art

[0002] Typically, processes running on nodes can help understand the performance of transport control protocol (TCP) connections between nodes (or endpoints). For example, a system can be deployed to capture the roundtrip time (RTT) of packets by measuring the start time when a packet is sent by a node and the acknowledgment time when an acknowledgment is received by a node. However, with the increase in the use of middleboxes to implement intermediary nodes between more conventional endpoints, measuring TCP performance at the endpoints must consider new performance measurement techniques to facilitate TCP connection optimization. Summary of the invention

[0003] One aspect of the present disclosure provides a method for estimating performance characteristics of a transmission control protocol. The method includes, at data processing hardware of a middlebox, sparsely sampling a plurality of packets from one or more transmission control protocol (TCP) connections between one or more source endpoints and one or more destination endpoints passing through the middlebox over a period of time. For each of the sampled plurality of packets, the method includes, by the data processing hardware, generating a timestamp when the packet is sampled, and by the data processing hardware recording a sequence number and an acknowledgment number of the corresponding packet. The method further includes, by the data processing hardware, generating an estimated performance characteristic of the one or more TCP connections between the one or more source endpoints and the one or more destination endpoints passing through the middlebox over the period of time based on at least two of a corresponding timestamp, a corresponding sequence number, or a corresponding acknowledgment number of one or more packets from the sampled plurality of packets.

[0004] In some embodiments, the method includes identifying, by the data processing hardware, a first sequence number recorded for a first packet of the sampled plurality of packets transmitted between the middlebox and the destination endpoint. Here, the method also includes determining, by the data processing hardware, whether the first sequence number recorded for the first packet matches any of the acknowledgment numbers recorded for the sampled plurality of packets transmitted between the destination endpoint and the middlebox. In this embodiment, when the first sequence number matches the acknowledgment number recorded for a second packet of the sampled plurality of packets, the method includes determining, by the data processing hardware, a first trip time between the middlebox and the destination endpoint based on a difference between the corresponding timestamps of the first packet and the second packet. In this embodiment, when the first sequence number does not match any of the acknowledgment numbers recorded for the sampled plurality of packets, the method includes estimating, by the data processing hardware, a corresponding timestamp of a first unsampled packet from the TCP connection by interpolating the acknowledgment numbers recorded for the sampled plurality of packets. Here, the first unsampled packet is associated with a corresponding acknowledgment number that matches the first sequence number. This embodiment further includes determining, by the data processing hardware, the first travel time between the middlebox and the destination endpoint based on a difference between a first timestamp generated for the first packet and a corresponding timestamp estimated for the first unsampled packet.

[0005] In some examples, the method includes identifying, by the data processing hardware, a second sequence number recorded for a third packet of the sampled plurality of packets, the third packet being transmitted between the middlebox and the source endpoint. Here, the method also includes determining, by the data processing hardware, whether the second sequence number recorded for the third packet matches any acknowledgment number recorded for the sampled plurality of packets transmitted between the source endpoint and the middlebox. In this example, when the second sequence number matches the acknowledgment number recorded for a fourth packet of the sampled plurality of packets, the method includes determining, by the data processing hardware, a second trip time between the middlebox and the source endpoint based on a difference between the corresponding timestamps generated for the third packet and the fourth packet. In this example, when the second sequence number does not match any of the acknowledgment numbers recorded for the sampled plurality of packets, the method includes, by interpolating the acknowledgment numbers recorded for the sampled plurality of packets, estimating, by the data processing hardware, a corresponding timestamp of a second unsampled packet from the TCP connection, the second unsampled packet being associated with a corresponding acknowledgment number matching the second sequence number, and determining the second trip time between the middlebox and the destination endpoint based on a difference between a third timestamp generated for the third packet and the corresponding timestamp estimated for the second unsampled packet. Generating the estimated performance characteristic of the TCP connection may also include calculating a round trip time between the source endpoint and the destination endpoint through the middlebox by summing the first trip time between the middlebox and the destination endpoint and the second trip time between the middlebox and the source endpoint.

[0006] In other configurations, the method includes estimating the corresponding time of the unrecorded acknowledgment number based on interpolation of the recorded acknowledgment numbers of the sampled plurality of packets. The method may include estimating the corresponding time of the unrecorded sequence number based on interpolation of the recorded sequence numbers of the sampled plurality of packets.

[0007] In some embodiments, generating the estimated performance characteristic of the TCP connection includes determining, by the data processing hardware, a slope of acknowledgment numbers over the period of time based on the acknowledgment numbers recorded for the sampled plurality of packets transmitted from the destination endpoint to the source endpoint through the middlebox, and generating a throughput based on the slope of acknowledgment numbers, wherein the throughput corresponds to a rate of packet transmission from the source endpoint to the destination endpoint through the middlebox. In other embodiments, generating the estimated performance characteristic of the TCP connection includes determining a slope of acknowledgment numbers over time based on the acknowledgment numbers recorded for the sampled plurality of packets transmitted from the source endpoint to the destination endpoint through the middlebox, and generating a throughput based on the slope of acknowledgment numbers, wherein the throughput corresponds to a rate of packet transmission from the destination endpoint to the source endpoint through the middlebox.

[0008] In some examples, the method includes, at a given moment, at the data processing hardware, receiving a first packet communicated over the TCP connection from the source endpoint, the first packet including a first sequence number and a first acknowledgment number, and receiving a second packet communicated over the TCP connection from the destination endpoint, the second packet including a second sequence number and a second acknowledgment number. Here, the method also includes estimating, by the data processing hardware, a corresponding window size for the source endpoint by determining a difference between the first sequence number associated with the first packet and the second acknowledgment number associated with the second packet. In these examples, the method may further include estimating, by the data processing hardware, a corresponding window size for the destination endpoint by determining a difference between the second sequence number associated with the second packet and the first acknowledgment number associated with the first packet.

[0009] In some embodiments, sparsely sampling the plurality of packets includes sampling less than all packets from the TCP connection between the source endpoint and the destination endpoint. Sparsely sampling the plurality of packets may include uniformly sampling packets from the TCP connection in a stateless and systematic manner. The source endpoint may be a user equipment (UE) in communication with a remote distributed network, and the destination endpoint corresponds to a server in communication with the remote distributed network.

[0010] Another aspect of the present disclosure provides a system for estimating performance characteristics of a transmission control protocol. The system includes data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. The operations include, at a middlebox, sparsely sampling a plurality of packets from one or more transmission control protocol (TCP) connections between one or more source endpoints and one or more destination endpoints passing through the middlebox over a period of time. For each of the plurality of sampled packets, the operations include generating a timestamp when the packet is sampled, and recording a sequence number and an acknowledgment number of the corresponding packet. The operations further include generating an estimated performance characteristic of the one or more TCP connections between the one or more source endpoints and the one or more destination endpoints passing through the middlebox over the period of time based on at least two of a corresponding timestamp, a corresponding sequence number, or a corresponding acknowledgment number of one or more packets from the plurality of sampled packets.

[0011] In some embodiments, the operations include identifying a first sequence number recorded for a first packet of the sampled plurality of packets transmitted between the middlebox and the destination endpoint. Here, the operations also include determining whether the first sequence number recorded for the first packet matches any of the acknowledgment numbers recorded for the sampled plurality of packets transmitted between the destination endpoint and the middlebox. In this embodiment, when the first sequence number matches the acknowledgment number recorded for a second packet of the sampled plurality of packets, the operations include determining a first trip time between the middlebox and the destination endpoint based on a difference between the corresponding timestamps of the first packet and the second packet. In this embodiment, when the first sequence number does not match any of the acknowledgment numbers recorded for the sampled plurality of packets, the operations include estimating a corresponding timestamp of a first unsampled packet from the TCP connection by interpolating the acknowledgment numbers recorded for the sampled plurality of packets. Here, the first unsampled packet is associated with a corresponding acknowledgment number that matches the first sequence number. This embodiment further includes determining the first travel time between the middlebox and the destination endpoint based on a difference between a first timestamp generated for the first packet and a corresponding timestamp estimated for the first unsampled packet.

[0012] In some examples, the operations include identifying a second sequence number recorded for a third packet of the sampled plurality of packets, the third packet being transmitted between the middlebox and the source endpoint. Here, the operations also include determining whether the second sequence number recorded for the third packet matches any acknowledgment number recorded for the sampled plurality of packets transmitted between the source endpoint and the middlebox. In this example, when the second sequence number matches the acknowledgment number recorded for a fourth packet of the sampled plurality of packets, the method includes determining, by the data processing hardware, a second trip time between the middlebox and the source endpoint based on a difference between the corresponding timestamps generated for the third packet and the fourth packet. In this example, when the second sequence number does not match any of the acknowledgment numbers recorded for the sampled plurality of packets, the operations include estimating a corresponding timestamp of a second unsampled packet from the TCP connection by interpolating the acknowledgment numbers recorded for the sampled plurality of packets, the second unsampled packet being associated with a corresponding acknowledgment number matching the second sequence number, and determining the second trip time between the middlebox and the destination endpoint based on a difference between a third timestamp generated for the third packet and the corresponding timestamp estimated for the second unsampled packet. Generating the estimated performance characteristic of the TCP connection may also include calculating a round trip time between the source endpoint and the destination endpoint through the middlebox by summing the first trip time between the middlebox and the destination endpoint and the second trip time between the middlebox and the source endpoint.

[0013] In other configurations, the operations include estimating corresponding times of unrecorded acknowledgment numbers based on interpolation of recorded acknowledgment numbers of the sampled plurality of packets. The operations may include estimating corresponding times of unrecorded sequence numbers based on interpolation of recorded sequence numbers of the sampled plurality of packets.

[0014] In some embodiments, generating the estimated performance characteristic of the TCP connection includes determining a slope of acknowledgment numbers over the period of time based on the acknowledgment numbers recorded for the sampled plurality of packets transmitted from the destination endpoint to the source endpoint through the middlebox, and generating a throughput based on the slope of acknowledgment numbers, wherein the throughput corresponds to a rate of packet transmission from the source endpoint to the destination endpoint through the middlebox. In other embodiments, generating the estimated performance characteristic of the TCP connection includes determining a slope of acknowledgment numbers over time based on the acknowledgment numbers recorded for the sampled plurality of packets transmitted from the source endpoint to the destination endpoint through the middlebox, and generating a throughput based on the slope of acknowledgment numbers, wherein the throughput corresponds to a rate of packet transmission from the destination endpoint to the source endpoint through the middlebox.

[0015] In some examples, the operations include, at a given moment, receiving a first packet communicated over the TCP connection from the source endpoint, the first packet including a first sequence number and a first acknowledgment number, and receiving a second packet communicated over the TCP connection from the destination endpoint, the second packet including a second sequence number and a second acknowledgment number. Here, the operations also include estimating a corresponding window size of the source endpoint by determining a difference between the first sequence number associated with the first packet and the second acknowledgment number associated with the second packet. In these examples, the operations may further include estimating a corresponding window size of the destination endpoint by determining a difference between the second sequence number associated with the second packet and the first acknowledgment number associated with the first packet.

[0016] In some embodiments, sparsely sampling the plurality of packets comprises sampling less than all packets from the TCP connection between the source endpoint and the destination endpoint. Additionally or alternatively, sparsely sampling the plurality of packets may comprise uniformly sampling packets from the TCP connection in a stateless and systematic manner. The source endpoint may be a user equipment (UE) in communication with a remote distributed network, and the destination endpoint corresponds to a server in communication with the remote distributed network.

[0017] Another aspect of the present disclosure provides a method for estimating the travel time of a transmission control protocol. The method includes, at data processing hardware of a middlebox, sparsely sampling a plurality of packets from one or more transmission control protocol (TCP) connections between endpoints communicating through the middlebox over a period of time. For each of the sampled plurality of packets, the method includes, by the data processing hardware, generating a timestamp when the packet was sampled. The method includes, by the data processing hardware, determining that a first sequence number of a first sampled packet of the sampled plurality of packets transmitted between the middlebox and the corresponding endpoint does not match any confirmation number of the sampled plurality of packets. The method also includes, by the data processing hardware, estimating a corresponding time of a confirmation number matching the first sequence number based on an interpolation of a plurality of confirmation numbers of the sampled plurality of packets. The method further includes, by the data processing hardware, determining the travel time between the middlebox and the corresponding endpoint by a difference between a first timestamp associated with the first packet and the estimated corresponding time.

[0018] This aspect may include one or more of the following optional features. In some examples, sparsely sampling the plurality of packets includes sampling less than all packets from the TCP connection between the endpoints through the middlebox. Optionally, sparsely sampling the plurality of packets may include uniformly sampling packets from the TCP connection in a stateless and systematic manner. The endpoints may include a user equipment (UE) communicating with a remote distributed network as a source endpoint and a server communicating with the remote distributed network as a destination endpoint.

[0019] The details of one or more embodiments of the present disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figures 1A-1C is a schematic diagram of an example communications network implementing a Transmission Control Protocol (TCP) connection.

[0021] Figure 1D is a schematic diagram of an example TCP connection between endpoints of a communications network.

[0022] Figure 2A-2E is a schematic diagram of an example of a performance evaluator for a middlebox of a communication network.

[0023] Figure 3 is a flow chart of an example arrangement of operations of a method of estimating TCP performance.

[0024] Figure 4 is a flow chart of an example arrangement of operations of a method of estimating TCP performance.

[0025] Figure 5 is a schematic diagram of an example computing device for implementing systems and methods for migrating virtual network functions.

[0026] Like reference numbers in the various drawings indicate like elements. DETAILED DESCRIPTION

[0027] The Transmission Control Protocol / Internet Protocol (TCP / IP) suite is mainly used for network communication. The TCP / IP protocol suite was originally designed based on end-to-end communication. For example, a source endpoint (such as a user) communicates with a destination endpoint (such as a server). This end-to-end design usually assumes that a gateway node (i.e., a router) configured to communicate between two separate networks forwards data packets of a TCP / IP packet stream to a destination endpoint (e.g., directly or through a jump between intermediate destinations). Based on this assumption, the router will not need to modify the packet during packet transmission (e.g., by modifying the header or payload). However, nowadays, with the development of network communication, the connection between nodes within the network and / or networks has become more complex. In other words, the network deploys hardware and / or software as nodes within the network environment to perform network functions (i.e., network services) other than routing packets. For example, these network functions include firewalls, network address translation (NAT), load balancing, virtual private network (VPN) tunnels, etc. for network protection. In order to perform these network functions, network administrators and / or network providers have implemented middleboxes.

[0028] Unfortunately, middleboxes may disrupt the usual end-to-end design of packets that are transported using TCP (i.e., transported by a TCP connection). For example, some middleboxes are configured to inspect and / or modify packets to perform network functions. In addition, middleboxes typically receive packets from many TCP connections that are multiplexed by the middlebox, and typically operate at the Internet Protocol (IP) layer. By operating at the IP layer, the middlebox does not employ TCP flow markings, such as, for example, the start or end of a flow marking or confirmation. In other words, middlebox activity cannot provide an indicator of packet flow performance that is the same or similar to a TCP flow (e.g., an end-to-end TCP connection). Without an indicator of packet flow performance, it is difficult for a network administrator or network service to evaluate and / or improve network communications.

[0029] When helping to configure, improve and / or optimize communications in a network environment, it is important and useful to determine the connection characteristics of a TCP connection (such as round-trip time, throughput, window size, etc.). In other words, the performance of a TCP connection may affect how entities can effectively communicate with each other and whether certain services (e.g., streaming audio / video at a target quality level) can actually be provided across a TCP connection. In some cases, the connection characteristics are determined or estimated at one or both ends of a TCP connection (i.e., at one or two stream endpoints such as a source endpoint and a destination endpoint). However, this typically requires custom functions at those stream endpoints, where custom functions may not always be possible, such as when the endpoints have minimal processing capabilities. For example, some Internet of Things (IoT) devices (such as home / household appliances) have low power or low processing capabilities. In some examples, determining and / or estimating the performance of a connection (e.g., a connection of two endpoints communicating via a middlebox) further burdens the connection itself. In other words, it is important that the system determines the connection characteristics without adversely affecting the connection. Otherwise, not only may the determined / estimated connection characteristics not be indicative of or accurately measure the actual connection characteristics that should be achieved (thereby resulting in an uninformed system configuration), but the system that burdens the connection while determining the connection characteristics may reduce throughput and / or cause harm to all parties involved in the communication.

[0030] To address these issues, embodiments of the present disclosure are directed to a system that generates TCP connection characteristics or estimates of TCP connection characteristics at a middlebox associated with a TCP connection with minimal impact on the connection itself and entities involved in the connection. Furthermore, because the middlebox only samples TCP packets / segments (e.g., a small portion of TCP packets / segments communicated via the TCP connection) in order to generate the estimate, the processing overhead of the middlebox is relatively small while having minimal (or imperceptible) impact on the performance of the TCP connection. As a result, the performance of other network connections managed by the middlebox remains unaffected.

[0031] refer to Figure 1A and 1B In some embodiments, the networked environment 100 is a distributed system (e.g., a remote network such as a cloud environment) with loosely coupled computing resources 110, 110an. The computing resources 110 may also be referred to as servers 110. These computing resources 110 may include data processing hardware 112 (e.g., such as a server) accessible to one or more clients 120, 120an. Figure 1B and 1C One or more central processing units (CPUs) shown in FIG. 1 ) and / or memory hardware 114 (e.g., as shown in FIG. Figure 1B and 1C10). For example, computing resources 110 are shown as multiple servers. Clients 120 can communicate with computing resources 110 hosted on physical network layer 102 through network 130. For example, clients 120 and servers 110 represent a collection of machines (e.g., hosts) that form endpoints of a physical network.

[0032] There can be many different types of networks 130, ranging from private networks and enterprise networks to public networks. Some more specific examples of networks include personal area networks (PANs), local area networks (LANs), wireless local area networks (WLANs), wide area networks (WANs), storage / system area networks (SANs), passive optical networks (PONs), enterprise private networks (EPNs), virtual private networks (VPNs), radio access networks (RANs), packet core networks, etc. In some configurations, clients 120 and end hosts (i.e., hosts of computing resources 110) can access (i.e., connect to) network 130 via hardware such as a network interface controller (NIC) (also known as a network interface card, network adapter, or LAN adapter) or a synchronous transport module (STM) (in the case of an optical network). For example, Figure 1A Each client 120 is depicted as having at least one NIC 122, 122an and each server 110 as having at least one NIC 116, 116an.

[0033] refer to Figure 1AIn some examples, the network 130 includes a physical network layer 102 and a virtual network layer 104, which can be overlaid on the physical network layer 102 to form a virtual network (also known as an overlay network) with its own Internet Protocol (IP) address space. The virtual network layer 104 enables the network 130 to be managed (e.g., by a network administrator), designed, and / or deployed in a manner that can decouple network functions from hardware appliances (e.g., proprietary hardware appliances). In other words, the virtual network layer 104 allows virtual network functions (also known as network function virtualization) so that network services can be operated by software to allow adaptability and scalability on traditional networks with dedicated hardware. For example, software-defined networking (SDN) of virtual networks removes the control plane from the network hardware and instead implements the control plane in software. In some embodiments, the virtual network layer 104 of the network 130 performs some network functions virtually via software, while performing other network functions with dedicated hardware in a hybrid virtualization approach. The virtual and / or hybrid approach can allow network providers or network administrators flexibility and / or optimization in network space, power, size, scalability, deployment time, and / or repair and maintenance. For example, the virtual network layer 104 is dynamically configured to allow underlying changes to the physical network layer 102 without disrupting network services.

[0034] The virtual network layer 104 can be configured to perform various network functions and / or mirror traditional network functions of the physical network. Some examples of these network functions include routing, network address translation (NAT), load balancing (LB), security (e.g., firewalls and intrusion detection / prevention systems), protocol conversion, WAN optimization, proxy, cache, etc. In other words, the virtual network function can be applicable to any data plane processing or control plane function within the network 130. In addition, Figure 1A The physical network layer 102 and the virtual network layer 104 are shown in dashed lines to indicate that some or all of the functionality of these layers 102, 104 may be physical and / or virtual. For example, components shown within the virtual network layer 104, such as the client virtual machines 140, the middlebox 160, and / or the backend virtual machines 150, may be implemented as dedicated physical hardware (i.e., physical layers) rather than operating on the virtual network layer 104. Depending on the network 130 and / or the services provided by the network 130, there may be various advantages to different types of physical layer and virtual layer configurations.

[0035] Continue to refer Figure 1A and 1B, the network environment 100 includes various endpoints. These endpoints may refer to physical endpoints (e.g., physical endpoints present on the physical network layer 102) or virtual endpoints (e.g., virtual endpoints present on the virtual network layer 104). Figures 1A-1C As shown, client 120 and / or server 110 may be physical endpoints that communicate with each other by means of network addresses. In other words, client 120 may be a source endpoint and server 110 may be a destination endpoint (or vice versa). Data 170 in the form of packets (also referred to as data packets 170 or packets 170) may be communicated between endpoints based on assigned network addresses.

[0036] In some examples, network environment 100 includes virtual network endpoints. Here, Figure 1AThe virtual network endpoints are described as virtual machines (VMs), such as client virtual machines (client VMs) 140, 140an and backend virtual machines (backend VMs) 150, 150an. A virtual machine (VM) generally refers to an emulation or image of one or more host computer systems that enables a user to experience the functionality of the host computer system without interfering with the capabilities of the underlying operating system of the corresponding host computer system. VMs can be created to perform dedicated tasks within one or more host environments. In some examples, the virtual network layer 104 allows multiple VMs to exist simultaneously within the host computer system. In order to allow multiple VMs to operate simultaneously, the VMs are typically sandboxed from the key operating system resources of the host computer system. VMs are advantageous for the virtual network layer 104 because they can be designed to be mapped to (e.g., access) one or more computing resources 110 of the physical network layer 102 via the backend VM 150. For example, one or more client VMs 140 are mapped to one or more backend VMs 150, where each backend VM 150 is a VM associated with a computing resource 110 of the physical network layer 102 (e.g., a distributed system). By mapping to specific computing resources 110, the client VM 140 in conjunction with the backend VM 150 can be designed for specific network services (e.g., network applications). In some configurations, the backend VM 150 operates as a host-based VM, where data can be contained on a server at the physical network layer 102 and accessible by or distributed to various remote clients 120. For example, a virtual network layer 104 with a VM allows a centrally managed network structure. Although network virtualization is often described with respect to VMs, network virtualization (e.g., with a virtual network layer 104) can be generalized to connect other types of network endpoints, including virtual private networks (VPNs), bare metal servers, proxies, etc. For example, the networking environment 100 supports a VPN gateway as a client that forwards requests from customer premises. Therefore, in some examples, VMs are more generally network endpoints to accommodate other types of network connections.

[0037] In some embodiments, to perform network functions, the network environment 100 includes at least one middlebox 160. Figure 1A As shown in FIG. 1 , the middlebox 160 may include corresponding data processing hardware 502 and memory hardware 504, wherein the memory hardware 504 stores executable programs for performing operations (e.g., executing Figure 3 and 4 The middlebox 160 may be physical hardware (e.g., a dedicated appliance on the physical network layer 102) or software configured to perform network functions (e.g., Figure 1A ). Middlebox 160 generally refers to any intermediary device that performs functions other than the normal, standard functions of an Internet Protocol (IP) router on a datagram path between a source endpoint / host (e.g., a client via client VM 140) and a destination endpoint / host (e.g., a server at physical network layer 102 via backend VM 150). In other words, middlebox 160 is generally responsible for network functions other than packet routing (e.g., network address translation (NAT), network address port translation (NAPT), load balancing (LB), firewalls, intrusion detection / prevention systems, protocol conversion, proxies, etc.). Although middlebox 160 is generally responsible for network functions other than routing, middlebox 160 may be part of a device that also handles routing, such that networking environment 100 includes a single device that supports routing and middlebox functions.

[0038] Within the networking environment 100, the middlebox 160 receives network flows from one or more clients 120 to perform network functions. A network connection with the middlebox 160 may be established based on a connection request from the client 120. In other words, a connection may be negotiated between the client 120 and the backend (i.e., the server 110) so that the middlebox 160 sits in the middle, processes, and potentially modifies packets belonging to the connection. Once a connection has been established with the middlebox 160 (e.g., a virtual connection is formed between the client VM 140 and the backend VM 150), the middlebox 160 receives network traffic (e.g., packets 170 of data) based on the configured network functions of the middlebox 160.

[0039] In some examples, the middlebox 160 performs NAT or NAPT on packets between the client 120 and the backend (e.g., the backend VM). In the case of NAPT, the middlebox 160 tracks each connection and the port mapping assigned to each connection. In other examples, the middlebox 160 is a layer 4 load balancer middlebox that forwards packets between the client 120 and the backend, while also potentially tracking the connection to ensure that future packets belonging to the connection continue to connect to the same backend. Alternatively, the middlebox 160 can be a layer 7 (i.e., application layer) load balancer, wherein the client 120 establishes a transmission control protocol (TCP) with the middlebox 160, and the middlebox 160 establishes a separate connection with the backend (i.e., on behalf of the client 120). For a layer 7 load balancer, the transport protocol state (e.g., TCP sequence number and window size) will be transmitted from the middlebox 160 to the corresponding end host.

[0040] As previously described, in some embodiments, an overlay network such as virtual network layer 104 is not required to perform middlebox functions. For example, in a network such as a physical Ethernet, software running on an end host can direct a connection to middlebox 160 by setting the destination medial access control (MAC) address to the MAC address corresponding to middlebox 160. Here, to offload the connection, the host will set the MAC address to the address of the destination (or the address of an appropriate gateway to reach the destination). In other words, middlebox 160 can communicate with physical machines instead of VMs. For example, in load balancing, client 120 and server 110 do not have to be VMs on virtual network layer 104, but rather connections between physical machines.

[0041] The middlebox 160 may differ from a router in that the middlebox 160 typically performs stateful network functions. More specifically, a stateful network function refers to a network function that tracks the operational state and / or characteristics of a network connection associated with the network function. For example, the middlebox 160 tracks connections so that the middlebox 160 is aware of the connection, checks packet context (e.g., periodically checks payloads), and / or associates new connections with existing connections. By performing stateful functions, the middlebox 160 includes a greater degree of data granularity to identify and / or recall connection events or connection data associated with a previous session of a client 120 or VM. These stateful functions may provide security and performance benefits of layers 102, 104, which may help stabilize network services.

[0042] refer to Figure 1B In some embodiments, the network environment 100 includes more than one type of network 130, 130ac in communication with each other. Here, the client 120 is depicted as a user equipment (UE) associated with the RAN 130, 130a. The RAN 130a includes three UEs 120a-c in communication with a base station (e.g., an evolved Node B (eNB)). In this example, the eNB is coupled to an evolved packet core (EPC) network 130, 130b that includes a middlebox 160. The EPC 130b can in turn communicate with external networks such as remote networks 130, 130c (e.g., a distributed system or cloud environment). Remote network 130c includes servers 110a-n configured to store and communicate packages of data 170 to and from clients 120a-c across networks 130a-c.

[0043] Figure 1CA simplified network environment 100 is shown to illustrate a network connection 180 between two endpoints 190. The network connection 180 passes through the middlebox 160 so that the packet 170 from the source endpoint 190 s flows to middlebox 160 (eg, shown as packets 170, 170 sx1-6 ), and flows from the middlebox 160 to the destination endpoint 190 d (For example, shown as package 170, 170 xd1-6 ). Using network connection 180, packet 170 may also be sent in the reverse direction from destination endpoint 190 d (shown as server 110) to source endpoint 190 s (eg, shown as client 120). For example, Figure 1C Depicts the destination endpoint 190 d The packets flowing to the middlebox 160 (eg, shown as packets 170, 170 dx1-6 ) and flows from the middlebox 160 to the source endpoint 190 s A package (eg, shown as package 170, 170 xs1-6 ). Here, the subscript of packet 170 identifies the direction of travel of packet 170. "D" refers to the destination endpoint 190 d "X" refers to the middlebox 160. "S" refers to the source endpoint 190 s In other words, "DX" corresponds to the destination endpoint 190 d Packet 170 traveling to middlebox 160, and "XS" corresponds to packets traveling from middlebox 160 to source endpoint 190 s The package is 170.

[0044] In order to s To destination endpoint 190 d The packets 170 are transmitted between the endpoints 190 (e.g., a source and a destination) using the Transmission Control Protocol (TCP). For example, the connection 180 is referred to as a TCP connection 180. TCP is a host-to-host protocol for packet-switched communication network 130. TCP is typically interfaced between a user or application process and the Internet Protocol (IP). TCP is configured to transmit a continuous stream of bytes (i.e., octets) between endpoints 190 (e.g., a source and a destination) by packaging a certain number of bytes into segments (also known as packets 170) for transmission. When a TCP connection 180 exists, TCP is configured to transmit a continuous stream of bytes (i.e., octets) in each direction (e.g., (1) from the source endpoint 190 to the destination). s To destination endpoint 190 d , and (2) from the destination endpoint 190 d To source endpoint 190 s ) transmits two byte streams (or packet streams) on the same network.

[0045] TCP also includes a specific structure for a given packet 170 (e.g., Figure 1C 170). The structure of packet 170 can ensure the reliability of the data (i.e., the payload of packet 170) and also communicate about the characteristics of TCP connection 180. To ensure the reliability of data that is damaged, lost, duplicated, or delivered out of order, TCP assigns a sequence number 172 to each byte transmitted and requires a positive acknowledgment (ACK) from the destination port (e.g., one of endpoints 190, depending on the direction in which packet 170 is being transmitted). In some examples, the ACK is its own packet 170, while in other examples, the acknowledgment occurs in the form of acknowledgment number 174. In other examples, when the acknowledging endpoint (e.g., specified by the destination port) sends a response to the original source endpoint 190, the ACK is sent. s When data is transmitted (e.g., as specified by the source port), the ACK includes its own ACK packet 170 as well as an acknowledgment number 174. When an ACK is not received within a timeout interval, TCP will retransmit the data (e.g., packet 170). The destination (e.g., one of the endpoints 190, depending on the direction in which the packets 170 are being transmitted) can use the sequence number 172 to reorder the packets 170 or eliminate duplicates when necessary. In addition to reliability, TCP allows the destination to specify the amount of traffic that can be received. In some examples, the destination communicates a window 178 with each ACK to indicate the range of acceptable sequence numbers 172 after the last successfully received packet 170 that the recipient is willing to receive. The window 178 refers to the acceptable data bandwidth (i.e., bytes) that can be transmitted to the destination at any one time.

[0046] In order to s and destination endpoint 190 d To establish a TCP connection 180 between the source endpoint 190, the endpoint 190 initially undergoes a three-way handshake. In the first step of the three-way handshake, the source endpoint 190 s (shown as client 120, for example) wants to establish a connection with destination endpoint 190 d (shown as server 110, for example). Here, source endpoint 190 s Send a packet 170 with a synchronized sequence number (SYN) to notify the destination endpoint 190 d Source endpoint 190 s Intends to start with destination endpoint 190 d Communication and source endpoint 190 s Which sequence number do you want to start with? In step 2 of the three-way handshake, the destination endpoint 190 d (e.g., server 110) responds with a SYN and ACK. Destination endpoint 190 dUse ACK to confirm the source endpoint 190 s The packet 170 is sent and the source endpoint 190 is notified using SYN. d It may start packet transmission with sequence number 172. In step three, source endpoint 190 s Confirm destination endpoint 190 d At this time, at the source endpoint 190 s and destination endpoint 190 d A TCP connection 180 is established between them.

[0047] Although the three-way handshake establishes a TCP connection 180 between endpoints 190, the function of the middlebox 160 as an intermediary device between these endpoints 190 destroys the traditional means of monitoring the TCP performance between the endpoints 190 of these connections. As previously described, some middleboxes 160 inspect and / or modify packets 170 to perform network functions. When packets 170 are modified, traditional techniques for monitoring TCP performance may not be able to rely on the fields / attributes of the standard packet structure. Additionally or alternatively, the middlebox 160 typically receives packets from many TCP connections 180 multiplexed by the middlebox, and typically operates at the Internet Protocol (IP) layer. By operating at the IP layer, the middlebox does not use TCP flow markers, such as, for example, the start or end of a flow marker or confirmation. In other words, the middlebox activity cannot provide an indicator of the same or similar packet flow performance as a TCP flow (e.g., an end-to-end TCP connection). In the absence of an indicator of packet flow performance, it is difficult for a network administrator or network service to evaluate and / or improve network communications.

[0048] Continue to refer Figures 1A-1C , the network environment 100 further includes a performance evaluator 200 at or coupled to the middlebox 160. The performance evaluator 200 is configured to estimate a performance characteristic 202 ( Figure 1C ).For example, Figure 1C The performance evaluator 200 is shown evaluating one or more performance characteristics 202, 202ac of the TCP connection 180 between the client 120 and the server 110. Some examples of performance characteristics 202 include travel time 202a (e.g., round trip time RTT), throughput 202b, and estimated window size 202c. Figure 1C As shown in , performance evaluator 200 estimates performance characteristics 202 by sampling packets 170 from TCP connection 180. For example, Figure 1C , the performance evaluator 200 receives the data from the middlebox 160 to the source endpoint 190 s Two packs of 170 xs2,6 , from source endpoint 190 s Two packets 170 travel to middlebox 160sx3,4 , from destination endpoint 190 d A packet 170 traveling to middlebox 160 dx1 , and travels from middlebox 160 to destination endpoint 190 d A package of 170 xd4 This illustrates that because performance evaluator 200 is located and / or capable of monitoring packets 170 and / or packet activity at middlebox 160 , performance evaluator 200 may monitor each segment of bidirectional TCP connection 180 between endpoints 190 .

[0049] In some embodiments, once the TCP connection 180 has been established, each endpoint 190 (eg, source endpoint 190 s or destination endpoint 190 d ) transmits bytes of data as packets 170. Because TCP connection 180 allows each endpoint 190 to transmit data, each endpoint 190 has its own timeline relative to sequence numbers 172 of packets 170 sent (e.g., Figure 1D In other words, source endpoint 190 s It will start with an initial sequence number (ISN) of 172, for the source endpoint 190 s To destination endpoint 190 d The ISN 172 will increment for each byte of data transmitted. Similarly, the destination endpoint 190 d will start with its own initial sequence number (ISN) 172 (e.g., most likely different from the source endpoint 190 s ISN 172) and for the destination endpoint 190 d To source endpoint 190 s Each byte of data communicated increments its own initial sequence number 172. Each of these sequence numbers 172 will be managed independently of each other so that the source endpoint 190 s The bytes transferred will not be incremented from the destination endpoint 190 d The sequence number of the transmitted packet 170 (or vice versa).

[0050] refer to Figure 1D , source endpoint 190 s The ISN 172 in sequence number 172 is equal to 1024 (in packet 170 sd1 ) is initialized, and the destination endpoint 190 d The ISN 172 in the sequence number 172 is equal to 4000 (in the packet 170 sd1 When the source endpoint 190 sTransfer 32 bytes of data to destination endpoint 190 d When the sequence number of 1024 is incremented by 32 bytes to 1056 (for example, from the first packet 170 sd1 To the second pack 170 sd2 is shown), and the destination endpoint 190 d The sequence number 172 remains at 4000 (for example, even though 32 bytes have been transferred from the source endpoint 190 using TCP connection 180). s transmission), until the destination endpoint 190 d For example, destination endpoint 190 d Transfer 328 bytes to source endpoint 190 s , incrementing its sequence number 172 from 4000 to 4328 (e.g., from the first packet 170 ds1 To the second pack 170 ds2 is shown), and the source endpoint 190 s The sequence number 172 of the source endpoint 190 remains at 1056. The incremented sequence number 172 will be visible as the sequence number 172 in subsequent packets 170 sent by the endpoint 190. For example, including s The first packet of 32 bytes of data sent is 170 sd1 will have ISN172 equal to 1024, and due to the source endpoint 190 s Previously in the first pack 170 sd1 The 32 bytes of data sent in the source endpoint 190 s The second packet sent is 170 sd2 will have a subsequently incremented sequence number 172 equal to 1056. Destination endpoint 190 d The sequence number 172 for each data packet 170 follows the same increment (eg, see the first packet 170 ds1 and the second pack 170 ds2 ). The payload 176 of the packet 170 generally refers to the bytes included in the packet 170. For example, the source endpoint 190 s The first packet sent is 170 sd1 includes a corresponding payload 176 equal to 32 bytes, and the destination endpoint 190 d The first packet sent is 170 ds1 The corresponding payload 176 is equal to 328 bytes. In some examples, the TCP packet 170 has a maximum number of bytes for TCP packaging. In other examples, the connection between the endpoints 190 itself further limits the maximum number of bytes of the payload 176 due to the limitations / constraints of the connection 180.

[0051] If Figure 1C and 1DAs shown in the packet structure in FIG. 1 , in addition to sequence number 172, packet 170 may also include an acknowledgment (ACK) number 174. ACK number 174 in packet 170 indicates that the sender of packet 170 has received sequence number 172 (e.g., from opposite endpoint 190) before ACK number 174 and expects to receive the next sequence number 172. For example, although there may be some delay between endpoints 190 (e.g., other intermediate hops or delays within TCP connection 180), when destination endpoint 190 d From source endpoint 190 s Receive the first packet with 32 bytes 170 sd1 When including the source endpoint 190 s The first pack of 170 sd1 The ACK number 174 of the sequence number 172 (ie, the sequence number 172 of 1024) is incremented by 1 byte to indicate the destination endpoint 190 d Intend to source endpoint 190 s The next expected byte received by the destination endpoint 190 d To source endpoint 190 s Next packet sent 170 ds1 Confirms receipt of 32 bytes. In other words, when the destination endpoint 190 d From the first packet 170 with sequence number 172 equal to 1024 sd1 When receiving 32 bytes, from the destination endpoint 190 d Next Instant Packet 170 of Communications ds1 The ACK number 174 will be equal to 1025 (that is, sequence number 1024 has been received and the destination endpoint 190 d Intend to receive 1025 next).

[0052] refer to Figures 2A-2D The performance evaluator 200 includes a sampler 210 and an estimator 220. The sampler 210 is configured to receive data from an endpoint 190 (eg, a source endpoint 190). s and destination endpoint 190 d ) between the TCP connection 180 sampling a plurality of packets 170. For example, Figure 1CThe sampler 210 is depicted with packets 170 sampled by the dashed box fed into the performance evaluator 200. Here, by sampling from the packet stream of the TCP connection 180, the performance evaluator 200 can reduce the burden (e.g., processor and / or resource cost) on the middlebox 160 or the TCP connection 180. For example, for a given TCP connection 180, the performance evaluator 200 can evaluate each packet 170 or substantially most of the packets 170 flowing through the middlebox 160. This approach will be accurate, but will still impose a heavy tax on the resources of the middlebox 160 (e.g., by measuring the establishment of a full TCP proxy for each flow (i.e., connection 180) at the middlebox 160). In other words, this approach will attempt to recreate the TCP endpoint for each flow at the middlebox 160. In addition, in addition to taxing the resources of the middlebox 160, this approach will introduce additional latency for the client 120 due to the window size at the proxy. Any additional latency may result in a reduction in user experience, especially when these TCP connections 180 affect media transmission (e.g., audio, video, or both in real-time communication). This sampling may become increasingly important as the number of clients and / or hosts increases and the number of network flows and connections received by the middlebox 160 in proportion, requiring greater programming resources to program and / or manage activities at the middlebox 160, particularly in functions with state tracking (e.g., connection tables) performed by the middlebox 160. For example, there may be millions of packets flowing through the middlebox 160. In these cases, the middlebox 160, acting as an intermediary device, may add latency, bottlenecks, and additional hops to network functions between endpoints 190.

[0053] In some examples, for each packet 170 sampled by the sampler 210, when the packet 170 is sampled at the middlebox 160, the sampler 210 generates a timestamp 212 for the sampled packet 170. In other words, the timestamp 212 corresponds to the time t when the middlebox 160 encounters the packet 170. In some implementations, the sampler 210 records the sequence number 172 and / or the acknowledgment number 174 of each sampled packet 170 along with the timestamp 212. The sampler 210 may also be configured to record and / or store other fields or attributes of the packet 170 (e.g., packet header information, payload size, flags, checksums, etc.). When a sampled packet 170 fails to include an attribute that the sampler 210 is configured to record, the sampler 210 may mark the occurrence and / or continue processing the packet 170 (e.g., recording other packet attributes and / or generating the timestamp 212). For example, the packet 170 fails to include the acknowledgment number 174. The sampler 210 may be configured to record and / or store a record of the sequence number 172, the acknowledgment number 174, the timestamp 212, and any other packet attributes of the sampled packets 170. Additionally or alternatively, the sampler 210 generates a curve 214 or trend line for each recorded sequence number 172 and / or acknowledgment number 174 at its corresponding timestamp 212. In other words, the sampler 210 may plot these numbers 172, 174 with respect to time. By generating the curve 214 or trend line, the sampler 210 may streamline the estimation of the performance characteristics 202 by the performance evaluator 200, or allow various network entities (such as network users, network administrators, etc.) to have visibility into the performance of the TCP connection 180 over time. As shown in FIG. Figure 2A As shown in , the sampler 210 may plot these recorded numbers 172 , 174 together in the same curve 214 or separately (not shown). Figure 2A The sampler 210 is depicted with the curves 214 numbered 172 , 174 versus time t to illustrate the generating and / or recording functionality of the sampler 210 , even though the performance evaluator 200 does not require the sampler 210 to generate the curves 214 to estimate one or more performance characteristics 202 .

[0054] Specifically refer to Figure 2A , the sampler 210 is shown as receiving (i.e., sampling) two packets 1703,9. Here, the sampler 210 generates a first timestamp 2123 for the first sampled packet 1703 and a second timestamp 2129 for the second sampled packet 1709. Figure 2AIn FIG. 1 , sampler 210 generates curve 214 as one possible means of recording and / or storing the numbers 172, 174 associated with two packets 1703, 9. For example, sequence number 172 and confirmation number 174 are shown as points connected on respective trend lines. In some examples, trend lines represent interpolation of known data points (e.g., recorded numbers 172, 174 and timestamp 212). Interpolation is often used in data sampling to generate a function representing the sampled data points (e.g., Figure 2A The function may be used to approximate or estimate the value of an unknown data point (e.g., an unsampled data point) based on the slope of the trend line of the sampled packets 170. In some examples, the performance evaluator 200 (e.g., at the estimator 220) estimates the performance characteristic 202 based on the interpolation of information (e.g., packet attributes) from the sampled packets 170. As an example, by using the trend line of the sequence number 174, the performance evaluator 200 estimates that the packet 1706 between the two packets 1703,9 may occur at the timestamp 212 associated with the midpoint between the sequence numbers 172 of the two packets 1703,9.

[0055] In some embodiments, the sampler 210 sparsely samples a plurality of packets 170 from the TCP connection 180. Sparse sampling rather than evaluating all packets 170 can help ensure that minimal resources of the middlebox 160 are consumed. In a general sense, sparsely sampling packets 170 from a TCP connection 180 means sampling less than all packets 170 communicated within a packet flow of the TCP connection 180. In some examples, sparse sampling refers to not sampling a majority of packets 170 communicated within a packet flow of the TCP connection 180. In some embodiments, the sampler 210 is configured to sparsely sample packets 170 of the TCP connection 180 by sampling between one percent and ten percent (1% to 10%) of the packets 170 of the TCP connection 180. In other embodiments, the sampler 210 is configured to sparsely sample packets 170 of the TCP connection 180 by sampling less than one percent (e.g., 0.1%) of the packets 170 of the TCP connection 180. Regardless of the sparseness of the sampling, the sampling can occur at the system sampling frequency. Sampling may occur randomly or uniformly (e.g., systematically at some different intervals). Here, sampling by sampler 210 may occur in a stateless manner. For example, sampler 210 does not consider the operating state and / or characteristics of TCP connection 180 during sampling.

[0056] In some configurations, the estimator 220 is configured to generate a source endpoint 190 passing through the middlebox 160 over a period of time based on at least two of the corresponding timestamp 212, the corresponding sequence number 172, or the corresponding acknowledgment number 174 of one or more packets from the sampled plurality of packets 170. s and destination endpoint 190 dThe functionality of the estimator 220 may vary depending on the type of performance characteristics 202 of the TCP connection 180 that the estimator 220 is estimating. Figure 2B An estimator 220 is shown that determines a performance characteristic 202 corresponding to an RTT 202a (or travel time 222). Figure 2C An estimator 2220 is shown determining the performance characteristic 202 corresponding to the throughput 202b. Figure 2D and 2E Describes determining the destination endpoint 190 d and source endpoint 190 s The estimated window size 202c corresponds to the performance characteristic 202 of the estimator 220.

[0057] refer to Figure 2B In some examples, to estimate the round trip time (RTT) 202a, the estimator 220 analyzes segments of the TCP connection 180 between the endpoints 190. In other words, the RTT 202a includes: (i) the source endpoint 190 s and (ii) the first trip time 222, 222a (eg, device / client-side trip time) of the TCP connection segment between the destination endpoint 190 and the middlebox 160; and d and the middlebox 160. In order to make an accurate estimate of the RTT 202a, the estimator 220 may sum the first travel time 222a and the second travel time 222b. Although this may not be very accurate, the estimator 220 may estimate the RTT 202a based on the travel time 222 of any of the TCP connection segments and double the travel time 222 to form an estimate of the RTT 202a between the endpoints 190 through the middlebox 160.

[0058] In some examples, for a travel time 222 (e.g., a first travel time 222a or a second travel time 222b), the estimator 220 identifies a sequence number 172 from a sample of packets 170 transmitted by the TCP connection 180 within a given TCP connection segment. Figure 2B The subscript "sx / xs" indicates the source endpoint 190 s At least one sampled packet 170 of a TCP connection segment between the middlebox 160 sx / xs , and the subscript "dx / xd" indicates the destination endpoint 190 d At least one sampled packet 170 of a TCP connection segment between the middlebox 160 dx / xd Based on the generated timestamp 212 from the sampler 210 sx / xs,212 dx / xd and the associated record numbers 172, 174, the estimator 220 determines that in the TCP connection segment (eg, at the source endpoint 190 s and middlebox 160) will receive an acknowledgment number 174 that matches sequence number 172. In other words, the time difference between when sequence number 172 is communicated within a TCP connection segment (e.g., the timestamp 212 when performance evaluator 200 records sequence number 172) and when an acknowledgment number 174 identifying sequence number 172 is acknowledged within the same TCP connection segment (e.g., the timestamp 212 when performance evaluator records the acknowledgment number 174 that matches sequence number 172), corresponds to the trip time 222 for the TCP connection segment. This is true regardless of the TCP connection segment.

[0059] The estimator 220 may attempt to match a known sequence number 172 that has been recorded by the sampler 210 or an estimated sequence number 172 based on an interpolation of two or more known sequence numbers 172 recorded by the sampler 210. Because the sampler 210 does not sample all packets 170 communicated in the TCP connection 180 between the endpoints 190, the estimator 220 may not receive a recorded acknowledgment number 174 that matches the sequence number 172. When a recorded acknowledgment number 174 does not match a sequence number 172 identified by the estimator 220 for determining the travel time 222, the estimator 220 is configured to estimate a corresponding time when the corresponding acknowledgment number 174 will match the identified sequence number 172 based on an interpolation of recorded acknowledgment numbers 174 for packets 170 sampled by the sampler 210. For example, by confirming an open point on a trend line instead of confirming a filled point on the trend line, Figure 2B It is shown that the confirmation number 174 that matches the sequence number 172 of the record from the sampler 210 is unknown. Here, the estimator 220 determines the corresponding time of the confirmation number 174 that matches the sequence number 172 in order to determine two travel times 222a-b. The first travel time 222a is shown as the difference between the estimated matching confirmation time t3 and the corresponding timestamp 212, t1 of the recorded sequence number 172. Similarly, the second travel time 222b is shown as the difference between the estimated matching confirmation time t4 and the corresponding timestamp 212, t2 of the recorded sequence number 172. In some configurations, the estimator 220 assumes that the sampler 210 has no recorded matching confirmation number 174 (e.g., due to infrequent or sparse sampling). In other words, the estimator 220 may not even be configured to attempt to identify the confirmation number 174 of the record that matches the sequence number 172.

[0060] refer to Figure 2C , the estimator 220 may determine whether any direction of the TCP connection 180 between the endpoints 190 (eg, from the source endpoint 190 sTo destination endpoint 190 d or from destination endpoint 190 d To source endpoint 190 s ) throughput 202b. In either direction, the estimator 220 determines the slope "m" of the acknowledgment numbers 174 over a period of time t based on the recorded acknowledgment numbers 174. Because the acknowledgment numbers 174 confirm the sequence of data received from the packets 170 via the sequence numbers 172, the slope m of the acknowledgment numbers 174 corresponds to the rate at which packets 170 are being transmitted or have been transmitted to the endpoint 190 communicating the acknowledgment numbers 174. In other words, when the destination endpoint 190 d When the communication confirmation number is 174, the slope m is equal to the source endpoint 190 s Sending to destination endpoint 190 d Conversely, when the source endpoint 190 s When the confirmation numbers 174 are being communicated, the slope m of these confirmation numbers 174 and the destination endpoint 190 d Sending to source endpoint 190 s The rate at which packets 170 are transmitted corresponds.

[0061] In some examples, such as Figure 2D Instead of sending a single packet 170 at a time and waiting for a response (e.g., an acknowledgment packet 170), each endpoint 190 of the TCP connection 180 sends a group of packets 170 (called a window) to the other endpoint 190. Each side maintains its respective window. Each endpoint 190 can acknowledge a packet 170 or a window of packets 170 to minimize latency due to acknowledgments. Because the size of the window may affect the TCP connection 180, the performance evaluator 200 is configured to generate an estimated window size 202c for each endpoint 190 to help optimize communications between the endpoints 190. In some examples, the estimator 220 generates the estimated window size 202c by selecting packets 170 that have been sampled by the sampler 210. Based on the sampled packets 170, the estimator 220 determines the difference between the recorded sequence number 172 and the acknowledgment number 174 (e.g., the recorded acknowledgment number 174 or the estimated acknowledgment number 174) at the same timestamp 212. When the acknowledgment number 174 is received by the destination endpoint 190, the estimated window size 202c is calculated. d communication (eg, with a destination endpoint 190 d When the packet 170 sent is associated with Figure 2D , the window size 202c estimated by the estimator 220 will be the destination endpoint 190 d In contrast, when the acknowledgment number 174 is received by the source endpoint 190 s communication (eg, with a source endpoint 190 s When the packet 170 sent is associated with Figure 2E, the window size 202c estimated by the estimator 220 will be the window size 202c of the source endpoint 190 s Regardless of which endpoint 190 the estimated window size 202c corresponds to, the estimator 220 generates the estimated window size 202c based on the determined difference between the sequence number 172 and the confirmation number 174 of the record. In other words, the estimator 220 determines the estimated window size 202c by the difference between the numbers 172, 174 at the same timestamp 212. For example, Figure 2D The boxes around numbers 172, 174 are depicted at the same time to illustrate the estimated window size 202c. In some embodiments, each time the sampler 210 samples the TCP packet flow, the sampler 210 is configured to sample packets 170 in each TCP flow direction (e.g., a pair of packets at the same timestamp 212), so that the estimator 220 can more efficiently generate the estimated window size 202c. In other embodiments, the sampler 210 samples only one packet 170 from either TCP flow direction, so that the estimator 220 interpolates the sampled packets 170 to generate the estimated window size 202c.

[0062] Figure 2D and 2E Depicted are different curves 214 that the sampler 210 and / or the estimator 220 may generate to determine the estimated window size 202c. Figure 2D In the example, sampler 210 and / or estimator 220 generates curve 214, which depicts the data from source endpoint 190. s Serial number 172 (for example, shown as Figure 2D The reference 172s on the trend line in the image together with the ... d The confirmation number 174 (for example, shown as Figure 2D Reference 174d) on the trend line in Figure 2E In the example, the sampler 210 and / or the estimator 220 generates a curve 214, which depicts the flow of data from the destination endpoint 190. d Serial number 172 (for example, shown as Figure 2E The reference 172d on the trend line in the source endpoint 190 s The confirmation number 174 (for example, shown as Figure 2E For any curve 214, the estimated window size 202c is equal to the distance between a point on the sequence number 172 trend line and a point on the confirmation number 174 trend line at the same time t.

[0063] Figure 3is a flow chart of an example arrangement of operations of a method 300 of estimating TCP performance characteristics 202. In operation 302, the method 300 collects data from a source endpoint 190 through a middlebox 160 over a period of time t. s and destination endpoint 190 d The method 300 sparsely samples a plurality of packets 170 from a TCP connection 180 between the plurality of packets 170. In operation 304, the method 300 performs operations 304a-b on each sampled packet 170 of the plurality of packets 170. In operation 304a, for each sampled packet 170, the method 300 generates a timestamp 212 when the corresponding packet 170 is sampled. In operation 304b, for each sampled packet 170, the method 300 records a sequence number 172 and an acknowledgment number 174 of the corresponding packet 170. Here, for the corresponding packet 170, an acknowledgment packet 174 may not exist. In operation 306, the method 300 generates a source endpoint 190 passing through the middlebox 160 within the period of time t based on at least two of the corresponding timestamp 212, the corresponding sequence number 172, or the corresponding acknowledgment number 174 of one or more packets from the plurality of packets 170 sampled. s and destination endpoint 190 d Estimated performance characteristics 202 of TCP connection 180 between.

[0064] Figure 4 4 is a flow chart of an example arrangement of operations of a method 400 for estimating a travel time of a TCP connection 180. At operation 402, the method 400 sparsely samples a plurality of packets 170 at a middlebox 160 from a TCP connection 180 between endpoints 190 communicating through the middlebox 160 over a period of time t. For each sampled packet 170 in the sampled plurality of packets 170, at operation 404, the method 400 generates a timestamp 212 when the corresponding packet 170 was sampled. At operation 406, the method 400 determines that a first sequence number 172 of a first sampled packet 170 in the sampled plurality of packets 170 transmitted between the middlebox 160 and the corresponding endpoint 190 does not match any acknowledgment number 174 of the sampled plurality of packets 170. At operation 408, the method 400 estimates a corresponding time t of an acknowledgment number 174 matching the first sequence number 172 based on an interpolation of the plurality of acknowledgment numbers 174 of the sampled plurality of packets 170. At operation 410 , the method 400 determines a travel time between the middlebox 160 and the corresponding endpoint 190 by a difference between the first timestamp 212 associated with the first packet 170 and the estimated corresponding time.

[0065] Figure 5is a schematic diagram of an example computing device 500 that can be used to implement the systems (e.g., performance evaluator 200) and methods (e.g., methods 300, 400) described in this document. Computing device 500 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown herein, their connections and relationships, and their functions are meant to be exemplary only and are not meant to limit implementations of the inventions described and / or claimed in this document.

[0066] The computing device 500 includes a processor 510 (e.g., data processing hardware), a memory 520 (e.g., memory hardware), a storage device 530, a high-speed interface / controller 540 connected to the memory 520 and a high-speed expansion port 550, and a low-speed interface / controller 560 connected to a low-speed bus 570 and the storage device 530. Each of the components 510, 520, 530, 540, 550, and 560 are interconnected using various buses and can be mounted on a general motherboard or in other appropriate ways. The processor 510 can process instructions for execution within the computing device 500, including instructions stored in the memory 520 or on the storage device 530, to display graphical information of a graphical user interface (GUI) on an external input / output device, such as a display 580 coupled to the high-speed interface 540. In other embodiments, multiple processors and / or multiple buses can be used as appropriate with multiple memories and memory types. Furthermore, multiple computing devices 500 may be connected, with each device providing portions of the necessary operations (eg, as a server bank, a group of blade servers, or a multi-processor system).

[0067] Memory 520 stores information non-temporarily within computing device 500. Memory 520 may be a computer-readable medium, a volatile memory unit, or a non-volatile memory unit. Non-temporary memory 520 may be a physical device for temporarily or permanently storing programs (e.g., sequences of instructions) or data (e.g., program state information) for use by computing device 500. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electrically erasable programmable read-only memory (EEPROM) (e.g., typically used for firmware, such as bootloaders). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disk or tape.

[0068] Storage device 530 can provide mass storage for computing device 500. In some embodiments, storage device 530 is a computer readable medium. In various embodiments, storage device 530 can be a floppy disk device, a hard disk device, an optical disk device or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configuration. In additional embodiments, a computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine readable medium, such as a memory 520, a storage device 530, or a memory on a processor 510.

[0069] The high-speed controller 540 manages bandwidth-intensive operations of the computing device 500, while the low-speed controller 560 manages less bandwidth-intensive operations. Such allocation of responsibilities is exemplary only. In some embodiments, the high-speed controller 540 is coupled to the memory 520, the display 580 (e.g., through a graphics processor or accelerator), and the high-speed expansion port 550, which can accept various expansion cards (not shown). In some embodiments, the low-speed controller 560 is coupled to the storage device 530 and the low-speed expansion port 590. The low-speed expansion port 590, which can include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled to one or more input / output devices, such as keyboards, pointing devices, scanners, or network devices such as switches or routers, for example, through a network adapter.

[0070] As shown in the figure, computing device 500 can be implemented in several different forms. For example, it can be implemented as a standard server 500a or multiple times in a group of such servers 500a, implemented as a laptop computer 500b, or implemented as part of a rack server system 500c.

[0071] Various implementations of the systems and techniques described herein can be implemented in digital electronic and / or optical circuits, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be special purpose or general purpose, coupled to receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0072] These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages, and / or in assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0073] The process and logic flow described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating outputs. The process and logic flow can also be performed by a dedicated logic circuit, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, or be operably coupled to receive data from the mass storage device or transmit data to the mass storage device, or both, and the mass storage device is, for example, a magnetic disk, a magneto-optical disk, or an optical disk. However, a computer does not have to have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0074] To provide interaction with a user, one or more aspects of the present disclosure may be implemented on a computer having a display device, such as a CRT (cathode ray tube) monitor, an LCD (liquid crystal display) monitor, or a touch screen for displaying information to a user, and optionally a keyboard and pointing device, such as a mouse or trackball, through which a user can provide input to the computer. Other types of devices may also be used to provide interaction with a user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including sound, voice, or tactile input. In addition, a computer may interact with a user by sending documents to and receiving documents from a device used by a user; for example, by sending a web page to a web browser in response to a request received from a web browser on a user's client device.

[0075] Several embodiments have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of the present disclosure. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. A method (300) for estimating performance characteristics of a transmission control protocol (TCP), characterized in that The method (300) comprises: At data processing hardware (502) of a middlebox (160), from a source endpoint (190 s ) and the destination endpoint (190 d ) sparsely samples a plurality of packets (170) from the TCP connection (180) between the two packets; For each packet (170) of the sampled plurality of packets (170): generating, by the data processing hardware (502), a timestamp (212) of when the packet (170) was sampled; and recording, by the data processing hardware (502), a sequence number (172) and an acknowledgment number (174) of the packet (170); and The data processing hardware (502) generates, based on at least two of a corresponding timestamp (212), a corresponding sequence number (172), or a corresponding confirmation number (174) of one or more packets (170) from the sampled plurality of packets (170), a source endpoint (190) passing through the middlebox (160) within the period of time. s ) and the destination endpoint (190 d ) between the TCP connection (180).

2. The method (300) according to claim 1, characterized in that: Further including: Identified by the data processing hardware (502) as a node between the middlebox (160) and the destination endpoint (190 d ) between the first packet (170) of the plurality of packets (170) of the sample transmitted between the first packet (170); determining, by the data processing hardware (502), whether the first sequence number (172) recorded for the first packet (170) matches the sequence number recorded for the first packet at the destination endpoint (190); d ) matches any one of the confirmation numbers (174) recorded in the plurality of sampled packets (170) transmitted between the middlebox (160); When the first sequence number (172) matches the confirmation number (174) recorded for a second packet (170) of the sampled plurality of packets (170), determining, by the data processing hardware (502), a connection between the middlebox (160) and the destination endpoint (190) based on a difference between the respective timestamps (212) generated for the first packet (170) and the second packet (170) d ) between the first travel time (222); and When the first sequence number (172) does not match any of the confirmation numbers (174) recorded for the sampled plurality of packets (170): estimating, by the data processing hardware (502), a corresponding timestamp (212) of a first unsampled packet (170) from the TCP connection (180) associated with a corresponding acknowledgment number (174) matching the first sequence number (172) by interpolating the acknowledgment numbers (174) recorded for the sampled plurality of packets (170); and Determining, by the data processing hardware (502), a first time stamp (212) generated for the first packet (170) and a corresponding time stamp (212) estimated for the first unsampled packet (170) between the middlebox (160) and the destination endpoint (190) based on a difference between the first time stamp (212) generated for the first packet (170) and the estimated time stamp (212) for the first unsampled packet (170) d ) between the first travel time (222).

3. The method (300) according to claim 2, characterized in that: Further including: a second sequence number (172) recorded by the data processing hardware (502) for a third packet (170) of the plurality of packets (170) identified as being sampled, the third packet (170) being transmitted between the middlebox (160) and the source endpoint (190); s ) are transmitted between; The data processing hardware (502) determines whether the second sequence number (172) recorded for the third packet (170) matches the sequence number recorded for the source endpoint (190). s ) matches any one of the confirmation numbers (174) recorded in the plurality of sampled packets (170) transmitted between the middlebox (160); When the second sequence number (172) matches the confirmation number (174) recorded for a fourth packet (170) of the sampled plurality of packets (170), the data processing hardware (502) determines, based on a difference between the respective timestamps (212) generated for the third packet (170) and the fourth packet (170), that the middlebox (160) and the source endpoint (190) are in communication with each other. s ) between the second travel time (222); and When the second sequence number (172) does not match any of the confirmation numbers (174) recorded for the sampled plurality of packets (170): estimating, by the data processing hardware (502), a corresponding timestamp (212) of a second unsampled packet (170) from the TCP connection (180) by interpolating the acknowledgment numbers (174) recorded for the sampled plurality of packets (170), the second unsampled packet (170) being associated with a corresponding acknowledgment number (174) matching the second sequence number (172); and Determining, by the data processing hardware (502), a time stamp between the middlebox (160) and the source endpoint (190) based on a difference between a third timestamp (212) generated for the third packet (170) and a corresponding timestamp (212) estimated for the second unsampled packet (170) s ) between the second travel time (222).

4. The method (300) according to claim 3, characterized in that: Generating the estimated performance characteristics (202) of the TCP connection (180) includes, by performing a d ) and the first travel time (222) between the middlebox (160) and the source endpoint (190 s ) between the source endpoint (190 s ) and the destination endpoint (190 d )’s round trip time (222).

5. The method (300) according to any one of claims 1 to 4, characterized in that: Further comprising estimating, by the data processing hardware (502), corresponding timestamps (212) of unsampled packets (170) from the TCP connection (180) by interpolating the acknowledgment numbers (174) recorded for the sampled plurality of packets (170).

6. The method (300) according to any one of claims 1 to 4, characterized in that: Further comprising estimating, by the data processing hardware (502), corresponding timestamps (212) of unsampled packets (170) from the TCP connection (180) by interpolating the sequence numbers (172) recorded for the sampled plurality of packets (170).

7. The method (300) according to any one of claims 1 to 4, characterized in that: Generating the estimated performance characteristics (202) of the TCP connection (180) includes: Based on the middlebox (160) from the destination endpoint (190 d ) is transmitted to the source endpoint (190 s ), determining a slope of the confirmation numbers (174) over the period of time; and A throughput is generated based on the slope of the acknowledgment numbers (174), the throughput being related to the number of acknowledgments received from the source endpoint (190) via the middlebox (160). s ) to the destination endpoint (190 d )'s packet (170) transmission rate corresponds to.

8. The method (300) according to any one of claims 1 to 4, characterized in that: Generating the estimated performance characteristics (202) of the TCP connection (180) includes: Based on the middlebox (160) from the source endpoint (190 s ) is transmitted to the destination endpoint (190 d ), determining a slope of the confirmation number (174) over time; and A throughput is generated based on the slope of the acknowledgment numbers (174), the throughput being related to the number of packets received from the destination endpoint (190) through the middlebox (160). d ) to the source endpoint (190 s )'s packet (170) transmission rate corresponds to.

9. The method (300) according to any one of claims 1 to 4, characterized in that: Further including, at a given moment: At the data processing hardware (502), from the source endpoint (190 s ) receiving a first packet (170) communicated via the TCP connection (180), the first packet (170) comprising a first sequence number (172) and a first acknowledgment number (174); At the data processing hardware (502), from the destination endpoint (190 d ) receiving a second packet (170) communicated via the TCP connection (180), the second packet (170) comprising a second sequence number (172) and a second acknowledgment number (174); and The data processing hardware (502) estimates the source endpoint (190) by determining a difference between the first sequence number (172) associated with the first packet (170) and the second acknowledgment number (174) associated with the second packet (170). s )'s corresponding window size.

10. The method (300) according to claim 9, characterized in that: The method further comprises estimating, by the data processing hardware (502), the destination endpoint (190) by determining a difference between the second sequence number (172) associated with the second packet (170) and the first acknowledgment number (174) associated with the first packet (170). d )'s corresponding window size.

11. The method (300) according to any one of claims 1 to 4, characterized in that: The number of the sampled packets (170) is less than the number of all packets (170) communicated within the packet flow of the TCP connection (180).

12. The method (300) according to any one of claims 1 to 4, characterized in that: Sparsely sampling the plurality of packets (170) includes uniformly sampling packets (170) from the TCP connection (180) in a stateless and systematic manner.

13. The method (300) according to any one of claims 1 to 4, characterized in that: The source endpoint (190 s ) is a user equipment (UE) communicating with a remote distributed network (130), and the destination endpoint (190 d ) corresponds to a server (110) in communication with the remote distributed network (130).

14. A system (100) for estimating performance characteristics of a transmission control protocol (TCP), characterized in that The system (100) comprises: Data processing hardware (502); and Memory hardware (504) in communication with the data processing hardware (502), the memory hardware (504) storing instructions that, when executed on the data processing hardware (502), cause the data processing hardware (502) to perform operations, the operations comprising: From a source endpoint (190) passing through a middlebox (160) over a period of time s ) and the destination endpoint (190 d ) sparsely samples a plurality of packets (170) from the TCP connection (180) between the two packets; For each packet (170) of the sampled plurality of packets (170): generating a timestamp (212) of when the packet (170) was sampled; and recording the sequence number (172) and confirmation number (174) of the packet (170); and Generate a source endpoint (190) passing through the middlebox (160) within the period of time based on at least two of a corresponding timestamp (212), a corresponding sequence number (172), or a corresponding confirmation number (174) of one or more packets (170) from the sampled plurality of packets (170); s ) and the destination endpoint (190 d ) between the TCP connection (180).

15. The system (100) according to claim 14, characterized in that The operations further include: Identified as being between the middlebox (160) and the destination endpoint (190 d ) between the first packet (170) of the plurality of packets (170) of the sample transmitted between the first packet (170); determining whether the first sequence number (172) recorded for the first packet (170) matches the sequence number recorded for the first packet at the destination endpoint (190); d ) matches any one of the confirmation numbers (174) recorded in the plurality of sampled packets (170) transmitted between the middlebox (160); When the first sequence number (172) matches the confirmation number (174) recorded for a second packet (170) of the sampled plurality of packets (170), determining a connection between the middlebox (160) and the destination endpoint (190) based on a difference between the respective timestamps (212) of the first packet (170) and the second packet (170). d ) between the first travel time (222); and When the first sequence number (172) does not match any of the confirmation numbers (174) recorded for the sampled plurality of packets (170): estimating a corresponding timestamp (212) of a first unsampled packet (170) from the TCP connection (180) associated with a corresponding acknowledgment number (174) matching the first sequence number (172) by interpolating the acknowledgment numbers (174) recorded for the sampled plurality of packets (170); and Determining a time stamp between the middlebox (160) and the destination endpoint (190) based on a difference between a first timestamp (212) generated for the first packet (170) and a corresponding timestamp (212) estimated for the first unsampled packet (170) d ) between the first travel time (222).

16. The system (100) according to claim 15, characterized in that The operations further include: identifying a second sequence number (172) recorded for a third packet (170) of the plurality of packets (170) sampled, the third packet (170) being transmitted between the middlebox (160) and the source endpoint (190); s ) are transmitted between; Determine whether the second sequence number (172) recorded for the third packet (170) matches the sequence number recorded for the source endpoint (190 s ) matches any one of the confirmation numbers (174) recorded in the plurality of sampled packets (170) transmitted between the middlebox (160); When the second sequence number (172) matches the confirmation number (174) recorded for a fourth packet (170) of the sampled plurality of packets (170), determining a connection between the middlebox (160) and the source endpoint (190) based on a difference between the respective timestamps (212) generated for the third packet (170) and the fourth packet (170) s ) between the second travel time (222); and When the second sequence number (172) does not match any of the confirmation numbers (174) recorded for the sampled plurality of packets (170): estimating a corresponding timestamp (212) of a second unsampled packet (170) from the TCP connection (180) by interpolating the acknowledgment numbers (174) recorded for the sampled plurality of packets (170), the second unsampled packet (170) being associated with a corresponding acknowledgment number (174) matching the second sequence number (172); and Determining a time difference between the middlebox (160) and the source endpoint (190) based on a difference between a third timestamp (212) generated for the third packet (170) and a corresponding timestamp (212) estimated for the second unsampled packet (170) s ) between the second travel time (222).

17. The system (100) according to claim 16, characterized in that Generating the estimated performance characteristics (202) of the TCP connection (180) includes, by performing a d ) and the first travel time (222) between the middlebox (160) and the source endpoint (190 s ) between the source endpoint (190 s ) and the destination endpoint (190 d )’s round trip time (222).

18. The system (100) according to any one of claims 14 to 17, characterized in that: The operations further include estimating corresponding times of unrecorded acknowledgment numbers (174) based on interpolation of recorded acknowledgment numbers (174) of the sampled plurality of packets (170).

19. The system (100) according to any one of claims 14 to 17, characterized in that: The operations further include estimating corresponding times of unrecorded sequence numbers (172) based on interpolation of recorded sequence numbers (172) of the sampled plurality of packets (170).

20. The system (100) according to any one of claims 14 to 17, characterized in that Generating the estimated performance characteristics (202) of the TCP connection (180) includes: Based on the middlebox (160) from the destination endpoint (190 d ) is transmitted to the source endpoint (190 s ), determining a slope of the confirmation numbers (174) over the period of time; and A throughput is generated based on the slope of the acknowledgment numbers (174), the throughput being related to the number of acknowledgments received from the source endpoint (190) via the middlebox (160). s ) to the destination endpoint (190 d )'s packet (170) transmission rate corresponds to.

21. The system (100) according to any one of claims 14 to 17, characterized in that: Generating the estimated performance characteristics (202) of the TCP connection (180) includes: Based on the middlebox (160) from the source endpoint (190 s ) is transmitted to the destination endpoint (190 d ), determining a slope of the confirmation number (174) over time; and A throughput is generated based on the slope of the acknowledgment numbers (174), the throughput being related to the number of packets received from the destination endpoint (190) through the middlebox (160). d ) to the source endpoint (190 s )'s packet (170) transmission rate corresponds to.

22. The system (100) according to any one of claims 14 to 17, characterized in that The operations further include, at a given time: From the source endpoint (190 s ) receiving a first packet (170) communicated via the TCP connection (180), the first packet (170) comprising a first sequence number (172) and a first acknowledgment number (174); From the destination endpoint (190 d ) receiving a second packet (170) communicated via the TCP connection (180), the second packet (170) comprising a second sequence number (172) and a second acknowledgment number (174); and The source endpoint (190) is estimated by determining a difference between the first sequence number (172) associated with the first packet (170) and the second acknowledgment number (174) associated with the second packet (170). s )'s corresponding window size.

23. The system (100) according to claim 22, characterized in that The operations further include estimating the destination endpoint (190) by determining a difference between the second sequence number (172) associated with the second packet (170) and the first acknowledgment number (174) associated with the first packet (170). d )'s corresponding window size.

24. The system (100) according to any one of claims 14 to 17, characterized in that The number of the sampled packets (170) is less than the number of all packets (170) communicated within the packet flow of the TCP connection (180).

25. The system (100) according to any one of claims 14 to 17, characterized in that Sparsely sampling the plurality of packets (170) includes uniformly sampling packets (170) from the TCP connection (180) in a stateless and systematic manner.

26. The system (100) according to any one of claims 14 to 17, characterized in that The source endpoint (190 s ) includes a user equipment (UE) communicating with a remote distributed network (130), and the destination endpoint (190 d ) includes a server (110) in communication with the remote distributed network (130).

27. A method (400) for estimating travel time of a transmission control protocol (TCP), characterized in that: The method (400) comprises: At data processing hardware (502) of a middlebox (160), sparsely sampling a plurality of packets (170) from the TCP connection (180) between endpoints (190) communicating through the middlebox (160) over a period of time; For each packet (170) of the sampled plurality of packets (170), generating, by the data processing hardware (502), a timestamp (212) of when the packet (170) was sampled; determining, by the data processing hardware (502), that a first sequence number (172) of a first sampled packet (170) of the plurality of sampled packets (170) transmitted between the middlebox (160) and the corresponding endpoint (190) does not match any acknowledgment number (174) of the plurality of sampled packets (170); estimating, by the data processing hardware (502), a corresponding time of an acknowledgment number (174) matching the first sequence number (172) based on interpolation of a plurality of acknowledgment numbers (174) of the sampled plurality of packets (170); and The data processing hardware (502) determines a travel time (222) between the middlebox (160) and the corresponding endpoint (190) by a difference between a first timestamp (212) associated with the first sample packet (170) and the estimated corresponding time.

28. The method (400) of claim 27, characterized in that: The number of the sampled packets (170) is less than the number of all packets (170) communicated within the packet flow of the TCP connection (180).

29. The method (400) of claim 27, characterized in that: Sparsely sampling the plurality of packets (170) includes uniformly sampling packets (170) from the TCP connection (180) in a stateless and systematic manner.

30. The method (400) according to any one of claims 27-29, characterized in that: The endpoint (190) includes a source endpoint (190) that communicates with the remote distributed network (130) s ) and communicates with the remote distributed network (130) as a destination endpoint (190 d ) server (110).

Images