Smart broadband service with intelligent packet control

Abstract

Systems and methods are provided to control network traffic. Preferential network traffic may be identified and prioritized based on indications from service providers. Data caps may be monitored for preferential traffic and actions may be taken when these caps are reached. Additionally, during network sessions, data may be selectively routed through preferential or non-preferential queues based on identified characteristics. Preferential resources, such as Low Latency, Low Loss, and Scalable Throughput (L4S) queues, may be allocated based on the past behavior of sending applications. Explicit Congestion Notification bits may help determine if a sending application should be granted access to preferential queues during congestion. Efficient and prioritized handling of network traffic, and optimized performance and resource allocation are provided.

Description

SMART BROADBAND SERVICE WITH INTELLIGENT PACKET CONTROLCross-References to Related Applications

[0001] This application claims priority to and the benefit of commonly owned U.S. NonProvisional Application No. 18 / 626,659, filed April 4, 2024 (titled “APPLICATION FLOW- AWARE BROADBAND SERVICE WITH DATA CAPS”); commonly owned U.S. Provisional Application No. 63 / 574,668, filed April 4, 2024, and U.S. Non-Provisional Application No. 18 / 667,655, filed May 17, 2024 (both titled “INTELLIGENT APPLICATION PRIORITY PACKET DELIVERY CONTROL”); and commonly owned U.S. Non-Provisional Application No. 18 / 806,438, filed August 15, 2024, (titled “QUEUE PROTECTION WITH SELECTIVE L4S USE”), the disclosures of which are hereby incorporated by reference herein in their entireties.

[0002] Also, the disclosure of commonly owned Application No. 18 / 626,668, filed April 4, 2024 (titled “CUSTOMER-CENTRIC, APPLICATION-FLOW AWARE BROADBAND SERVICE”), is hereby incorporated by reference herein in its entirety.Background

[0003] This disclosure is directed to systems and methods for enforcing data caps for preferential network traffic; systems and methods for determining, based on one or more characteristics of a portion of a network session data, whether to provide the portion of network session data using a first queue for preferential network traffic or a second queue for non-preferential network traffic; and content delivery, including low- to ultralow-latency content delivery via real-time transport protocol (RTP).Summary

[0004] When a network link is congested, latency on the network link generally increases. Traditionally, this has often occurred primarily because of certain congestion control mechanisms utilized by a sender transmitting data on the network link, rather than due to a lack of available capacity on the network link. Such congestion control mechanisms attempt to detect currently available capacity on the network link, to allow the sender to adjust its data transmission rate accordingly. However, such congestion control mechanisms oftencause queuing delay, e.g., application service providers often send data too quickly for the network to queue it up.

[0005] For example, as stated in Internet Engineering Task Force (IETF), “Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: Architecture,” RFC 9330, January 2023, (referred to herein as RFC 9330), the contents of which are hereby incorporated by reference herein in their entirety, “queuing remains a major, albeit intermittent, component of latency. For instance, spikes of hundreds of milliseconds are not uncommon, even with state-of-the-art Active Queue Management (AQM). ... It has been demonstrated that, once access network bitrates reach levels now common in the developed world, increasing link capacity offers diminishing returns if latency (delay) is not addressed.” RFC 9330 further states that “Queuing delay degrades performance intermittently. ... It occurs i) when a large enough capacity-seeking (e.g., TCP) flow is running alongside the user’s traffic in the bottleneck link, which is typically in the access network, or ii) when the low latency application is itself a large capacity-seeking or adaptive rate flow (e.g., interactive video).”

[0006] The L4S standard has been introduced to help address these issues. As stated in RFC 9330, “This document describes the L4S architecture, which enables Internet applications to achieve low queuing latency, low congestion loss, and scalable throughput control. L4S is based on the insight that the root cause of queuing delay is in the capacity-seeking congestion controllers of senders, not in the queue itself. With the L4S architecture, all Internet applications could (but do not have to) transition away from congestion control algorithms that cause substantial queuing delay and instead adopt a new class of congestion controls that can seek capacity with very little queuing. These are aided by a modified form of Explicit Congestion Notification (ECN) from the network. With this new architecture, applications can have both low latency and high throughput. The architecture primarily concerns incremental deployment. It defines mechanisms that allow the new class of L4S congestion controls to coexist with ‘Classic’ congestion controls in a shared network. The aim is for L4S latency and throughput to be usually much better (and rarely worse) while typically not impacting Classic performance.”

[0007] Traditional single-queue buffering of Internet packets at a network component such as an access network router suffers from Head-of-line (HoL) blocking, effectively making highly latency-sensitive traffic wait in a queue behind less latency-sensitive traffic, which adversely affects the customer’s quality of experience (QoE). The L4S mechanism helps address this issue using dual queueing in the wide area network (WAN), with one queuededicated to low latency packets and the other queue dedicated to classic traffic and makes reasonable assumptions about performance of network-dependent low latency applications such as gaming, AR / VR, voice, etc. to deliver an improved service.

[0008] However, application service providers and / or Internet service providers (ISPs) may decide to mark all of their traffic as L4S-capable, in an effort to minimize latency of their traffic, whether or not such marking is accurate. Moreover, since the dual queuing mechanism of L4S is designed to implement both L4S and non-L4S service flows, it may starve the non-L4S service flow if all application service providers and / or ISPs mark their traffic as L4S.

[0009] A further complication is that an ISP may want to accelerate some application- provider-marked L4S-capable traffic based on other considerations, such as a general policy related to Quality-of-Experience (QoE). L4S allows application service providers to mark their traffic as L4S-compliant or L4S-capable. Net neutrality in some jurisdictions might require ISPs to treat traffic marked as L4S-capable from all application service providers in the same manner, regardless of the application service provider. This also creates a situation where application service providers can abuse the network by marking all their traffic using the L4S DiffServ codepoints. While current mechanisms for queue protection allow an ISP to move layer 2 traffic from the preferred (low latency) queue to the non-preferred (classic) queue when a “fairness” policy is violated, this would typically occur only instantaneously, justified based on current traffic patterns and queue congestion, to ensure that the ISP is not intentionally favoring one application service provider’s traffic over another’s.

[0010] Accordingly, there is a need for a mechanism to cause application service providers and / or ISPs to be more selective in terms of how much and which network traffic is to be treated preferentially in at least some circumstances.

[0011] To help overcome these issues, systems and methods are provided herein for receiving, at a first networking equipment, network traffic over a network; identifying a portion of the network traffic that corresponds to preferential network traffic over a network, based on an indication from an application service provider or an ISP; accessing a preferential network traffic data cap for data provided over the network to the first networking equipment over a particular period of time; determining whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap; and, based on determining that the amount of preferential network traffic provided to the first networking equipment over the particular period of time equals or exceeds the preferential network trafficdata cap, causing identification of an action to be performed on the portion of the network traffic that corresponds to preferential network traffic.

[0012] Such aspects enable enforcing data caps for preferential traffic (e.g., L4S-capable network traffic marked by an ISP or application service provider as preferential) separately from regular (e.g., non-L4S-capable and / or not marked as preferential by an ISP or application service provider), to control an amount of network traffic that is to be processed in the low latency service flow. This allows subsets of such network traffic to be treated preferentially, while avoiding a situation where too much (or all) traffic is requested to be processed in the low latency service flow, which is desirable because such a situation causes network congestion, which detracts from the improvements provided by L4S. Further, these techniques allow individual application service providers and / or ISPs flexibility in terms of actions to be taken when customers exceed a data cap.

[0013] The enforcement of such data caps is likely to motivate application service providers (and / or ISPs and / or subscribers) to adjust priority packet flows based on a true need for ultralow latency and low loss for packet delivery for aspects of the application as opposed to other aspects that might be acceptable without such low latency and low loss of packets. The metering and enforcing data caps for different traffic types (e.g., preferential, and non- preferential) may be performed by an ISP based on tonnage at the broadband service subscriber’s household. The techniques disclosed herein may minimize queuing for preferential traffic through protection policies.

[0014] In some embodiments, the first networking equipment is located at a particular location and provides a local area network (LAN) at the particular location, and second networking equipment is associated with providing a wide area network (WAN). The systems and methods described herein may be further configured to transmit an indication to the second networking equipment indicating that the amount of preferential network traffic equals or exceeds the preferential network traffic data cap, wherein the indication causes the second networking equipment to perform the identification of the action to be performed. In some embodiments, the first networking equipment may comprise a plurality of networking equipment devices, and / or the second networking equipment may comprise a plurality of networking equipment devices.

[0015] In some embodiments, the first network equipment is associated with providing a WAN, and the first network equipment causes the identification of the action to be performed.

[0016] In some embodiments, identifying the portion of the network traffic that corresponds to preferential network traffic comprises determining whether data associated with the portion of the network traffic comprises bits indicative of the portion of the network traffic being designated as preferential by the application service provider or the ISP.

[0017] In some embodiments, the portion of network traffic designated as preferential by the application service provider, or the ISP is L4S-capable.

[0018] In some embodiments, determining the amount of preferential network traffic provided to the first networking equipment over the particular time period comprises determining an amount of at least one of network traffic designated as preferential by the application service provider or network traffic designated as preferential by the ISP provided to the first networking equipment over the particular time period.

[0019] In some embodiments, the portion of the network traffic is designated as preferential by the application service provider; accessing the preferential network traffic data cap for data provided over the network to the first networking equipment comprises accessing an application service provider preferential network traffic data cap for network traffic designated as preferential by the application service provider; and determining whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap comprises determining whether an amount of application service provider network traffic designated as preferential provided to the first networking equipment over the particular time period equals or exceeds the application service provider preferential network traffic data cap. In some embodiments, the preferential network traffic (e.g., L4S) data cap associated with the first networking equipment may be applied to a specific application service provider’s traffic or to the sum total of all application service providers that mark their traffic as L4S-capable and communicate with the first networking equipment.

[0020] In some embodiments, the portion of the network traffic is designated as preferential by the ISP; accessing the preferential network traffic data cap for data provided over the network to the first networking equipment comprises accessing an ISP preferential network traffic data cap for network traffic designated as preferential by the ISP; and determining whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap comprises determining whether an amount of ISP network traffic designated as preferential by the ISP provided to the first networking equipment over the particular time period equals or exceeds the ISP preferential network traffic data cap.

[0021] In some embodiments, the portion of the network traffic is designated as preferential by the ISP; accessing the preferential network traffic data cap for data provided over the network to the first networking equipment comprises accessing an ISP preferential network traffic data cap for network traffic designated as preferential by the ISP; and determining whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap comprises determining whether an amount of ISP network traffic designated as preferential by the ISP provided to the first networking equipment over the particular time period equals or exceeds the ISP preferential network traffic data cap.

[0022] In some embodiments, a first queue is provided to process preferential network traffic intended for or transmitted by the first networking equipment, and a second queue is provided to process non -preferential network traffic intended for or transmitted by the first networking equipment; and the identified action comprises modifying the bits to indicate that the portion of the network traffic is non-preferential network traffic, to cause the non- preferential network traffic to be transmitted via the second queue.

[0023] In some embodiments, a first queue is provided to process preferential network traffic intended for or transmitted by the first networking equipment, and a second queue is provided to process non-preferential network traffic intended for or transmitted by the first networking equipment; and the identified action comprises maintaining the portion of the network traffic as preferential network traffic, to cause the portion of the network traffic to be transmitted via the first queue.

[0024] In some embodiments, the portion of the network traffic is designated as preferential by the ISP; and the method further comprises maintaining the portion of the network traffic as preferential network traffic based on determining that an amount of current network traffic having bits indicative of the ISP designating the current network traffic as preferential is below a threshold.

[0025] In some embodiments, the portion of the network traffic is associated with a particular data provider; second networking equipment, associated with providing a wide area network (WAN), is configured to receive an indication from the first networking equipment that the bits do not indicate that the portion of the network traffic is at least one of preferential or experiencing congestion, despite the second networking equipment having caused the bits to indicate that the portion of the network traffic is at least one of preferential or experiencing congestion; and the second networking equipment is configured to, based on receiving the indication from the first networking equipment that the bits do not indicate that the portion ofthe network traffic is preferential or experiencing congestion, refrain from causing subsequent network traffic from the particular data provider and intended for the first networking equipment to include an indication that the subsequent network traffic is at least one of preferential or experiencing congestion. In some embodiments, if a sender marks packets for L4S-capable, and the receiver does not receive the marked packets due to network equipment along one of the hops dropping the L4SZECN bits, the receiver’s response may be an ECN marking of 00 in the RTCP or acknowledgment / negative acknowledgment (ACK / NACK) packets. In this circumstance, the sender may determine to set the ECN bits to 00 since ECN / L4S is not supported end-to-end.

[0026] In some embodiments, determining whether the data associated with the portion of the network traffic comprises bits indicative of the portion of the network traffic being preferential network traffic comprises: based on determining that the bits indicate that the portion of the network traffic is experiencing congestion, identifying a prior packet of the portion of the network traffic, and determining whether the prior packet of the portion of the network traffic comprises bits indicating whether the portion of the network traffic was designated as preferential by the application service provider or the ISP.

[0027] In some embodiments, second networking equipment is associated with the ISP and is further configured to: ingest the network traffic; determine whether the portion of the network traffic originated outside of the network of the ISP; based on determining that the portion of the network traffic originated outside of the network of the ISP, determine whether the data associated with the portion of the network traffic comprises, in error, the bits indicative of the portion of the network traffic being preferential network traffic; and based on determining that the data associated with the portion of the network traffic comprises, in error, the bits indicative of the portion of the network traffic being preferential network traffic, modifying the bits to indicate that the portion of the network traffic is non-preferential network traffic.

[0028] In some embodiments, the portion of the network traffic that corresponds to preferential network traffic is associated with a cellular network.

[0029] However, while L4S attempts to do justice to highly latency-sensitive applications, it does not provide for treating different portions of a same network session (e.g., a video game) differently based on their characteristics. For example, L4S may process all video game traffic in a preferential manner, even if certain portions of the video game (e.g., a menu screen) may not need preferential treatment as such portions may not be critical to the QoE. This may cause finite network resources to be dedicated to portions of a network session tofacilitate preferential treatment of such portions, even though such portion may be sufficiently provided in a non-preferential manner without impacting the QoE, on the basis that other portions of the network session require the preferential treatment, which may not be an optimal use of network resources. What is needed is a context-aware mechanism for selectively treating portions of the same network traffic session as preferential or non- preferential, based on real-time analysis of the user’s current network session.

[0030] To help overcome these issues, systems, apparatuses, and methods are provided herein for receiving, via a network, a request from a device connected to a local area network (LAN), wherein portions of data are configured to be selectively provided to the device via the network using a first queue for preferential network traffic or a second queue for non- preferential network traffic; based on the request, establishing a network session with the device during which network session data is provided to the device via the network; during the network session: identifying one or more characteristics of a portion of the network session data; determining, based on the one or more characteristics of the portion of the network session data, whether to provide the portion of the network session data to the device using the first queue or the second queue; and based on determining to provide the portion of the network session data using the first queue, providing the portion of the network session data to the device using the first queue. Such aspects enable network resources (e.g., latency priority) to be conserved for portions of a network session for which the network resources will have the most impact on the user experience (e.g., combat portions of a video game), by treating portions of the network traffic (e.g., portions of the video game such as a menu, advertisement, or cut scene) less likely to impact the user experience as non-preferential (e.g., not given latency priority).

[0031] In some embodiments, identifying the one or more characteristics of the portion of the network session data comprises determining the portion of the network session data is latency sensitive; determining whether to provide the portion of the network session data to the device using the first queue or the second queue is based on whether the portion of the network session data is latency sensitive; and based on determining that the portion of the network session data is latency sensitive, providing the portion of the network session data to the device using the first queue.

[0032] In some embodiments, the network session comprises a video gaming session and the network session data comprises data for a video game played by a user of the device during the video gaming session; and determining whether the portion of the network session data is latency sensitive comprises determining whether the portion of the network sessiondata corresponds to one or more characters within the video game being controlled by the user or to a cutscene or menu screen of the video game.

[0033] In some embodiments, determining whether the portion of the network session data is latency sensitive is further based at least in part on at least one of a type of the video game or a type of a video game portion in which the one or more characters are being controlled by the user.

[0034] In some embodiments, the determining whether the portion of the network session data is latency sensitive is based at least in part on identifying metadata associated with the video game and determining whether the metadata indicates that the portion of the network session data corresponds to a portion of the video game that is latency sensitive. In some embodiments, the determining whether the portion of the network session data is latency sensitive is based at least in part on a game engine calling APIs on the sender / transmission system.

[0035] In some embodiments, determining whether to provide the portion of the network session data to the device using the first queue or the second queue based at least in part on a bandwidth of the network and a jitter associated with providing the portion of the network session data over the network.

[0036] In some embodiments, the network session data is provided by an application service provider during the network session, and the portion of the network session data is a first portion; and the systems and methods may be further configured to identify one or more characteristics of a second portion of the network session data; determine, based on the one or more characteristics of the second portion of the network session data, whether to provide the second portion of the network session data to the device using the first queue or the second queue; and based on determining to provide the second portion of the network session data using the second queue, provide the second portion of the network session data to the device using the second queue.

[0037] In some embodiments, identifying the one or more characteristics of the portion of the network session data comprises determining a threshold level of localization accuracy or a threshold level of localization precision required for the portion of the network session data; and determining to provide the portion of the network session data using the first queue based on determining the localization accuracy required for the portion of the network session data exceeds the threshold level of localization accuracy or the localization precision required for the portion of the network session data exceeds the threshold level of precision. In some embodiments, a frequency of localization updates may be increased based at least in part ondetermining that a portion of network session data is latency sensitive, and / or based at least in part on determining that the localization precision exceeds the threshold level of precision, and / or based at least in part on determining that the localization accuracy exceeds the threshold level of accuracy.

[0038] In some embodiments, determining the localization accuracy required for the portion of the network session data exceeds the threshold level of localization accuracy or the localization precision required for the portion of the network session data exceeds the threshold level of precision based on at least one of a proximity of the device to one or more objects in an environment of the device or a speed of the device.

[0039] In some embodiments, the device is an extended reality (XR) device, and the systems and methods further comprise causing data transmitted by the XR device to be provided to a remote server using the first queue, based at least in part on the determining that the localization accuracy required for the portion of the network session data exceeds the threshold level of localization accuracy or the localization precision required for the portion of the network session data exceeds the threshold level of precision.

[0040] In some embodiments, causing data transmitted by the device to be provided to a remote server using the first queue is further performed based on determining a distance between a user wearing the XR device and a physical object in an XR environment surrounding the user, and / or based on determining a distance between a user wearing the XR device and a virtual object in the XR environment surrounding the user, and / or based on determining a distance between the virtual object in relation to other objects in the XR environment. In some embodiments, causing data transmitted by the device to be provided to a remote server using the first queue for preferential network traffic is further based on determining whether one or more virtual objects rendered by the XR device are within a threshold distance of a physical object in an environment surrounding the XR device. In some embodiments, causing data transmitted by the device to be provided to a remote server using the first queue is further performed based at least in part on whether the XR device is located indoors and not outdoors.

[0041] In some embodiments, based on determining to provide the portion of the network session data using the first queue, the systems and methods disclosed herein may cause bits of the portion of the network session data to be marked with an indication that the portion of the network session data is to be provided to the device using the first queue.

[0042] In some embodiments, the first queue is for the preferential network traffic is for L4S network traffic.

[0043] In some embodiments, the network session comprises a multiplayer video gaming session in which users of a plurality of devices, including the device and one or more other devices, are playing a video game with each other over the network; the one or more other devices are remote from the device of the LAN; and the systems and methods disclosed herein further comprise, based on determining that a particular device of the other devices is not capable of being provided data over the network using the first queue, providing data to each of the plurality of devices using the first queue for a remainder of the network session.

[0044] In some embodiments, determining that the particular device of the other devices is not capable of being provided data over the network using the first queue comprises analyzing a network packet received from the particular device.

[0045] In some embodiments, the device is a robotic device, and the systems and methods further comprise causing data transmitted by the device to be provided to a remote server using the first queue for preferential network traffic is based on determining whether one or more objects are within a threshold distance of the robotic device.

[0046] In some embodiments, the device is associated with a controller, and the systems and methods further comprise further causing network session data, corresponding to input received via the controller in relation to the portion of the network session data, to be processed using the first queue for preferential network traffic.

[0047] When a network link is congested, for example, at the “last mile” of data traffic before delivery to a client device, latency on the network link may increase during heavy traffic. Traditional single-queue buffering of Internet packets at a network component, such as an access network router, suffers from Head-of-Line (HoL) blocking. This may cause high latency-sensitive traffic to wait in a queue behind less latency-sensitive traffic, which adversely affects the customer’s quality of experience (QoE).

[0048] Some congestion control mechanisms utilized by a sender transmitting data on the network link sometimes exacerbate the congestion problem. Congestion control mechanisms sometimes attempt to detect currently available capacity on the network link, to allow the sender to adjust its data transmission rate accordingly. In one approach, RFC 9330 introduces the Low Latency, Low Loss, Scalable Throughput (L4S) service, addressing latency issues in high bit-rate networks. It identifies queueing delay as a performance degrader, occurring when large capacity-seeking flows, such as TCP or interactive video, run alongside user traffic. L4S uses dual queueing in the WAN or LAN, dedicating one queue to low latency traffic and another to classic traffic, aiming to improve network-dependent applications’ performance. The approach is based on the idea that queueing delay’s root cause lies in thecapacity-seeking congestion controllers of senders, not the queue itself. L4S encourages transitioning from congestion control algorithms causing substantial queueing delay to controls seeking capacity with minimal queueing, aided by a modified form of Explicit Congestion Notification (ECN).

[0049] However, the L4S approach has limitations. It assumes all applications will transition to the new congestion controls, which may not be feasible due to legacy systems and compatibility issues. It also assumes applications will not attempt to send all traffic to the L4S queue. Further, the dual queueing system causes resource allocation challenges thus impacting the performance of classic traffic.

[0050] In the L4S approach, some network traffic, for example, some video game traffic or XR network traffic, may be processed in a preferential manner. The non-preferential queue may be sufficient for much of the network traffic without impacting the QoE, thus not causing suboptimal use of network resources. For example, portions of the network traffic such as a menu, advertisement, or cutscene of a video game, may be less likely to impact the user experience. By adding data that does not require low-latency treatment to the non- preferential queue, network resources (e.g., latency priority) may be conserved for data that will have the most impact on the user experience (e.g., combat portions of a video game).

[0051] For purposes of illustration, packets associated with a network traffic flow may be recognized based on a 5-tuple, for example, source IP address / port number, destination IP address / port number and the protocol in use determination as to one or more of the above described time period, data quantity or percentage may be application dependent. An explicit congestion notification (ECN) bit may be used to signal a request for a low latency queue.

[0052] However, if applications repeatedly request the low latency queue for network traffic flows even when low latency is not required, then the low latency queue may become congested.While L4S attempts to do justice to highly latency-sensitive applications, it does not provide for incentivizing preferential queue requests only for data that needs it, or for explicitly imposing a penalty for preferential queue requests for data that does not need it.

[0053] To help address the limitations and problems of these and other approaches, according to an aspect of the disclosure, in some embodiments, a queue assigner may maintain a record of previous preferential queue requests by an application for processing by a network node. For example, state data associated with an application flow may be determined by a network node. The queue assigner associated with a network may utilize the state data to determine whether preferential treatment should be applied to the applicationflow. The queue assigner may be located at a (or implemented as part of a) device such as a cable modem (CM) in the upstream with respect to data being uploaded, and / or with respect to data being downloaded, may be located at (or implemented as part of) a cable modem termination system CMTS, base station of a cellular network, BNG (broadband gateway) or the like. Such a device may sometimes be referred to as the network node or receiving device. Thus, the application that sends traffic may be upstream or downstream of the network node for which prioritized queueing is to be decided. The terms “application,” “sending application,” or “sender application” may also refer to a server (e.g., an application server) that provides a data flow (e.g., downstream traffic). For the processing for the network node or receiving device, a decision has to be made as to whether a packet should be added to the preferential queue (sometimes referred to as “LL,” “low latency,” or “L4S queue”) or to the non-preferential queue (sometimes referred to a “classic queue”).

[0054] If the application is currently requesting processing by a network node using the preferential queue for its network traffic, the queue assigner may grant preferential status to a flow of network traffic associated with the requesting application only if one or more conditions for prior requests for preferential queue use of the application is satisfied. For example:—the queue assigner may grant the preferential status for the flow of network traffic only if the application requested no preferential queue request within a threshold look-back time period, for example, about 0.001 to about 30 seconds, or about 0.000,001 to 90 seconds;—the queue assigner may grant the preferential status for the flow of network traffic only if the application requested less than a threshold amount of data, for example, preferential queue request(s) are for no more than a percentage, for example, about 20%-65% of the application’s current total network data traffic (or request for traffic delivery) for processing by the network node, or no more than a specified number of packets, for example, about 1,024 to 65,536 packets, within the threshold look-back time period;—the queue assigner may grant the preferential status for the flow of network traffic only if the application’s preferential queue request(s) is for no more than a percentage, for example, about 20%-65%%, or about 5%-98%, of its total network traffic requests for processing by the network node;— the queue assigner may grant the preferential status for the flow of network traffic only if, based on a combination of more than one of the factors of the above examples (e.g., [(no more than a percentage) AND (no more than a threshold amount)], [(no more than a percentage) OR (no more than a threshold amount)]), criteria are met. For example, if asending application requested L4S queue processing, within a threshold period of time, for example, within about 10-15 seconds, or within about 0.001-90 seconds, for no more than a threshold percent, by way of illustration about 10-15% or 3-50%, of its data flows, then it may receive “credit” for its current data flow.

[0055] In an embodiment, an application that judiciously or selectively uses L4S may get some credit as a reward for its avoidance of the precious L4S queue resource. This credit may be applied to a subsequent data flow associated with the application. The credit may be in the form of a reduction in the queueing score of the subsequent data flow during a burst arrival, or a shortening of the decay time of the queueing score. When the application requests use of the L4S queue because of a traffic burst, then this credit (i.e., reduction of the queueing score or hastening the time of its deemed demise) works to its advantage by calculating faster expiry for congestion ascribed to the data flow associated with the application. In this way, a packet from another data flow may be chosen for ejection from the L4S queue rather than a packet from the data flow associated with the application that has selectively used L4S in the past. The disclosure of U.S. Non-Provisional Application No. 18 / 744,496, filed June 14, 2024 (titled “DYNAMIC SYSTEMS AND METHODS FOR MEDIA-AWARE LOW- TO ULTRA-LOW LATENCY, REAL-TIME TRANSPORT PROTOCOL CONTENT DELIVERY”), is hereby incorporated by reference herein in its entirety. See, also, the aforementioned U.S. Non-Provisional Application No. 18 / 667,655 (titled “INTELLIGENT APPLICATION PRIORITY PACKET DELIVERY CONTROL”).

[0056] The amount of credit received by a data flow for past non-resource grabbing behavior of the sending application may be proportional to the past behavior. For example, the queue assigner may grant the preferential status for a quantity of data packets from the sender in the current data flow based on the quantity or percentage of packets in the most recent data flow of the sender that did not request L4S queue processing, or based on the quantity or percentage of packets in one or more data flows of the sender within a past period of time that did not request L4S queue processing.

[0057] The preferential status granted, may be in the form of reducing a queueing score decay timer, automatically being added to the L4S queue, or one or more of the other rewards discussed herein. For example, a database accessed by a queue assigner device or module may keep track of how many packets associated with a sending application have been given credit over a period of time, what percentage of packets associated with a sending application have been given credit over a period of time, how many packets of a data flow have beengiven credit, over a period of time, and / or what percentage of packets associated with a data flow have been given credit over a period of time.

[0058] In some embodiments, if one or more of the prior preferential queue request conditions is met, then the queue assigner may grant the preferential status for the flow of network traffic, which may be thought of as a kind of credit to the application’s current request for the preferential queue The preferential status for the request may take the form of the queue assigner reducing a timer expiration value for the congestion. In this way, requests for preferential queue processing of network traffic for this application would be given preference over those of other applications because a queueing score for the network traffic of this application would be given a lower value, or a queueing score value with a faster expiration time. The queueing score may be a value that indicates, approximates, or serves as a proxy for a level of network traffic congestion for processing by the network node. For example, the queueing score may be calculated as:(the rate of data flow) x (the level of congestion) where both the rate of data flow and the level of congestion is measured at the instant each packet of the data flow arrives.

[0059] In an embodiment, if the queueing score is zero or below a threshold then all requested preferential queue requests may be granted since there is likely no congestion for processing by the network node.

[0060] In some embodiments, further limitations may be imposed on the “credit” given to the current application flow based on prior application flow(s).The queue assigner may limit the granting of the credit for preferential status of the current application data flow based on several factors. For example, the length of the look-back time period — the period during which resource-grabbing behavior of prior application flows is assessed— may be set depending on the value of the queueing score. As discussed, the queueing score may reflect, or serve as an approximation or proxy for, the amount of congestion at the network node that is created / caused by a particular application flow Accordingly, the queue assigner may lengthen the look-back period if the queueing score is high, making it more difficult to receive preferential treatment of the data flow.

[0061] In some embodiments, an application provider may provide network traffic flows associated with two or more applications flows (e.g., each data flow may be distinguished by its respective 5-tuple). For example, more than one application may be running between asource and a destination IP (even if port numbers and protocol field are different). By way of illustration, this may occur if a server is providing cloud gaming service and another service such as chat / messenger. In such situations, a network service provider may transfer the credits of one flow to another flow— credit accrued to a first application flow for requesting no preferential queue processing within a threshold time may be applied to a data flow sent by a second application. When a voice chat selectively uses L4S, the “credit” for nonresource grabbing behavior may be transferred to cloud gaming. In this example, a burst in cloud gaming traffic may benefit from infrequent (selective) use of L4S by the voice chat. The network node may implement this by comparing the source and destination IP addresses of different flows that are active (based on the macro timer) at any time.

[0062] In some embodiments, determining whether to provide the portion of the application flow to the network node using the preferential queue or the classic queue may also take into consideration a bandwidth of the network and a jitter associated with providing the portion of the data traffic over the network, as well as a traffic processing capacity of the network node.

[0063] A method, system, apparatus, non-transitory computer-readable medium, and means for implementing the method are disclosed for selectively enabling use of preferential resource based on past resource-requesting behavior. A method may include, for instance, by way of illustration: detecting a characteristic of network traffic transmitted by an application, wherein the characteristic indicates a request for processing via a preferential queue; identifying a prior network traffic transmission profile of the application; based at least on the prior network traffic transmission profile of the application, determining that the application has not, within a threshold time period, previously transmitted prior network traffic that indicated a prior request for the processing via the preferential queue; and based at least on the determining that the application has not, within the threshold time period, previously transmitted the prior application data flow that indicated the request for the processing via the preferential queue, providing the network traffic to the network node using the preferential queue. For example, the characteristic of network traffic transmitted by the application may be a request for expedited processing by the \ network node via the L4S queue. This request may be detected, for example, based on ECN bits of data packets of the network traffic. One or more of the operations of such a method may be performed by the device that is the network node itself as part of its queue management.

[0064] Such a method may also include: based on the determining that the application has not, within the threshold time period, previously transmitted the prior network traffic thatindicated the request for the processing via the preferential queue: computing a queueing parameter indicating a queue congestion for processing by the network node, and adjusting the queueing parameter to enable the providing the data to the network node using the preferential queue.

[0065] For example, this queueing parameter may be a queueing score calculated based on a combining (e.g., multiplying) a rate of flow of the data provided to the network node via the network and a level of congestion for processing by the network node. The adjusting the queueing parameter may entail changing a decay time of the queueing score. This may be performed using a decay timer that approximates how long it will take the congestion to dissipate at this network node. The queueing score may be computed in other ways, for example, based on the number of packets that have been received per unit of time, or the number of packets that are in one or more queues awaiting processing.

[0066] In such a method, a queueing parameter may be computed, which indicates congestion for processing by the network node, and the threshold time period may be adjusted based on the queueing parameter such that a longer threshold time period is selected when the computing of the queueing parameter indicates a more severe queue congestion.

[0067] The preferential queue may be an L4S queue or other type of low latency queue, and the network traffic transmitted by the application may be a data flow comprising data packets. The characteristic of the network traffic transmitted by the application may be detected by accessing an Explicit Congestion Notification control bit of packets of the data flow.

[0068] A queue assigner, a device separate from or integrated with the network device that processes the data flow, may determine that the application has not, within the threshold time period, previously transmitted network traffic that indicated a prior request for the processing via the preferential queue. For example, the queue assigner may be part of, or may be at, one of a cable modem termination system (CMTS), a cable modem (CM), a base station (BS) of a cellular network or a Broadband Network Gateway (BNG) of a digital subscriber line (DSL) network.

[0069] The data transmitted by the application may include video data or game rendering data.

[0070] Also contemplated is a computer-implemented method that may include, for example, identifying that a network traffic flow associated with a sender and provided via a network to a network node requested preferential queue processing; determining that a previous application data flow associated with the application and provided to the networknode requested default queue processing; based at least on the determining that the previous application flow associated with the sender provided to the network node was default queue requesting, selectively enabling use of the preferential queue by directing the network traffic flow to the preferential queue of the network node.

[0071] Also described is a method for low-latency to ultralow-latency content delivery that may entail, by way of illustration: for at least one application source or sender: based at least in part on receiving an indication of selective preferential transport of a first image unit, maintaining a priority score for the preferential transport. Based at least in part on receiving no indication of selective default transport of the first image unit, increasing the priority score; and subsequently, for the at least one source or sender: determining that present network congestion restricts access to the preferential transport of a second image unit, prioritizing preferential transport of the second image unit based at least in part on the priority score.

[0072] In addition, disclosed is a method for low latency content delivery. Such a method may include for at least one source or sender: based at least in part on receiving an indication of selective preferential transport of a first fragment of a segment of content, decreasing a priority score for the preferential transport. Based at least in part on receiving no indication of the selective preferential transport of the first fragment of the segment of the content, increasing the priority score; and subsequently, for the at least one source or sender: determining that present network congestion restricts access to the preferential transport of a second fragment, and prioritizing preferential transport of the second fragment based at least in part on the priority score.

[0073] The present invention is not limited to the combination of the elements as listed herein and may be assembled in any combination of the elements as described herein. These and other capabilities of the disclosed subject matter will be more fully understood after a review of the following figures, detailed description, and claims.Brief Description of the Drawings

[0074] The present disclosure, in accordance with one or more various embodiments, is described in detail with reference to the following figures. The drawings are provided for purposes of illustration only and merely depict typical or example embodiments. These drawings are provided to facilitate an understanding of the concepts disclosed herein and should not be considered limiting of the breadth, scope, or applicability of these concepts. Itshould be noted that for clarity and ease of illustration, these drawings are not necessarily made to scale.

[0075] FIG. 1 shows an illustrative architecture for enforcing a data cap for preferential network traffic, in accordance with some embodiments of this disclosure.

[0076] FIGS. 2A-2B show illustrative block diagrams for providing a dual queue service configuration, in accordance with some embodiments of this disclosure.

[0077] FIG. 3 shows an illustrative diagram of network traffic in a system, in accordance with some embodiments of this disclosure.

[0078] FIG. 4 shows illustrative marking of explicit congestion notification (ECN) bits, in accordance with some embodiments of this disclosure.

[0079] FIG. 5 shows an illustrative flowchart for metering and data cap policy enforcement at an edge router or WAN networking equipment.

[0080] FIG. 6 shows an illustrative flowchart . 6 shows an illustrative flowchart 600 for metering and data cap policy enforcement at the subscriber / home gateway level, in accordance with some embodiments of this disclosure.

[0081] FIG. 7 shows a block diagram for an illustrative architecture of a 5G mobile network, in accordance with some embodiments of this disclosure.

[0082] FIG. 8 shows a block diagram for an illustrative architecture of a digital subscriber line access multiplexer (DSLAM) for an IP service provider, in accordance with some embodiments of this disclosure.

[0083] FIGS. 9A-9E are illustrative block diagrams and flowcharts for transmitting data based on L4S markings in data packets, in accordance with some embodiments of this disclosure.

[0084] FIGS. 10-11 show illustrative devices and systems for enforcing a data cap for preferential network traffic, in accordance with some embodiments of this disclosure.

[0085] FIG. 12 is a flowchart of a detailed illustrative process for enforcing a data cap for preferential network traffic, in accordance with some embodiments of this disclosure.

[0086] FIG. 13 shows an example of selectively treating portions of data as preferential or non-preferential during a network session, in accordance with some embodiments of this disclosure.

[0087] FIG. 14 shows an illustrative architecture of a system for selectively treating portions of data as preferential or non-preferential during a network session, in accordance with some embodiments of this disclosure.

[0088] FIG. 15 shows an illustrative architecture of a system for selectively treating portions of data as preferential or non-preferential during a network session, in accordance with some embodiments of this disclosure.

[0089] FIG. 16 shows an illustrative architecture for a system for selectively transmitting data using a preferential queue or a non-preferential queue, during a multiplayer session for multiplayer gaming, in accordance with some embodiments of this disclosure.

[0090] FIG. 17 is an illustrative example of using SLAM on an XR headset (e.g., indoors or outdoors), enabling a virtual objects to interact at close range with the physical world, to facilitate relatively more precise localization, in accordance with some embodiments of this disclosure.

[0091] FIG. 18 is an example of using SLAM on a device (e.g., indoors or outdoors ) to enable a virtual object to interact at a large distance with the physical world, to facilitate relatively less precise localization, in accordance with some embodiments of this disclosure.

[0092] FIG. 19A is an example of using SLAM to transmit network traffic associated with a robotic lawn mower when there are physical objects close to the robotic lawn mower, in accordance with some embodiments for this disclosure.

[0093] FIG. 19B is an example of using SLAM to transmit network traffic associated with a robotic lawn mower when there are no physical objects, or minimal physical objects, close to the robotic lawn mower, in accordance with some embodiments for this disclosure.

[0094] FIG. 20 is an illustrative architecture for a system having interfaces with SLAM functionalities and / or sub-functionalities, for enabling L4S in a XR device using cloud-based SLAM, in accordance with some embodiments for this disclosure.

[0095] FIG. 21 is an illustrative architecture for a system having interfaces with SLAM functionalities and / or sub-functionalities, for enabling L4S in a robotic device using cloudbased SLAM, in accordance with some embodiments for this disclosure.

[0096] FIG. 22 is an illustrative architecture for a system for selectively treating portions of network traffic preferentially during a virtual conference, in accordance with some embodiments for this disclosure.

[0097] FIG. 23 A-23B show illustrative examples for selectively treating portions of network traffic preferentially when remotely controlling construction equipment, in accordance with some embodiments for this disclosure.

[0098] FIG. 24 shows an illustrative architecture for a system for prioritized stream delivery with L4S based on remote vehicle control with proximity of objects, in accordance with some embodiments of this disclosure.

[0099] FIG. 25 shows a flowchart of an illustrative process for selectively enabling L4S for a video streaming request based on whether it is for a live view of an on-premises camera, in accordance with some embodiments of this disclosure.

[0100] FIG. 26 shows a flowchart of an illustrative process for selectively enabling L4S for a video streaming request based on priority metadata of the video, in accordance with some embodiments of this disclosure.

[0101] FIG. 27 shows an illustrative example of selectively treating network traffic as preferential in the context of placing wagers over a network, in accordance with some embodiments of this disclosure.

[0102] FIG. 28 is a flowchart of a detailed illustrative process for preferential treatment of portions of network traffic, in accordance with some embodiments of this disclosure.

[0103] FIG. 29 is an illustration of queue assignment of data packets of network traffic sent by an application, according to an example of an aspect of some embodiments of the present disclosure;

[0104] FIG. 30 is a flowchart showing processing in accordance with an aspect of some embodiments of the present disclosure;

[0105] FIG. 31 shows pseudocode for quantifying network traffic congestion and its decay, in accordance with an aspect of some embodiments of the present disclosure;

[0106] FIG. 32 shows variables that may be used for pseudocode described, in accordance with an aspect of some embodiments of the present disclosure;

[0107] FIGS. 33A-33D show pseudocode for determining whether to assign data traffic to the preferential queue or to the non-preferential queue, in accordance with an aspect of some embodiments of the present disclosure; and

[0108] FIG. 34 is a flowchart showing processing in accordance with an aspect of some embodiments of the present disclosure.

[0109] FIG. 35 is a flowchart that shows processing for providing credit for priority access to the preferential resource in accordance with an aspect of some embodiments of the present disclosure;

[0110] FIG. 36 is a flowchart showing processing for selective enablement of preferential processing of application traffic in accordance with an aspect of some embodiments of the present disclosure.

[0111] FIG. 37 is a flowchart showing processing for assigning credit to a third application flow in accordance with an aspect of some embodiments of the present disclosure.

[0112] The drawings are intended to depict only typical aspects of the subject matter disclosed herein, and therefore should not be considered as limiting the scope of the disclosure. Those skilled in the art will understand that the structures, systems, devices, and methods specifically described herein and illustrated in the accompanying drawings are nonlimiting exemplary embodiments and that the scope of the present invention is defined solely by the claims.Detailed Description

[0113] FIG. 1 shows an illustrative architecture for a system 100 for enforcing a data cap for preferential network traffic, in accordance with some embodiments of this disclosure. System 100 may comprise service provider network 102, physical location 104 (e.g., a home of user 110, a place of business, a school, or any other suitable location, or any combination thereof), networking equipment 106 and 108 (e.g., a modem, router, switch, gateway, wireless access point, mesh access point, extender, hub, and / or any other suitable networking equipment), devices 112 and 114. In some embodiments, modem 106, router 107 and / or networking equipment 122 may comprise a traffic analysis module 121 and / or a traffic flow identification and policy enforcement module (TIPE) module 123. In some embodiments, cloud server 124 comprises a traffic generating application 125. System 100 may comprise any suitable combination of hardware and / or software to provide the functionalities described herein.

[0114] Service provider network 102 may include, for example, any suitable software and / or hardware (e.g., networking equipment, servers, and / or databases) and / or any suitable infrastructure (e.g., physical cable transmission lines, fiber-optic transmission channels or mediums or channels, satellites) to provide core, regional, access networks and / or backhaul (and / or any other suitable portion of the network) of one or more Internet service providers (ISPs), to facilitate a telecommunications network. In some embodiments, the ISP may be provided by a business or other organization that provides access to the Internet for a fee. For example, service provider network 102 may correspond to or comprise a wide area network (WAN), to facilitate Internet connectivity (or connectivity over any other suitable public or private network) between networked devices worldwide or over any other suitable geographic region or location(s), to enable such devices to exchange information and resources. In some embodiments, a WAN or service provider network 102 may be used to connect LANs (and / or other types of communication) to enable electronic communications between remotely located devices. In the example of FIG. 1, the local area network (LAN),e.g., a small-scale network for data exchange between a group of computers or other devices at a single location, provided at location 104 by way of networking equipment 106 and / or 108, may not be considered as part of the WAN provided by service provider network 102. Service provider network 102 may provide broadband, high bandwidth Internet access.

[0115] In some embodiments, networking equipment 122 and cloud server 124 may be located remote from location 104. The devices, servers, and networking equipment of system 100 may communicate over a wired connection and wireless connection. For example, devices 112, 114 and networking equipment 106 and 108 may be equipped with antennas for transmitting and receiving electromagnetic signals at frequencies within the electromagnetic spectrum, e.g., radio frequencies, to communicate with each other over a network in a localized area. The network within location 104 may correspond to, e.g., a wireless fidelity (Wi-Fi) network, such as, for example, 802.1 In, 802.1 lac, 802.1 lax, Wi-Gig / 802.1 lad, 802.11 (Wi-Fi 7) at a fronthaul of a telecommunications network, to provide wireless networking technology allowing electronic devices to connect to one another and / or the Internet from a shared network access point.

[0116] The devices of system 100 may communicate over a wired LAN and / or may communicate wirelessly over a wireless LAN (WLAN) and to transmit data to and receive data from the Internet and may be present within an effective coverage area of the localized network. The Internet is a global system of interconnected computer networks and devices employing common communication protocols, e.g., the transmission control protocol (TCP), user datagram protocol (UDP) and the Internet protocol (IP) in the TCP / IP or UDP / IP suite.

[0117] Router 108 may be configured to forward or route data packets from the Internet connection, received by way of modem 106, to devices within the localized network of system 100 and receive data packets from such devices. In some embodiments, router 108 may include a built-in modem to provide access to the Internet for the household (e.g., received by way of cable or fiber connections included in backhaul portions of a telecommunications network), built-in switches or hubs to deliver data packets to the appropriate devices within the Wi-Fi network, built-in access points to enable devices to wirelessly connect to the Wi-Fi network, and / or system 100 may include one or more standalone modems, switches, routers and access points. In some embodiments, modem 106 and / or router 108 may be leased from and / or installed at location 104 (e.g., the customer’s premises) by the ISP as part of a managed Wi-Fi install, to give service network provider 102 visibility into LAN and WAN network traffic associated with data transmitted to or receive from modem 106 of location 104.

[0118] In some embodiments, one or more applications and / or media assets may be provided to user 110 by way of wired or wireless signals transmitted through the LAN at location 104. For example, user 110 may be playing a video game 116 (e.g., “Call of Duty”) via smart television 116 and / or a video game console, each of which may be connected to the Internet via the LAN within location 104 to provide such video game. As another example, tablet 114 may simultaneously be connected to the Internet via the LAN to provide a video conferencing application (e.g., Zoom) 118 to user 110.

[0119] In some embodiments, devices 112 and 114 may be, for example a headset; a mobile device such as, for example, a smartphone or tablet; a laptop computer; a personal computer; a desktop computer; a smart television; a smart watch or wearable device; smart glasses; extended reality (XR) head-mounted display (HMD); a stereoscopic display; a wearable camera; XR glasses; XR goggles; a near-eye display device; a robot; an autonomous cleaning device; or any other suitable user equipment or device capable of connecting to the Internet or other suitable network; or any combination thereof.

[0120] In some embodiments, traffic analysis module 121 and TIPE module 123 may be implemented in conjunction to achieve one of more of the functionalities described herein. For example, traffic analysis module 121 may compute results of network traffic analysis in the LAN and / or WAN in real-time for a subscriber home (e.g., at location 104) and / or perform a metering function with respect to preferential and / or non-preferential network traffic, and TIPE module 123 may be configured to apply a policy when a data cap is met, based on an indication received from traffic analysis module 121.

[0121] FIGS. 2A-2B show illustrative block diagrams for providing a dual queue service configuration, in accordance with some embodiments of this disclosure. System 100 may provide (e.g., in the WAN) a queue for low latency (e.g., L4S) network traffic and a queue for classic traffic, based at least in part using networking equipment 122 of FIG. 1. For example, the low latency queue of system 100 may be associated with low latency service flow 206 and low latency service flow 210, and the classic queue of system 100 may be associated with classic queue of service flow 208 and 212, as discussed in more detail in as White et al., “Low Latency DOCSIS: Technology Overview,” Cable Labs, 2019 Fall Technical Forum SCTE-ISBE (hereinafter “White et al.), the contents of which are hereby incorporated by reference herein in their entirety. A downstream aggregate service flow (ASF) over service flow 206, 210 between subscriber 204 and service provider network 202 may include low latency service flow 206 and classic service flow 210, and an upstream ASF between subscriber 204 and service provider network 202 may include low latency serviceflow 210 and classic service flow 212. In some embodiments, networking equipment modem 106, modem 106 and / or router 108 108 and / or other networking equipment may provide one or more buffers or other suitable memory at which the low latency queue and the classic queue may be stored. In some embodiments, system 100 may employ per-flow queues and / or per-flow AQMs, in addition to or in the alternative to dual-queuing.

[0122] In some embodiments, service provider network 202 may correspond to service provider network 102 of FIG. 1, networking equipment modem 106 and / or router 108, and subscriber 204 may correspond to networking equipment 106, 108 of user 110 at location 104 of FIG. 1. FIG. 2B may correspond to an architecture for a cellular network, and service provider network 214 which may correspond to service provider network 102 of FIG. 1, and client device or user equipment 216 may correspond to service provider network 102 of FIG. 1, networking equipment 122 and / or cloud server 124.

[0123] L4S provides an end-to-end solution to provide certain traffic flows, such as, for example, gaming or voice, with reduced latency. With L4S, the data source and / or data recipient may execute congestion control algorithms to efficiently utilize available capacity while minimizing latency and packet loss, where the data source may use congestion feedback received from the recipient to optimize data transmission. With L4S, the header of an IP packet may indicate, via an explicit congestion notification (ECN), whether the IP packet supports L4S and whether congestion is being experienced, e.g., marking specific packets as having queuing delay that exceeds a threshold. L4S may be implemented at the transport layer by the service provider network and / or application service providers at client and server. In some embodiments, L4S may be enabled by operating system (OS) providers, such as, for example, Google and Apple.

[0124] As stated in RFC 9330, “The Dual-Queue Coupled AQM ... acts like a 'semi- permeable' membrane that partitions latency but not bandwidth. As such, the two queues are for transitioning from Classic to L4S behaviour, not bandwidth prioritization.” RFC 9330 further states that “Two separate queues are used to isolate L4S queuing delay from the larger queue that Classic traffic needs to maintain full utilization” and “The two queues act as if they are a single pool of bandwidth in which flows of either type get roughly equal throughput without the scheduler needing to identify any flows.”

[0125] RFC 9330 further states that “the scheduler can serve the L4S queue with priority (denoted by the T' on the higher priority input), because the L4S traffic isn't offering up enough traffic to use all the priority that it is given. Therefore, for latency isolation on short timescales (sub -round-trip), the prioritization of the L4S queue protects its low latency byallowing bursts to dissipate quickly; but for bandwidth pooling on longer timescales (roundtrip and longer), the Classic queue creates an equal and opposite pressure against the L4S traffic to ensure that neither has priority when it comes to bandwidth — the tension between prioritizing L4S and coupling the marking from the Classic AQM results in approximate perflow fairness.”

[0126] As further stated in White et al., AQM can ensure that the Classic queue is not starved: “To enable the Low Latency Queue to rapidly dequeue an arrived burst of traffic, the Inter-Service-Flow scheduler gives a higher weight to the Low Latency Queue than it does to the Classic Queue. The coupling to the Low Latency AQM counterbalances the weighted scheduler by making low-latency applications leave space for Classic traffic. This ensures that the weighted scheduler does not give priority over bandwidth, as a traditional weighted scheduler would.” Further, as stated in Internet Engineering Task Force (IETF), “Dual-Queue Coupled Active Queue Management (AQM) for Low Latency, Low Loss, and Scalable Throughput (L4S),” RFC 9332, January 2023, (referred to herein as RFC 9332), the contents of which are hereby incorporated by reference herein in their entirety: “The scheduling weight of the Classic queue should be small (e.g., 1 / 16) ... if L4S traffic is over-aggressive or unresponsive, the scheduler weight for Classic traffic will at least be large enough to ensure it does not starve in the short term” and “The scheduler draining the two queues MUST give L4S packets priority over Classic, although priority MUST be bounded in order not to starve Classic traffic” and “The L4S queue has latency priority within sub-round-trip timescales, but over longer periods the coupling from the Classic to the L4S AQM ... ensures that it does not have bandwidth priority over the Classic queue.”

[0127] As described in more detail below, system 100 may be configured to deliver a customized broadband to a customer (e.g., user 110 of FIG. 1) based on received user input. For example, system 100 may orchestrate network flows such that customer-specified traffic (e.g., assuming the traffic is L4S-capable) is directed to the low latency service flow associated with the low latency queue, and / or that traffic that is not customer specified (e.g., whether L4S-capable or not L4S-capable) is directed to the classic service flow. In some embodiments, any traffic that is L4S is directed to the low latency service flow (regardless of whether the traffic is specified by the user as preferred), or only traffic that is both specified by the user and that is L4S-capable is directed to the low latency service flow. Such directing of traffic to the low latency service flow based on user input gives the user some measure of control to cause certain packets to be processed with minimal delay, e.g., by shifting a portion of network traffic indicated by the customer as high importance to a low latency service flowor dedicated quality of service (QoS) service flow while keeping the remaining traffic in the classic or default QoS service flow. For example, as part of the L4S mechanism, system 100 may facilitate L4S packets being given latency priority over classic packets for at least certain periods of time, to minimize latency for such packets. In some embodiments, system 100 may use L4S in conjunction with one or more other techniques, e.g., Differentiated services (Diffserv), to forward packets via low latency service flow at the expense of packets over the classic service flow. In some embodiments, if a particular traffic flow is not L4S-capable, based on receiving selection of a particular traffic flow by a user, system 100 may cause an ISP and / or application service provider to be informed of the request, which may cause the ISP and / or application service provider to configure the network traffic to be L4S-capable. In some embodiments, system 100 may receive or transmit API calls requesting that an ISP or application service provider enables L4S for certain network traffic, e.g., user-specified traffic.

[0128] FIG. 4 shows illustrative marking of explicit congestion notification (ECN) bits, in accordance with some embodiments of this disclosure. In some embodiments, to determine whether a packet should be assigned to a low latency service flow (e.g., 206, 210, 218 of FIGS. 2A-2B), ISPs and application service providers of low latency traffic (e.g., cloud gaming) of system 100 may mark portions of their traffic with a codepoint, e.g., a differentiated services (DiffServ) codepoint or any other suitable codepoint. This codepoint indicates the ISP’s and / or application service provider’s ability to perform scalable congestion control, e.g., to respond to a congestion notification in a graceful manner that does not aggressively reduce throughput. For example, the ISP or application service provider may use the DiffServ field information to shift a packet to the low latency service flow in a “weakest link” of the network, such as, for example, the access network. In some embodiments, the ISP or application service provider may signal congestion using an ECN field when appropriate, to produce a graceful degradation in throughput from the application service provider’s server. In some embodiments, an ISP and / or application service provider may allow a customer to indicate that network traffic to a particular device and / or for a particular application, e.g., based on a particular service type associated with the application, should be provided with latency priority, e.g., assigned to the low latency service flow.

[0129] In some embodiments, ECN may be contained within the DiffServ codepoint to indicate whether or not congestion is experienced by marking the two least-significant bits in the DiffServ in the IP header identifying a data packet. For example, the most significant six bits in the DiffServ field may contain the differentiated services code Point (DSCP) bits, andthe state of the two ECN bits indicates whether or not the packet is an ECN-capable packet and whether or not congestion has been experienced. A sender of network traffic may indicate a packet as ECN-capable or non-ECN-capable based on whether the sender is ECN- capable. If an ECN-capable packet experiences congestion at the egress queue of a switch, router, and / or other network component, such switch, router, and / or other network component may mark the packet as experiencing congestion. When the packet reaches the ECN-capable receiver (destination endpoint), the receiver echoes the congestion indicator to the sender (source endpoint) by sending a packet marked to indicate congestion, and after receiving the congestion indicator from the receiver, the source endpoint reduces the transmission rate to relieve the congestion,” as described in “Understanding CoS Explicit Congestion Notification,” Juniper Product and Release Support, November 29, 2023, the contents of which are hereby incorporated by reference herein in their entirety.

[0130] As shown in FIG. 4, in some embodiments, two ECN bits in the DiffServ field provide four codes that determine if a packet is marked as an ECN-capable transport (ECT) packet, meaning that both endpoints of the transport protocol are ECN-capable, and if there is congestion experienced (CE). Historically, codes 01 and 10 had the same meaning, namely that the sending and receiving endpoints of the transport protocol are ECN-capable, and there was no difference between these codes. Recent work, however, earmarks ECT(l) as the bit pattern for L4S-capable traffic. System 100 may modify such interpretation of the ECN bits by assigning distinct meanings to ECT(0) and ECT(l) in order to designate at least two different traffic classes. In some embodiments, ECT(l), e.g., bit pattern 01, may be used to indicate L4S-capable traffic, and ECT(0), e.g., bit pattern 10, may be used to indicate that the sender is capable of receiving explicit congestion notification (though the sender may not be compliant with L4S). In some embodiments, L4S-capable traffic marked by an application server is assigned to ECT(l) and L4S-capable traffic marked by an ISP (e.g., based on customer preferences) is assigned to ECT(0). For example, ECT(0) is used as an internal reference for network traffic that is designated as preferential by the ISP rather than the application provider. In some embodiments, ISPs can independently choose which bit combination represents one of the two classes described above, or multiple ISPs can agree on the definition and / or choice of ECT(0) and ECT(l). In some embodiments, system 100 may add one or more extra bits to be added, specifically to indicate whether such a data packet having such bits was marked by an ISP operator or application service provider providing L4S enablement.

[0131] In some embodiments, the same bits that are used for designating whether the clientserver are ECN-capable may also be used for marking whether congestion is actually experienced in the network (bits 11 : CE). Thus, if a packet has the ECN bits marked as CE, then, in order to classify it as either marked by the application service provider or by the ISP (to meter the traffic accurately and apply policy), the ISP may perform a lookup of that flow identifier (ID) from a prior packet belonging to the same network traffic flow to check its traffic class, to determine whether the sender packet was marked with 10 or 01.

[0132] FIG. 5 shows an illustrative flowchart 500 for metering and data cap policy enforcement at an edge router or WAN networking equipment, in accordance with some embodiments of this disclosure. In various embodiments, the individual steps of process 500 may be implemented by one or more components of the devices, methods, and systems of FIGS. 1-4 and 6-12 and may be performed in combination with any of the other processes and aspects described herein. Although the present disclosure may describe certain steps of process 500 (and of other processes described herein) as being implemented by certain components of the devices, methods, and systems of 1-4 and 6-12, this is for purposes of illustration only, and it should be understood that other components of the devices, methods, and systems of 1-4 and 6-12 may implement those steps instead. In some embodiments, system 100 (e.g., traffic analysis module 121 and / or TIPE module 123 and / or other suitable combination of hardware and software components) may be configured to perform the method of FIG. 5.

[0133] In some embodiments, system 100 may deliver customized broadband to a customer (e.g., user 110 of FIG. 1). For example, system 100 may orchestrate network flows such that customer preferred traffic and / or certain traffic specified by an application service provider is treated as higher priority than the other traffic, such as, for example, by shifting a portion of network traffic indicated by the customer as high importance to a low latency service flow or dedicated quality of service (QoS) service flow (e.g., queue of service flow 206, 210, and / or 218 of FIGS. 2A-2B), while keeping the remaining traffic in the classic or default QoS service flow (e.g., queue of service flow 208, 212, and / or 220 of FIGS. 2A-2B).

[0134] In some embodiments, system 100 (e.g., traffic analysis module 121 and / or TIPE module 123) may be configured to enforce data caps for customer-preferred traffic or for application provider-marked traffic separately from regular (non-L4S-capable) traffic. For example, system 100 (e.g., traffic analysis module 121 and / or TIPE module 123) may perform metering of multiple different network traffic classes, e.g., customer-preferred L4S- capable traffic, application service provider-marked but non-customer-preferred L4S-capabletraffic, non-L4S-capable traffic, and / or any other suitable network traffic. In some embodiments, system 100 may perform metering of network traffic (e.g., using traffic analysis module 121 of FIG. 1) to measure the tonnage of each of these network traffic classes at the subscriber (home) level.

[0135] In the example of FIG. 5, packets in the traffic class that is L4S-capable and customer preferred may be denoted as having ECN bits ECT(0), which may be used by the ISP as an internal reference, and the packets in the traffic class that is L4S-capable marked by an application service provider, but not customer preferred, may be denoted as having ECN bits ECT(l). As shown in FIG. 5, for accurate metering of the (e.g., three) traffic classes, a router at the edge of the ISP network (e.g., executing the TIPE module 123) may perform one or more modifications to ECN bits in the DiffServ field of the IP Header.

[0136] At 502, system 100 (e.g., TIPE module 123 executing at least in part at service provider network 102) may inspect a next data packet in a queue, to check whether such data packet is marked as L4S-capable, e.g., whether ECN bits are in ECT (1) configuration. At 504, system 100 (e.g., TIPE module 123) may determine, based on the inspection at 502, whether the data packet is L4S-capable. If so, processing may proceed to 506; otherwise, processing may return to 502. At 506, if system 100 (e.g., TIPE module 123) determines, at 504, that the data packet is L4S-capable, then networking equipment (e.g., router 122 of FIG. 1) may use a flow ID, e.g., 5-tuple{IP address(source, destination), protocol, port(source, destination)} to determine whether the customer has set preferential treatment for this application flow, where such preferential treatment may have been set based on, for example, the techniques described in the above-mentioned commonly owned application (Attorney docket no. 003597-2998-101).

[0137] At 508, after determining at 506 that the customer has configured this type of network traffic flow as preferred, system 100 (e.g., TIPE module 123) subsequently checks whether a data cap (e.g., a preferential data cap) has already been reached. In some embodiments, the data cap may be enforced based at least in part on a timing policy, e.g., a daily limit, a weekly limit, a monthly limit, an annual limit, or a limit over any other suitable period of time. In some embodiments, this information may be provided via an explicit message, sent from traffic analysis module 121 sent from a home gateway, cable modem (CM), and / or router to an edge router (e.g., at which TIPE module 123 may be executed at least in part) when a data cap is attained. Based on determining, at 508, that a data cap has not been attained, system 100 (e.g., TIPE module 123) may, at 510, set the ECN bits to ECT(0),to indicate that this packet belongs to a traffic class that is designated by the ISP as preferential.

[0138] On the other hand, if system 100 (e.g., TIPE module 123) determines at 508 that the data cap has been attained, system 100 may perform one or more actions, e.g., based at least in part on a business rule in a service-level agreement (SLA) between the subscriber and the ISP. For example, if system 100 (e.g., TIPE module 123) determines, at 512, that the customer SLA permits this traffic flow to continue to use the low latency queue after meeting or exceeding the data cap, then the ECN bits may be set to ECT(0) at 514. On the other hand, after determining at 512 that the application network traffic flow is no longer allowed to be in the priority lane after the data cap has been attained, then system 100 (e.g., TIPE module 123) clears, at 516, the ECN bits by setting the ECN bits to 00, to mark the packet as belonging to a regular, non-L4S-capable flow. In some embodiments, this action is equivalent to “bleaching” the ECN bits once the data cap(s) have been met or exceeded.

[0139] At 518, after determining at 506 that the packet does not belong to a flow ID that has been marked as preferred by the customer, system 100 (e.g., TIPE module 123) may make another policy decision. Based on determining that a policy indicates that the application service provider-marked L4S-capable traffic to enter the low latency queue, system 100 (e.g., TIPE module 123) may, at 520, set the ECN bits to ECT(l) to indicate that this packet belongs to a traffic class of application service provider-marked L4S-capable traffic, albeit not customer-preferred L4S-capable traffic. Alternately, a negative determination at 518 may cause system 100 (e.g., TIPE module 123) to, at 522, “bleach” the ECN bits because an internal policy (e.g., of the ISP) is to neither meter, nor provide preferential treatment to, application service provider-marked L4S-capable traffic, but instead to treat the packet as belonging to regular, non-L4S-capable network traffic flow. At 524, if one or more uninspected packets are determined to be in the queue, processing may return to 502 to process such packet(s). In some embodiments, at 524, system 100 may process the packet, inspected at 502 and subsequently processed at 504-522, or any suitable subset thereof, to use a first queue (e.g., a low latency queue of service flow 206, 210 for L4S- capable traffic) to transmit or receive the data packet if the data packet comprises a header with ECN bits set to ECT(0) or ECT(l), or may use a second queue (e.g., classic queue of service flow 208, 212 for non-L4S-capable traffic) if the data packet comprises a header with ECN bits set to 00.

[0140] FIG. 6 shows an illustrative flowchart 600 for metering and data cap policy enforcement at the subscriber / home gateway level, in accordance with some embodiments ofthis disclosure. In various embodiments, the individual steps of process 600 may be implemented by one or more components of the devices, methods, and systems of FIGS. 1-5 and 7-14 and may be performed in combination with any of the other processes and aspects described herein. Although the present disclosure may describe certain steps of process 600 (and of other processes described herein) as being implemented by certain components of the devices, methods, and systems of FIGS. 1-5 and 7-14, this is for purposes of illustration only, and it should be understood that other components of the devices, methods, and systems of FIGS. 1-5 and 7-14 may implement those steps instead. In some embodiments, system 100 (e.g., traffic analysis module 121 and / or TIPE module 123 and / or other suitable combination of hardware and software components) may be configured to perform the method of FIG. 6.

[0141] Based on receiving one or more packets of network traffic at 602, system 100 (e.g., traffic analysis module 121 of FIG. 1) may, at 604, check whether the ECN bits of such network packet are marked as L4S-capable, e.g., comprise an indication of ECT(0) or ECT(l). Based on determining that the ECN bits of such network packet are marked, processing may proceed to 606; otherwise, processing may proceed to 608. At 606, based on determining that ECN bits of such packet are marked with ECT(l), processing may proceed to 610, or based on determining that ECN bits of the network packet are marked with ECT(0), processing may proceed to 612. At 610, since ECT(l) is marked, system 100 (e.g., traffic analysis module 121 of FIG. 1) adds “packetBytes” to an accumulated value of “byteCount” of application service provider-marked L4S-capable traffic tonnage. At 612, since ECT(0) is marked, system 100 (e.g., traffic analysis module 121 of FIG. 1) adds “packetBytes” to an accumulated value of “byteCount” of ISP-marked customer-preferred L4S-capable traffic tonnage.

[0142] On the other hand, if at 606, the system determines that the ECN bits are marked with an indication (e.g., 11) indicating congestion is being experienced, system 100 (e.g., traffic analysis module 121 of FIG. 1) uses, at 614, the network traffic flow ID 5-tuple to determine a marking of ECN bits of a prior packet belonging to the same network traffic flow ID, e.g., to determine whether this flow was application service provider-marked or ISP- marked as customer-preferred. In some embodiments, ECN bits marked with 11 is an indication that the network traffic should be treated preferentially. Based on determining, at 616, that the earlier or prior packet included a bit marking of ECT(l), processing may proceed to 618. On the other hand, based on determining, at 616, that the earlier or prior packet included a bit marking of ECT(0), processing may proceed to 620. At 618, since ECT(l) is marked, system 100 (e.g., traffic analysis module 121 of FIG. 1) adds“packetBytes” to an accumulated value of “byteCount” of application service provider- marked L4S-capable traffic tonnage. At 620, since ECT(O) is marked, system 100 (e.g., traffic analysis module 121 of FIG. 1) adds “packetBytes” to an accumulated value of “byteCount” of ISP-marked customer-preferred L4S-capable traffic tonnage. Accordingly, the metering function increments the appropriate accumulated “byteCount” variable using the “packetBytes” of the current packet.

[0143] If system 100 (e.g., traffic analysis module 121) determines at 604 that the ECN bits are not marked as L4S-capable, processing may proceed to 608. For example, if the ECN bits in the packet indicate that it is not L4S-capable traffic, then the “byteCount” of the non-L4S traffic tonnage is incremented by the ‘packetBytes.’” At 622, system 100 (e.g., traffic analysis module 121) determines whether the data cap has been reached (or exceeded) after adding packetBytes to the accumulated byteCount. An affirmative determination at 622 causes processing to proceed to 624; otherwise, processing may return to 602. At 624, as the accumulated value of each of the“byteCount” variables attains or exceeds the data cap value, then an indication or message is sent to the TIPE module 123 at networking equipment 122 (e.g., an edge router), and TIPE module 123, based on receiving this indication or message, may enforce a new policy that is in force within the ISP network when a data cap(s) is reached. For example, TIPE module 123 may perform one or more of the techniques described at 512-516 or 518-522 in relation to performing an action related to an SLA or policy when one or more data caps are reached.

[0144] In some embodiments, in FIGS. 5-6, an ISP may implement metering for three or more (or any other suitable number of) traffic classes, or it may implement metering for a subset of such traffic classes, and the ISP may change the ECN bits accordingly. For example, if an ISP is implementing metering for only customer-preferred traffic because it has implemented dual-queuing policies for only customer preferred traffic to be sent to the low latency queue, the ISP can “bleach” the ECN bits from all L4S-capable application service provider-marked traffic that is not customer-preferred. Conversely, if the ISP is implementing metering and dual queuing only for application service provider-marked L4S- capable traffic vs. non-L4S-capable traffic, specifically for enforcing data caps on the L4S- capable prioritized traffic, the ISP may not mark any traffic within its network and may ensure that the application service provider-marked traffic is not “bleached” by a network router. The ISP may, alternatively, implement metering for all of the three or more (or any other suitable number of) traffic classes, but implement dual queuing such that only onetraffic class is diverted to the low latency queue, or two traffic classes are diverted to the low latency queue.

[0145] In some embodiments, the ISP may also apply dual-queuing policies temporally. For example, application provider-marked L4S-capable traffic could be sent to the low latency queue (e.g., 206, 210 of FIG. 2) only if there is currently no customer-preferred L4S-capable traffic, or if the instantaneous throughput of the customer-preferred L4S-capable traffic is below a threshold.

[0146] In some embodiments, the data cap may be a preferential data cap (e.g., to which any L4S-capable traffic counted against, or to which only customer-preferred traffic is counted against or to which only application provider-marked traffic is counted against. In some embodiments, multiple data caps may be used at the same time, and different actions may be performed based on whether one or more of such data caps are reached or exceeded. For example, a combined byteCount of all L4S traffic, e.g., application provider-marked traffic and customer-preferred traffic, may be compared to a first data cap, application- provider-marked traffic may be compared to a second data cap, and customer-preferred traffic may be compared to a third data cap. As an example, if the first data cap is exceeded, certain actions (e.g., shown at 514 or 516) may be performed with respect to both application- provider-marked traffic and customer-preferred traffic; if the second data cap is exceeded, certain actions (e.g., shown at 514 or 516) may be performed with respect to application- provider-marked traffic only; and if the third data cap is exceeded, certain actions (e.g., shown at 514 or 516) may be performed with respect to customer-preferred traffic only.

[0147] FIG. 7 shows a block diagram 700 for an illustrative architecture of a 5G mobile network, in accordance with some embodiments of this disclosure. In some embodiments, system 100 may implement or otherwise be in communication with such a 5G network. As shown in FIG. 7, for 5G mobile networks, user plane function (UPF) 702 provides an interconnect point between the mobile infrastructure and the Data Network (DN) 704, e.g., encapsulation and decapsulation of general packet radio service (GPRS) tunnelling protocol for the user plane (GTPU). The UPF further provides a protocol data unit (PDU) session anchor point, which provides mobility within and between radio access technologies (RATs), including sending one or more end marker packets to the gNodeB (gNB). The UPF further provides for packet routing and forwarding, including performing the role of an uplink classifier / UU-CL (directing flows to specific data networks based on traffic matching filters) and a branching point, when acting as an intermediate UPF (I-UPF) multi-homed to more than one PDU session anchor (PSA). The UPF further provides application detection usingservice data flow (SDF) traffic filter templates or 3 -tuple (protocol, server-side IP address and port number) packet flow description (PFD) received from session management function (SMF) 706. The UPF further provides per-flow QoS handling, including transport level packet marking for uplink (UL) and downlink (DL), rate limiting and reflective QoS differentiated services codepoint (DSCP) marking on the DL. The 5G SMF 706 is a critical 5G core component that performs a fundamental role in the 5G Service-Based Architecture (SB A). The SMF is primarily responsible for interacting with the decoupled data plane; creating, updating, and removing Protocol Data Unit (PDU) sessions; and managing session context with the UPF.

[0148] In some embodiments, the techniques described herein for enforcing data caps for preferential traffic may be enforced at an ingest point of a 5G network, which is the 5G UPF, part of the 5G core’s SMF leveraging the N4 interface, as shown in FIG. 7. The SMF interfaces, along with the policy control function (PCF) 708, as part of the mobile core, which may be leveraged to provide the L4S management and enforcement. The PCF’s functions may include providing policy rules for control plane functions (e.g., network slicing, roaming, and mobility management); accessing subscription information for policy decisions made by a unified data repository (UDR); activating policy and charging control (PCC) rules for a PDU session in the SMF; and / or providing transparency and control over the consumption of network resources during real-time service delivery. In some embodiments, one or more of the policy control function or the user plane function may be used to set or monitor data cap(s).

[0149] FIG. 8 shows a block diagram 800 for an illustrative architecture of a DSLAM for an IP service provider, e.g., an ISP, in accordance with some embodiments of this disclosure. For digital subscriber line (DSL) networks, broadband remote access server (BRAS) 802 is the ingest point into the ISP’s IP network 808 from the Internet 810. BRAS 802 may be configured to enforce QoS policies; provide layer 3 connectivity and route IP traffic through an ISP’s backbone network to the Internet; aggregate circuits from one or more link access devices, such as, for example, DSLAMs 804; and / or provide layer 2 connectivity through transparent bridging and / or or permanent point-to-point protocol (PPP) sessions over Ethernet or asynchronous transfer mode (ATM) sessions, as shown at 806. In some embodiments, BRAS 802 may be used to implement the methods described herein for assigning ECN markings to bits of headers of packets of network traffic.

[0150] FIGS. 9A-9E are illustrative block diagrams and flowcharts for transmitting data based on L4S markings in data packets, in accordance with some embodiments of thisdisclosure. System 900 comprises real-time transport protocol (RTP) delivery system 902, RTP client 904, TCP delivery system 908, TCP client 914, Internet 910, and operator’s network 912. System 900 is an example of handling L4S markings for a TCP and / or UDP sender, where the sender is an application service outside of an operator’s network 912 and / or the sender is inside the operator’s network 912. RTP delivery system 902 comprises RTP sender 918 and UDP socket address 920, and RTP client 904 comprises UDP socket 922 and transmission receiver 924. TCP delivery system 908 comprises TCP sender 926 and TCP socket 928. TCP client 914 comprises TCP socket 930 and transmission receiver 932.

[0151] As shown in FIGS. 9A-9B, the techniques described herein may be performed in relation to data transmitted by or received at an RTP / UDP sender (e.g., an application service provider) outside an operator’s network. At 901, RTP sender 918 of RTP delivery system 902 formats a data packet for transmission, and at 903, RTP sender 918 codes the ECN bits for the packet with ECT(0) or ECT(l), e.g., indicative of either ISP-marked L4S-capable network traffic or application service provider-marked L4S-capable network traffic. At 905, RTP sender 918 transmits the packet of 903 as an RTP packet over UDP socket 920, and at 907, the RTP packet enters a first hop within operator’s network 912. At 909, a network device (e.g., networking equipment 122 of FIG. 1 or modem 106 and / or router 108 of FIG. 1) receives and checks the RTP packet for an ECN marking. At 911, the network device determines whether the ECT bits are set to 01, e.g., indicating that the packet is L4S-capable. If so, processing may proceed to 913; otherwise, processing may proceed to 915. At 913, the method as defined in FIG. 5 may be performed. At 915, the network device determines whether the ECT bits are set to 11, e.g., indicating that congestion is being experienced. If so, processing may proceed to 919, where RTP receiver (e.g., RTP client 904) receives the RTP packet. Otherwise, at 917, the network device sets the ECN bits to 00, e.g., marking the packet as belonging to a regular, non-L4S-capable flow prior to proceeding to 919.

[0152] At 921, RTP receiver 912 determines whether the packet data indicates an error; if so, processing proceeds to 923; otherwise processing proceeds to 925. At 923, RTP client 904 sends RTP sender 918 an RTCP packet with a retransmit request for synchronization source (SSRC) and packet number count with ECN bits equal to a received RTP packet code. At 925, RTP client 904 sends RTP sender 918 an RTCP packet with ECN bits that is equal to a received RTP packet code. After 923, at 927, RTP sender 918 receives the RTCP retransmit request pack with SSRC and a sequence number of a dropped packet, and at 928, determines whether the ECN bits are set to 00 (e.g., non-ECN-capable transport). If so, at 929, RTP sender 918 retransmits the dropped RPT packet with ECN bits 00. Otherwise, at 931, RTPsender 918 determines whether the ECN bits are set to 01 (e.g., indicative of L4S-capable transport); if so, processing proceeds to 933; otherwise, RTP delivery system 902 handles the congestion at 935, e.g., using the L4S’s dual queue and ECN mechanism. At 933, RTP delivery system 902 retransmits the dropped RTP packet with ECT bits 01.

[0153] After 925, at 937, RTP sender 918 determines whether the ECN bits are set to 00. If so, processing proceeds to 939; otherwise processing proceeds to 941. At 939, RTP sender 918 transmits a next packet with ECN bits 00, e.g., indicative of non-ECN-capable transport. At 943, having determined at 941 that the ECN bits are set to 01, RTP sender 918 transmits a next packet with ECT bits 01, e.g., indicative of L4S-capable transport. At 945, having determined at 941 that the ECN bits are not set to 01, RTP delivery system 902 handles the congestion, e.g., using the L4S’s dual queue and ECN mechanism.

[0154] As shown in FIGS. 9 A and 9C, the techniques described herein may be performed in relation to data transmitted by or received at an RTP / UDP sender (e.g., an application service provider) inside an operator’s network. As shown in FIG. 9C, steps 940, 942, and 944 may be performed in the same or similar manner as 901, 903, and 905, respectively, of FIG. 9B. At 946, a network device (e.g., router 108 of FIG. 1) check whether an ECN marking, a source IP address, and a destination IP address to determine if the packet originated inside an operator’s network. At 948, the network device may determine whether the ECN bits are set to 10. If so, processing may proceed to 950, where the method defined in FIG. 6 may be performed. Otherwise, processing may proceed to 952. Steps 952, 954, 956, 958, 960, 962, 964, 966, and 968, of FIG. 9C may be performed in a similar manner as in steps 915, 917, 919, 921, 923, 925, 927, 928, and 929, respectively. At 970, RTP sender 918 determines whether the ECN bits are set to 10 (e.g., indicative of ECN-capable transport); if so, processing proceeds to 972; otherwise, RTP delivery system 902 handles the congestion at 974, e.g., using the L4S’s dual queue and ECN mechanism. At 972, RTP delivery system 902 retransmits the dropped RTP packet with ECT bits 10. Steps 976 and 978 may be performed in a similar manner as 937 and 939 of FIG. 9B.

[0155] After 962, at 976, RTP sender 918 determines whether the ECN bits are set to 00. If so, processing proceeds to 978; otherwise processing proceeds to 980. At 978, RTP sender 918 transmits a next packet with ECN bits 00, e.g., indicative of non-ECN-capable transport. At 982, having determined at 980 that the ECN bits are set to 10, RTP sender 918 transmits a next packet with ECT bits 10, e.g., indicative of L4S-capable transport. At 984, having determined at 980 that the ECN bits are not set to 10, RTP delivery system 902 handles the congestion, e.g., using the L4S’s dual queue and ECN mechanism.

[0156] As shown in FIGS. 9A and 9D, the techniques described herein may be performed in relation to data transmitted by or received at a TCP sender (e.g., an application service provider) outside an operator’s network. At 947, TCP sender 926 of TCP delivery system 908 formats a data packet for transmission, and at 949, TCP sender 926 codes the ECN bits for the packet with ECT(0) or ECT(l), e.g., indicative of either ISP-marked L4S-capable network traffic or application service provider-marked L4S-capable network traffic. At 951, TCP sender 926 transmits the packet of 949 as an TCP packet over TCP socket 928, and at 953, the TCP packet enters a first hop within operator’s network 912. At 955, a network device (e.g., networking equipment 122 of FIG. 1 or router 108 or cable modem 106 of FIG. 1) receives and checks the TCP packet for an ECN marking. At 957, the network device determines whether the ECT bits are set to 01, e.g., indicating that the packet is L4S-capable. If so, processing may proceed to 959; otherwise, processing may proceed to 961. At 959, the method as defined in FIG. 5 may be performed. At 961, network device determines whether the ECT bits are set to 11, e.g., indicating that congestion is being experienced. If so, processing may proceed to 965, where TCP receiver (e.g., TCP client 914) receives the TCP packet. Otherwise, at 963, the network device sets the ECN bits to 00, e.g., marking the packet as belonging to a regular, non-L4S-capable flow prior to proceeding to 965.

[0157] At 967, TCP receiver 914 determines whether the packet data indicates an error; if so, processing proceeds to 969; otherwise processing proceeds to 971. At 969, TCP receiver or client 914 sends TCP sender 926 a NACK packet with ECN bits equal to a received TCP packet code. At 971, TCP client 914 sends TCP sender 926 an ACK packet with ECN bits equal to the received packet code. At 973, TCP sender 926 receives NACK packet for the previously transmitted packet.

[0158] At 975, TCP delivery system 908 determines whether the ECN bits are set to 00 (e.g., non-ECN-capable transport). If so, at 977, TCP sender 926 retransmits the last transmit TCP packet with ECN bits 00. Otherwise, at 979, TCP sender 926 determines whether the ECN bits are set to 01 (e.g., indicative of L4S-capable transport); if so, processing proceeds to 980; otherwise, TCP delivery system 908 handles the congestion at 981, e.g., using the L4S’s dual queue and ECN mechanism. At 980, TCP sender 926 retransmits the last transmit TCP packet with ECN bits 00.

[0159] After 971, at 983, upon determining that ECN bits are set to 00, processing proceeds to 985. Otherwise, processing proceeds to 987. At 985, TCP sender 926 transmits a next packet with ECN bits 00, e.g., indicative of non-ECN-capable transport. At 989, having determined at 987 that the ECN bits are set to 01, TCP sender 926 transmits a next packetwith ECT bits 01, e.g., indicative of L4S-capable transport. At 991, having determined at 987 that the ECN bits are not set to 01, TCP delivery system 908 handles the congestion, e.g., using the L4S’s dual queue and ECN mechanism.

[0160] As shown in FIGS. 9 A and 9E, the techniques described herein may be performed in relation to data transmitted by or received at a TCP sender (e.g., an application service provider) outside an operator’s network. As shown in FIG. 9E, steps 947, 949, and 951 may be performed as in FIG. 9D. At 986, a network device (e.g., router 108 and / or modem 106 of FIG. 1) may check the TCP packet for an ECN marking, a source IP address, and a destination IP address to determine whether such packet originated inside the operator’s network. If so, processing may proceed to 988, where the network device may determine whether the ECT bits are set to 10. If so, processing may proceed to 990, where the method shown in FIG. 6 may be performed. A negative determination at 988 causes processing to proceed to 961. Steps 961, 963, 965, 967, 969, 971, 973, 975, and 977 may be performed in a similar manner as in FIG. 9D. At 992, upon determining that ECT bits are set to 10 in the packet, processing may proceed to 980; otherwise, processing may proceed to 981. Steps 980. 981, 983, and 985 may be performed in a similar manner as in FIG. 9E. At 994, upon determining that the ECT bits are set to 10, e.g., indicating that the packet has an attribute of being an ECN-capable transport, processing may proceed to 995, where TCP sender 926 transmits a next packet with ECT bits 10, e.g., to TCP client 914; otherwise, a negative determination at 994 causes processing to proceed to 996. At 996, upon determining that the ECT bits are set to 11, e.g., indicating congestion experienced, TCP delivery system 908 handles the congestion, e.g., using the L4S’s dual queue and ECN mechanism. Otherwise, at 887, upon determining that the ECT bits are set to 11, TCP delivery system 908 transmits a next packet with ECN bits sent to 00.

[0161] L4S is a relatively new standard. Network equipment manufacturers such as commercial router companies, DOCSIS companies, and mobile equipment companies have started supporting L4S in their equipment. Consumer electronics companies are also starting to provide support in consumer grade routers, and operating systems and other software are starting to offer L4S support through APIs. However, some equipment along the path between the sender and receiver, and / or the receiver, may not provide L4S support. As an example, a TCP or UDP packet may be marked by an application service provider’s sender and transmitted to the client device. For example, if the L4S ECN bits are dropped or the marking is lost, the receiver’s TCP ACK or NACK response is not L4S marked. The same is true for RTP or real-time control transport protocol (RTCP) packets. For example, if an RTPpacket is marked as L4S and transmitted over the network and that marking is lost, the L4S ECN marking is not included in the RTCP response. Today, the sender will continue marking the L4S packets to be sent to that receiver. If an RTP packet is sent from a sender to a receiver and the ECN L4S marking is lost, the RTCP response packet from the sender will not be L4S-ECN marked.

[0162] For example, if the packet sender receives a response packet that is not ECN marked, this indicates that, at any hop along the route to the client, at least one network equipment the packet traversed did not support the ECN markings, and / or indicates that the client device does not support ECN. As soon as a sender receives a response with no ECN marking, the sender may disable the ECN marking for any future packets. This reduces the possibility of network congestion for L4S traffic flows and provides an elegant solution for a graceful way of the sender handling the network operator’s remarking of L4S-transmitted packets due to the data cap enforcement.

[0163] FIGS. 10-11 show illustrative devices, systems, servers, and related hardware for enforcing a data cap for preferential network traffic, in accordance with some embodiments of this disclosure. FIG. 10 shows generalized embodiments of illustrative computing devices 1000 and 1001, which may correspond to, e.g., a smart phone; a tablet; a laptop computer; a personal computer; a desktop computer; a smart television; a smart watch or wearable device; smart glasses; a stereoscopic display; a wearable camera; virtual reality (VR) glasses; VR goggles; a stereoscopic display; augmented reality (AR) glasses; an AR HMD; a VR HMD; or any other suitable computing device; or any combination thereof. In another example, computing device 1001 may be a user television equipment system or device. In some embodiments, computing devices 1000 and 1001 may correspond to, e.g., device 112 or device 114 of FIG. 1.

[0164] User television equipment device 1001 may include set-top box 1015. Set-top box 1015 may be communicatively connected to microphone 1016, Audio output equipment (e.g., speaker or headphones 1014), and display 1012. In some embodiments, microphone 1016 may receive audio corresponding to a voice of a user providing input. In some embodiments, display 1012 may be a television display or a computer display. In some embodiments, set- top box 1015 may be communicatively connected to user input interface 1010. In some embodiments, user input interface 1010 may be a remote-control device. Set-top box 1015 may include one or more circuit boards. In some embodiments, the circuit boards may include control circuitry, processing circuitry, and storage (e.g., RAM, ROM, hard disk, removable disk, etc.). In some embodiments, the circuit boards may include an input / outputpath. More specific implementations of computing devices are discussed below in connection with FIG. 11. In some embodiments, computing device 1000 may comprise any suitable number of sensors (e.g., gyroscope or accelerometer, etc.), and / or a GPS module (e.g., in communication with one or more servers and / or cell towers and / or satellites) to ascertain a location of computing device 1000. In some embodiments, computing device 1000 comprises a rechargeable battery that is configured to provide power to the components of the device.

[0165] Each one of computing device 1000 and computing device 1001 may receive content and data via input / output (VO) path 1002. VO path 1002 may provide content (e.g., broadcast programming, on-demand programming, Internet content, content available over a local area network (LAN) or wide area network (WAN), and / or other content) and data to control circuitry 1004, which may comprise processing circuitry 1006 and storage 1008. Control circuitry 1004 may be used to send and receive commands, requests, and other suitable data using VO path 1002, which may comprise VO circuitry. VO path 1002 may connect control circuitry 1004 (and specifically processing circuitry 1006) to one or more communications paths (described below). VO functions may be provided by one or more of these communications paths but are shown as a single path in FIG. 10 to avoid overcomplicating the drawing. While set-top box 1015 is shown in FIG. 3 for illustration, any suitable computing device having processing circuitry, control circuitry, and storage may be used in accordance with the present disclosure. For example, set-top box 1015 may be replaced by, or complemented by, a personal computer (e.g., a notebook, a laptop, a desktop), a smartphone (e.g., computing device 1000), an XR device; a tablet; a network-based server hosting a user-accessible client device; a non-user-owned device; any other suitable device; or any combination thereof.

[0166] Control circuitry 1004 may be based on any suitable control circuitry such as processing circuitry 1006. As referred to herein, control circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or supercomputer. In some embodiments, control circuitry may be distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, control circuitry 1004 executes instructions for the system (e.g., system 100 of FIG. 1) or application stored in memory (e.g., storage1008). Specifically, control circuitry 1004 may be instructed by the system or application to perform the functions discussed above and below. In some implementations, processing or actions performed by control circuitry 1004 may be based on instructions received from the system or application.

[0167] In client / server-based embodiments, control circuitry 1004 may include communications circuitry suitable for communicating with a server or other networks or servers. The system or application may be a stand-alone application implemented on a device or a server. The system or application may be implemented as software or a set of executable instructions. The instructions for performing any of the embodiments discussed herein of the system or application may be encoded on non-transitory computer-readable media (e.g., a hard drive, random-access memory on a DRAM integrated circuit, read-only memory on a BLU-RAY disk, etc.). For example, the instructions may be stored in storage 1008 and executed by control circuitry 1004 of a computing device 1000.

[0168] In some embodiments, the system or application may be a client / server application where only the client application resides on device 1000 (e.g., device 112 or 114), and a server application resides on an external server (e.g., server 1104). For example, the system or application may be implemented partially as a client application on control circuitry 1004 of device 1000 and partially on server 1104 as a server application running on control circuitry 1111. Server 1104 may be a part of a local area network with one or more of computing devices 1000, 1001 or may be part of a cloud computing environment accessed via the Internet. In a cloud computing environment, various types of computing services for performing searches on the Internet or informational databases, providing video communication capabilities, providing storage (e.g., for a database) or parsing data are provided by a collection of network-accessible computing and storage resources (e.g., server 1104 and / or an edge computing device), referred to as “the cloud.” Device 1000 may be a cloud client that relies on the cloud computing capabilities from server 1104 to determine whether processing (e.g., at least a portion of virtual background processing and / or at least a portion of other processing tasks) should be offloaded from the mobile device and facilitate such offloading. When executed by control circuitry of server 1104, the system or application may instruct control circuitry 1111 to perform processing tasks for the client device and facilitate enforcement of data cap and / or metering. The client application may instruct control circuitry 1004 to determine whether processing should be offloaded. In some embodiments, data structure 700, 702, 704 of FIGS. 7A-7C may be located at server 1104 and / or database 1105 and / or at computing device 1107, 1108 and / or 1110.

[0169] Control circuitry 1004 may include communications circuitry suitable for communicating with a server, edge computing systems and devices, a table or database server, or other networks or servers The instructions for carrying out the above-mentioned functionality may be stored on a server (which is described in more detail in connection with FIG. 11. Communications circuitry may include a cable modem, an integrated services digital network (ISDN) modem, a digital subscriber line (DSL) modem, a telephone modem, Ethernet card, or a wireless modem for communications with other equipment, or any other suitable communications circuitry. Such communications may involve the Internet or any other suitable communication networks or paths (which is described in more detail in connection with FIG. 11). In addition, communications circuitry may include circuitry that enables peer-to-peer communication of computing devices, or communication of computing devices in locations remote from each other (described in more detail below).

[0170] Memory may be an electronic storage device provided as storage 1008 that is part of control circuitry 1004. As referred to herein, the phrase “electronic storage device” or “storage device” should be understood to mean any device for storing electronic data, computer software, or firmware, such as random-access memory, read-only memory, hard drives, optical drives, digital video disc (DVD) recorders, compact disc (CD) recorders, BLU-RAY disc (BD) recorders, BLU-RAY 3D disc recorders, digital video recorders (DVR, sometimes called a personal video recorder, or PVR), solid state devices, quantum storage devices, gaming consoles, gaming media, or any other suitable fixed or removable storage devices, and / or any combination of the same. Storage 1008 may be used to store various types of content described herein as well as the system or application data described above. Nonvolatile memory may also be used (e.g., to launch a boot-up routine and other instructions). Cloud-based storage, described in more detail in relation to FIG. 11, may be used to supplement storage 1008 or instead of storage 1008.

[0171] Control circuitry 1004 may include video generating circuitry and tuning circuitry, such as one or more analog tuners, one or more MPEG-2 decoders or MPEG-2 decoders or decoders or HEVC decoders or any other suitable digital decoding circuitry, high-definition tuners, or any other suitable tuning or video circuits or combinations of such circuits. Encoding circuitry (e.g., for converting over-the-air, analog, or digital signals to MPEG or HEVC or any other suitable signals for storage) may also be provided. Control circuitry 1004 may also include scaler circuitry for upconverting and downconverting content into the preferred output format of computing device 1000. Control circuitry 1004 may also include digital-to-analog converter circuitry and analog-to-digital converter circuitry for convertingbetween digital and analog signals. The tuning and encoding circuitry may be used by computing device 1000, 1001 to receive and to display, to play, or to record content. The tuning and encoding circuitry may also be used to receive video communication session data. The circuitry described herein, including for example, the tuning, video generating, encoding, decoding, encrypting, decrypting, scaler, and analog / digital circuitry, may be implemented using software running on one or more general purpose or specialized processors. Multiple tuners may be provided to handle simultaneous tuning functions (e.g., watch and record functions, picture-in-picture (PIP) functions, multiple-tuner recording, etc.). If storage 1008 is provided as a separate device from computing device 1000, the tuning and encoding circuitry (including multiple tuners) may be associated with storage 1008.

[0172] Control circuitry 1004 may receive instruction from a user by way of user input interface 1010. User input interface 1010 may be any suitable user interface, such as a remote control, mouse, trackball, keypad, keyboard, touchscreen, touchpad, stylus inputjoystick, voice recognition interface, or other user input interfaces. Display 1012 may be provided as a stand-alone device or integrated with other elements of each one of computing device 1000 and computing device 1001. For example, display 1012 may be a touchscreen or touch- sensitive display. In such circumstances, user input interface 1010 may be integrated with or combined with display 1012. In some embodiments, user input interface 1010 includes a remote-control device having one or more microphones, buttons, keypads, any other components configured to receive user input or combinations thereof. For example, user input interface 1010 may include a handheld remote-control device having an alphanumeric keypad and option buttons. In a further example, user input interface 1010 may include a handheld remote-control device having a microphone and control circuitry configured to receive and identify voice commands and transmit information to set-top box 1015.

[0173] Audio output equipment 1014 may be integrated with or combined with display 1012. Display 1012 may be one or more of a monitor, a television, a liquid crystal display (LCD) for a mobile device, amorphous silicon display, low-temperature polysilicon display, electronic ink display, electrophoretic display, active matrix display, electro-wetting display, electro-fluidic display, cathode ray tube display, light-emitting diode display, electroluminescent display, plasma display panel, high-performance addressing display, thin- film transistor display, organic light-emitting diode display, surface-conduction electronemitter display (SED), laser television, carbon nanotubes, quantum dot display, interferometric modulator display, or any other suitable equipment for displaying visual images. A video card or graphics card may generate the output to the display 1012. Audiooutput equipment 1014 may be provided as integrated with other elements of each one of computing device 1000 and computing device 1001 or may be stand-alone units. An audio component of videos and other content displayed on display 1012 may be played through speakers (or headphones) of audio output equipment 1014. In some embodiments, audio may be distributed to a receiver (not shown), which processes and outputs the audio via speakers of audio output equipment 1014. In some embodiments, for example, control circuitry 1004 is configured to provide audio cues to a user, or other audio feedback to a user, using speakers of audio output equipment 1014. There may be a separate microphone 1016 or audio output equipment 1014 may include a microphone configured to receive audio input such as voice commands or speech. For example, a user may speak letters, words, terms, or numbers that are received by the microphone and converted to text by control circuitry 1004. In a further example, a user may voice commands that are received by a microphone and recognized by control circuitry 1004. Camera 1018 may be any suitable video camera integrated with the equipment or externally connected. Camera 1018 may be a digital camera comprising a charge-coupled device (CCD) and / or a complementary metal-oxide semiconductor (CMOS) image sensor. Camera 1018 may be an analog camera that converts to digital images via a video card.

[0174] The system or application may be implemented using any suitable architecture. For example, it may be a stand-alone application wholly implemented on each one of computing device 1000 and computing device 1001. In such an approach, instructions of the application may be stored locally (e.g., in storage 1008), and data for use by the application is downloaded on a periodic basis (e.g., from an out-of-band feed, from an Internet resource, or using another suitable approach). Control circuitry 1004 may retrieve instructions of the application from storage 1008 and process the instructions to provide the functionality, and generate any of the displays, discussed herein. Based on the processed instructions, control circuitry 1004 may determine what action to perform when input is received from user input interface 1010. For example, movement of a cursor on a display up / down may be indicated by the processed instructions when user input interface 1010 indicates that an up / down button was selected. An application and / or any instructions for performing any of the embodiments discussed herein may be encoded on computer-readable media. Computer-readable media includes any media capable of storing data. The computer-readable media may be non- transitory including, but not limited to, volatile and non-volatile computer memory or storage devices such as a hard disk, floppy disk, USB drive, DVD, CD, media card, register memory, processor cache, Random Access Memory (RAM), etc.

[0175] Control circuitry 1004 may allow a user to provide user profile information or may automatically compile user profile information. For example, control circuitry 1004 may access and monitor network data, video data, audio data, processing data, historical interactions by the user, and / or any other suitable data. Control circuitry 1004 may obtain all or part of other user profiles that are related to a particular user (e.g., via social media networks), and / or obtain information about the user from other sources that control circuitry 1004 may access. As a result, a user can be provided with a unified experience across the user's different devices.

[0176] In some embodiments, the system or application is a client / server-based application. Data for use by a thick or thin client implemented on each one of computing device 1000 and computing device 1001 may be retrieved on-demand by issuing requests to a server remote to each one of computing device 1000 and computing device 1001. For example, the remote server may store the instructions for the application in a storage device. The remote server may process the stored instructions using circuitry (e.g., control circuitry 1004) and generate the displays discussed above and below. The client device may receive the displays generated by the remote server and may display the content of the displays locally on computing device 1000. This way, the processing of the instructions is performed remotely by the server while the resulting displays (e.g., that may include text, a keyboard, or other visuals) are provided locally on computing device 1000. Computing device 1000 may receive inputs from the user via input interface 310 and transmit those inputs to the remote server for processing and generating the corresponding displays. For example, computing device 1000 may transmit a communication to the remote server indicating that an up / down button was selected via input interface 310. The remote server may process instructions in accordance with that input and generate a display of the application corresponding to the input (e.g., a display that moves a cursor up / down). The generated display is then transmitted to computing device 1000 for presentation to the user.

[0177] In some embodiments, the system or application may be downloaded and interpreted or otherwise run by an interpreter or virtual machine (run by control circuitry 1004). In some embodiments, system or application may be encoded in the ETV Binary Interchange Format (EBIF), received by control circuitry 1004 as part of a suitable feed, and interpreted by a user agent running on control circuitry 1004. For example, the system or application may be an EBIF application. In some embodiments, the system or application may be defined by a series of JAVA-based files that are received and run by a local virtual machine or other suitable middleware executed by control circuitry 1004. In some of such embodiments (e.g., thoseemploying MPEG-2, MPEG-4, HEVC or any other suitable digital media encoding schemes), the system or application may be, for example, encoded and transmitted in an MPEG-2 object carousel with the MPEG audio and video packets of a program.

[0178] FIG. 11 is a diagram of an illustrative system 1100 for providing recommendations for enforcing a data cap for preferential network traffic, in accordance with some embodiments of this disclosure. Computing devices 1107, 1108, 1110 (which may correspond to, e.g., computing device 1000 or 1001) may be coupled to communication network 1109. Communication network 1109 may be one or more networks including the Internet, a mobile phone network, mobile voice, or data network (e.g., a 5G, 4G, or LTE network), cable network, public switched telephone network, satellite network, or other types of communication network or combinations of communication networks. Paths (e.g., depicted as arrows connecting the respective devices to the communication network 1109) may separately or together include one or more communications paths, such as a satellite path, a fiber-optic path, a cable path, a path that supports Internet communications (e.g., IPTV), free- space connections (e.g., for broadcast or other wireless signals), or any other suitable wired or wireless communications path or combination of such paths. Communications with the client devices may be provided by one or more of these communications paths but are shown as a single path in FIG. 11 to avoid overcomplicating the drawing. In some embodiments, communication network 1109 may correspond to service provider network 102. Networking equipment 111 may correspond to, for example, networking equipment 106, 108, and / or 122 of FIG. 1.

[0179] LAN networking equipment 1115 may correspond to, for example, networking equipment 106 and / or 108 (e.g., router, gateway, switch, and / or modem and / or other suitable equipment) of FIG. 1. LAN networking equipment 1115 may comprise control circuitry 1121, EO path 1122, and storage 1124. WAN networking equipment 1117 may correspond to, for example, networking equipment 122 (e.g., a backbone or carrier router or CMTS other suitable networking equipment) of FIG. 1. WAN networking equipment 1117 may comprise control circuitry 11131, EO path 1132, and storage 1134.

[0180] Although communications paths are not drawn between computing devices, these devices may communicate directly with each other via communications paths as well as other short-range, point-to-point communications paths, such as USB cables, IEEE 1394 cables, wireless paths (e.g., Bluetooth, infrared, IEEE 702-1 lx, etc.), or other short-range communication via wired or wireless paths. The computing devices may also communicate with each other directly through an indirect path via communication network 1109.

[0181] System 1100 may comprise media content source 1102, one or more servers 1104, and / or one or more edge computing devices. In some embodiments, system or application may be executed at one or more of control circuitry 1111 of server 1104 (and / or control circuitry of computing devices 1107, 1108, 1110 and / or control circuitry of one or more edge computing devices). In some embodiments, media content source 1102 and / or server 1104 may be configured to facilitate network traffic between computing devices 1107, 1108, 1110 and / or any other suitable computing devices, and / or host or otherwise be in communication (e.g., over network 1109) with one or more application services. In some embodiments, cloud server 124 may correspond to, for example, media content source 1102 and / or server 1104. In some embodiments, server 1104 may perform actions to facilitate enforcement of data caps described herein.

[0182] In some embodiments, server 1104 may include control circuitry 1111 and storage 1114 (e.g., RAM, ROM, Hard Disk, Removable Disk, etc.). Storage 1114 may store one or more databases. Server 1104 may also include an input / output path 1112. I / O path 1112 may provide 3D object data, context data for the 3D environment, natural language descriptions, machine learning model inputs and / or outputs, device information, or other data, over a local area network (LAN) or wide area network (WAN), and / or other content and data to control circuitry 1111, which may include processing circuitry, and storage 1114. Control circuitry 1111 may be used to send and receive commands, requests, and other suitable data using I / O path 1112, which may comprise I / O circuitry. I / O path 1112 may connect control circuitry 1111 (and specifically control circuitry) to one or more communications paths.

[0183] Control circuitry 1111 may be based on any suitable control circuitry such as one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi -core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or supercomputer. In some embodiments, control circuitry 1111 may be distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, control circuitry 1111 executes instructions for an emulation system application stored in memory (e.g., the storage 1114). Memory may be an electronic storage device provided as storage 1114 that is part of control circuitry 1111.

[0184] FIG. 12 is a flowchart of a detailed illustrative process 1200 for enforcing a data cap for preferential network traffic, in accordance with some embodiments of this disclosure.In various embodiments, the individual steps of process 1200 may be implemented by one or more components of the devices, methods, and systems of FIGS. 1-11 and may be performed in combination with any of the other processes and aspects described herein. Although the present disclosure may describe certain steps of process 1200 (and of other processes described herein) as being implemented by certain components of the devices, methods, and systems of FIGS. 1-11, this is for purposes of illustration only, and it should be understood that other components of the devices, methods, and systems of FIGS. 1-11 may implement those steps instead.

[0185] At 1202, control circuitry (e.g., control circuitry 1121 of FIG. 11 of LAN networking equipment 1115 and / or control circuitry 1131 of WAN networking equipment 1117 of FIG. 11), I / O circuitry (e.g., 1002 of FIG. 10 and / or 1122 and 1132 of FIG. 11 of LAN networking equipment 1115 and WAN networking equipment 1117, respectively, of FIG. 11) and / or a network interface, may receive, at a first networking equipment, network traffic over a network (e.g., communication network 1109). The first networking equipment may comprise, for example, a router, modem, and / or gateway 106 and / or 108 of FIG. 1, which may correspond to networking equipment 115, and which may provide a LAN, and / or networking equipment 122 of FIG. 1, which may correspond to networking equipment 1117, on the WAN and configured to provide a first queue of a first service flow for low latency network traffic and a second queue of a second service flow for classic network traffic.

[0186] At 1204, the control circuitry (and / or the I / O circuitry and / or the network interface) may identify a portion of the network traffic at the first networking equipment (e.g., LAN networking equipment 1115) that corresponds to preferential network traffic, based on an indication from an application service provider or ISP. For example, the control circuitry may analyze header information, IP address information, source information, destination information, DPI (deep packet inspection), packet sizes, and interarrival times, and / or codepoint information of the network traffic, or may employ any other suitable technique, to determine whether the network traffic is L4S-capable (e.g., as marked or designated by an application service provider or an ISP). In some embodiments, the ISP may mark L4S- capable network traffic as preferential based at least in part on user preferences or user input received from a user (e.g., user 110 of location 104 of FIG. 1) associated with the first networking equipment (e.g., networking equipment 106 and / or 108 of FIG. 1). For example, the control circuitry may determine that network traffic is preferential network traffic designated by an application service provider based on ECN bits in a header of the network traffic having a pattern of 01 (codepoint name ECT(l)), as such bits may have been modifiedor inserted by the application service provider. As another example, the control circuitry may determine that network traffic is preferential network traffic designated by an ISP based on ECN bits in a header of the network traffic having a pattern of 10 (codepoint name ECT(0)), as such bits may have been modified or inserted by the ISP.

[0187] At 1206, the control circuitry (and / or the VO circuitry and / or the network interface) may access a preferential network traffic data cap associated with the first networking equipment. For example, the data cap may be a certain amount of data (e.g., 100 GB or any other suitable amount of data) allocated for application service provider network traffic, ISP network traffic, or a combination thereof. In some embodiments, the data cap may be an amount of data permitted to be used during a particular period of time (e.g., a month or a year). In some embodiments, the data cap may be stored at one of more of the networking equipment or a remote server, and / or may be stored in association with user profiles or accounts, e.g., with an ISP. The process of FIG. 12 may be employed in any suitable type of network for which data caps may be imposed.

[0188] At 1208, the control circuity may determine whether the metering of the preferential network traffic requires ISP action (e.g., modifying bits in header / DiffServ codepoint). For example, the control circuitry may determine that preferential traffic corresponding to customer preferred traffic is to be metered separately from preferential traffic designated as such by an application service provider. In some embodiments, the data cap may be an aggregate of both application service provider marked preferential network traffic and ISP marked preferential network traffic, or separate data caps may be employed for each of application service provider marked preferential network traffic and ISP marked preferential network traffic.

[0189] Upon determining, at 1208, that the network traffic is customer preferred, processing may proceed to 1210, where the first networking equipment associated with the LAN may transmit instructions to the second networking equipment associated with the WAN to change ECN bits in the header to correspond to the DiffServ codepoint (e.g., to ECT(0)) for such type of network traffic.

[0190] At 1212, the control circuitry may determine whether an amount of preferential network traffic associated with the first networking equipment at the LAN over the particular period of time equals or exceeds the preferential network data cap. For example, the amount of preferential network traffic (e.g., L4S-capable network traffic as marked by an application service provider and / or an ISP) may be determined based on performing a metering function for traffic tonnage (e.g., at cable modem 106 or router 108 or a gateway of FIG. 1), and / orWAN networking equipment (e.g., networking equipment 122 of FIG. 1). In some embodiments, the cable modem 106, router 108 and / or gateway may be owned by the ISP, to facilitate visibility into the network traffic at the LAN.

[0191] In some embodiments, the data caps may be enforced as a total (upstream and downstream tonnage) or separately for upstream tonnage and downstream tonnage. In some embodiments, data caps for the preferential network traffic and total network traffic associated with the LAN may be enforced separately for upstream and downstream, or as a total of upstream or downstream traffic. In some embodiments, the ISP may separately meter and enforce policy for three classes of traffic from (and to) application servers including regular non-L4S-capable traffic, L4S-capable traffic marked by an application service provider, and L4S-capable traffic marked by an ISP based on customer selection.

[0192] Upon determining at 1212 that the data cap is reached or exceed, processing may proceed to 1216; otherwise, processing may proceed to 1214. At 1214, the control circuitry may process the network traffic identified at 1204 as preferential network traffic, e.g., using the first queue (e.g., the low latency L4S-capable queue as part of service flow 206, 210 of FIG. 2A), to provide latency priority and / or bandwidth priority to such traffic.

[0193] At 1216, the control circuitry and / or I / O circuitry and / or network interface may cause identification of an action to be performed on the preferential network traffic. For example, the first networking equipment (e.g., providing the LAN) may transmit an indication to the second networking equipment (e.g., on the WAN) of the action to be performed, or the second networking equipment may perform the determination at 1216 and identify the action to be performed.

[0194] For example, when a subscriber is determined to have met or exceeded a data cap for their preferred traffic, the ISP may perform an action such as, for example, mark or maintain a codepoint of ECT(l) in a header of a data packet of the network traffic, to continue to route the customer-designated preferred traffic to the preferred-traffic queue based on an agreement / SLA with the customer (e.g., for extra payment, movement to higher subscription tier), or re-route the preferred traffic back to the non-preferred traffic queue (e.g., mark the codepoint as non-ECT).

[0195] 1216 may be performed by the first networking equipment (e.g., providing the LAN at location 104 of FIG. 1) and / or the second networking equipment (e.g., on the WAN). For example, the first networking equipment may cause the action to be identified by transmitting an indication to the second networking equipment that the preferential network data cap has been exceeded in relation to a particular customer associated with detected network traffic, toallow the second networking equipment to determine which action is to be taken and implement such action. In some embodiments, first networking equipment may signal to the second networking equipment that a client device does not support L4S, and thus should be processed using the classic service flow and queue for classic network traffic.

[0196] In some embodiments, when network traffic is ingested, a check may be performed to determine (e.g., based on an IP address of a source and destination) whether the data traffic is to stay inside the ISP’s network or is to leave the operator’s network. Based on this determination, one or more ECT bits may be modified or disabled.

[0197] In some embodiments, traffic may be marked as preferential network traffic by an ISP based on determining that the customer has made the choice to give this preferential treatment, and the client-server of the application were capable of supporting L4S, e.g., responding to congestion control in a scalable manner. The control circuitry may determine whether one or more data caps (e.g., for preferential network traffic) are attained or exceeded, and based on determining, handle the traffic when data caps are attained or exceeded. For example, the SLA may allow a customer to continue to use the low latency queue for their preferred traffic even when the data cap tonnage is attained. Alternately, the L4S-capable traffic may be processed using the classic or default queue when the data cap is met. For example, the traffic analysis module 121 may provide the metering function, and signal to TIPE module 123 when a data cap is met, so that the TIPE module 123 can apply a new policy as defined by the SLA between the ISP and the customer. In some embodiments, application service providers may or may not allow a customer to prioritize traffic to a particular device or based on a particular service type.

[0198] FIG. 13 shows an example of selectively treating portions of data as preferential or non-preferential during a network session, in accordance with some embodiments of this disclosure.Latency requirements in remote rendered gaming (cloud gaming), varies based on interactivity. As an example, first person shooter (FPS) games, such as video game 1300, often have an extremely low latency requirement. FPS games require fast reflexes and precision targeting, and small delays in input response can significantly impact gameplay. In some embodiments, when providing an FPS video game that is remotely rendered, system 100 may use an ultra-low latency delivery system SCReAM, to enable the L4S bit in the realtime protocol (RTP) and real-time transport control protocol (RTCP) packets, and / or along with selecting an optimal edge server based on the user’s location.

[0199] System 100 may determine that portion 1302 of video game 1300 that is being transmitted to, or that is to be transmitted to, a device (e.g., device 112 or 114 of FIG. 1) corresponds to a menu of the video game, e.g., prior to initiating the actual gameplay of the video game or when the actual gameplay is paused, or otherwise does not require a relatively higher latency. Thus, system 100 may cause portion 1302 to be provided to the client device (e.g., device 112 or 114 of FIG. 1) using a non-preferential service flow. In some embodiments, when using L4S or another technique to give preferential treatment to certain portions of network traffic, system 100 (e.g., which may comprise a game engine or may be in communication with a game engine) may transmit requests through APIs to the ultra-low latency delivery system (e.g., system 100, using low latency queue 206 of FIG. 2 A) on when to set the L4S bits and when not to. For example, at portion 1304, which may occur after the menu portion 1302 of video game 1300 ends, system 100 may determine that FPS gameplay combat (or another latency-intensive portion of gameplay) is occurring against other players and / or characters controlled by the CPU, and thus system 100 may employ the first low- latency queue when providing data corresponding to portion 1304 to the client device. At portion 1306, which may occur at a time after portion 1304 occurs within video game 1300, the system may determine that portion 1306 corresponds to video game 1300 switching to a cutscene (e.g., a narrative scene which may be prerecorded and does not involve the user providing inputs or involves some user inputs), or during which an advertisement or other supplemental content is displayed, for which the second queue for non-preferential traffic may be determined to be sufficient to transmit such data over the network. For example, even if a cutscene allows some level of interactivity by the user (e.g., selecting a particular button on a controller at a certain time), the system may determine that such cut scene is still not latency sensitive and should be provided using the second non-preferential network traffic queue. At portion 1308, which may occur at a time after portion 1306 within video game 1300, the system may determine that portion 1308 has returned to FPS gameplay combat, and thus may enable L4S for transmission of such data or otherwise treat such data preferentially. At portion 1310, which may occur at a time after portion 1308 within video game 1300, the system may determine that portion 1310, while corresponding to actual gameplay of video game 1300, may nonetheless be delivered using the second, non-preferential queue, since content of such scene corresponds to exploration of a world in video game 1300 (or other relatively less latency sensitive portion), rather than combat as in portions 1304 and 1308, and thus L4S may be disabled. For example, data corresponding to portion 1310 during the network session may be processed using the second queue for non-preferential traffic.

[0200] In some embodiments, L4S packet markings (or other preferential treatment) may be enabled during gameplay of portions of the game (e.g., determined by the game developer) such as, for example, when the player reaches a specific location within a level or during a boss fight. In some embodiments, the system and / or game engine may use pre-existing gaming metadata to determine when to signal the start of applying the L4S packet markings. In some embodiments, the metadata may be written to an XML file (or any other suitable file or data structure), and networking equipment or another suitable device may read that file and / or send or receive such file via an API. For example, a game engine may make an API call to “enable or disable” priority or preferential treatment (e.g., L4S and / or another suitable mechanism) for a particular portion of network session data.

[0201] In some embodiments, the system may perform audio and / or visual analysis in real time and / or any other suitable analysis to identify what type of scene is being played in the video game. In some embodiments, such analysis and / or metadata may indicate to the system whether a portion of the network session data is latency sensitive (or not). For example, in a cloud gaming session, combat gameplay may be considered latency sensitive, while a cutscene, or an advertisement, or levels of the game where the user is just exploring a map, for example, may not be latency sensitive. As another example, in a football video game, plays during the game (e.g., a field goal or extra point being kicked, or a play from the line of scrimmage in which the offense of one team is competing with the defense of the opposing team) may be considered latency sensitive, but a time after the play, or a time when a coach or fans are shown in gaps between plays, may not be considered latency sensitive.

[0202] In some embodiments, whether a portion of network session data is latency sensitive (or not) may depend at least in part on a threshold latency (and / or a threshold for another network characteristic) that is required for a given portion of a video for optimal QoE, and the system may use such threshold in determining whether to employ the first queue or the second queue. In some embodiments, the threshold latency may be a single value or a range of values. In some embodiments, the system may use inputs from the users (e.g., how quickly inputs are being received currently or historically for that portion from the user and / or other users, or audio or other input received from the user) to determine what type of scene is being played, e.g., combat or exploration. FIG. 13 shows examples of gameplay modes where the system and / or game engine may automatically enable L4S (or other preferential treatment of network traffic) based on the in-game latency requirements.

[0203] FIG. 14 shows an illustrative architecture of a system 1400 for selectively treating portions of data as preferential or non-preferential during a network session, in accordancewith some embodiments of this disclosure. In some embodiments, system 1400 may comprise or correspond to a cloud gaming system where L4S enablement is controlled by the game engine. In some embodiments, system 1400 utilizes a cloud-based architecture that uses SCReAM (RTP), or any other suitable technique, for extreme low latency video delivery from a cloud game instance to the client device, with the ability to turn L4S on or off within the same packet flow (e.g., during a network session). In some embodiments, game engine 1403 may control when L4S packets are turned on or off based on the latency interaction requirements within the game. In some embodiments, system 1400 may be implemented at least in part by one or more portions of system 100.

[0204] As shown in FIG. 14, UDP socket address: port 1 1402 of cloud gaming service instance 1401 may receive, from UDP socket 1406 (IP address 1 : port 1) of remote client device 1404 and over network 1405, RTCP packets with SRC:<remote client device 1 IP address> with L4S (or other suitable preferential treatment protocol) enabled or disabled. Such packets may have been received by UDP socket 1406 from transmission receiver 1434. Cloud gaming service instance 1401 may implement a network congestion control module 1408, which may receive, at RTCP response router 1410, RTCP packets with SRC IP (e.g., a client device IP address of client device 1404 having sent RTCP packets (e.g., related to gameplay of a video game provided by game engine 1403). RTCP response router 1410 forwards RTCP packets SRC1 to packet response datastore 1412, and RTP packet SRC n to packet response datastore for SRC n 1414 (e.g., from various computing devices at one or more locations). Datastore 1412 provides an indication of a congestion window (CWND) and round-trip time (RTT), e.g., bytes in flight) for SRC1 to RTCP reporting system 1416.Datastore 1414 provides an indication of a congestion window and RTT for SRC n to RTCP reporting system 1416. In some embodiments, UDP socket 1402 may provide indications of new or removed SRC IP connections to network congestion control module.

[0205] RTCP reporting system 1416 may receive one or more inputs. For example, RTCP reporting system 1416 may transmit to transmission scheduler 1418, CWND RTT (bytes in flight) for a worst case SRC. Additionally, or alternatively, transmission scheduler 1418 may receive from game engine 1403 an indication of whether to enable or disable L4S for a portion of network traffic (e.g., based on latency interaction requirements within the game being provided by game engine 1403), and priority queue RTP packets 1420 transmits RTP multiplexed encoded video and audio packets to transmission scheduler 1418. Based on such inputs, transmission scheduler 1418 may provide an indication to UDP socket 1402 of whether to enable or disable L4S preferential treatment of a particular portion of networktraffic, and UDP socket 1402 may transmit, to UDP socket 1406 of remote client device 1404 RTP multiplexed encoded video and audio packets with L4S enabled or disabled.

[0206] Priority queue RTP packets 1420 may transmit a queue length to rate controller 1422. Game engine 1403 may transmit raw image data for the video game being provided to the client device to video encoder 1424 and may transmit raw audio data to audio encoder 1426. Video encoder 1424 may encode the raw image data and provide the encoded video to multiplexer 1428. Audio encoder 1426 may encode the raw audio data and provide the encoded audio to multiplexer 1428. Multiplexer 1428 may provide a multiplexed bitrate to rate controller 1422 and may provide multiplexed encoded video and audio packets to RTP sender 1430, which, as described above, may be provided to transmission scheduler 1418 and subsequently to UDP socket 1402 and client device 1404. Game engine 1403 may receive controller input via UDP socket 1 address: port 2 1432 from UDP socket 1407 (IP address 1 : port 2) over network 1405, and game engine 1403 may transmit haptic data in relation to portions of the video game to UDP socket 1432, for delivery to UDP socket 1407 of client device 1404 over network 1405.

[0207] UDP socket 1406 of client device 1404 may transmit the RTP multiplexed encoded video and audio packets to transmission receiver 1434, and transmission receiver 1434 may transmit such packet to demultiplexer 1436, which may obtain an encoded video Packetized Elementary Stream (PES) and encoded audio PES from the RTP multiplexed encode video and audio packets , and provide encoded video PES to video decoder 1438 and encoded audio PES to audio decoder 1440. Video decoder 1438 may provide decoded video frames to video renderer 1442, and audio decoder 1440 may provide decoded audio frames to audio Tenderer 1444. Video renderer 1442 may cause the rendered video frames to be provided via display 1446 of client device 1404 to a user, and audio renderer 1444 may provide rendered audio frames to audio output equipment (e.g., speaker or headphones) 1448 to the user.

[0208] Controller data transmission module 1450 of client device 1404, e.g., provided by a game remote play client application, may be configured to receive haptic data from UDP socket 1407, and transmit controller inputs received from the user (operating game controller) of client device 1404 to UDP socket 1407, for transmission to cloud gaming service instance 1401. The haptic data may be transmitted to game controller 1452, e.g., to provide vibration sensations to the user that is gaming during certain portions of the gameplay.

[0209] In some embodiments, system 1400 may enable L4S (or other preferential treatment of network traffic) for the controller data transmission (transmitted based on inputs received from game controller 1452) when the received packets are L4S-enabled (or enabled for otherpreferential treatment of network traffic) for the video / audio streams, disable disabling L4S (or other preferential treatment of network traffic) on the controller packets when L4S is not enabled (or other preferential treatment of network traffic is not enabled) on the incoming audio / video streams. For example, in cloud gaming, remote console gaming, remote vehicle control, remote control of construction equipment, and / or in other suitable applications, the controller return may be L4S enabled based on the incoming packets L4S enablement. For example, in FIG. 13, during FPS gameplay combat at portion 1304 of video game 1300, inputs received from the user’s controlled may be processed using the first queue for preferential network traffic, to help ensure a low latency gaming experience, whereas controller inputs received when portion 1302 of video game 1300 is being provided to the user may be processed using the second queue for non-preferential network traffic.

[0210] FIG. 15 shows an illustrative architecture of a system 1500 for selectively treating portions of data as preferential or non-preferential during a network session, in accordance with some embodiments of this disclosure. In some embodiments, system 1500 may be used for delivering data in a console-based gaming system during a multiplayer video gaming session, e.g., where the game console can support multiple remote users playing a multiplayer game running on the game console as if all players were local in the same room. Some nonlimiting examples of local remote play game titles are Mortal Combat, Mario Cart and PlayStation All-Stars Battle Royale. The L4S enablement may be controlled by, e.g., a game engine of system 1500. In some embodiments, system 1500 uses SCReAM (RTP), or any other suitable technique, for extreme low latency video delivery from the cloud game instance to the client device, with the ability to turn L4S on or off within the same packet flow (e.g., during a network session). The game engine may be in control of when L4S packets are turned on or off based on the latency interaction requirements within the game. In some embodiments, one RTP sender and one video encoder may be used to deliver the same stream to multiple client devices. In some embodiments, if any one of such multiple client devices does not have end-to-end L4S capability, L4S may be disabled for all remote client devices. In some embodiments, system 1500 may be used to transmit a separate stream may be transmitted via an adaptive bitrate (ABR) server to a one or more participants that are not in watch-only mode, as described in more detail in commonly-owned U.S. Patent Application No. 18 / 428,257 filed January 31, 2024 (titled “LOCAL OR REMOTE CONSOLE OR PCBASED GAMEPLAY WITH LOCAL OR REMOTE PARTICIPANTS”), in the name of Adeia Guides Inc., the contents of which is hereby incorporated by reference herein in its entirety.

[0211] System 1500 of FIG. 15 may be implemented at least in part in a similar manner to system 1400 of FIG. 14. For example, system 1500 may implement game console (acting as a server) 1401 (and / or which may utilize P2P communications with clients in some embodiments), and system 1500 include one or more of the following portions of system 1400 of FIG. 14: game engine 1403, client device 1404, UDP socket 1407, network congestion control module 1408, RTCP response router 1410, packet response datastore for SRC1 1412, packet response datastore for SRC n 1414, RTCP reporting system 1416, transmission scheduler 1418, priority queue RTP packets 1420, rate controller 1422, video encoder 1424, audio encoder 1426, multiplexer 1428, RTP sender 1430, UPD socket 1 1432, transmission receiver 1434, demultiplexer 1436, video decoder 1438, audio decoder 1440, video Tenderer 1442, audio Tenderer 1444, display 1446, and audio output equipment (e.g., speaker or headphones) 1448.

[0212] System 1500 may include controller VO mapper 1502, which may interface with controller handler 1504 corresponding to controller 1452; controller handler 1506 corresponding to controller 1516; controller handler 1508 corresponding to controller 1518; and controller handler 1510 corresponding to controller 1520. For example, controller I / O mapper 1502 may receive controller input from controller 1452 being operated by a first user and transmit such controller input to controller handler 1504, which in turn transmits such input to game engine 1403, and controller I / O mapper 1502 may transmit haptic data (received from controller handler 1504) to controller 1452. Controller I / O mapper 1502 may receive controller input from controller 1516 being operated by a second user and transmit such controller input to controller handler 1506, which in turn transmits such input to game engine 1403, and controller I / O mapper 1502 may transmit haptic data (received from controller handler 1506) to controller 1516. Controller I / O mapper 1502 may receive controller input from controller 1518 being operated by a third user and transmit such controller input to controller handler 1508, which in turn transmits such input to game engine 1403, and controller I / O mapper 1502 may transmit haptic data (received from controller handler 1508) to controller 1518. Controller I / O mapper 1502 may receive controller input from controller 1520 being operated by a fourth user and transmit such controller input to controller handler 1510, which in turn transmits such input to game engine 1403, and controller I / O mapper 1502 may transmit haptic data (received from controller handler 1510) to controller 1520. In some embodiments, the first, second, third, and fourth users, operating the controllers 1452, 1516, 1518, and 1520, respectively, may be located in different geographic areas (or the same geographic area), and may be playing the video game providedby game engine 1403 over a network (e.g., network 1405). In some embodiments, UDP socket 1432, UDP socket 1522, UDP socket 1524, and UDP socket 1526 may be used to receive haptic data for, and transmit controller input received from, controllers 1452, 1516, 1518, and 1520, respectively.

[0213] As shown in FIG. 15, game console 1401 may receive, from UDP socket 1406 (IP address 1 : port 1) of remote client device 1404 and over network 1405, RTCP packets with SRC:<remote client device 1 IP address> with L4S (or other suitable preferential treatment protocol) enabled or disabled. Such packets may have been received by UDP socket 1406 from transmission receiver 1434. Game engine 1403 may receive controller input via UDP socket 1 address: port 2 1432 from UDP socket 1407 (IP address 1 : port 2) over network 1405, and game engine 1403 may transmit haptic data in relation to portions of the video game to UDP socket 1432, for delivery to UDP socket 1407 of client device 1404 over network 1405.

[0214] Similarly, UDP socket 1528 (IP address 2: port 1) of remote client device 2 (remote client device 1532) may receive RTP multiplexed encoded video and audio packets with L4S (or other suitable preferential treatment protocol) enabled or disabled. Game engine 1403 may receive controller input via UDP socket 2 address: port 3 1522 from UDP socket 1530 (IP address n: port 2) over network 1405, and game engine 1403 may transmit haptic data in relation to portions of the video game to UDP socket 1522, for delivery to UDP socket 1530 of client device 1532 over network 1405.

[0215] Similarly, UDP socket 1552 (IP address 3: port 2) of remote client device 3 (remote client device 1556) may receive RTP multiplexed encoded video and audio packets with L4S (or other suitable preferential treatment protocol) enabled or disabled. Game engine 1403 may receive controller input via UDP socket 2 address: port 4 1524 from UDP socket 1553 (IP address n: port 2) over network 1405, and game engine 1403 may transmit haptic data in relation to portions of the video game to UDP socket 1524, for delivery to UDP socket 1553 of client device 1556 over network 1405.

[0216] UDP socket 1528 of client device 1532 may transmit the RTP multiplexed encoded video and audio packets to transmission receiver 1534, and transmission receiver 1534 may transmit such packet to demultiplexer 1536, which may obtain an encoded video PES and encoded audio PES from the RTP multiplexed encode video and audio packets, and provide encoded video PES to video decoder 1538 and encoded audio PES to audio decoder 1540. Video decoder 1538 may provide decoded video frames to video Tenderer 1542, and audio decoder 1540 may provide decoded audio frames to audio Tenderer 1544. Video Tenderer1542 may cause the rendered video frames to be provided via display 1546 of client device 1532 to the second user, and audio Tenderer 1544 may provide rendered audio frames to audio output equipment (e.g., speaker or headphones) 1548 to the second user.

[0217] Controller data transmission module 1550 of client device 1532, e.g., provided by a game remote play client application, may be configured to receive haptic data from UDP socket 1530, and transmit controller inputs received from the user (operating game controller) of client device 1532 to UDP socket 1530, for transmission to game console 1401. The haptic data may be transmitted to game controller 1516, e.g., to provide vibration sensations to the user that is gaming during certain portions of the gameplay.

[0218] UDP socket 1552 of client device 1556 may transmit the RTP multiplexed encoded video and audio packets to transmission receiver 1554, and transmission receiver 1554 may transmit such packet to demultiplexer 1555, which may obtain an encoded video PES and encoded audio PES from the RTP multiplexed encode video and audio packets , and provide encoded video PES to video decoder 1558 and encoded audio PES to audio decoder 1560. Video decoder 1558 may provide decoded video frames to video Tenderer 1562, and audio decoder 1560 may provide decoded audio frames to audio Tenderer 1564. Video Tenderer 1562 may cause the rendered video frames to be provided via display 1566 of client device 1556 to a third user, and audio Tenderer 1564 may provide rendered audio frames to audio output equipment (e.g., speaker or headphones) 1568 to the user.

[0219] Controller data transmission module 1570 of client device 1556, e.g., provided by a game remote play client application, may be configured to receive haptic data from UDP socket 1553, and transmit controller inputs received from the user (operating game controller) of client device 1556 to UDP socket 1553, for transmission to game console 1401. The haptic data may be transmitted to game controller 1518, e.g., to provide vibration sensations to the fourth user that is gaming during certain portions of the gameplay. While not shown, a remote client device similar to client devices 1404, 1532, and 1556 may be employed in relation to the fourth user operating controller 1520. In some embodiments, L4S (or another suitable preferential treatment technique) may be enabled for network data corresponding to controller inputs received from controller 1518, based on incoming RTP audio / video network packets being L4S enabled (or another suitable preferential treatment technique), and L4S (or disabled for another suitable preferential treatment technique) may be disabled for network data corresponding to controller inputs received from the user, based on incoming RTP audio / video network packets being L4S disabled (or disabled for another suitable preferential treatment technique).

[0220] FIG. 16 shows an illustrative architecture for a system 1600 for selectively transmitting data using a preferential queue or a non-preferential queue, during a multiplayer session for multiplayer gaming, in accordance with some embodiments of this disclosure. System 1600 may provide a multiplayer session server 1602 for multiplayer gaming with a network delivery optimization server 1604, which may implement a fairness function 1606. The multiplayer gaming session may be provided by one or more of multiplayer session server 1602, network delivery optimization server 1606, and regional server 1608 over network 1605 to any suitable number of devices, e.g., game console player 1612 at a first client device, game console player 1614 at a second client device ... game console n 1616 at n client device(s). Multiplayer session server 1602 may establish a player session packet RTT with a <(session)ID> network delivery optimization server 1604.

[0221] In a multiplayer locally rendered gaming session, all participants might not have L4S support along the entire route from the multiplayer server to the game device. To maintain fairness, if any player in a multiplayer gaming session does not have end-to-end L4S support, the multiplayer server may disable all L4S packet markings for all players in that multiplayer gaming session. This may be done by system 1600 examining a first response packet received from each client device. For example, in the case of TCP, such response packet may be an acknowledgment / negative acknowledgment (ACK / NACK) response packet. For example, TCP socket address:portl 1616 of game console 1612 may transmit TCP multiplayer ACK / NACK packets with L4S enabled or disabled to TCP socket address: portl 1618 of regional server 1608, based on an indication received from TCP receiver 1620 of game console 1612, which in turn may be based on an indication received from game engine 1610 (e.g., based on latency requirements for a current portion of the video game being played). TCP socket 1618 may transmit to TCP sender 1640, the TCP multiplayer ACK / NACK packets and an indication of whether L4S is enabled or disabled for a particular portion of network traffic, and based on such received data, TCP sender 1640 may transmit an indication of whether L4S is enabled or disabled, and the indication of a packet RTT, to network delivery optimization server 1604 and / or multiplayer session server 1602.

[0222] As another example, TCP socket address: port 2 1622 of game console 1612 may transmit TCP multiplayer ACK / NACK packets with L4S enabled or disabled to TCP socket address: portl 1624 of regional server 1608, based on an indication received from TCP sender 1626 of game console 1612, which in turn may be based on an indication received from game engine 1610 (e.g., based on latency requirements for a current portion of the videogame being played). TCP socket 1624 may transmit to TCP receiver 1642, the TCP multiplayer ACK / NACK packets and an indication of whether L4S is enabled or disabled for a particular portion of network traffic, and based on such received data, TCP receiver 1642 may transmit an indication of whether L4S is enabled or disabled, and the incoming multiplayer packets, to network delivery optimization server 1604 and / or multiplayer session server 1602.

[0223] In another example, in RTP, such response packet that is examined may be an RTCP packet. For example, UDP socket address:portl 1628 of game console 1612 may transmit RTCP multiplayer packets with L4S enabled or disabled to UDP socket address: portl 1630 of regional server 1608, based on an indication received from RTP receiver 1632 of game console 1612, which in turn may be based on an indication received from game engine 1610 (e.g., based on latency requirements for a current portion of the video game being played). UDP socket 1630 may transmit, to RTP sender 1644, the RTCP multiplayer packets and an indication of whether L4S is enabled or disabled for a particular portion of network traffic, and based on such received data, RTP sender 1644 may transmit an indication of whether L4S is enabled or disabled, and the indication of a packet RTT, to network delivery optimization server 1604 and / or multiplayer session server 1602.

[0224] As another example, UDP socket address:port2 1634 of game console 1612 may transmit RTP incoming multiplayer packets with L4S enabled or disabled to UDP socket address: port2 1636 of regional server 1608, based on an indication received from RTP sender 1638 of game console 1612, which in turn may be based on an indication received from game engine 1610 (e.g., based on latency requirements for a current portion of the video game being played). UDP socket 1636 may transmit, to RTP receiver 1646, the RTCP multiplayer packets (and an indication of whether L4S is enabled or disabled for a particular portion of network traffic, and based on such received data, RTP receiver 1646 may transmit an indication of whether L4S is enabled or disabled, and incoming multiplayer packets, to network delivery optimization server 1604 and / or multiplayer session server 1602.

[0225] In some embodiments, if the first response packet is not L4S marked, all L4S markings may be disabled for all players in the multiplayer gaming session. In some embodiments, a packet indicating whether L4S is supported and / or enabled may be transmitted at an initial handshake stage, e.g., when a first request is sent, the system may determine whether L4S should not be utilized for the current network session.

[0226] As shown in FIG. 16, the architecture of system 1600 may provide for a multiplayer regional server 1608 which includes a multiplayer session instance of a multiplayer session.The multiplayer session is served by multiplayer session server 1602. Such server 1602 may perform matchmaking, authentication, and / or other multiplayer functions to establish and facilitate the multiplayer session instance. System 1600 may optimize delivery of multiplayer session data and may include a network delivery optimization subsystem which implements the fairness function 1606. In some embodiments, each multiplayer session on both the server and the client device includes both an RTP sender and receiver or a TCP sender and receiver. On the server, the RTP sender or the TCP sender transmits the multiplayer merged game data to a client device’s game engine. On the client device, the RTP or TCP sender sends the player’s game play data and input to the multiplayer server. Based on the calculated TCP or RTP packet round-trip times (RTTs), system 1600 may calculate both jitter and latency associated with network traffic for a device, and certain sessions may be L4S-enabled or L4S-disabled to enable fairness across sessions of the multiplayer session.

[0227] For example, multiplayer session server 1602 and / or network delivery optimization server 1604 may transmit indications to RTP sender 1644 and TCP sender 1640 of whether certain network traffic should be treated preferentially, and such indication may be propagated to the client device and game engine. For example, outgoing multiplayer packets and an indication regarding whether L4S is enabled or disabled for such packets may be forwarded to RTP sender 1644 from multiplayer session server 1602 and / or network delivery optimization server 1604, and RTP sender 1644 may transmit, to UDP socket address 1630, RTP multiplayer packets and the indication regarding whether L4S is enabled or disabled for such packets. UDP socket address 1630 may transmit, to UDP socket address 1628, RTP outgoing multiplayer packets associated with L4S being enabled or disabled, and UDP socket address 1628 may transmit the RTP incoming multiplayer packets, and the indication regarding whether L4S is enabled or disabled for such packets, to RTP receiver 1632, which in turn may be transmitted to game engine 1610.

[0228] For game console player 2 1614 participating in the multiplayer session, TCP socket address 1656, TCP socket address 1662, UDP socket address 1668, UDP socket address 1674, TCP receiver 1660, TCP sender 1666, RTP receiver 1672, and RTP sender 1678 may be implemented in a similar manner as TCP socket address 1616, TCP socket address 1622, UDP socket address 1634, UDP socket address 1634, TCP receiver 1620, TCP sender 1626, RTP receiver 1632, and RTP sender 1638, respectively. TCP socket address 1658, TCP socket address 1664, UDP socket address 1670, UDP socket 1676, TCP sender 1680, TCP receiver 1682, RTP sender 1684, and RTP receiver 1686 of regional server 1608 may be implemented in a similar manner as TCP socket address 1618, TCP socket address 1624,UDP socket address 1630, UDP socket 1636, TCP sender 1640, TCP receiver 1642, RTP sender 1644, and RTP receiver 1646 of regional server 1608.

[0229] Similarly, for game console player n 1616 participating in the multiplayer session, TCP socket address 1651, TCP socket address 1657, UDP socket address 1663, UDP socket address 1669, TCP receiver 1655, TCP sender 1661, RTP receiver 1667, and RTP sender 1673 may be implemented in a similar manner as TCP socket address 1616, TCP socket address 1622, UDP socket address 1634, UDP socket address 1634, TCP receiver 1620, TCP sender 1626, RTP receiver 1632, and RTP sender 1638, respectively. TCP socket address 1653, TCP socket address 1659, UDP socket address 1665, UDP socket 1671, TCP sender 1675, TCP receiver 1677, RTP sender 1679, and RTP receiver 1681 of regional server 1608 may be implemented in a similar manner as TCP socket address 1618, TCP socket address 1624, UDP socket address 1630, UDP socket 1636, TCP sender 1640, TCP receiver 1642, RTP sender 1644, and RTP receiver 1646 of regional server 1608.

[0230] FIG. 17 is an illustrative example 1700 of using SLAM on an XR head-mounted device (HMD) or headset (e.g., indoors or outdoors), enabling virtual objects to interact at close range with the physical world, to facilitate relatively more precise localization, in accordance with some embodiments of this disclosure. In some embodiments, in example 1700 of FIG. 17, the system may enable L4S (or other preferential treatment of network traffic mechanism), optionally along with one or more of higher framerate, resolution video and higher bandwidth for transmitting the inertial measurement unit (IMU) and video data from the XR device to the cloud. The cloud-based SLAM system may additionally or alternatively enable L4S for the packets transmitting the localization from the cloud to the XR device. As shown in FIG. 17, the system may use SLAM in AR for rendering virtual object interacting with objects at a relatively close distance (e.g., in a particular type of location, such as a home or otherwise indoors, and / or within a threshold distance). In some embodiments, in example 1700, the XR device may be rendering virtual objects in close proximity to real world objects and may take into account the distance of the user wearing or using the XR device and the virtual objects being rendered, to determine whether a portion of network session data is latency sensitive. For example, if multiple objects are relatively close to the XR device within an XR environment, the system may consider the associated network session data to be latency sensitive and process it using the first queue for preferential network traffic.

[0231] As described in more detail in commonly-owned U.S. Application No. 18 / 128,934, filed March 30, 2023 (titled “DATA TRANSMISSION THROTTLING AND DATAQUALITY UPDATING FOR A SLAM DEVICE”), in the name of Adeia Guides Inc., the contents of which is hereby incorporated by reference herein in its entirety, the system may implement distributed SLAM delivery optimization for localization, e.g., when a spatial map already exists for a particular location. Localization and mapping may be performed within the network, and the device may transmit camera and IMU sensor data, and the network edge may perform map building and localization. In some embodiments, the images captured by the camera may be encoded using a codec, such as, for example, H.264, high efficiency video coding (HEVC), versatile video coding (VVC) or any other suitable codec, and multiplexed with IMU sensor data and transmitted to the SLAM system running at the network edge. Both the bitrate and resolution of the encoding directly effects the accuracy of the localization. Depending on the level of accuracy needed for a particular portion of network traffic being transmitted or to be transmitted, the encoded bitrate and / or resolution may be raised or lowered. When a device is far away from any object, a much lower bitrate can be used. As moving objects move into a closer range of the device or the device moves into a closer of the objects, the bitrate can be raised offering a better localization accuracy. This bitrate may be dynamically adjusted based on the changing proximity of the objects to the device. As can be seen in comparing FIGS. 17and 18, a higher level of localization accuracy may be desirable in FIG. 17 than in FIG. 18.

[0232] FIG. 18 is an example 1800 of using SLAM on a device (e.g., indoors or outdoors) to enable a virtual object to interact at a large distance with the physical world, to facilitate relatively less precise localization, in accordance with some embodiments of this disclosure. For example, at the sender device (e.g., an XR device such as, for example, a tablet), L4S may be disabled, optionally along with a relatively lower framerate, resolution video and lower bandwidth for transmitting the IMU and video data from the XR device to the cloud. In some embodiments, the cloud-based SLAM system additionally or alternatively disable L4S for the packets transmitting the localization from the cloud to the XR device. For example, the example of FIG. 18 may be used for rendering virtual objects interacting with objects at a relatively far distance. In some embodiments, in example 1800, the XR device may be rendering virtual objects in close proximity to real world objects and may take into account the distance of the user wearing or using the XR device and the virtual objects being rendered, to determine whether a portion of network session data is latency sensitive. For example, if no objects are relatively close to the XR device and / or objects are detected at beyond a threshold distance within an XR environment in relation to the user wearing orusing the XR device, the system may consider the associated network session data not to be latency sensitive and process it using the second queue for non-preferential network traffic.

[0233] FIG. 19A is an example 1900 of using SLAM to transmit network traffic associated with a robotic lawn mower when there are physical objects (e.g., bushes, stones, flowers) close to the robotic lawn mower, requiring the localization to be more precise and frequent, in accordance with some embodiments for this disclosure. For example, L4S may be enabled at the sender (the robotic lawn mower), optionally along with a relatively higher framerate, resolution video and higher bandwidth for transmitting the IMU and video data from the mower to the cloud. The cloud-based SLAM system may additionally or alternatively enable L4S for the packets transmitting the localization from the cloud to the mower.

[0234] FIG. 19B is an example 1902 of using SLAM to transmit network traffic associated with a robotic lawn mower when there are no physical objects, or minimal physical objects, close to the robotic lawn mower, allowing the localization to be less precise and / or frequent, in accordance with some embodiments for this disclosure. For example, L4S may be disabled at the sender (lawnmower) optionally along with lower framerate, resolution video and lower bandwidth for transmitting the IMU and video data from the mower to the cloud. The cloudbased SLAM system may additionally or alternatively disable L4S for the packets transmitting the localization from the cloud to the mower.

[0235] FIG. 20 is an illustrative architecture for a system 2000 having interfaces with SLAM functionalities and / or sub-functionalities, for enabling L4S in a XR device 2001 using cloud-based SLAM, in accordance with some embodiments for this disclosure. In some embodiments, system 2000 may be a SLAM system for enabling L4S packet markings based on speed, object tracking, and proximity of obstacles to an XR device or a vehicle or other device. The SLAM architecture is an expansion on MapLab which has been converted to run in the cloud with optimization for video encoding resolution and framerate based on bandwidth, localization requirements and object tracking. As discussed in relation to FIGS. 16-17, for an XR device (e.g., connected to an LAN at location 104), a lower localization precision may be sufficient, or a higher localization precision may be required, based on a speed of the XR device and / or other objects and a proximity to other objects. In some embodiments, L4S packets for the video and IMU data may be enabled on the video and IMU uplink flow and and / or may be enabled for localization within the mapped space on the downlink flow based on the localization update frequency requirements and the precision and / or accuracy of the localization. System 2000 may be utilized for robotics control and / or any other suitable application, e.g., system 2000 may be leveraged in autonomous driving,based on proximity of other moving or stationary objects around the vehicle and / or speed of the vehicle.

[0236] As shown in FIG. 20, XR device 2001 may comprise distributed SLAM client 2002. XR device 2001 may comprise network congestion control 2004 may receive, from UDP socket 2006 over network 2005, RTCP packets, which may be received from UDP socket 2008 of distributed SLAM network edge server 2010. UDP socket 2006 may receive, from UDP socket 2008 over network 2005, an indication of whether RTCP packets are enabled or disabled for L4S (or other suitable protocol to treat portions of network traffic preferentially). Transmission scheduler 2012 of XR device 2001 may transmit, to UDP socket 2006, RTP multiplexed encoded video and IMU packets and an indication of whether L4S (or another protocol to treat the packets preferentially) is enabled for such packets. Network congestion control module 2004 may transmit to transmission scheduler 2012, CWND, RTT (bytes in flight), and network congestion control module 2004 may receive, from transmission scheduler 2012, synchronization source identifier (SSRC), transmission timestamps (TSTX) of RTP packets, RTPSN (sequence number) of RTP packets, and RTPsize(RTP packet size).

[0237] The RTP multiplexed encoded video and IMU packets may be received by transmission scheduler 2012 from priority queue for RTP packets 2014, and priority queue 2014 may have received such data from RTP sender 2016, which in turn may have received such data from multiplexer 2018. Priority queue 2014 may provide, to rate control module 2020, queue lengths, and rate control module 2020 may provide a target rate to video encoder 2022. Video encoder 2022 may receive raw image data from camera 2024 and may encode such data to obtain encoded video data to be provided to multiplexer 2018. IMU 2026 may provide IMU data to IMU data bitrate computation module 2028, which may provide the IMU data to multiplexer 2018, and which may compute and provide to rate control module 2020 the IMU data bitrate. Session handler 2030 may comprise localization handler API and may transmit data indicating an RTP receiver address: port, and data indicating whether data packets are to be L4S-enabled or disabled, to transmission scheduler 2012. Session handler 2030 may provide one or more notifications (e.g., localization accuracy notifications), localization accuracy requests, and / or localization data to one or more 2D or 3D applications.

[0238] UDP socket 2008 may transmit RTP multiplexed encoded video and IMU packets to transmission receiver 2034 of distributed SLAM network edge server 2010, and transmission receiver 2034 may transmit RTCP packets to UDP socket 2008. Transmission receiver 2034 may transmit, to demultiplexer 2036, RTP multiplexed encoded video and IMU packets, and demultiplexer 2036 may perform demultiplexing of such data to obtain encoded video datawith timestamp(s) and IMU data (e.g., acceleration, angular rate, inclination) with timestamp(s), and provide such data to timing synchronizer 2042. Video decoder 2038 may decode the encoded video data with timestamp(s) to obtain decoded video frame(s) with timestamp(s), and PNG encoder 2040 may be used to obtain PNG image with timestamp(s), to be provided to timing synchronizer 2042. Timing synchronizer 2042 may provide a synchronized PNG image, and synchronized IMU data, to visual inertial odometry module 2044 and image-IMU synchronizer 2046.

[0239] Feature tracking module 2048 may receive the PNG image and IMU data, and such data may be provided to map builder 2050 and localization module 2052, based on which map builder 2050 may generate a new map 2054 (e.g., of an environment surrounding a user of XR device 2001), and such map may be merged at 1256 (e.g., with previously generated maps at the same location and / or recently generated) and stored at local edge spatial map data datastore 2058. Localization module 2052 may transmit a bitrate request to rate control module 2020 and may transmit an encoding properties request to video encoder 2022, and receive, from transmission receiver 2034, an indication of whether to enable or disable L4S for a particular portion of network traffic; a bitrate notification from rate control module 2020 based on the transmitted bitrate request; an encoding properties response from video encoder 2022 based on the transmitted encoding properties request. Localization module 2052 may store, based on the received bitrate notification, data in the bitrate to location accuracy table 2064. Localization module 2052 may transmit localization data, and an indication of whether to enable or disable L4S, to UDP socket 2060, and UDP socket 2060 may transmit localization data, and an indication of whether L4S is enabled or disabled for the particular portion of network traffic, to UDP socket 2062 of XR device 2001. UDP socket 2062 may transmit localization data to session handler 2030 and may receive localization sender address port data from session handler 2030. Localization module 2052 may receive map data 2069 from datastore 2058.

[0240] Session handler 2066 may receive from session handler 2030, data related to session setup request RTP sender address:port and codec type, and session handler 2066 may transmit, to session handler 2030, a session setup response with RTP receiver address:port and localization data sender address port. Localization module 2052 may receive a localization accuracy request to session handler 2030 and transmit a localization accuracy notification to session handler 2030 based on the request.

[0241] FIG. 21 is an illustrative architecture for a system 2100 having interfaces with SLAM functionalities and / or sub-functionalities, for enabling L4S in a robotic device 2101using cloud-based SLAM, in accordance with some embodiments for this disclosure. In some embodiments, system 2000 may be a SLAM system for enabling L4S packet markings based on speed, object tracking, and proximity of obstacles to a robot or other device. In some embodiments, system 2100 may be implemented at least in part using the open-source ROS (robot operating system) where MapLab has leveraged and modified SLAM based subfunctions in its implementation, or using any suitable custom implementation thereof, or using any other suitable technique. In some embodiments, the architecture for system 2100 is similar to that of system 2000 (e.g., for an XR device) where video encoding resolution and framerate may be altered based on the bandwidth available and the localization requirements. As discussed in relation to FIGS. 18 and 19A-19B, for a device such as robot, lower localization precision and / or accuracy may be sufficient, or higher localization precision and / or accuracy may be required, based on speed and proximity to other objects or obstacles and obstacle avoidance and / or a size of the robot and / or the other objects or obstacles. L4S packets for the video and IMU data may be enabled on the video and IMU uplink flow and / or may be enabled on the localization within the mapped space on the downlink flow.

[0242] As shown in FIG. 21, system 2100 may be implemented in a similar manner as system 2000 of FIG. 20, except that system 2100 may employ robotic device 2101 instead of (or in addition to) XR device 2001 of FIG. 20. Robotic device 2101 may comprise distributed SLAM client 2102. Distributed SLAM client 2102 of FIG. 21 may be implemented in a similar manner as distributed SLAM client 2002 of FIG. 20. System 2100 of FIG. 21 may comprise distributed SLAM network edge server 2110 which may be implemented in a similar manner as distributed SLAM network edge server 2010 of FIG. 20. Session handler 2030 may provide one or more notifications (e.g., localization accuracy notifications), localization accuracy requests, and / or localization data to robotic control robot operating system (ROS), based on which ROS 2104 may transmit instructions to control robotic actuators and motors 2106 (and / or any other suitable components of robotic device 2101), e.g., based on whether a current portion of network traffic requires relatively higher or relatively lower latency.

[0243] In some embodiments, in the examples of FIGS. 17-21, the system may minimize localization latency, e.g., by using the first queue to process preferential network traffic, when SLAM client and / or SLAM network edge determines that XR device 2001 or robotic device 2101 is in close proximity to one or more other objects. This may enable minimizing the latency at which localization updates are received.

[0244] FIG. 22 is an illustrative architecture for a system 2200 for selectively treating portions of network traffic preferentially during a virtual conference, in accordance with some embodiments for this disclosure. System 2200 may comprise meeting attendee client device 1 2202, video conferencing service or server(s) 2204, meeting attendee client device 2 2206, ... meeting attendee client device n. Video conferencing service or server(s) 2204 may facilitate a video conference between any suitable number of remote participants.

[0245] Video conferencing is another area where portions of network traffic may be treated preferentially set. For example, manual settings when setting up a meeting may be based on a set of selected invitees (e.g., based on their physical location such as participating from a different country, where such information may be obtained from an email service server or using any other suitable technique) to a meeting or it could be prioritized for all meeting attendees at the meeting invite level. In some embodiments, system 2200 may set automatic settings based on if a user has enabled video or not. In some embodiments, based on the participant activity within the video conference, network traffic may be treated preferentially (or not). For example, L4S may be disabled if video conference system 2200 detects the participant is distracted (e.g., based on head pose and eye tracking or window pose, and / or based on audio input or lack thereof received from the user), or if system 2200 detects the user has stepped away from his or her device that is joined to the virtual conference. Such detection mechanisms are discussed in more detail in commonly owned U.S. Application No. 18 / 207,348, filed June 8, 2023 (titled “AUTOMATED MEETING RECORDINGS AND REPLAY BASED ON USER ACTIVITY AND ATTENDANCE”), in the name of Adeia Guides Inc., the contents of which is hereby incorporated by reference herein in its entirety.

[0246] In some embodiments, enabling / disabling L4S packet markings may be triggered based on specific events or when a specific functionality is invoked, e.g., when a participant invokes a functionality to share his or her screen with other remote participants of the virtual conference, or when annotation is taking place on a whiteboard, for example. In some embodiments, an application administrator (e.g., IT personnel) may allow the automatic enablement for L4S markings for features of an application (e.g., the feature of the application that is used or invoked triggers the utilization of L4S Packet markings). In one example, the feature associated with the application (e.g., video conferencing application) is an interactive whiteboard. FIG. 22 may provide video conference system 2200 for marking packets to be treated preferentially based on meeting priority, user priority, device priority, side group priority, head pose, user’s gaze, window focus, attendee’s proximity to the client device, and / or any other suitable factors.

[0247] As shown in FIG. 22, client device 2202 may receive camera input 2210 and / or microphone input 2212 of a user participating in the video conference. The raw audio captured by the microphone may be provided to audio encoder 2214, and the incoming raw video from camera 2210 may be provided to video encoder 2216. Multiplexer 2215 may receive encoded audio from audio encoder 2214 and encoded video from video encoder 2216 and perform multiplexing on such encoded video and audio data and provide the multiplexed encoded audio / video stream to WebRTC sender 2218. UDP socket 2222 may receive an indication of whether to enable or disable L4S for a portion of video conference network traffic from network delivery optimization module 2220, and the multiplexed audio / video stream, and may provide such data to UDP socket 2222, which in turn may provide such data to UDP socket 2224 of server 2204. The indication of whether to enable or disable L4S for a portion of video conference network traffic may be based on any suitable data provided to delivery optimization module 2220, e.g., eye tracking data 2228 gleaned from eye tracking sensor 2226, head / pose orientation data 2230, and / or facial recognition data 2231.

[0248] Meeting and side group manager 2232 may transmit a request to meeting manager 2234 of server 2204 to join a meeting (e.g., with user_ID, device_ID, priority) and / or a side group meeting with user id, group ID and priority. Meeting and side group manager 2232 may transmit an indication of a meeting or side group priority, and device priority, to network delivery optimization module 2220. Email application 2236 and email plugin 2238 may analyze a user’s meeting invite schedule and / or invite list and / or user priority or meeting priority preferences and transmit such data to meeting and side group manager 2232. A unique device ID 2240 (e.g., CPU ID, mac address, and / or any other suitable identifying data) for client device 2202 may be transmitted to network delivery optimization module 2220. Meeting and side group manager 2232 may transmit to a user account data 2242 for the user of client device 2202, stored at server 2204, data comprising a save device request with user id, and list of device IDs and priority. Network delivery optimization module 2220 may communicate with network delivery optimization module 2244 of server 2204 whether to enable or disable L4S (or other suitable preferential treatment protocol) with one or more portions of a particular session lD.

[0249] UDP socket 2246 of client device 2202 may receive, from UDP socket 2248 of server 2204, multiplexed encoded remote rendered video / audio stream(s) with L4S enabled or disabled. For example, such video and / or audio stream(s) may have originated at locations corresponding to one or more of client device 2206 or 2208 participating in the virtual meeting. Such data may be provided from UDP socket 2246 to WebRTC receiver 2250,which in turn may provide such data to demultiplexer 2252, which may separate the combined audio and video signals, e.g., to transmit encoded video stream(s) to video decoder 2254, and encoded audio stream(s) to audio decoders 2256 and 2258. Video decoder 2254 decodes the video stream(s) and transmits the decoded video data to video Tenderer 2260, which provides an output of the video conference at display 2262 to the user of client device 2202. Decoded audio output by audio decoders 2256 and 2258 may be transmitted to audio mixer 2264, and audio mixer 2264 transmits audio output to audio Tenderer 2266 and 2268 for output to the user of client device 2202 via audio output equipment (e.g., speakers or headphones) 2270.

[0250] Meeting attendee client device 2206 may receive, at UDP socket 2271, multiplexed encoded audio and video stream packets, e.g., from UDP socket 2273 of server 2204. For example, such packets may have been received at UDP socket 2224 of server 2204, transmitted to meeting video processor 2274, and combined at 1476 with other video and / or audio streams of other conference participants, and such combined data may be transmitted to WebRTC sender user instance 2278, and forwarded to UDP socket 2271 along with an indication to enable or disable L4S for such packets. In some embodiments, network delivery optimization 2244 may provide an indication to enable or disable L4S to webRTC sender user instance 2280, which in turn may transmit an indication to enable or disable L4S to UDP socket 2248, along with the multiplexed encoded remote rendered video and audio stream (e.g., captured at client device 2206 and / or 2208). WebRTC sender user instance 2280 may receive a packet received response (RTCP or other suitable protocol) from UDP socket 2248.

[0251] UDP socket 2271 of meeting attendee client device 2206 may transmit the multiplexed encoded audio / video packets to meeting application 2282, which may forward such data, along with an indication to enable or disable L4S, to UDP socket 2284, and such data may be forwarded to UDP socket 2286 of server 2204, which in turn may forward such data to meeting video processor 2274. UDP socket 2288 of client device 2208 may receive multiplexed audio / video stream packets with L4S enabled or disabled to UDP socket 2288, which may forward such data to meeting application 2290, which in turn may forward such data to UDP socket 2292. WebRTC sender user instance 2294 may provide an indication to enable or disable L4S for multiplexed encoded audio / video stream packets to UDP socket 2298. In some embodiments, meeting application 2290 of client device 2208 and meeting application 2282 may transmit a request to join a meeting with a user ID, device lD, priority data and / or other suitable data, and / or a side group meeting with user ID, group ID, and priority data and / or other suitable data, to meeting manager 2234.

[0252] As shown in FIG. 23 A, a remote-control device controlled by operator 2302 may be configured to control construction equipment (e.g., excavation equipment) or any other suitable equipment, from a remote location (from 60 miles away or less, or any other suitable distance away) over a mobile network or any other suitable network. There may be cases where extreme low latency may not be required for controlling such construction equipment, and others where it may be required. For example, as shown at 2304, if a bulldozer or dozer (or any other suitable construction equipment) is moving from one work site to another with no objects in close proximity, a relatively low latency may be sufficient for network communications between the remote control and the bulldozer, e.g., since the equipment may be moving slowly, 50-100 ms latency may be sufficient, and thus L4S may be disabled for the network communications. On the other hand, as shown at 2306, if the system determines that the bulldozer is operating in close proximity of another object or the dozer has reached its working site, the latency may need to be decreased on both the video delivery from the dozer and the controller input from the remote-control operator, and thus L4S may be enabled for the network communications.

[0253] FIG. 23 A shows a sequence of images of the remote operator 2302 with multiple camera views 2304 and 2306 of the bulldozer. 2304 is an example of a remote-control bulldozer relocating to another working location and / or having no equipment (or minimal other objects) in close proximity. 2306 is an example of a remote-control dozer moving into an area with other equipment / objects. In each of these cases, L4S (or other preferential treatment of network traffic) may be enabled or disabled based on “working mode” or proximity to other objects. FIG. 23B is an example of L4S stream enablement (or enablement of other preferential treatment of network traffic) based on work site proximity and obstacle proximity.

[0254] As shown in FIG. 23B, construction equipment 2308 may transmit, to base station 2301, equipment audio visual data streams with L4S disabled, e.g., due to construction equipment 2308 having relocating to another working location and / or having no equipment (or minimal other objects) in close proximity, and construction equipment 2308 may transmit, to base station 2301, an indication to disable L4S for haptics data streams, and base station 2301 may transmit an indication that controller data streams have L4S disabled. Similar data flows may be transmitted and received between construction equipment 2310 and base station as for construction equipment 2308. On the other hand, based on determining that construction equipment 2312 and 2314 are proximate to work stie 2316 and / or each other and / or other objects (e.g., of a certain size, or a certain number of objects), base station 2301may transmit indications to construction equipment 2312, 2314 to enable L4S preferential treatment for audio visual data streams and haptic data streams and controller data streams. Data streams transmitted by or received by remote control operators 2322 and 2324 for construction equipment 2312 and 2314, respectively, may be L4S enabled (e.g., based on such construction equipment having proximity to a work site or other objects). On the other hand, data streams transmitted by or received by remote control operators 2320 and 2318 for construction equipment 2308 and2310, respectively, may be L4S disabled (e.g., based on such construction equipment lacking proximity to a work site or other objects).

[0255] FIG. 24 shows an illustrative architecture for a system 2400 for prioritized stream delivery with L4S based on remote vehicle control with proximity of objects, in accordance with some embodiments of this disclosure. Equipment control system 2401 may comprise network congestion control 2402, which may be configured to receive RTCP packets with SRC IP address from UDP socket 2404, based on RTP multiplexed encoded video and audio packets received from transmission scheduler 2406. Proximity detection module 2408 may, based on measurements from sensors (e.g., lidar sensors 2410), transmit an indication whether to enable or disable L4S for certain network traffic. UDP socket address 2404 may transmit RTP multiplexed encoded video and audio packets that are L4S enabled or disabled to UDP socket 2428 of remote-control system 2405, and UDP socket address 2404 may receive RTCP packets with SRC: <remote client device 1 IP address> L4S enabled or disabled from UDP socket 2428.

[0256] Priority queue for RTP packets 2412 may receive the RTP multiplexed encoded video and audio packets from RTP sender 2414, which in turn was received by RTP sender 2414 from multiplexer 2416. Multiplexer 2416 may combine encoded video received from video encoder 2418 and encoded audio received from audio encoder 2420 to obtain the multiplexed bitrate transmitted to rate controller 2422 and to obtain the multiplexed encoded video and audio packets transmitted to RTP sender 2414. Video encoder 2418 may encode camera data received from camera 2424 and audio data received from microphone 2426. Priority queue 2412 may transmit a queue length to rate controller 2422, which may provide encoding properties and the target rate to encoders 2418 and / or 2420 and may receive audio bitrate information from audio encoder 2420.

[0257] UDP sockets 2430 and 2432 of equipment control system 2401 may receive, from UDP socket 2434 and UDP socket 2436, respectively, of remote control system 2405, controller input (e.g., received from remote operator 2476 of construction equipment controlled by control system 2401) that is enabled or disabled with respect to L4S. UDPsockets 2430 and 2432 of equipment control system 2401 may transmit, to UDP socket 2434 and UDP socket 2436, respectively, of remote control system 2405, haptics data that is enabled or disabled with respect to L4S. UDP sockets 2430 and 2432 may receive such haptics data from equipment and haptics feedback controller 2438, and UDP sockets 2430 and 2432 may transmit to equipment and haptics feedback controller 2438 controller input received from UDP sockets 2434 and 2436. Such haptics data may be received by equipment and haptics feedback controller 2438 from equipment controller actuators 2440, and the controller inputs may be provided to controller actuators 2440.

[0258] Equipment control system 2401 may comprise network congestion control 2442, which may be configured to receive RTCP packets with SRC IP address from UDP socket 2444, based on RTP multiplexed encoded video and audio packets received from transmission scheduler 2446. UDP socket address 2444 may transmit RTP multiplexed encoded video and audio packets that are L4S enabled or disabled to UDP socket 2450 of remote-control system 2405, and UDP socket address 2444 may receive RTCP packets with SRC: <remote client device 1 IP address> L4S enabled or disabled from UDP socket 2450. Priority queue for RTP packets 2452 may receive the RTP multiplexed encoded video and audio packets from RTP sender 2454, which in turn was received by RTP sender 2454 from multiplexer 2456. Multiplexer 2456 may combine encoded video received from video encoder 2458 to obtain the multiplexed bitrate transmitted to rate controller 2462 and to obtain the multiplexed encoded video and audio packets transmitted to RTP sender 2454. Video encoder 2458 may encode camera data received from camera 2424. Priority queue 2452 may transmit a queue length to rate controller 2462, which may provide encoding properties and the target rate to encoder 2458.

[0259] UDP socket 2428 may transmit the RTP multiplexed encoded video and audio packets, and the indication of whether L4S is enabled or disabled, to transmission receiver 2464, which may transmit the RTCP packets to UDP socket 2428. Transmission receiver 2464 may forward the RTP multiplexed encoded video and audio packets to demultiplexer 2466, which may obtain encoded video PES and audio PES and transmit the encoded video PES to video decoder 2468 and transmit encoded audio PES to audio decoder 2470. Video decoder 2468 may transmit decoder video frame(s) to video Tenderer 2472 and may transmit decoded audio frame(s) to audio Tenderer 2474. Remote control operator 2476 may receive the rendered video frame and rendered audio frames of the construction equipment (e.g., at the work site or traveling between work sites) from video Tenderer 2472 and audio Tenderer

[0260] UDP socket 2434 may transmit haptic data to controller data transmission module 2478, and UDP socket 2434 may receive the controller input from controller data transmission module 2478. Such controller input may be received by controller data transmission module 2478 from remote control operator 2476. Similarly, controller data transmission n 2480 may receive haptic data from UDP socket 2436 and controller data transmission n 2480 may transmit controller input, received from remote operating equipment 2476, to UDP socket 2436, and controller data transmission n 2480 may transmit the haptic data to remote control operator 2476. In some embodiments, L4S (or another suitable preferential treatment technique) may be enabled for network data corresponding to controller inputs received from controller data transmission module 2478 and / or controller data transmission module 2480, based on incoming RTP audio / video network packets being L4S enabled (or another suitable preferential treatment technique), and L4S (or disabled for another suitable preferential treatment technique) may be disabled for network data corresponding to controller inputs received from the user, based on incoming RTP audio / video network packets being L4S disabled (or disabled for another suitable preferential treatment technique).

[0261] UDP socket 2450 may transmit the RTP multiplexed encoded video and audio packets, and the indication of whether L4S is enabled or disabled, to transmission receiver 2482, which may transmit the RTCP packets to UDP socket 2450. Transmission receiver 2482 may forward the RTP multiplexed encoded video and audio packets to demultiplexer 2484, which may obtain encoded video PES transmit the encoded video PES to video decoder 2468. Video decoder 2486 may transmit decoder video frame(s) to video Tenderer 2488. Remote control operator 2476 may receive the rendered video frame of the construction equipment (e.g., at the work site or traveling between work sites) from video Tenderer 2488.

[0262] FIG. 25 shows a flowchart of an illustrative process 2500 for selectively enabling L4S for a video streaming request based on whether it is for a live view of an on-premises camera, in accordance with some embodiments of this disclosure. In various embodiments, the individual steps 2502-2526 of process 2500 may be implemented by one or more components of the devices, methods, and systems of FIGS. 1-4, 13-24, and 26-28 and may be performed in combination with any of the other processes and aspects described herein. Although the present disclosure may describe certain steps 2502-2526 of process 2500 (and of other processes described herein) as being implemented by certain components of the devices, methods, and systems of FIGS. 1-4, 13-24, and 26-28, this is for purposes ofillustration only, and it should be understood that other components of the devices, methods, and systems of FIGS. 1-4, 13-24, and 26-28 may implement those steps 2502-2526 instead.

[0263] FIG. 26 shows a flowchart of an illustrative process 2600 for selectively enabling L4S for a video streaming request based on priority metadata of the video, in accordance with some embodiments of this disclosure. In various embodiments, the individual steps 2602- 2624 of process 2600 may be implemented by one or more components of the devices, methods, and systems of FIGS. 1-4, 13-25, and 27-28 and may be performed in combination with any of the other processes and aspects described herein. Although the present disclosure may describe certain steps 1802-1824 of process 2600 (and of other processes described herein) as being implemented by certain components of the devices, methods, and systems of FIGS. 1-4, 13-25, and 27-28, this is for purposes of illustration only, and it should be understood that other components of the devices, methods, and systems of FIGS. 1-4, 13-25, and 27-28 may implement those steps 1802-1824 instead.

[0264] As shown in FIGS. 25-26, the techniques described herein may leverage key characteristics of a request or of the metadata associated with the request to selectively enable L4S packet markings during transport of a video stream from (or associated with) an onpremises camera (e.g., in-home camera, surveillance camera, or any other suitable camera or sensor) and / or storage associated with such on-premises camera.

[0265] As shown in FIG. 25, when a client (a user) requests a video from the on-premises camera at 2502, the media server (e.g., cloud server 124 of FIG. 1) checks (at 1704) whether the desired video is a live view or a pre-recorded view. If the requested view is live, then the media server architects a connection between the video camera and the user (at 2506). This may be done using webRTC as a peer-to-peer connection, with the media server as an intermediary (a relay), another protocol such as WebSockets, and / or using any other suitable technique. Once established, the media server (at 2508) directs the on-premises camera to turn on L4S capability marking for this stream, as there is an expectation of low latency delivery for this stream. On the other hand, if the requested video stream is pre-recorded (e.g., saved earlier as a clip when motion was detected), then the media server retrieves (at 2510) the stream from storage, and L4S-capable packet markings may not be required as there may not be an expectation of low latency, as shown at 2512-2514. In this example, if the network experienced congestion and converted the ECN bits to CE, then the sender (e.g., the onpremises camera or the media server / video storage) may reduce the video quality, thus throttling throughput.

[0266] At 2516, upon determining that the client device has requested video streaming, processing may proceed to 2518, where the on-premises camera may be caused to set ECN bits in DiffServ field of packetized video to ECT(l), i.e., 01, and at 2520, video segments may be sent to the client device. At 2522, the system may determine whether the client requests video streaming, and if so, at 2524 may cause storage or the media server to set ECN bits in DiffServ field of packetized video to 00. At 2526, the system may cause video segments to be sent to the client.

[0267] FIG. 26 depicts an embodiment in which the user requests, for example, a prerecorded video stream. In the example of FIG. 26, at 2602, inferencing may be performed on such pre-recorded stream to tag higher priority segments or areas in the video. For example, portions of video with higher priority may include areas where motion, faces, and / or objects of interest have been detected. At 2604, the system may cause priority metadata to be preserved in transcoded video (e.g., MPEG-7 metadata multiplexed with a video stream). At 2606, a media server for on-premises video camera(s) may receive a streaming request for stored video, and at2608, the media server may forward the request to a storage server with parameters for locating or seeking the desired video segments. At 2610, the media server may establish a connection with the storage server (e.g., server 1104 of FIG. 11) and the client, and at 2612, configure L4S capability as a dynamic setting based on priority of video segments. For example, the media server may then mark the packetized video of higher priority areas as L4S-capable, thus ensuring that they are delivered with lower latency. For example, regions where saliency is relatively low may not benefit from the enhanced higher priority queue treatment, while higher priority video frames benefit from preferential treatment. Lower latency delivery of more salient video thus reduces packet drop probability in network congestion which in turn reduces frame loss and enhances the customer experience. In some embodiments, near-real time inferencing may be available, to implement the techniques of FIG. 26 in live streaming applications.

[0268] At 2614, the system may determine whether the client requests video streaming. If so, processing may proceed to 2616, where the system may determine whether a current packet at the media server prepared from transcoded video buffer is tagged as higher priority. If so, processing proceeds to 2618; otherwise, processing proceeds to 2620. At 2618, the storage or media server (e.g., media content source 1102 of FIG. 11) sets ECN bits in Diffserv field of packetized video to ECT(l), i.e., 01. Otherwise, at 2620, the storage or media server (e.g., media content source 1102 of FIG. 11) sets ECN bits in Diffserv field of packetized video to 00. At 2622, the system may send video segments to the client device,based on the processing performed at 2618. At 2624, the system may send video segments to the client device, based on the processing performed at 2620.

[0269] FIG. 27 shows an illustrative example 2700 of selectively treating network traffic as preferential in the context of placing wagers (e.g., micro-betting) over a network, in accordance with some embodiments of this disclosure. For example, L4S capability (or another mechanism for treating network traffic preferentially) may be turned on selectively enabled in micro-betting applications. For example, in a circumstance where a small timewindow exists when a micro-bet is open for an outcome that may occur in the immediate future, API calls made based on input received from the bettors to a micro-betting platform for placing bets (e.g., bettors stakes and outcomes on which the stakes are placed), as well as the API calls acknowledging the placement of these bets, must be executed with expediency. Thus, if a chain of events to determine an outcome is expected to begin at time t = T, then the betting app client can turn on L4S marking for packets that represent bet placement at time t = T -X, where X is the total amount of time that L4S capability is turned on, while time t = T represents the time when the platform no longer takes any more bets on the outcome.

[0270] In some embodiments, based at least in part on receiving and accepting a bet, the platform returns its acknowledgement with L4S packet marking turned on, if the bet placement packets were received with L4S on. By implementing this mechanism, the betting platform and client apps may give preferential treatment to the placement of bets in comparison to other types of data such as status / score updates, video, chat, and / or any other data. If upstream congestion were experienced in the network and CE bits were turned on, then, while the current bet would still go through, the client app may become aware of this and reduce sending further data to the platform such as frequent requests for status updates or other data. If the CE bits were turned on in the downstream when the platform were acknowledging the successful placement of a bet, then the platform may become aware of the downstream congestion and may reduce the frequency of its information updates to the client.

[0271] In some embodiments, a betting platform (or an odds generation platform) may selectively turn on L4S capability packet markings when an odds update is being sent to clients. Odds affect the actual winnings in a bet. Thus, if a user bets on an outcome with stagnant odds, then their bet may not be accepted. To ensure that the odds are speedily updated for users looking to place bets on an outcome associated with an event, the platform can send odds updates with L4S capability turned on. This minimizes rejection of bets by the platform due to stagnant odds.

[0272] In some embodiments, the techniques described herein apply to explicit congestion notification, whether that leads to a scalable response (requirement from a sender for L4S capability), or any other response, albeit non-scalable (applicable for ECT(l) or ECT(O) markings by sender). The table below provides a non-limiting summary of applications enabled by the techniques described herein, along with the suggested responses by a sender when the network marks the “CE” bits due to congestion.

[0273] FIG. 28 is a flowchart of a detailed illustrative process 2800 for preferential treatment of portions of network traffic, in accordance with some embodiments of this disclosure. In various embodiments, the individual steps of process 2800 may be implemented by one or more components of the devices, methods, and systems of FIGS. 1-4, 10-11, and 13-27and may be performed in combination with any of the other processes and aspects described herein. Although the present disclosure may describe certain steps ofprocess 2800 (and of other processes described herein) as being implemented by certain components of the devices, methods, and systems of FIGS. 1-4, 10-11, and 13-27, this is for purposes of illustration only, and it should be understood that other components of the devices, methods, and systems of FIGS. 1-4, 10-11, and 13-27may implement those steps instead.

[0274] At 2802, control circuitry (e.g., control circuitry 1121 of FIG. 11 of LAN networking equipment 1915 and / or control circuitry 1131 of WAN networking equipment 1117 of FIG. 11) and / or I / O circuitry (e.g., 1121 of FIG. 11 and / or 1132 of FIG. 11 of LAN networking equipment 1115 and WAN networking equipment 1117, respectively, of FIG. 11) and / or a network interface, may establish a network session with a device during which network session data is provided to the device via the network. For example, the control circuitry may provide the FPS video game 1300 of FIG. 21 over the network to the user (e.g., user 110 of FIG. 1) of the device. The network session may last for any suitable period of time, e.g., a continuous period of time while user 110 of FIG. 1 is playing the video game online during a particular day. In some embodiments, the network may correspond to an LAN and / or WAN and / or any other suitable network (e.g., communications network 1109 of FIG. 11). In some embodiments, the network session may be established automatically or based on a request received from the device. Such request may be received from, for example, a device (e.g., device 112 or 114 of FIG. 1, an XR device, an autonomous cleaning or mowing device, a remote controller device, or any other suitable device, or any combination thereof). In some embodiments, the device may be connected to an LAN (e.g., a Wi-Fi network) at a particular location (e.g., location 104 of FIG. 1, which may be a home or residence of user 110 or any other suitable type of location). For example, router, modem, and / or gateway 106 and / or 108 may be used to provide such LAN, to enable devices 112 and 114 to connect to the Internet and access any suitable application or service (e.g., a video conference call via Zoom 118 or a “Call of Duty” video game online mode 116). At 2802, the control circuitry may, based on the request, establish a network session with the device during which network session data is provided to the device via the network. For example, the control circuitry may provide the FPS video game 1300 over the network to the user (e.g., user 110 of FIG. 1) of the device. The network session may last for any suitable period of time, e.g., a continuous period of time while user 110 of FIG. 1 is playing the video game online during a particular day.

[0275] At 2802, the control circuitry (e.g., 212 of LAN networking equipment 1115 and / or 1131 of WAN networking equipment 1117) provides a first queue for preferential network traffic and a second queue for non-preferential traffic. For example, the first queue maycomprise a buffer for a low latency service flow (e.g., 206, 210, 218 of FIGS. 2A-2B), such as, for example, for L4S-capable traffic, and the second queue may comprise a buffer for a classic service flow (e.g., service flow 206, 210, and / or 218 of FIGS. 2A-2B), such as, for example, for non-L4S-capable traffic.

[0276] At 2806, the control circuitry may, during the network session, identify one or more characteristics of a current portion of the network session data. For example, the control circuitry may perform analysis in real time or near real time to identify audio and / or visual characteristics of the current portion (e.g., portion 1302 of video game 1300). Additionally, or alternatively, the control circuitry may obtain and analyze metadata for the current portion of the network traffic, e.g., metadata indicating that portion 1302 of FIG. 13 is a menu of the video game 1300, metadata indicating that portion 1304 is a combat portion of video game 1300, and / or metadata indicating that portion 1306 is an advertisement portion. As another example, such as during an XR session or an autonomous vehicle or robot, the control circuitry may determine characteristics of an environment surrounding the device or vehicle, e.g., a number of objects in a vicinity of a device or vehicle, a type of objects in the vicinity of the vehicle, weather conditions in the vicinity of the device or vehicle, whether the device or vehicle is indoors or outdoors, or any other suitable characteristics. For example, the device may use any suitable number of object detection techniques to identify number and / or types of objects, e.g., machine learning techniques and / or heuristic-based techniques.

[0277] At 2808, the control circuitry may, during the network session, determine, based on the one or more characteristics, to provide the current portion using the first queue or the second queue. For example, the control circuitry (e.g., 212 of LAN networking equipment 1115 and / or 1131 of WAN networking equipment 1117) may determine whether the first queue or the second queue (e.g., at WAN networking equipment 1117 and / or other suitable networking equipment) should be used for the current portion of network traffic. For example, the control circuitry may determine, at such networking equipment, whether the current portion of network traffic is intended for the first queue or the second queue enroute to its destination (e.g., device 112 or 114 of FIG. 1). Such determination may be based at least in part on whether the identified character! stic(s) at 2806 indicates that the current portion of the network session is latency sensitive (or not). In some embodiments, whether a portion of network traffic is latency sensitive depends on whether the portion of the data requires a threshold latency for an optimal quality of service. In the example of FIG. 13, the control circuitry may determine that, for the menu portion 1302 of video game 1300 and the cut scene portion 1306, a relatively lower level of latency is acceptable, e.g., since menu 1302is not a core part of the gameplay, and that is would not significantly alter the gaming experience if input to switch to another menu option has a relatively higher latency, or that cut scene portion 1306, or that cut scene portion 1306 is prerecorded and does not require fast responses to user inputs or any responses to user inputs. In such a circumstance, processing may proceed to 2812. On the other hand, if the character! stic(s) identified at 2806 indicate that the current portion of network session data does require at least a threshold level of latency (and / or other network characteristics, such as, for example, bandwidth, jitter, RTT, or any other suitable characteristic to improve QoE), processing may proceed to 2810. For example, the control circuitry may determine that portion 1304 or 1308 is a current portion of the network session data being transmitted to the client device, and since such portions correspond to a FPS combat portion of video game 1300 (e.g., as determined based on metadata and / or analysis of a current audio or visual portion of the transmitted data), at least a threshold level of latency is required for an optimal QoE.

[0278] As another example, the control circuitry may determine that the XR device (shown in FIG. 17) is outdoors (e.g., in an open field) and / or otherwise is not in the vicinity of any objects or less than a threshold number of objects and / or particular types of objects, and thus processing using the second queue may be suitable. On the other hand, the control circuitry may determine that the XR device (shown in FIG. 17) is indoors (e.g., in small room) and / or otherwise is in the vicinity of objects or more than a threshold number of objects and / or particular types of objects, and thus processing using the first queue may be optimal.

[0279] As another example, the control circuitry may determine that the an autonomous robot (shown in FIG. 19B) or a vehicle or construction equipment (shown in FIGS. 23A-23B) is not in the vicinity of any objects or less than a threshold number of objects and / or particular types of objects, and thus processing using the second queue may be suitable. On the other hand, the control circuitry may determine that the autonomous robot (shown in FIG. 19A) or a vehicle or construction equipment (shown in FIGS. 23A-23B) is in the vicinity of objects or more than a threshold number of objects and / or particular types of objects, and thus processing using the first queue may be optimal.

[0280] At 2810, the control circuitry may provide the current portion of network session data using the first queue (e.g., 206, 210 of FIG. 2A or 218 of FIG. 2B) for L4S network traffic, and / or any other suitable mechanism for treating network traffic preferentially. For example, the control circuitry may transmit, at the networking equipment (e.g., networking equipment 1115 and / or 1117) a current portion of the network session data intended for the first queue. On the other hand, at 2812, the control circuitry may provide the current portionof network session data using the second queue (e.g., 208, 212 of FIG. 2A or 2820 of FIG. 2B) for non-L4S network traffic, and / or any other suitable mechanism for treating network traffic non-preferentially. For example, the control circuitry may transmit, at the networking equipment (e.g., networking equipment 1115 and / or 1117) a current portion of the network session data intended for the second queue.

[0281] At 2814, the control circuitry may determine whether the network session (established at 2802) remains ongoing. For example, the control circuitry may determine whether a next portion of data transmitted to the client device (or other client device on the LAN of FIG. 1) is associated with a same application service provider as the previous portion(s) of data, and if so, processing may proceed to 2806. If a next portion of data transmitted to a client device (or other client device on the LAN of FIG. 1) is related to a different application service, processing may return to 2802.

[0282] In some embodiments, the process of FIG. 28 may be performed simultaneously or in parallel for multiple types of network traffic requested by devices on the LAN. In some embodiments, as part of the process of FIG. 28, the control circuitry may determine whether a particular network traffic data cap has been reached, e.g., per application or per device. For example, the LAN at location 104 may be associated with a data cap for network traffic categorized for gaming, either as specified by user 110 or as specified by service provider network 102 of FIG. 1. If a data cap is reached for a particular device or application, network traffic may be provided using the second, non-preferential network traffic queue.

[0283] Network traffic is controlled in a manner that provides preferential processing as a reward for requests made in a particular manner. For example, if one application flow has not recently requested prioritized processing from a network, when the application later makes a request for prioritized processing, it may be more likely to be granted. Also, for example, if a network node is busy, only an application data flow from an application that has not recently requested prioritized processing may be granted prioritized processing. In some implementations, each application data flow may be scored based on request frequency and levels of network congestion. For example, the credit may be applied in the form of the application data flow being processed more quickly despite network congestion. The decision may be made by a specific network node part. The applications that send the data may vary, for example the applications may be video or games. Further, for example, all data sent by all types of applications may be given a turn for quick processing so as to avoid preferential treatment of any one application type. If application data flows associated with an application usually do not request preferential processing but sends a request for prioritized processingfor a particularly large data chunk, the credit may decrease for future data sent by the application. If the application sends a request indicating that normal processing is acceptable, its credit may increase for future data sent by the application or device. When the network is busy, for example, such credit may be used to decide which data is processed preferentially.

[0284] FIG. 29 is an illustration of a queue assigner 2901 that may assign data packets received from a sender, such as application 2911, to a preferential queue 2915, which may be an L4S queue or to a non-preferential or classic queue 2913 (sometimes referred to as a “standard queue”). Packets at the head of each queue are processed by network node 2909, which may be a destination node or may be an intermediate network node that performs some processing, such as forwarding packets toward a destination — a node other than the ultimate destination node for the packet.

[0285] As discussed, L4S may enable Internet applications to achieve low queueing latency, low congestion loss, and scalable throughput control. Internet applications may transition away from congestion control algorithms that cause substantial queueing delay and instead adopt a new class of congestion controls that can seek capacity with very little queueing. These are aided by a modified form of Explicit Congestion Notification (ECN) from the network — an architecture focused on incremental deployment. This architecture defines mechanisms that allow the new class of L4S congestion controls to coexist with “classic” congestion controls in a shared network. The aim is for L4S latency and throughput to be usually much better while typically not impacting significantly processing performance of packets on the classic queue. With L4S, network service providers have introduced dual queueing in their network (especially the access network, which may have limited capacity). While most queue-building traffic (that is, traffic that causes congestion at a network node) passes through a regular / default queue, some traffic may use a low latency queue. This “priority lane” is typically expected to be used by ultra-low latency, non-queue-building traffic. However, applications that are regarded as queue-building may also benefit from this queue if they use it selectively. The scheduler can add data traffic to the L4S, when current traffic is insufficient, to use the priority queue.

[0286] Ultra-low-latency video, such as cloud gaming, often requires round-trip time (RTT) latency of 10-100 milliseconds, while low-latency OTT video streaming, such as live sports, may typically require RTT latency of 1-10 seconds. L4S delivery is often enabled for an entire flow belonging to the former traffic class (ultra-low-latency), while the latter class of traffic, being queue-building, may not need L4S. However, according to an implementation of the present disclosure, both classes of traffic would be able selectively to leverage L4Sdelivery. This is connected to the insight that while video encoders deliver an aggregate bitrate based on the target bitrate input, the instantaneous bitrate can vary drastically and may need assistance from the network for delivery when traffic surges. Hence, L4S may be selectively enabled based on the size of a picture / frame or Packetized Elementary Stream (PES) packet for ultra-low-latency RTP delivery or Common Media Application Format (CMAF) fragment size for low-latency delivery.

[0287] Low queueing delay depends on hosts sending their data smoothly, either at a low rate or responding to explicit congestion notifications. So low queueing latency is something hosts create themselves, not something the network gives them. This tends to ensure that selfinterest alone does not drive flows to mismark their packets for the low latency queue. However, traffic from an application that produces congestion may be classified for a low latency queue, whether accidentally or maliciously.

[0288] QProt is an algorithm to preserve low latency in data flows that are transmitted pursuant to the Data Over Cable Service Interface Specification (DOCSIS), an international telecommunications standard that permits the addition of high-bandwidth data transfer to an existing cable television (CATV) system. QProt protects traffic in the low latency queue from the harm due to excess queueing that would otherwise be caused by such anomalous behavior. The QProt algorithm maintains per-flow-state, where a “flow” is usually defined using an end-to-end (layer-4) 5-tuple. The flow-state may be defined using a queueing score that decays over time. The queueing score may be transformed into time units so that it represents the flow-state’s expiry time. A higher queueing score would thus have a longer expiry time. Non-queue-building flows tend to release their flow-state rapidly-usually expiring reasonably early in the gap between the packets of a normal flow. Then the memory can be recycled for packets from other flows that arrive in between. Thus, only queuebuilding flows hold flow state persistently and raise issues.

[0289] The responsibility that each flow bears for queueing delay may be quantified in a variety of ways. The QProt algorithm generates such a score based on the product of the rate of each flow and the level of congestion, both measured at the instant each packet arrives. The instantaneous flow rate is represented at each discrete event when a packet arrives by the packet’s size, which accumulates faster the more packets arrive within each unit of time. The level of congestion may be normalized to a dimensionless number between 0 and 1 (probNative). This fractional congestion level may be used. However, queueing delay alone may be used. The unit of the resulting queueing score according to the QProt algorithm is “congested bytes” per second, which distinguishes it from just bytes per second.

[0290] Then, during the periods between bursts, no application flow accumulates any queueing score-the high rate of incoming packets may thus be benign. The queueing score may thus represent the share of “blame” for queueing that each flow bears. The scoring algorithm may use the same internal variable, probNative, that the Active Queue Management (AQM) for the low latency queue uses to mark packets using Explicit Congestion Notification (ECN) — described below in more detail. In this way, the queueing score accumulates the size of each arriving packet of a flow but scaled by the value of probNative (in the range 0 to 1) at the instant the packet arrives. So, a data flow’s score may accumulate faster the higher the degree of queueing and the faster that the flow's packets arrive when there is queueing.

[0291] For example, the queueing score may be a term such as qLscore, calculated as: qLscore = min(buckets[bckt_id].t_exp - now + probNative * pkt sz where: buckets[bckt_id].t_exp represents the current expiry time of the data flow; and thus buckets[bckt_id].t_exp - now represents the leftover time (accumulated as a result of fast arrival of packets) of an unexpired / persistent flow; probNative * pkt_szAGING may represent the quantity added to the queueing score by the current packet. AGING may be calculated as:AGING = pow(2, (LG AGING-30) ) * T RES; where:T RES is the resolution of t exp (typically measured in ns).It may represent the time of each “tick” that is counted for the flow state to expire. It may expire when t exp catches up with current time (now).LG AGING may be a constant (default: 19). Note that AGING need not depend on the flow and although calculated from some input parameters it is a property of the system.

[0292] To trigger the ejection of a packet from the low latency (LL) queue back to the classic queue, the following logic may be used: if ( ( qdelay > CRITIC ALqL ) / / Test if qdelay over threshold... / / ...and if flow’s q’ing score scaled by qdelay / CRITICALqL / / ...exceeds CRITICALqLSCORE&& ( qdelay * qLscore > CRITICALqLPRODUCT ) ) / / Recall that CRITICALqLPRODUCT = CRITICALqL * CRITICALqLSCOREIf the queue delay threshold is exceeded, the flow’s queueing score may be temporarily scaled up by the ratio of the current queue delay to the threshold queueing delay, CRITICALqL. If this scaled up score exceeds another constant threshold CRITICALqLSCORE, the packet may be ejected. The higher the qLscore, the greater the likelihood may be of the packet getting ejected, when the low latency queue delay exceeds a certain threshold. It will be understood that terms such as “critical” and other such terms as used in the illustrative pseudocode provided herein are intended merely as variable identifiers. These are choices in labeling meant to highlight one or more processes described herein and do not necessarily imply that such terms are critical to the disclosure as a whole or to any aspect thereof.

[0293] In an embodiment, an application that judiciously or selectively uses L4S may get some credit as a reward for its avoidance of the precious L4S queue resource. This credit may be applied to a subsequent data flow associated with the application in one or more ways.

[0294] In an implementation of such an approach, a fine timer (e.g., measured in nanoseconds) may be used to count up very fast. The timer may count up fast enough that it is typically expired (when it catches up with qLscore) between packet interarrival times of non-queue-building data flows. On the other hand, for queue-building data flows (i.e., flows that result in congestion at the network node), the timer may be unable to count to expiry between packet arrivals, leading to accumulation in the timer value until a threshold is reached. It is as such times, when a packet burst arrives requesting use of the L4S queue, the network node may determine whether this data flow has persistent state as indicated by a different timer than the previously described expiry timer. This timer, henceforth called Macro Timer, may be used to determine the time since the previous burst which had requested the L4S queue. If it is determined that a threshold amount of time has passed since the previous use of the L4S queue by this data flow or by a data flow associated with the application that has received the credit, then a reduced timer expiry may be set for each new packet arrival of the data flow. If the packet meets the criteria for ejection, then it is moved to the classic queue for processing. On the other hand, if ejection criteria are not met, then the packet may be processed by the L4S queue. Each new packet in the burst may be treated inthis manner. The “credit,” i.e., the amount by which the expiry time for each new packet is reduced may be constant, or it may be a variable as explained below.

[0295] FIG. 30 is a flowchart showing an example of steps according to an aspect of the present disclosure. As shown at 3001, a traffic flow may request L4S queue processing during a data burst. For example, the ECN bits of a packet of data flow may indicate that L4S queue processing is requested. At 3003, the system, for example, the queue assigner 2901 shown in FIG. 29, may determine whether the next packet in the burst has been received. If yes, then at 3005, the system may determine whether the previous use of the L4S queue by a data flow associated with the application or other sender, excluding the current burst, occurred within a threshold period of time. That is, in an implementation, only application data flows that were received at the network node prior to the current data burst may be assessed to determine whether the current data flow is to be given credit.

[0296] Whether a data flow is associated with an application may be determined in a variety of ways, for example, using the 5-tuple discussed herein. A previous packet may be deemed to have been sent as part of the current burst (and thus may be excluded from receiving credit) if the queueing timer has not expired — indicating congestion.

[0297] If it is determined that the period since the previous use of the L4S queue by a data flow associated with the current application flow excluding the current burst, exceeds the threshold period of time (e.g., the sender has not used the L4S queue in a while), then at 3007 a credit may be applied for the packet. This credit may be in the form of reducing the timer expiry: the “blame” attributed to this particular application flow for causing traffic congestion at this network node is deemed to be less than the actual blame per the definition of the queueing score-the queueing score as calculated for packets of the present data flow may thus be deemed to be lower (e.g., to decay more quickly). This may result in a lower likelihood of the packets of the application flow being redirected to the standard or classic queue. On the other hand, if the previous data flow associated with the sender application was less than a threshold time ago, then at 3009 the application flow associated with this sender may be deemed not to be entitled to the credit: The timer expiry for the queueing score may thus be calculated. For example, a regular decay timer for the queueing score may be used to determine whether the packets of the flow are entitled to the L4S queue processing.

[0298] In either case, however the queueing score expiry timer is calculated, at 3011 packets that meet the criteria for ejection from the L4S queue are moved to the classic, non- preferential queue, and at 3013 are processed accordingly. Packets that are determined to qualify for the L4S queue are processed at 3015 by the L4S queue.

[0299] QProt algorithm has a timer with a “tick time” that may be several orders of magnitude shorter than typical packet interarrival times (for non-queue-building flows). According to an implementation, a new timer, Macro Timer, may be used that may have a “tick time” several orders of magnitude larger than typical packet interarrival times.

[0300] In an implementation, a reduced expiry of the flow state timer may be calculated by reducing the queueing score of the flow. The flow state timer is the expiry timer as defined in the QProt algorithm (not the Macro Timer). The Macro Timer may be cleared and re-started every time a flow state expires, i.e., the flow is no longer persistent based on the expiry timer. Thus, there may be a coupling between the expiry timer and the Macro Timer even though they operate on very different timescales.If (buckets[bckt_id].t_exp == now) record {buckets[bckt_id].id, t macro = 0}As shown in the pseudocode, the value of the flow-id (an appropriately concise representation of the 5-tuple) is recorded and t macro could be set to the current time and started. Also, this logic may run every single “tick time” of the expiry timer. Thus, as soon as a flow’s expiry timer expires, t macro may be set to current time, and it begins to count up.

[0301] When a packet arrives as part of a data flow, the following logic may be applied:If(buckets[bckt_id].t_exp > now) {If(t_macro > T THRESH) qLscore = min(buckets[bckt_id].t_exp - now + probNative * pkt_sz / AGING - ll selective, qLSCORE MAX)}According to the first line of the pseudocode, the system may check whether the flow state has not expired before proceeding to the next step. If the flow state has expired, then there is no need for providing a “credit” to this flow because there is likely little to no significant congestion at this node. On the other hand, if the flow state is indeed persistent, then the queue assigner 2901 may check whether t macro has counted up to a value beyond a threshold T THRESH. This is to ensure that this flow is deserving of the credit because it hasnot used the L4S queue resource in a while — for at least the threshold period of time set by T THRESH threshold.

[0302] If the threshold time has indeed passed, then the system may award credit to the flow in one of several ways. In an implementation, the queue assigner 2901 may calculate qLscore as known but may also subtract an extra term ll selective to reduce its value. This is shown in FIG. 31. FIG. 32 describes variables that may be used for pseudocode provided, according to some embodiments. If the qdelay is greater than CRITICALql, it means that there is significant congestion at the network resource. Then qLscore is reduced as described herein.

[0303] In some embodiments, ll selective may be a constant. In this way, the same “credit” may be given to each packet arriving in the new burst. In an embodiments, the “credit” may be “used up” over time: the credit may be reduced with the arrival of each new packet of the data flow when the flow state is persistent (due to the burst). In this case, ll selective may be modeled as: ll_selective = { K / (t_exp - now) “} where: t exp represents the expiry timer of the specific flow holding persistent state for which the new packet arrived.K is a constant.T exp - now represents an accumulation of the previous qLscore as it is being decayed with each tick.

[0304] With fast arrival of new packets in the burst, t exp will increase (even if it increases more slowly due to subtraction of ll selective), therefore (t exp - now) will increase faster than the time ticks. The resulting effect will be that ll selective will reduce with the arrival of each new packet in the burst. The exponent a controls the rate at which ll selective reduces with each packet.

[0305] In another implementation, the credit for good behavior by the flow of the sender may be awarded by dynamically adjusting the threshold for congestion at the receiving node, e.g., CRITICALqLPRODUCT. A statistical function, such as mean or median, e.g., beta qLScore, of previous packets’ good behavior, e.g., qLScore, may be calculated.

[0306] In a flow-specific implementation, a dynamic credit adjustment may be made to the CRITICALqLPRODUCT based on the behavior of previous packets over time. In the pseudocode shown below, a new variable, beta qLScore, may be used. On each packet’s arrival, the CRITICALqLPRODUCT may be updated as follows:CRITICALqLPRODUCT =CRITICALqL * (CRITICALqLSCORE + |CRITICALqLSCORE - beta_qLScore|)Thus, the CRITICALqLPRODUCT, which in an implementation, is a static threshold value, instead may be set as CRITICALqL * CRITICALqLSCORE. The CRITICALqLPRODUCT is used in determining whether the flow is not using the low latency queue in a “fair” manner and whether the flow gets moved to the non-LL queue (classic queue). Upon each packet’s arrival, the new variable beta qLScore gets updated by calling a new the method updateBetaScore(qLScore), which may calculate an adjustment beta score based on previous packets’ history. This function updateBetaScore could be a calculation like median, mean or any other definition of a beta score to use for a credit for a good application’s low latency use. As shown in FIGS. 33A-33D, when the next packet arrives, the CRITICALqLPRODUCT may be dynamically calculated based on the value of the beta qLScore. This will apply the credit in the last section of pseudocode below when the check is made to determine whether the flow is violating good L4S queue use behavior.

[0307] When a flow burst from a sender is experienced at a network node for the first time, there may be no indication in memory of previous selective request for use of the low latency queue by the sender. Thus, performance is likely to suffer in its initial burst as the data flow associated with the sender may not receive any “credit.” In an embodiment, the sender may indicate its selective use of the L4S resource by using a specific (currently unused) bit pattern, for example, in the 6-bit Differential Services (Diffserv) field of the IP Header of packets of its data flow.

[0308] In a related vein, in another implementation, an application may indicate to the ISP via an API call that it uses selective L4S enablement. For example, this may be implemented in a Network-as-a-Service (NaaS) architecture by an ISP.

[0309] In yet another implementation that addresses the issue of the debut of a sender’s flow at a receiving node, the ISP may identify the application flow type (and therefore its characteristic of selectively using L4S) using methods such as source IP / source port inspection, traffic distribution, Deep Packet Inspection (DPI), or the like, so that it can recognize that the flow is likely to use selective L4S. Machine learning techniques may be used to classify the data flow based on such and related data. In this case, a default value selective may be used in the initial burst.

[0310] It is also contemplated that a credit generated by a data flow of one sender application may be applied to benefit a data flow of another sender application. For example, in some embodiments, an application provider may have a plurality of flows to a specific end user. Each flow may be identified by its 5-tuple. It is possible that more than one application is running between a source and a destination IP (even if port numbers and protocol fields are different). This could occur, for example, if a server is providing cloud gaming service and another service such as chat / messenger. In such situations, a network service provider may transfer the credits of one flow to another flow. For example, the credits developed, say, when a voice call selectively uses L4S may be transferred to a data flow associated with cloud gaming. In this example, a burst in cloud gaming traffic may benefit from infrequent (selective) use of L4S by the voice chat. The network node may implement this by comparing the source and destination IP addresses of different flows that are active (based on the Macro Timer) at any time.

[0311] FIG. 34 is a flowchart that shows the processing according to an aspect of the present disclosure. At 3402, application data flow from a sender, such as application 2911, is received for processing at a network node. For example, the application flow may be associated with an application running on a server and the network node may be device 2909 at which processing of the network traffic sent by the application 2911 is requested. Queue assigner 2901 may determine whether congestion exists at the network node 2909 at which processing is requested. In an implementation, if no congestion is detected, then all data flows requesting preferential processing may be so processed, or the preferential queue may be ignored or not enabled at all because of a lack of data traffic.

[0312] Queue assigner 2901 at 3404 may then determine whether the application flow indicates that processing at the L4S queue has been requested for the application flow. If so, then at 3406, the queue assigner 2901 may retrieve from prior L4S queue request database 2903 data regarding one or more prior data flows associated with the application associated with the application flow.

[0313] At 3408, the queue assigner 2901 determines whether, within a threshold time period, a prior application data flow indicated a request for preferential queue processing. Queue assigner 2901 may consult a database, such as prior L4S queue request database 2903, that stores information for previously received network traffic for one or more senders. Such a database may keep track of the time for each application data flow received, whether the data flow requested preferential processing, the sender application associated with the data flow, the percentage or number of packets requesting such preferential processing in eachdata flow, and other statistical information about preferential queue processing requests. In an implementation, in addition, data regarding one or more prior data flows associated with a client device that has transmitted a server request to the sender application 2911 may be retrieved. For example, a client device 112 shown in FIG. 1 may be a gaming device that requests data processing by a remote server; this remote server may be the sender 2911 that sends data to the network node 2909. In this implementation, data regarding preferential queue processing requests for network traffic flows requested by the client device 112 may be kept track of. For example, such data may be exchanged between various queue assigner devices of a network. Queue assigner 2901 would then receive such data regarding preferential queue processing requests for network traffic flows requested by the client device 112 from another queue assigner on the network, and determine, accordingly, whether to award credit to the current data flow received from the sender 2911. In addition, the database may also maintain a record of whether packets of each data flow were actually processed in accordance with the request, which is whether they were processed in a preferential queue.

[0314] In an embodiment, if the data packets of the prior application data flow were not actually added to the preferential queue pursuant to the request for preferential queue processing, then such a prior data flow would not be counted as having requested referential processing. That is, in this embodiment only if the prior application data flow that requested preferential queue processing was actually granted preferential queue processing would prevent getting credit for the current data flow of the sender.

[0315] In an implementation, in addition to, or instead of, giving a credit to a sender for prior data flows not requesting preferential processing, the sender of prior data flows that did request preferential processing may be penalized. For example, the time indicated by the congestion decay timer may be increased as a penalty for a sender associated with a prior data flow that did request preferential queue processing.

[0316] If at 3408 the queue assigner 2901 determines that within a threshold time period the application has requested preferential queue processing for one or more prior data flows, then processing moves to 3412, where the packets of the current application data flow are processed according to the timer for decay. As discussed, the timer for decay may be a timer that decays, for example over a period measured in microseconds, the queueing score — a quantification of congestion at the network node. This queueing score may be an approximation or estimation for how long a current network congestion is estimated, or deemed, to last in view of the current traffic buildup. In this way, the probability that packets of the current data flow will be processed on the classic queue 2913 are increased. In animplementation, if the determination at 3408 is “yes,” then the request for preferential queue processing for the current packets may be ignored and the packets of the data flow may be processed on the classic cue without regard to the congestion decay timer.

[0317] On the other hand, if at 3408 it is determined that the application 2911 has not requested, within a threshold period of time, preferential queue processing for one or more prior data flows associated with the sender 2911, then at 3410, the application may be given credit for judicious use of the preferential queue, typically applied to the timer for expiry of flow state, The queue assigner 2901 decreases the time indicated by a timer for decay of congestion. In this way, the probability that packets of the current application data flow will be processed on the preferential queue 2915 are increased. In an implementation, if the determination at 3408 is “no,” then the request for preferential queue processing for the current packets may granted and the packets of the data flow may be processed on the preferential queue without regard to the congestion decay timer. At 3412, the network traffic of this application flow may be processed according to the timer for expiry of flow state. In this way, it may be determined whether packets will receive preferential processing at the network node.

[0318] FIG. 35 is a flowchart that shows processing according to an aspect of the present disclosure for providing credit for priority access to the preferential resource (e.g., the 14S queue of a network node). At 3502, a queueing parameter may be computed for assessing or predicting network congestion at the network node that is or will be receiving the network traffic from the sender, such as application 2911.

[0319] One or more of the operations shown as 3504, 3506 and 3508 may be conducted to implement the priority that the credit represents. At 3504, a timer that counts down to how much time is left until the queueing parameter is decayed. In an implementation, this timer may be decreased based on the quantity of the credit. As discussed, the quantity of credit awarded to the current application data flow may be based on the time period that elapsed since the sender application previously requested preferential processing (e.g., requested L4S queue for a previous data flow), the amount or percentage of a previous application data flow for which preferential processing was requested, the amount or percentage of a previous application data flow for which preferential processing was actually provided by the current network node, the amount or percentage of a previous application data flow for which preferential processing was actually provided by any network node, and the like, or a combination of the foregoing. Or the quantity of credit awarded to the current data flow maybe a constant — either credit is awarded for past non-resource grabbing behavior or credit is not awarded.

[0320] Based on, or based in part on, the credit awarded, at 3506 the queueing parameter may be adjusted. For example, the queueing parameter may be decreased by about 35%-66%, or by about 15%-98%, based on the credit. As discussed, the quantity of credit awarded to the current application data flow may be based on a variety of factors.

[0321] Or, as shown at 3508, the macro timer for determining the time until the next data flow associated with the sender application may then be re-started.

[0322] FIG. 36 is a flowchart that shows processing according to an aspect of the present disclosure. At 3602, the application flow from a sender, such as application 2911, is received for processing at a network node. For example, as discussed, the sender may be an application running on a server and the network node may be device 2909 at which processing of the network traffic sent by the application 2911 is requested. Queue assigner 2901 may determine whether congestion exists at the network node 2909 at which processing is requested. In an implementation, if no congestion is detected, then all data flows requesting preferential processing may be so processed, or the preferential queue may be ignored or not enabled at all because of a lack of data traffic.

[0323] Queue assigner 2901 may identify at 3604 that the network traffic indicates that processing at the L4S queue has been requested for the network traffic. If so, then at 3606, the queue assigner 2901 may retrieve from prior L4S queue request database 2903, data regarding one or more prior application data flows associated with the sender to identify a prior data flow associated with the sender.

[0324] At 3608, the queue assigner 2901 determines whether the sender 2911 has sent a prior application data flow that indicated a request for preferential queue processing. As discussed, this determination may be based on more than one previous data flow associated with the sending application. For example, it may be based on whether two or about 3-100 of the most recent data flows associated with the sending application have requested preferential processing. Or, it may be based on an the arithmetic mean, or median, of about 2-100 of the most recent data flows associated with the sending application have requested preferential processing. Queue assigner 2901 may consult a database, such as prior L4S queue request database 2903, that stores information for previously received network traffic for one or more senders. If yes, then normal L4S processing may continue at 3612.

[0325] On the other hand, if preferential processing has not been requested previously, or has not been requested for the number, mean or median of data flows, or for the number,mean or median of packets thereof, as discussed previously, then at 3610, the preferential processing requested may be selectively enabled. In an embodiment, the data packets would be granted access to the preferential queue. In another embodiment, priority (e.g., credits) would be given for accessing the preferential queue, as discussed herein.

[0326] FIG. 37 is a flowchart that shows processing according to an aspect of the present disclosure. At 3702, network traffic from a sender, such as application 2911, is received for processing at a network node. Queue assigner 2901 may identify that the flow indicates that processing at the L4S queue has been requested for the network traffic. If so, then at 3704, the queue assigner 2901 may retrieve from prior L4S queue request database 2903 data regarding one or more prior application data flows associated with the sender to identify a prior data flow associated with the sender. It may be determined that the sender has requested preferential processing for its prior application data flow, or that it has done so within a threshold period of time. Accordingly, its priority score would be maintained (e.g., it would receive no credit).

[0327] At 3706, another application data flow — the second application data flow— may be received from the sender. Queue assigner 2901 may consult a database and determine that this second application data flow indicates that processing at the L4S queue has not been requested. Accordingly, at 3708, sender’s future data flow’s priority score would be increased (e.g., it would receive credit).

[0328] At 3710, a third application data flow may be received from the sender, and it may be determined that this third application flow indicates that processing at the L4S queue is requested. It may be determined at 3712 whether, based on current network conditions, the third data flow would be denied preferential processing, or would be denied preferential processing for most of its packets, or for any of its packets. If so, then 3712, the priority score (e.g., credit awarded) at 3708 may be applied to the third data flow for priority access to the preferential resource (e.g., to grant access to the L4S queue or to increase the probability of access for the L4S queue). At 3714, the third data flow is processed according to the previous operation (e.g., with credit, as shown in 3712, or without credit, if 3712 is skipped).

[0329] One or more actions of the methods FIGS. 30-37 may be incorporated into or combined with one or more actions of any other process or embodiments described herein. These and other methods described herein, or portions thereof, may be saved to a memory or storage (e.g., of the systems shown in FIG. 10) or locally as one or more instructions or routines, which may be executed by any suitable device or system having access to the memory or storage to implement these methods.

[0330] In some embodiments, such preferential treatment may be selectively turned on and off based on determining whether to employ L4S (or not) for the portions of the data traffic. In an embodiment, the system of awarding credit for past good behavior of a sender may be selectively enabled or disabled.

[0331] In an embodiment, some applications’ data flows may be granted the credit described above based on past behavior for use in a preferentially queue, while other applications’ data flows may not be granted the credit based on similar past behavior. For example, network traffic identified as cloud gaming and in-home console / personal computer (PC) streaming to remote players, may be granted the granted credit selectively, as described above. While other applications that may be deemed less latency sensitive may never be granted credit. For example, system may determine that for cutscenes or a menu screen or advertisements of the video game, the second queue may be sufficient.

[0332] In a similar vein, some applications may be exempt from past behavior examination and always receive access to the L4S queue upon requesting it.

[0333] In some embodiments, L4S packet markings (or other preferential treatment) may be enabled during gameplay of portions of the game (e.g., determined by the game developer) such as, for example, when the player reaches a specific location within a level or during a boss fight. In some embodiments, the system and / or game engine may use pre-existing gaming metadata to determine when to signal the start of applying the L4S packet markings. In some embodiments, the metadata may be written to an XML file (or any other suitable file or data structure), and networking equipment or another suitable device may read that file and / or send or receive such file via an API. For example, a game engine may make an API call to “enable or disable” priority or preferential treatment (e.g., L4S and / or another suitable mechanism) for a particular portion of traffic. Similarly, the method of awarding credit based on past behavior may be selectively enabled or disabled based on such metadata.

[0334] In some embodiments, the system may perform audio and / or visual analysis in real time and / or any other suitable analysis to identify what type of scene is being played in the video game. In some embodiments, such analysis and / or metadata may indicate to the system whether a portion of the session data is latency sensitive (or not). For example, in a cloud gaming session, combat gameplay may be considered latency sensitive, while a cutscene, or an advertisement, or levels of the game where the user is just exploring a map, for example, may not be latency sensitive. As another example, in a football video game, plays during the game (e.g., a field goal or extra point being kicked, or a play from the line of scrimmage) may be considered latency sensitive, but a time after the play, or a time when a coach or fansare shown in gaps between plays, may not be considered latency sensitive. Similarly, the method of awarding credit based on past behavior may be selectively enabled or disabled based on such metadata.

[0335] This specification discloses embodiments which include, but are not limited to, the following items:

[0336] 1. A method comprising: receiving, at a first networking equipment, network traffic over a network; identifying a portion of the network traffic that corresponds to preferential network traffic, based on an indication from an application service provider or an Internet service provider (ISP); accessing a preferential network traffic data cap for data provided over the network to the first networking equipment over a particular time period; determining whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap; and based on determining that the amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap, causing identification of an action to be performed on the portion of the network traffic that corresponds to preferential network traffic.

[0337] 2. The method of item 1, wherein the first networking equipment is located at a particular location and provides a local area network (LAN) at the particular location, and second networking equipment is associated with providing a wide area network (WAN), the method further comprising: transmitting an indication to the second networking equipment indicating that the amount of preferential network traffic equals or exceeds the preferential network traffic data cap, wherein the indication causes the second networking equipment to perform the identification of the action to be performed.

[0338] 3. The method of item 1, wherein the first network equipment is associated with providing a wide area network (WAN), and the first network equipment causes the identification of the action to be performed.

[0339] 4. The method of item 1, wherein identifying the portion of the network traffic that corresponds to preferential network traffic comprises: determining whether data associated with the portion of the network traffic comprises bits indicative of the portion of the network traffic being designated as preferential by the application service provider or the ISP.

[0340] 5. The method of item 4, wherein the portion of network traffic designated as preferential by the application service provider, or the ISP is low latency, low loss, and scalable throughput (L4S)-capable.

[0341] 6. The method of item 4, wherein determining the amount of preferential network traffic provided to the first networking equipment over the particular time period comprises determining an amount of at least one of network traffic designated as preferential by the application service provider or network traffic designated as preferential by the ISP provided to the first networking equipment over the particular time period.

[0342] 7. The method of item 4, wherein: the portion of the network traffic is designated as preferential by the application service provider; accessing the preferential network traffic data cap for data provided over the network to the first networking equipment comprises accessing an application service provider preferential network traffic data cap for network traffic designated as preferential by the application service provider; and determining whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap comprises determining whether an amount of application service provider network traffic designated as preferential provided to the first networking equipment over the particular time period equals or exceeds the application service provider preferential network traffic data cap.

[0343] 8. The method of item 4, wherein: the portion of the network traffic is designated as preferential by the ISP;accessing the preferential network traffic data cap for data provided over the network to the first networking equipment comprises accessing an ISP preferential network traffic data cap for network traffic designated as preferential by the ISP; and determining whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap comprises determining whether an amount of ISP network traffic designated as preferential by the ISP provided to the first networking equipment over the particular time period equals or exceeds the ISP preferential network traffic data cap.

[0344] 9. The method of item 4, wherein the first networking equipment is located at a particular location and provides a local area network (LAN) at the particular location, and at the least one bit is designated by the ISP as preferential based at least in part on input received from, or preferences indicated by, a user associated with first networking equipment.

[0345] 10. The method of item 4, wherein: a first queue is provided to process preferential network traffic intended for or transmitted by the first networking equipment, and a second queue is provided to process non-preferential network traffic intended for or transmitted by the first networking equipment; and the identified action comprises modifying the bits to indicate that the portion of the network traffic is non-preferential network traffic, to cause the non-preferential network traffic to be transmitted via the second queue.

[0346] 11. The method of item 4, wherein: a first queue is provided to process preferential network traffic intended for or transmitted by the first networking equipment, and a second queue is provided to process non-preferential network traffic intended for or transmitted by the first networking equipment; and the identified action comprises maintaining the portion of the network traffic as preferential network traffic, to cause the portion of the network traffic to be transmitted via the first queue.

[0347] 12. The method of item 4, wherein: the portion of the network traffic is designated as preferential by the ISP; andthe method further comprises maintaining the portion of the network traffic as preferential network traffic based on determining that an amount of current network traffic having bits indicative of the ISP designating the current network traffic as preferential is below a threshold.

[0348] 13. The method of item 4, wherein: the portion of the network traffic is associated with a particular data provider; second networking equipment, associated with providing a wide area network (WAN), is configured to receive an indication from the first networking equipment that the bits do not indicate that the portion of the network traffic is at least one of preferential or experiencing congestion, despite the second networking equipment having caused the bits to indicate that the portion of the network traffic is at least one of preferential or experiencing congestion; and the second networking equipment is configured to, based on receiving the indication from the first networking equipment that the bits do not indicate that the portion of the network traffic is preferential or experiencing congestion, refrain from causing subsequent network traffic from the particular data provider and intended for the first networking equipment to include an indication that the subsequent network traffic is at least one of preferential or experiencing congestion.

[0349] 14. The method of item 4, wherein determining whether the data associated with the portion of the network traffic comprises bits indicative of the portion of the network traffic being preferential network traffic comprises: based on determining that the bits indicate that the portion of the network traffic is experiencing congestion, identifying a prior packet of the portion of the network traffic, and determining whether the prior packet of the portion of the network traffic comprises bits indicating whether the portion of the network traffic was designated as preferential by the application service provider or the ISP.

[0350] 15. The method of item 4, wherein second networking equipment is associated with the ISP and is further configured to: ingest the network traffic; determine whether the portion of the network traffic originated outside of the network of the ISP;based on determining that the portion of the network traffic originated outside of the network of the ISP, determine whether the data associated with the portion of the network traffic comprises, in error, the bits indicative of the portion of the network traffic being preferential network traffic; and based on determining that the data associated with the portion of the network traffic comprises, in error, the bits indicative of the portion of the network traffic being preferential network traffic, modifying the bits to indicate that the portion of the network traffic is non- preferential network traffic.

[0351] 16. The method of item 1, wherein the portion of the network traffic that corresponds to preferential network traffic is associated with a cellular network.

[0352] 17. A system comprising: first networking equipment; and control circuitry configured to: receive, at the first networking equipment, network traffic over a network; identify a portion of the network traffic that corresponds to preferential network traffic, based on an indication from an application service provider or an Internet service provider (ISP); access a preferential network traffic data cap for data provided over the network to the first networking equipment over a particular time period; determine whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap; and based on determining that the amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap, cause identification of an action to be performed on the portion of the network traffic that corresponds to preferential network traffic.

[0353] 18. The system of item 17, wherein the first networking equipment is located at a particular location and provides a local area network (LAN) at the particular location, and second networking equipment is associated with providing a wide area network (WAN), and the control circuitry is further configured to:transmit an indication to the second networking equipment indicating that the amount of preferential network traffic equals or exceeds the preferential network traffic data cap, wherein the indication causes the second networking equipment to perform the identification of the action to be performed.

[0354] 19. The system of item 17, wherein the first network equipment is associated with providing a wide area network (WAN), and the first network equipment causes the identification of the action to be performed.

[0355] 20. The system of item 17, wherein the control circuitry is further configured to identify the portion of the network traffic that corresponds to preferential network traffic by: determining whether data associated with the portion of the network traffic comprises bits indicative of the portion of the network traffic being designated as preferential by the application service provider or the ISP.

[0356] 21. The system of item 20, wherein the portion of network traffic designated as preferential by the application service provider, or the ISP is low latency, low loss, and scalable throughput (L4S)-capable.

[0357] 22. The system of item 20, wherein the control circuitry is further configured to determine the amount of preferential network traffic provided to the first networking equipment over the particular time period by determining an amount of at least one of network traffic designated as preferential by the application service provider or network traffic designated as preferential by the ISP provided to the first networking equipment over the particular time period.

[0358] 23. The system of item 20, wherein: the portion of the network traffic is designated as preferential by the application service provider; the control circuitry is further configured to access the preferential network traffic data cap for data provided over the network to the first networking equipment by accessing an application service provider preferential network traffic data cap for network traffic designated as preferential by the application service provider; andthe control circuitry is further configured to determine whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap by determining whether an amount of application service provider network traffic designated as preferential provided to the first networking equipment over the particular time period equals or exceeds the application service provider preferential network traffic data cap.

[0359] 24. The system of item 20, wherein: the portion of the network traffic is designated as preferential by the ISP; the control circuitry is further configured to access the preferential network traffic data cap for data provided over the network to the first networking equipment by accessing an ISP preferential network traffic data cap for network traffic designated as preferential by the ISP; and the control circuitry is further configured to determine whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap by determining whether an amount of ISP network traffic designated as preferential by the ISP provided to the first networking equipment over the particular time period equals or exceeds the ISP preferential network traffic data cap.

[0360] 25. The system of item 20, wherein the first networking equipment is located at a particular location and provides a local area network (LAN) at the particular location, and at the least one bit is designated by the ISP as preferential based at least in part on input received from, or preferences indicated by, a user associated with first networking equipment.

[0361] 26. The system of item 20, wherein: a first queue is provided to process preferential network traffic intended for or transmitted by the first networking equipment, and a second queue is provided to process non-preferential network traffic intended for or transmitted by the first networking equipment; and the identified action comprises modifying the bits to indicate that the portion of the network traffic is non-preferential network traffic, to cause the non-preferential network traffic to be transmitted via the second queue.

[0362] 27. The system of item 20, wherein: a first queue is provided to process preferential network traffic intended for or transmitted by the first networking equipment, and a second queue is provided to process non-preferential network traffic intended for or transmitted by the first networking equipment; and the identified action comprises maintaining the portion of the network traffic as preferential network traffic, to cause the portion of the network traffic to be transmitted via the first queue.

[0363] 28. The system of item 20, wherein: the portion of the network traffic is designated as preferential by the ISP; and the control circuitry is further configured to maintain the portion of the network traffic as preferential network traffic based on determining that an amount of current network traffic having bits indicative of the ISP designating the current network traffic as preferential is below a threshold.

[0364] 29. The system of item 20, wherein: the portion of the network traffic is associated with a particular data provider; second networking equipment, associated with providing a wide area network (WAN), is configured to receive an indication from the first networking equipment that the bits do not indicate that the portion of the network traffic is at least one of preferential or experiencing congestion, despite the second networking equipment having caused the bits to indicate that the portion of the network traffic is at least one of preferential or experiencing congestion; and the second networking equipment is configured to, based on receiving the indication from the first networking equipment that the bits do not indicate that the portion of the network traffic is preferential or experiencing congestion, refrain from causing subsequent network traffic from the particular data provider and intended for the first networking equipment to include an indication that the subsequent network traffic is at least one of preferential or experiencing congestion.

[0365] 30. The system of item 20, wherein the control circuitry is further configured to determine whether the data associated with the portion of the network traffic comprises bits indicative of the portion of the network traffic being preferential network traffic by:based on determining that the bits indicate that the portion of the network traffic is experiencing congestion, identifying a prior packet of the portion of the network traffic, and determining whether the prior packet of the portion of the network traffic comprises bits indicating whether the portion of the network traffic was designated as preferential by the application service provider or the ISP.

[0366] 31. The system of item 20, wherein second networking equipment is associated with the ISP and is further configured to: ingest the network traffic; determine whether the portion of the network traffic originated outside of the network of the ISP; based on determining that the portion of the network traffic originated outside of the network of the ISP, determine whether the data associated with the portion of the network traffic comprises, in error, the bits indicative of the portion of the network traffic being preferential network traffic; and based on determining that the data associated with the portion of the network traffic comprises, in error, the bits indicative of the portion of the network traffic being preferential network traffic, modifying the bits to indicate that the portion of the network traffic is non- preferential network traffic.

[0367] 32. The system of item 17, wherein the portion of the network traffic that corresponds to preferential network traffic is associated with a cellular network.

[0368] 33. A non-transitory computer-readable medium having non-transitory computer- readable instructions encoded thereon that, when executed by control circuitry, cause the control circuitry to: receive, at the first networking equipment, network traffic over a network; identify a portion of the network traffic that corresponds to preferential network traffic, based on an indication from an application service provider or an Internet service provider (ISP); access a preferential network traffic data cap for data provided over the network to the first networking equipment over a particular time period;determine whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap; and based on determining that the amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap, cause identification of an action to be performed on the portion of the network traffic that corresponds to preferential network traffic.

[0369] 34. The non-transitory computer-readable medium of item 33, wherein the first networking equipment is located at a particular location and provides a local area network (LAN) at the particular location, and second networking equipment is associated with providing a wide area network (WAN), and the control circuitry is further configured to: transmit an indication to the second networking equipment indicating that the amount of preferential network traffic equals or exceeds the preferential network traffic data cap, wherein the indication causes the second networking equipment to perform the identification of the action to be performed.

[0370] 35. The non-transitory computer-readable medium of item 33, wherein the first network equipment is associated with providing a wide area network (WAN), and the first network equipment causes the identification of the action to be performed.

[0371] 36. The non-transitory computer-readable medium of item 33, wherein the control circuitry is further configured to identify the portion of the network traffic that corresponds to preferential network traffic by: determining whether data associated with the portion of the network traffic comprises bits indicative of the portion of the network traffic being designated as preferential by the application service provider or the ISP.

[0372] 37. The non-transitory computer-readable medium of item 36, wherein the portion of network traffic designated as preferential by the application service provider, or the ISP is low latency, low loss, and scalable throughput (L4S)-capable.

[0373] 38. The non-transitory computer-readable medium of item 36, wherein the control circuitry is further configured to determine the amount of preferential network trafficprovided to the first networking equipment over the particular time period by determining an amount of at least one of network traffic designated as preferential by the application service provider or network traffic designated as preferential by the ISP provided to the first networking equipment over the particular time period.

[0374] 39. The non-transitory computer-readable medium of item 36, wherein: the portion of the network traffic is designated as preferential by the application service provider; the control circuitry is further configured to access the preferential network traffic data cap for data provided over the network to the first n...

Claims

What is claimed is:

1. A method comprising: receiving, at a first networking equipment, network traffic over a network; identifying a portion of the network traffic that corresponds to preferential network traffic, based on an indication from an application service provider or an Internet service provider (ISP); accessing a preferential network traffic data cap for data provided over the network to the first networking equipment over a particular time period; determining whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap; and based on determining that the amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap, causing identification of an action to be performed on the portion of the network traffic that corresponds to preferential network traffic.

2. The method of claim 1, wherein the first networking equipment is located at a particular location and provides a local area network (LAN) at the particular location, and second networking equipment is associated with providing a wide area network (WAN), the method further comprising: transmitting an indication to the second networking equipment indicating that the amount of preferential network traffic equals or exceeds the preferential network traffic data cap, wherein the indication causes the second networking equipment to perform the identification of the action to be performed.

3. The method of any one of claims 1-2, wherein the first network equipment is associated with providing a wide area network (WAN), and the first network equipment causes the identification of the action to be performed.

4. The method of any one of claims 1-3, wherein identifying the portion of the network traffic that corresponds to preferential network traffic comprises:determining whether data associated with the portion of the network traffic comprises bits indicative of the portion of the network traffic being designated as preferential by the application service provider or the ISP.

5. The method of claim 4, wherein the portion of network traffic designated as preferential by the application service provider, or the ISP is low latency, low loss, and scalable throughput (L4S)-capable.

6. A system comprising: first networking equipment; and control circuitry configured to: receive, at the first networking equipment, network traffic over a network; identify a portion of the network traffic that corresponds to preferential network traffic, based on an indication from an application service provider or an Internet service provider (ISP); access a preferential network traffic data cap for data provided over the network to the first networking equipment over a particular time period; determine whether an amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap; and based on determining that the amount of preferential network traffic provided to the first networking equipment over the particular time period equals or exceeds the preferential network traffic data cap, cause identification of an action to be performed on the portion of the network traffic that corresponds to preferential network traffic.

7. A computer-implemented method, comprising: establishing a network session with a device during which network session data is provided to the device via a network, wherein the device is connected to a local area network (LAN), and wherein portions of data are configured to be selectively provided to the device via the network using a first queue for preferential network traffic or a second queue for non- preferential network traffic, the first queue and the second queue being provided by at least one networking equipment; during the network session:identifying one or more characteristics of a portion of the network session data; determining, based on the one or more characteristics of the portion of the network session data, whether to provide the portion of the network session data to the device using the first queue for preferential network traffic or the second queue for non-preferential network traffic; and based on determining to provide the portion of the network session data using the first queue for preferential network traffic, providing the portion of the network session data to the device using the first queue for preferential network traffic.

8. The method of claim 7, wherein: identifying the one or more characteristics of the portion of the network session data comprises determining whether the portion of the network session data is latency sensitive; determining whether to provide the portion of the network session data to the device using the first queue for preferential network traffic or the second queue for non-preferential network traffic is based on whether the portion of the network session data is latency sensitive; and based on determining that the portion of the network session data is latency sensitive, providing the portion of the network session data to the device using the first queue for preferential network traffic.

9. The method of claim 8, wherein: the network session comprises a video gaming session and the network session data comprises data for a video game played by a user of the device during the video gaming session; and determining whether the portion of the network session data is latency sensitive comprises determining whether the portion of the network session data corresponds to one or more characters within the video game being controlled by the user or to a cutscene or menu screen of the video game.

10. The method of claim 9, wherein determining whether the portion of the network session data is latency sensitive is further based at least in part on at least one of a type of the video game or a type of a video game portion in which the one or more characters are being controlled by the user.

11. The method of claim 9, wherein the determining whether the portion of the network session data is latency sensitive is based at least in part on identifying metadata associated with the video game and determining whether the metadata indicates that the portion of the network session data is latency sensitive.

12. A system comprising: control circuitry configured to: establish a network session with a device during which network session data is provided to the device via a network, wherein the device is connected to a local area network (LAN), and wherein portions of data are configured to be selectively provided to the device via the network using a first queue for preferential network traffic or a second queue for non- preferential network traffic, the first queue and the second queue being provided by at least one networking equipment; during the network session: identify one or more characteristics of a portion of the network session data; determine, based on the one or more characteristics of the portion of the network session data, whether to provide the portion of the network session data to the device using the first queue for preferential network traffic or the second queue for non- preferential network traffic; and based on determining to provide the portion of the network session data using the first queue for preferential network traffic, provide the portion of the network session data to the device using the first queue for preferential network traffic.

13. A computer-implemented method comprising: detecting a characteristic of current network traffic transmitted by an application, wherein the characteristic indicates a request for processing via a preferential queue; identifying a prior network traffic transmission profile of the application; based at least on the prior network traffic transmission profile of the application, determining that the application has not, within a threshold time period, previously transmitted prior network traffic that indicated a prior request for the processing via the preferential queue; andbased at least on the determining that the application has not, within the threshold time period, previously transmitted the prior network traffic that indicated the request for the processing via the preferential queue, providing a credit for the current network traffic for accessing a device using the preferential queue.

14. The method of claim 13, further comprising: computing a queueing parameter indicating a queue congestion for processing by the device, wherein the providing the credit for the current network traffic for the accessing the device using the preferential queue comprises adjusting the queueing parameter to enable providing the current network traffic to the device using the preferential queue.

15. The method of claim 14, wherein the queueing parameter comprises a queueing score calculated based on combining a rate of flow of traffic provided to the device via the network and a level of congestion for processing by the device, and the adjusting the queueing parameter comprises changing a decay time of the queueing score.

16. The method of any one of claims 13-15, further comprising: computing a queueing parameter indicating congestion for processing by the device, and wherein the providing the credit for the current network traffic for the accessing the device using the preferential queue comprises awarding an amount of credit according to a quantity or percentage of the previously transmitted prior network traffic that indicated the request for the processing via the preferential queue.

17. The method of any one of claims 13-16, wherein the preferential queue is a Low Latency, Low Loss, and Scalable Throughput queue, and the network traffic transmitted by the application is a data flow, and wherein the detecting the characteristic of the current network traffic transmitted by the application comprises accessing an Explicit Congestion Notification control bit of packets of the data flow.

18. A system comprising: a memory; and processing circuitry configured:to detect a characteristic of current network traffic transmitted by an application, wherein the characteristic indicates a request for processing via a preferential queue; to identify, based at least on accessing the memory, a prior network traffic transmission profile of the application; based at least on the prior network traffic transmission profile of the application, to determine that the application has not, within a threshold time period, previously transmitted prior network traffic that indicated a prior request for the processing via the preferential queue; and based at least on the determining that the application has not, within the threshold time period, previously transmitted the prior network traffic that indicated the request for the processing via the preferential queue, to provide a credit for the current network traffic for accessing a device using the preferential queue.

Images