Hierarchical quality of service traffic management

Abstract

A network node (700) of a network (100) receives one or more packet flows corresponding to a user of a network (100) and determines one or more marking thresholds based on a rate allocation corresponding to the user. The network node (700) marks a field of a packet of the one or more packet flows with one of a plurality of predefined drop precedence values based on a comparison between a random value and the one or more marking thresholds. The rate allocation may be received from another network node that determines the rate allocation based on estimated throughput demand.

Description

[0001] HIERARCHICAL QUALITY OF SERVICE TRAFFIC MANAGEMENT

[0002] RELATED APPLICATIONS

[0003] This application claims the benefit of U.S. Provisional Application No. 63 / 556840, filed 22 February 2024, the entire disclosure of which being hereby incorporated by reference herein.

[0004] TECHNICAL FIELD

[0005] The present disclosure generally relates to the field of communication networks and is more particularly directed to network traffic management using Hierarchical Quality of Service (HQoS).

[0006] BACKGROUND

[0007] Hierarchical resource sharing and Quality of Service (QoS) in microwave fronthaul networks are currently implemented by simple Weighted Fair Queuing (WFQ) and Strict Priority (SP) scheduling among limited number of traffic classes and limited levels (1 -2) of hierarchy. The resource sharing is enforced by each network node individually without being aware of the topology, the load of various links, and thus the network-level objectives of QoS. The link-by-link approach has the advantage that it is supported by most networking devices including nodes in the microwave fronthaul.

[0008] For private Wide Area Networks (WANs), Google has proposed Bandwidth Enforcer (BwE). BwE is able to optimize resource allocation of traffic classes organized in a hierarchy and applies bandwidth enforcers to shape the traffic of different classes close to the sources. This approach relies on continuous measurements and periodically (10-20 secs) reconfigures the enforcers to optimize the network behavior.

[0009] Other approaches include a distributed method for addressing the same problem as BwE that relies on special packet marking. This special packet marking may mark a packet with a rank or a Packet Value (PV) that is assigned to each packet and expresses the importance of the packet in the traffic mix. The solution also includes a scheduler that drops the less important packets according to the marking when congestion is encountered. This solution can also implement HQoS and can support the achievement of network-wide resource sharing objectives. However, such solutions typically require a new header field introduced to store the packet marking and a scheduler that is aware of this field, e.g., to perform packet pairing, support scheduling logic, and so on. These limitations make deployment of such approaches difficult in practice. SUMMARY

[0010] Embodiments of present disclosure perform HQoS network traffic management using a packet marking process in the data plane of a plurality of packet flows and a rate allocation process outside of the data plane of the packet flows.

[0011] Particular embodiments include a method of marking packets performed by a network node of a network. The method comprises receiving one or more packet flows corresponding to a user of the network. The method further comprises determining one or more marking thresholds based on a rate allocation corresponding to the user. The method further comprises marking a field of a packet of the one or more packet flows with one of a plurality of predefined drop precedence values based on a comparison between a random value and the one or more marking thresholds.

[0012] In some embodiments, the field is a Differentiated Services Code Point field, Multiprotocol Label Switching experimental bits field, or Explicit Congestion Notification field.

[0013] In some embodiments, determining the one or more marking thresholds based on the rate allocation comprises setting a first marking threshold to the rate allocation.

[0014] In some embodiments, determining the one or more marking thresholds based on the rate allocation comprises setting a second marking threshold to the rate allocation adjusted by a constant factor.

[0015] In some embodiments, the one or more marking thresholds define a plurality of ranges, each range having a respective length and the length of each range controls a probability of marking the packet with a respective one of the plurality of predefined drop precedence values.

[0016] In some embodiments, the method further comprises forwarding the packet comprising the marked field to a further network node.

[0017] In some embodiments, the method further comprises measuring a rate of the one or more packet flows. In some such embodiments, the method further comprises generating the random value from a range limited by the rate of the one or more packet flows.

[0018] In some embodiments, the method further comprises sending the rate of the one or more packet flows to a further network node that is outside of a data plane of the one or more packet flows and receiving the rate allocation from the further network node.

[0019] In some embodiments, the one or more marking thresholds define a plurality of Random Early Detection, RED, algorithm ranges, each of the RED ranges corresponding to a respective one of the predefined drop precedence values.

[0020] Other embodiments include a network node configured to receive one or more packet flows corresponding to a user of the network. The network node is further configured to determine one or more marking thresholds based on a rate allocation corresponding to the user. The network node is further configured to mark a field of a packet of the one or more packet flows with one of a plurality of predefined drop precedence values based on a comparison between a random value and the one or more marking thresholds.

[0021] In some embodiments, the network node is further configured to perform any one of the packet marking methods described above.

[0022] In some embodiments, the network node comprises processing circuitry and a memory. The memory contains instructions executable by the processing circuitry whereby the network node is configured to perform packet marking as described above.

[0023] Other embodiments include a computer program comprising instructions which, when executed on processing circuitry of a network node, cause the processing circuitry to carry out any one of the packet marking methods described above.

[0024] Yet other embodiments include a carrier containing said computer program. The carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0025] Embodiments of the present disclosure also include a method of resource allocation performed by a network node of a network. The method comprises obtaining, from a plurality of other network nodes in the network, rate measurements of a plurality of packet flows for a plurality of users of the network. The method further comprises estimating throughput demand for each user based on the rate measurements. The method further comprises determining a rate allocation at each of the other network nodes for each user based on the estimated throughput demands. The method further comprises sending the rate allocations to the plurality of other network nodes.

[0026] In some embodiments, sending the rate allocation to the plurality of other network nodes controls drop precedence packet marking performed by the other network nodes.

[0027] In some embodiments, the method further comprises periodically re-estimating the throughput demand for each user based on periodically updated rate measurements.

[0028] In some embodiments, estimating the throughput demand for each user is further based on resource availability in the network.

[0029] In some embodiments, the method further comprises determining a traffic aggregate for each user. Estimating the throughput demand for each user is further based on a resource sharing target between the traffic aggregates. In some such embodiments, the method further comprises determining the resource sharing target between the traffic aggregates based on a number of the users, traffic volume, and a resource sharing policy. In some such embodiments, the resource sharing policy indicates a sharing target between different tenants of the network. In some such embodiments, the resource sharing policy indicates a sharing target between users of the different tenants of the network.

[0030] In some embodiments, the resource sharing policy is represented within the network node as a graph of rate transformers. In some such embodiments, each rate transformer maps an outgoing aggregated rate to incoming rates of different traffic groups taking part in an aggregate traffic of the plurality of packet flows.

[0031] In some embodiments, the network node is outside of a data plane of the plurality of packet flows.

[0032] In some embodiments, estimating the throughput demand for each user is further based on a resource sharing target between the traffic aggregates.

[0033] In some embodiments, determining the rate allocations at each of the other network nodes comprises determining the rate allocations at each of the other network nodes in descending congestion order from most congested other network node to least congested other network node.

[0034] Other embodiments include a network node configured to obtain, from a plurality of other network nodes in the network, rate measurements of a plurality of packet flows for a plurality of users of the network. The network node is further configured to estimate throughput demand for each user based on the rate measurements. The network node is further configured to determine a rate allocation at each of the other network nodes for each user based on the estimated throughput demands. The network node is further configured to send the rate allocations to the plurality of other network nodes.

[0035] In some embodiments, the network node is further configured to perform any one of the resource allocation methods described above.

[0036] In some embodiments, the network node comprises processing circuitry and a memory. The memory contains instructions executable by the processing circuitry whereby the network node is configured to perform resource allocation as described above.

[0037] Other embodiments include a computer program comprising instructions which, when executed on processing circuitry of a network node, cause the processing circuitry to carry out any5 one of the resource allocation methods described above.

[0038] Yet other embodiments include a carrier containing said computer program. The carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0039] BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Aspects of the present disclosure are illustrated by way of example and are not limited by the accompanying figures with like references indicating like elements. In general, the use of a reference numeral should be regarded as referring to the depicted subject matter according to one or more embodiments, whereas discussion of a specific instance of an illustrated element will append a letter designation thereto (e.g., discussion of a RAN node 20, generally, as opposed to discussion of particular instances of RAN nodes 20a, 20b).

[0041] FIG. 1 illustrates an example HQoS scheduler for six packet flows. FIG. 2 illustrates a marker graph corresponding to the HQoS scheduler in FIG. 1 .

[0042] FIG. 3 is a schematic block diagram illustrating an example Radio Access Network comprising a plurality of network nodes.

[0043] FIG. 4 is a schematic block diagram illustrating an example network comprising a plurality of network nodes.

[0044] FIG. 5 is a graph illustrating example TVFs.

[0045] FIG. 6 is a code snippet illustrating an example resource allocation algorithm.

[0046] FIG. 7 is a code snippet illustrating an example algorithm for determining a congestion threshold value.

[0047] FIG. 8 is a code snippet illustrating an example algorithm for finding a throughput allocation for a flow at a given congestion level.

[0048] FIG. 9 is a code snippet illustrating an example inverse rate transformation algorithm.

[0049] FIG. 10 is a flow diagram illustrating an example resource allocation process.

[0050] FIG. 1 1 is a flow diagram illustrating an example packet marking process.

[0051] FIG. 12 is a schematic diagram illustrating an example of DSCP-based RED AQM.

[0052] FIG. 13 is a table illustrating expected ideal rates in a first scenario.

[0053] FIG. 14 is a graph illustrating experimentally obtained throughput results for the first scenario using a DSCP marking algorithm.

[0054] FIG. 15 is a graph illustrating experimentally obtained throughput results for the first scenario using a traditional PV marking algorithm.

[0055] FIG. 16 is a graph illustrating experimentally obtained throughput results for the first scenario using a DSCP marking algorithm with DCTCP congestion control and ECN marking enabled.

[0056] FIG. 17 is a graph illustrating experimentally obtained throughput results for the first scenario using a traditional PV marking algorithm with DCTCP congestion control enabled.

[0057] FIG. 18 is a table illustrating fairness index calculations for the first scenario.

[0058] FIG. 19 is a table illustrating expected ideal rates in a second scenario.

[0059] FIG. 20 is a graph illustrating experimentally obtained throughput results for the second scenario using a DSCP marking algorithm.

[0060] FIG. 21 is a graph illustrating experimentally obtained throughput results for the second scenario using a traditional PV marking algorithm.

[0061] FIG. 22 is a graph illustrating experimentally obtained throughput results for the second scenario using a DSCP marking algorithm with DCTCP congestion control and ECN marking enabled. FIG. 23 is a graph illustrating experimentally obtained throughput results for the second scenario using a traditional PV marking algorithm with DCTCP congestion control enabled.

[0062] FIG. 24 is a table illustrating fairness index calculations for the second scenario.

[0063] FIG. 25 is a table illustrating expected ideal rates in a third scenario.

[0064] FIG. 26 is a graph illustrating experimentally obtained throughput results for the third scenario using a DSCP marking algorithm.

[0065] FIG. 27 is a graph illustrating experimentally obtained throughput results for the third scenario using a traditional PV marking algorithm.

[0066] FIG. 28 is a graph illustrating experimentally obtained throughput results for the third scenario using a DSCP marking algorithm with DCTCP congestion control and ECN marking enabled.

[0067] FIG. 29 is a graph illustrating experimentally obtained throughput results for the third scenario using a traditional PV marking algorithm with DCTCP congestion control enabled.

[0068] FIG. 30 is a table illustrating fairness index calculations for the third scenario.

[0069] FIG. 31 is a flow diagram illustrating an example packet marking method performed by a network node.

[0070] FIG. 32 is a flow diagram illustrating an example resource allocation method performed by a network node.

[0071] FIG. 33 is a schematic block diagram of an example network node.

[0072] FIG. 34 is a schematic block diagram illustrating an example wireless network environment.

[0073] FIG. 35 is a schematic block diagram illustrating an example UE.

[0074] FIG. 36 is a schematic block diagram illustrating an example network node.

[0075] FIG. 37 is a schematic block diagram illustrating an example virtualization environment.

[0076] DETAILED DESCRIPTION

[0077] Currently known link-by-link HQoS solutions (e.g., per-link or per-hop HQoS) cannot provide network-wide optimal resource sharing, e.g., when the User Equipments (UEs) of two operators with the same weight are unevenly distributed in multiple cell sites using the same core site to access a service or the Internet. Known link-by-link approaches can lead to unfairness between operators and among UEs of each operator.

[0078] In a traditional HQoS scheduler, as illustrated in FIG. 1 , packets are directed through a hierarchy of physical buffers. Packets leave the buffers under the control of a scheduler. Depending on the implementation, the schedulers might be interconnected directly. In this example, the HQoS scheduler may be described as a tree-like structure on top of traffic groups. Each scheduling node implements WFQ or SP scheduling among the input traffic groups. Subflows are the lowest level of traffic grouping (on the left side of FIG. 1 ). There are also four rate transformation nodes in this example, each of which corresponds to a scheduler. The rate transformation nodes are denoted as WFQ(1 ), WFQ(2), SP(3) and WFQ(4). The subflows are simply connected to the WFQ and SP policy nodes in the hierarchy. WFQ is generally implemented by a deficit round robin algorithm among queues with weights. SP is a simple strict priority scheduler between queues with different priority values.

[0079] As shown in marker graph of FIG. 2, the output of the final rate transformation node, WFQ(4), may be connected to a marker node that is preconfigured with a Throughput Value Function (TVF) for the six packet flows. The marker node marks the packet with a PV based on the transformed rate received from WFQ(4) and the TVF.

[0080] In the marker graph, the packet value computation for a given packet of a subflow “s” starts with a rate measurement update. Assuming that the rate of subflow s is S, a random rate value R is taken between 0 and S. R is then sent to WFQ or SP nodes in the marker graph that transform the rate value according to the weights of subflows and input traffic aggregates. The output rate is then forwarded to a rate transformer at the higher level. Finally, the obtained rate value is translated to a packet value by applying the Throughput Value Function (TVF) expressing the highest level resource sharing policy.

[0081] The main problem with the packet marking-based resource sharing solution (e.g., a rankbased solution in which a PV is assigned to each packet) is that there is no field in the packet headers in which the PV values can be stored. Accordingly, regular marking may be used, e.g., using Differentiated Services Code Point (DSCP) values indicating various packet drop precedence. However, the number of potential packet markings (e.g., up to 3 drop precedence marking for an Assured Forwarding (AF) traffic class) is much more reduced than the number of PVs that may typically be used (e.g., a thousand). Thus, fixed mapping from PV to Drop Precedence at the entry points does not follow the load situation of the system. That is, the share a given flow would get with these fixed settings at the bottleneck link is not responsive to the current load.

[0082] In addition, known PV-based approaches require a special scheduler implemented by all the nodes that can act as a bottleneck. Though the scheduler can be implemented in programmable packet processing hardware, this approach is far too complex for commodity / cheap fixed function switches used in most deployed networks.

[0083] In contrast to traditional approaches, embodiments of the present disclosure take into account the current load situation (besides the PV) when setting the drop precedence values for a given flow. Although one or more embodiments of the present disclosure will similarly use a marking graph with one or more of the same or similar parameters, in contrast to traditional approaches, embodiments of the present disclosure use PV to calculate rate allocation for all the subflows crossing a given link.

[0084] More specifically, particular embodiments use a centralized logic to collect the current actual offered load at network edges and calculate the would-be optimal bandwidth shares of the different flows based on these values according to the network-wide resource availability and a hierarchical resource sharing (e.g., HQoS) policy to be applied. It then uses the resulting optimal bandwidth shares to map to a certain packet marking (e.g., a DSCP value) expressing the drop precedence of the packet. The solution may use a limited number of drop precedence levels. In this regard, three levels of drop precedence were experimentally verified to be suitable, though different embodiments may use two or more. The calculation may be repeated periodically and the mappings may be updated to cater to dynamic changes in load conditions. The network Active Queue Management (AQM) may then be configured such that more optimal resource shares are approached with the applied packet markings. Generally speaking, it should be assumed that preferred embodiments include a few number of DSCP drop precedence levels and the presence of a DSCP-aware AQM (e.g., simple Random Early Detection (RED) / Weighted RED (WRED); with different RED profiles for different DSCP values).

[0085] Among other things, particular embodiments of the present disclosure may advantageously enable more optimal network-wide resource sharing relative to traditional approaches. Further, said network-wide resource sharing may be in accordance with an HQoS policy in real packet networks using regular networking hardware. Numerical results indicate a good approximation of the optimal allocation (see below). The method maps allocated throughput to few DSCP values and assume simple RED or WRED AQM in the network nodes, supported by commodity switches.

[0086] By marking packets with, e.g., DSCP tags instead of policing as in BwE, available network resources can be used more elastically. In some embodiments, if some sources underuse their desired throughput, other sources can get higher throughput than their desired throughput allocation by transmitting packets with high drop precedence. This is in contrast to BwE in which the rate of each source is strictly set by the bandwidth enforcer.

[0087] FIG. 3 is a schematic block diagram illustrating an example network comprising a plurality of network nodes. In this example, the network is a Radio Access Network (RAN) 10 that is shared between two network operators. The RAN 10 comprises a first RAN node 20a, a second RAN node 20b, and a switch 30. Each of the RAN nodes 20a-b serves a respective cell to a plurality of UEs, some of which belong to Operator 1 (01 ) and some of which belong to Operator 2 (02). Each of the RAN nodes 20a-b is connected to the switch 30 via a respective link 25a-b. Each link has a bandwith capacity of 1 Gbps in this example. The switch 30 is configured to direct upstream flows to the appropriate operator core network in accordance with an HQoS policy. Correspondingly, the HQoS policy needs to be applied to downstream traffic entering at the switch site 30 and exiting the RAN 10 at the RAN nodes 20a-b.

[0088] In existing shared networking infrastructures, the expectations on how network resources are shared among operators, slices, and the UEs of end-users are often described as an HQoS policy combining weighted fair and strict priority policies in a hierarchy. Implementation-wise, each RAN node 20 applies the same HQoS policy on each of the links independently to ensure the HQoS requirements among the parties. Ensuring HQoS requirements link-by-link is not the same as ensuring the HQoS policy at network-level because the network is composed of links of varying capacities as data packets travel along the network topology. Further, the distribution of UEs and the load they generate will tend to vary.

[0089] According to one hypothetical scenario involving the RAN 10 of FIG. 3, an equal sharing policy needs to be implemented between 01 and 02, end user fairness within each operator needs to be ensured, and both links A-B and A-C have 1 Gbps capacity. 01 has 1 UE being served by RAN node 20a and 3 UEs being served by RAN node 20b. 02 has 3 UEs being served by RAN node 20a and 1 UE being served by RAN node 20b. Applying the HQoS policy link-by-link leads to resource allocation where the sole user of 01 served by RAN node 20a and the sole user of 02 being served by RAN node 20b have 500 Mbps of allowed throughput while all other UEs have only 166Mbps. In this case, the links 25a, 25b are shared among 01 and 02 equally; i.e., with each operator being allocated 500 Mbps. However, the requirement for end-user fairness within each operator is not met.

[0090] In this scenario, a better allocation would be to provide 250Mbps per UE. Network-wide, such an allocation would not only ensure an equal share between 01 and 02, but would also satisfy user fairness within each operator. It is apparent that this latter case is closer to the expectations defined by the HQoS policy. In contrast to traditional approaches at HQoS, embodiments of the present invention are able to achieve an allocation that is closer to such expectations, e.g., by ensuring the policy is applied network-wide instead of link-by-link.

[0091] To accomplish improved allocation over traditional approaches, one or more embodiments of the present disclosure may include an offline resource allocation process that periodically recalculates throughput allocation among the flows (e.g., traffic aggregate, traffic of a UE) according to the HQoS policy to be implemented in the system. To calculate the resource allocation, a modified water filling algorithm may be applied. The flow throughput ratios at the network level may be closest to the ratios defined by the HQoS policy.

[0092] The throughput allocation is used to determine the value of the DSCP field (or similar field expressing the drop precedence). DSCP is mapped to enable higher level of flexibility in resource usage, compared to rate limiter-based bandwidth enforcers. AQM may be implemented at any potential bottlenecks (preferably each) that can support different drop precedences (e.g., RED with different drop profiles for different DSCP codes). As previously noted, the number of drop precedences may be quite limited, with as few as three being experimentally verified and two being conceptually possible.

[0093] FIG. 4 is a schematic block diagram illustrating an example network 100 supporting wireless communication. The network 100 comprises a core network 160 and a RAN 10. Consistent with FIG. 3, the RAN 10 comprises a first RAN node 20a, a second RAN node 20b, and a switch 30. Each RAN node 120 may support one or more Radio Access Technologies (RATs), e.g., Long Term Evolution (LTE) and / or New Radio (NR). Each of the RAN nodes 20a and 20b serves a respective coverage area 130a and 130b within which a UE 1 10 may access the core network 160. The core network 160 comprises one or more core network nodes 140 that support the UE 1 10 in accessing core network services and / or support accessing resources provided by an external data network 170 (e.g., the Internet).

[0094] Generally speaking, the network 10 may be viewed as a graph comprising nodes, edges, and edge capacities. A flow from the core network 160 toward the UE 1 10 is generally referred to the downlink direction, whereas a flow from the UE 1 10 toward the core network 160 is generally referred to as the uplink direction. Although not shown, the network 10 may further comprise one or more intermediate nodes that support communication and are located in the flow of traffic. The network 100 may additionally or alternatively comprise one or more supporting nodes that are not in the flow of traffic, yet nonetheless support communication from an out-of-band location (e.g., control nodes supporting mobility, charging, and other network functions).

[0095] The resource sharing policy between operators may be defined by operator specific TVFs, e.g., as shown in FIG. 5. Within each operator, a HQoS resource sharing hierarchy with weighted fair (and strict priority) scheduler nodes may be applied to express the resource sharing policy between slices, subslices, and UEs (or application flows). The hierarchy may introduce multiple layers and may, for example, handle large number of traffic aggregates in each level. The HQoS policy of each operator may, in some embodiments, be mapped to a PV marker graph. The PV graph may be used to perform rate transformations that encode WF and SP policies to get a final rate contribution that is then mapped to a PV. A packet may then be tagged with the PV and this PV may be used by a bottleneck nodes to make a decision to drop or retain the packet depending on this PV. In the example of FIG. 5, higher throughput generally maps to lower PVs.

[0096] This behavior may be emulated using a single process, e.g., using the algorithm shown in FIG. 6. Although the algorithm of FIG. 6 bears some conceptual similarities to a water filling algorithm, instead of directly increasing the flow rates, the algorithm of FIG. 6 finds the congestion level expressed by the congestion threshold value and allocates flow rates according to the policy belonging to the given congestion level. The algorithm systematically starts with the most congested link where the Congestion Threshold Value (CTV) (i.e. , the congestion cut-off value) is maximal and thus the average per-flow throughput is minimal. Note that a larger CTV would denote a higher congestion level, thereby reducing per-flow rates the most. The algorithm starts with the largest permitted CTV and iteratively decreases it until the first fully utilized link is found (which would thus be the most congested).

[0097] After finding the most congested link, the rates of those flows crossing the link are saved and used to calculate the weights in the HQoS policy. This may avoid underutilization by flows due to unevenly distributed load and capacities in the network topology. After the most congested edge is processed, the most congested edge may be considered ready and removed from further consideration. The process may then be repeated for remaining edges until all edges are processed in the network topology graph.

[0098] Finding the maximum cutoff value for a link (i.e., the link CTV) in a system with multiple operators applying HQoS policies is not simple. An example of an algorithm for determining the link CTV is shown in FIG. 7. The algorithm of FIG. 7 implements a binary search to find the minimal CTV that leads to a throughput allocation fully utilizing the link with an error factor epsilon. If v test is below kappa, the link is not congested. For each flow crossing the link, the algorithm calculates the allocated rate given by the tested CTV (v test) by calling the function “findCutOfflnputRate.” An example algorithm for performing the “findCutOfflnputRate” function is illustrated in FIG. 8. The algorithm of FIG. 8 relies on a PV marker graph as in traditional approaches. According to embodiments of the present disclosure however, the algorithm for performing the “findCutOfflnputRate” according to embodiments of the present disclosure evaluate in the opposite direction. That is, the input to the PV marker graph is a CTV value (i.e., the value “v test” in FIG. 8) and applies the steps of traditional approaches inversely, as shown in FIG. 8 and FIG. 9.

[0099] More specifically, the algorithm of FIG. 8 considers a HQoS marking graph. It first maps the CTV to a rate value (outgoing rate value) by applying the inverse of a TVF describing the policy at operator level. The rate value is then propagated back to the flow rates, by applying the inverse of traditional approaches. Each rate transformator in the marking graph maps the outgoing rate to incoming rates of the transformator’s input subflows / traffic aggregates. Calculating the input rate for a subflow in this graph is done by applying the inverse rate transformations (CutoffBP) along a predefined path in the marking graph. The algorithm of FIG. 8 follows the path to the input subflow and executes the rate transformator nodes one after another. After reaching the starting nodes in the marking graph, the rate for the given subflow and the corresponding cutoff value is computed. The CutoffBP for a WF node is described by FIG. 9, which is the inverse of traditional approaches. The drop precedence field (e.g., the DSCP field) of a packet may be set according to a determined rate allocation. The determined rate allocation may be determined using one or more of the techniques described herein, e.g., using the processes illustrated in FIG. 10 and FIG. 1 1 . FIG. 10 illustrates an example of a resource allocation process 200 that is able to execute offline from the data plane. The resource allocation process 200 uses rate information from the data plane as input and provides allocated throughput information to the data plane as output. FIG. 1 1 illustrates a data plane process 300 that executes at line-rate. The data plane process 300 provides rate information to the resource allocation process and periodically receives resource allocation information in return.

[0100] Turning to FIG. 10, the resource allocation process 200 may be an independent process that periodically reads flow rates from the data plane and estimates the flow demands (e.g., all flows are greedy S(u)=max. capacity, or u’s demand is S(u) = k*R(u) where k>1 .0 is a constant factor). Thus, the resource allocation process 200 may be executed to determine the A(u) parameter of the marking algorithm running in the data plane for each u. The new A(u) parameters may be updated in the data plane (e.g., storing them in a match-action table (SRAM) or in registers (SRAM)). The resource allocation process recalculates the A(u) parameters of the data plane process in every TupdateAiiocationPeriod time period.

[0101] Accordingly, the resource allocation process 200 comprises initializing a timer (i.e. , TupdateAiiocationPeriod, in this example) (step 210). The resource allocation process 200 further comprises detecting that the timer has elapsed (step 220). The resource allocation process 200 further comprises reading, from the data plane, rate measurements R(u) for all users u (step 230).

[0102] When a packet arrives the data plane process first update the data rate of u (R(u)). After that it reads A(u) from a data plane object. After that it takes a random number (r) between 0 and the rate of the user / flow (R(u)). When the r is higher than the allocated throughput of the user (A(u)) then it sets the drop precedence (e.g., DSCP) to level high. When the r is between the allocated resource (A(u)) and c*A(u) where 0<c<1 is constant, the DSCP level is set to medium. In other cases the DSCP level is low. In our example, c was set to 0.9. Note c is a configuration parameter of this process.

[0103] For different DSCP values, the networking node applies different RED profiles to obtain a drop / Explicit Congestion Notification (ECN) marking probability (P). FIG. 12 illustrates an example of how P may be applied in the early drop / marking mechanism of RED. As shown in FIG. 12, as average queue sizes increase, P tends to increase, with higher DSCP values causing P to increase more quickly than lower DSCP values. All the traffic directed to the same egress port share the same egress buffer / queue in the traffic management engine of the network node, (i.e., single queue per egress port is enough). As mentioned above, the benefits and validity of one or more solutions proposed herein have been experimentally verified. To do so, the traditional approach to computing a Jain fairness index was extended to accommodate hierarchical considerations. That is, a traditional Jain fairness index only checks the fairness of the flows. However, for purposes of verifying one or more of the embodiments disclosed herein, the traditional Jain fairness index was extended to perform an evaluation of fairness on every level in the resource sharing hierarchy. For example, when the flows get equal share, but the share between the operators is not fair, the original index for flows gives 1 .0 value that is the maximal. So, the hierarchical fairness index (e.g., as described by equations 1 and 2, below) takes into consideration the lower-level fairness. traf fic(group, level) = X; ingroup of that ievei i (Equation 1 )

[0104] The fairness index calculates the aggregated throughput of the hierarchical levels and calculates the Jain index on it. If the level has sublevels, then the value is multiplied by the sublevel fairness index. For example: with two operators, the throughput of the flows of the operatori and operator2 are aggregated and the Jain index of them are calculated. These values are multiplied by the Jain index of flows of the operatori and the Jain index of flows of the operator2. The indices decrease if the share between the operators is not fair or if the share between the flows of an operator is not fair.

[0105] An improvement over traditional approaches was experimentally verified using a Netbench packet-level simulator of Transmission Control Protocol (TCP) flows under several test scenarios. Each of the scenarios discussed below uses a simple network 10 as shown in FIG. 4. In these scenarios, links 25a, 25b from the RAN nodes 20a, 20b, respectively, to the switch 30 are bottleneck links having a 1 Gbps capacity each. Flows corresponding to two operators are using the network 10. The value 0.9 was used for the c parameter described above in each case. The link delays are 500000 ns. In each scenario, traffic enters the network 10 at the switch 30 and terminates within the network 10 at RAN nodes 20a, 20b (with traffic subsequently flowing to UEs 1 10 served by the RAN nodes 20a, 20b, not shown).

[0106] FIG. 13 illustrates the expected ideal rates that can be produced using a traditional resource allocation as opposed to embodiments as proposed herein in a first scenario. In the first scenario, the total number of flows between the operators is symmetric. In particular, there are 4 flows per operator in the example of FIG. 13, with RAN node 20a handling fewer flows for operator 1 than for operator 2. Correspondingly, RAN node 20b is handling fewer flows for operator 2 than for operator 1 . As can be seen from FIG. 13, proposed embodiments will result in a throughput distribution in which every flow would receive a 250 Mbps rate. In contrast, a traditional link-by-link solution would result in each 1 Gbps link 25a, 25b being unevenly shared between flows, notably, by a 3:1 ratio. Although both approaches result in the operator share being fair (i.e. , with 1 :1 ratio), the traditional approach would result in each operator’s throughput being shared among user flows unfairly. Embodiments of the present disclosure that take a broader view (e.g., network-wide) result in a fairer resource share on both operator and flow levels.

[0107] The resource allocation solution of one or more embodiments proposed herein was used in simulations using DSCP marking as proposed herein as well as traditional PV marking. The simulations used the NewReno congestion control algorithm. Using the proposed DSCP marking, NewReno slightly underutilized the link and the flow rates fluctuated between 150 and 250 Mbps as shown in FIG. 14. These fluctuations were caused by the instability of NewReno. Notwithstanding, the flow rates were generally close to the expected ideal level of 250Mbps per flow.

[0108] When a traditional PV marking algorithm using a packet value range of 0 to 65000 was used with the same TCP flows under NewReno congestion control, less underutilization was observed, as shown in FIG. 15. However, the rate fluctuations were more significant, caused by the larger buffer used in this traditional approach.

[0109] When a DSCP marking algorithm in accordance with one or more embodiments of the present disclosure was used with Data Center TCP (DCTCP) congestion control and ECN marking, the throughput allocation was observed to be extremely close to ideal, as shown in FIG. 16. That is, a throughput very close to the ideal throughput of 250Mbps per flow was observed. The same was observed using traditional PV marking, as shown in FIG. 17. Thus, the improved fairness of using a PV marking approach was experimentally verified to be obtainable using embodiments of the present disclosure without the need for a special PV field or PV-aware schedulers in the network 10.

[0110] The Jain fairness index was then computed at both the operator and flow levels using a traditional Jain evaluation, as well as using a complete hierarchy fairness evaluation as discussed above using Equation 1 and Equation 2. The results of these fairness computations are shown in FIG. 18 and confirm that the proposed approach improves fairness over other traditional alternatives discussed above (e.g., BwE).

[0111] FIG. 19 illustrates the expected ideal rates that can be produced using a traditional resource allocation as opposed to embodiments as proposed herein in a second scenario. In the second scenario, Operatori has users being served by both RAN nodes 20a, 20b, whereas Operator2 only has users being served by RAN node 20a and has no users being served by RAN node 20b. Thus, link 25b is only used by Operatori . In this user distribution, a perfect resource allocation according to the HQoS policy cannot be satisfied, and Operatori will have more resource than Operator 2. However, the difference in resource allocation and the level of unfairness is significant when comparing traditional approaches against embodiments proposed herein, as shown in the table of FIG. 19.

[0112] The per-flow throughput figures show similar character as in the previous scenario when evaluated under similar simulations. That is, the DSCP-based approach and the PPV-based method perform similarly and DCTCP can help in reaching almost perfect resource allocation according to the HQoS policy, as shown in FIGS. 20-23. Jain fairness index and the hierarchical fairness index reinforce this observation, as shown in the table of FIG. 24.

[0113] FIG. 25 illustrates the expected ideal rates that can be produced using a traditional resource allocation as opposed to embodiments as proposed herein in a third scenario. In the third scenario, 43 flows are distributed very unevenly between the two operators. Operator 1 only has one user flow from the switch 30 to RAN node 20a and two user flows from the switch 30 to RAN node 20b. Operator 2 has 40 user flows, i.e. , 20 user flows to RAN node 20a and 20 user flows to RAN node 20b. As shown in the table of FIG. 25, the results are similar. Indeed, at a glance it may even be hard to judge which allocation is the better.

[0114] As in previous scenarios, the DSCP-based marking approach and the PPV-based marking approach perform similarly, with DCTCP helping to stabilize results as shown in FIGS. 26-29. The Jain fairness index and the complete hierarchical fairness indices shown in the table of FIG. 30 demonstrate that at least a slight improvement was achievable even under more dramatically unbalanced conditions.

[0115] In view of the above, FIG. 31 illustrates an example method 400 of marking packets performed by a network node (e.g., a RAN node 20, a core network node 140, a switch 30) of a network 100 according to one or more embodiments of the present disclosure. The method 400 comprises receiving one or more packet flows corresponding to a user of the network 100 (block 410). The method 400 further comprises determining one or more marking thresholds based on a rate allocation corresponding to the user (block 420). The method 400 further comprises marking a field of a packet of the one or more packet flows with one of a plurality of predefined drop precedence values based on a comparison between a random value and the one or more marking thresholds (block 430).

[0116] Correspondingly, FIG. 32 illustrates an example method 500 of resource allocation according to one or more embodiments of the present disclosure. The method 500 is performed by a network node (e.g., a RAN node 20, a core network node 140, a switch 30) of a network 100 and comprises obtaining, from a plurality of other network nodes in the network 100, rate measurements of a plurality of packet flows for a plurality of users of the network (block 510). The method 500 further comprises estimating throughput demand for each user based on the rate measurements (block 520). The method 500 further comprises determining a rate allocation at each of the other network nodes for each user based on the estimated throughput demands (block 530). The method 500 further comprises sending the rate allocations to the plurality of other network nodes (block 540).

[0117] An example of a network node 700 suitable for performing one or more of the embodiments described herein (e.g., the method 400, the method 500) is illustrated in FIG. 33. The network node 700 comprises processing circuitry 710, memory circuitry 720, and interface circuitry 730. The processing circuitry 710 is communicatively coupled to the memory circuitry 720 and the interface circuitry 730, e.g., via one or more buses. The processing circuitry 710 may comprise one or more microprocessors, microcontrollers, hardware circuits, discrete logic circuits, hardware registers, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), applicationspecific integrated circuits (ASICs), or a combination thereof. For example, the processing circuitry 710 may be programmable hardware capable of executing software instructions stored, e.g., as a machine-readable computer program 740 in the memory circuitry 720. The memory circuitry 720 of the various embodiments may comprise any non-transitory machine-readable media known in the art or that may be developed, whether volatile or non-volatile, including but not limited to solid state media (e.g., SRAM, DRAM, DDRAM, ROM, PROM, EPROM, flash memory, solid state drive, etc.), removable storage devices (e.g., Secure Digital (SD) card, miniSD card, microSD card, memory stick, thumb-drive, USB flash drive, ROM cartridge, Universal Media Disc), fixed drive (e.g., magnetic hard disk drive), or the like, wholly or in any combination.

[0118] The interface circuitry 730 may be a controller hub configured to control the input and output (I / O) data paths of the network node 700. Such I / O data paths may include data paths for exchanging signals over a network. The interface circuitry 730 may be implemented as a unitary physical component, or as a plurality of physical components that are contiguously or separately arranged, any of which may be communicatively coupled to any other or may communicate with any other via the processing circuitry 710. For example, the interface circuitry 730 may comprise a transmitter 732 configured to send wireless communication signals and a receiver 734 configured to receive wireless communication signals.

[0119] The network node 700 may be configured (e.g., by the processing circuitry 710) to perform any of the methods 400, 500 described above.

[0120] Still other embodiments include a control program 740 comprising instructions that, when executed on processing circuitry 710 of a network node 170, cause the network node 170 to carry out any of the methods 400, 500 described above. Yet other embodiments include a carrier containing the control program 740. The carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0121] The features described herein may be applied in a variety of technical contexts, depending on the embodiment. FIG. 34 shows an example of a communication system 1 100 in accordance with some embodiments.

[0122] In the example, the communication system 1100 includes a telecommunication network 1 102 that includes an access network 1 104, such as a radio access network (RAN), and a core network 1 106, which includes one or more core network nodes 1108. The access network 1 104 includes one or more access network nodes, such as network nodes 1 110a and 1 1 10b (one or more of which may be generally referred to as network nodes 1 110), or any other similar 3rd Generation Partnership Project (3GPP) access nodes or non-3GPP access points. Moreover, as will be appreciated by those of skill in the art, a network node is not necessarily limited to an implementation in which a radio portion and a baseband portion are supplied and integrated by a single vendor. Thus, it will be understood that network nodes include disaggregated implementations or portions thereof. For example, in some embodiments, the telecommunication network 1 102 includes one or more Open-RAN (ORAN) network nodes. An ORAN network node is a node in the telecommunication network 1 102 that supports an ORAN specification (e.g., a specification published by the O-RAN Alliance, or any similar organization) and may operate alone or together with other nodes to implement one or more functionalities of any node in the telecommunication network 1 102, including one or more network nodes 11 10 and / or core network nodes 1 108.

[0123] Examples of an ORAN network node include an open radio unit (O-RU), an open distributed unit (O-DU), an open central unit (O-CU), including an O-CU control plane (O-CU-CP) or an O-CU user plane (O-CU-UP), a RAN intelligent controller (near-real time or non-real time) hosting software or software plug-ins, such as a near-real time control application (e.g., xApp) or a non-real time control application (e.g., rApp), or any combination thereof (the adjective “open” designating support of an ORAN specification). The network node may support a specification by, for example, supporting an interface defined by the ORAN specification, such as an A1 , F1 , W1 , E1 , E2, X2, Xn interface, an open fronthaul user plane interface, or an open fronthaul management plane interface. Moreover, an ORAN access node may be a logical node in a physical node. Furthermore, an ORAN network node may be implemented in a virtualization environment (described further below) in which one or more network functions are virtualized. For example, the virtualization environment may include an O-Cloud computing platform orchestrated by a Service Management and Orchestration Framework via an O-2 interface defined by the O-RAN Alliance or comparable technologies. The network nodes 1 1 10 facilitate direct or indirect connection of user equipment (UE), such as by connecting UEs 1 1 12a, 1 112b, 1 1 12c, and 11 12d (one or more of which may be generally referred to as UEs 11 12) to the core network 1 106 over one or more wireless connections.

[0124] Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 1 100 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired or wireless connections. The communication system 1 100 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0125] The UEs 1 1 12 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with the network nodes 1 1 10 and other communication devices. Similarly, the network nodes 11 10 are arranged, capable, configured, and / or operable to communicate directly or indirectly with the UEs 1 1 12 and / or with other network nodes or equipment in the telecommunication network 1 102 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in the telecommunication network 1102.

[0126] In the depicted example, the core network 1106 connects the network nodes 1 1 10 to one or more host computing systems, such as host 1 1 16. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 1106 includes one more core network nodes (e.g., core network node 1 108) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 1108. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier Deconcealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0127] The host 1 1 16 may be under the ownership or control of a service provider other than an operator or provider of the access network 1 104 and / or the telecommunication network 1 102. The host 11 16 may host a variety of applications to provide one or more service. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0128] As a whole, the communication system 1 100 of FIG. 34 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.1 1 standards (WiFi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and / or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0129] In some examples, the telecommunication network 1102 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 1 102 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 1 102. For example, the telecommunications network 1102 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC) / Massive loT services to yet further UEs.

[0130] In some examples, the UEs 11 12 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 1 104 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 1 104. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).

[0131] In the example, the hub 1 1 14 communicates with the access network 1 104 to facilitate indirect communication between one or more UEs (e.g., UE 1 1 12c and / or 1 1 12d) and network nodes (e.g., network node 1 1 10b). In some examples, the hub 1 1 14 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 1 1 14 may be a broadband router enabling access to the core network 1 106 for the UEs. As another example, the hub 1 1 14 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 1 110, or by executable code, script, process, or other instructions in the hub 1 1 14. As another example, the hub 1 1 14 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 1 1 14 may be a content source. For example, for a UE that is a VR device, display, loudspeaker, or other media delivery device, the hub 1 1 14 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 1 1 14 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, the hub 1 1 14 acts as a proxy server or orchestrator for the UEs, in particular if one or more of the UEs are low energy loT devices.

[0132] The hub 11 14 may have a constant / persistent or intermittent connection to the network node 1 1 10b. The hub 1 114 may also allow for a different communication scheme and / or schedule between the hub 1 1 14 and UEs (e.g., UE 11 12c and / or 1 1 12d), and between the hub 1 1 14 and the core network 1 106. In other examples, the hub 11 14 is connected to the core network 1 106 and / or one or more UEs via a wired connection. Moreover, the hub 1 114 may be configured to connect to an M2M service provider over the access network 1 104 and / or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with the network nodes 1 1 10 while still connected via the hub 1 1 14 via a wired or wireless connection. In some embodiments, the hub 1 114 may be a dedicated hub - that is, a hub whose primary function is to route communications to / from the UEs from / to the network node 1 1 10b. In other embodiments, the hub 1 1 14 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 1 1 10b, but which is additionally capable of operating as a communication start and / or end point for certain data channels.

[0133] FIG. 35 shows a UE 1200 in accordance with some embodiments. The UE 1200 presents additional details of some embodiments of the UE 1 112 of Figure 1 . As used herein, a UE refers to a device capable, configured, arranged and / or operable to communicate wirelessly with network nodes and / or other UEs. Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage / playback device, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), an Augmented Reality (AR) or Virtual Reality (VR) device, wireless customer-premise equipment (CPE), vehicle, vehiclemounted or vehicle embedded / integrated wireless device, etc. Other examples include any UE identified by the 3rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB-loT) UE, a machine type communication (MTC) UE, and / or an enhanced MTC (eMTC) UE.

[0134] A UE may support device-to-device (D2D) communication, for example by implementing a 3GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and / or operates the relevant device. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent a device that is not intended for sale to, or operation by, an end user but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).

[0135] The UE 1200 includes processing circuitry 1202 that is operatively coupled via a bus 1204 to an input / output interface 1206, a power source 1208, a memory 1210, a communication interface 1212, and / or any other component, or any combination thereof. Certain UEs may utilize all or a subset of the components shown in FIG. 35. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0136] The processing circuitry 1202 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine-readable computer programs in the memory 1210. The processing circuitry 1202 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field- programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 1202 may include multiple central processing units (CPUs).

[0137] In the example, the input / output interface 1206 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and / or output devices. Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof. An input device may allow a user to capture information into the UE 1200. Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like. The presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user. A sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.

[0138] In some embodiments, the power source 1208 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used. The power source 1208 may further include power circuitry for delivering power from the power source 1208 itself, and / or an external power source, to the various parts of the UE 1200 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 1208. Power circuitry may perform any formatting, converting, or other modification to the power from the power source 1208 to make the power suitable for the respective components of the UE 1200 to which power is supplied.

[0139] The memory 1210 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 1210 includes one or more application programs 1214, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 1216. The memory 1210 may store, for use by the UE 1200, any of a variety of various operating systems or combinations of operating systems.

[0140] The memory 1210 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUlCC), integrated UICC (iUICC) or a removable UICC commonly known as ‘SIM card.’ The memory 1210 may allow the UE 1200 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 1210, which may be or comprise a device-readable storage medium.

[0141] The processing circuitry 1202 may be configured to communicate with an access network or other network using the communication interface 1212. The communication interface 1212 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 1222. The communication interface 1212 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a network node in an access network). Each transceiver may include a transmitter 1218 and / or a receiver 1220 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, the transmitter 1218 and receiver 1220 may be coupled to one or more antennas (e.g., antenna 1222) and may share circuit components, software or firmware, or alternatively be implemented separately.

[0142] In the illustrated embodiment, communication functions of the communication interface 1212 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, near-field communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may be implemented in according to one or more communication protocols and / or standards, such as IEEE 802.1 1 , Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol / internet protocol (TCP / IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.

[0143] Regardless of the type of sensor, a UE may provide an output of data captured by its sensors, through its communication interface 1212, via a wireless connection to a network node. Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE. The output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).

[0144] As another example, a UE comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection. In response to the received wireless input the states of the actuator, the motor, or the switch may change. For example, the UE may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input.

[0145] A LIE, when in the form of an Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare. Non-limiting examples of such an loT device are a device which is or which is embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door / window sensor, a flood / moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal- or item-tracking device, a sensor for monitoring a plant or animal, an industrial robot, an Unmanned Aerial Vehicle (UAV), and any kind of medical device, like a heart rate monitor or a remote controlled surgical robot. A UE in the form of an loT device comprises circuitry and / or software in dependence of the intended application of the loT device in addition to other components as described in relation to the UE 1200 shown in FIG. 35.

[0146] As yet another specific example, in an loT scenario, a UE may represent a machine or other device that performs monitoring and / or measurements, and transmits the results of such monitoring and / or measurements to another UE and / or a network node. The UE may in this case be an M2M device, which may in a 3GPP context be referred to as an MTC device. As one particular example, the UE may implement the 3GPP NB-loT standard. In other scenarios, a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and / or reporting on its operational status or other functions associated with its operation.

[0147] In practice, any number of UEs may be used together with respect to a single use case. For example, a first UE might be or be integrated in a drone and provide the drone’s speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone. When the user makes changes from the remote controller, the first UE may adjust the throttle on the drone (e.g. by controlling an actuator) to increase or decrease the drone’s speed. The first and / or the second UE can also include more than one of the functionalities described above. For example, a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.

[0148] FIG. 36 shows a network node 1300 in accordance with some embodiments. As used herein, network node refers to equipment capable, configured, arranged and / or operable to communicate directly or indirectly with a LIE and / or with other network nodes or equipment, in a telecommunication network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)), O-RAN nodes or components of an O-RAN node (e.g., O-RU, O-DU, O-CU).

[0149] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units, distributed units (e.g., in an O-RAN access node) and / or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0150] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi-cell / multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, SelfOrganizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and / or Minimization of Drive Tests (MDTs).

[0151] The network node 1300 includes a processing circuitry 1302, a memory 1304, a communication interface 1306, and a power source 1308. The network node 1300 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which the network node 1300 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, the network node 1300 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 1304 for different RATs) and some components may be reused (e.g., a same antenna 1310 may be shared by different RATs). The network node 1300 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 1300, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 1300.

[0152] The processing circuitry 1302 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other network node 1300 components, such as the memory 1304, to provide network node 1300 functionality.

[0153] In some embodiments, the processing circuitry 1302 includes a system on a chip (SOC). In some embodiments, the processing circuitry 1302 includes one or more of radio frequency (RF) transceiver circuitry 1312 and baseband processing circuitry 1314. In some embodiments, the radio frequency (RF) transceiver circuitry 1312 and the baseband processing circuitry 1314 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 1312 and baseband processing circuitry 1314 may be on the same chip or set of chips, boards, or units.

[0154] The memory 1304 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and / or any other volatile or nonvolatile, non-transitory device-readable and / or computer-executable memory devices that store information, data, and / or instructions that may be used by the processing circuitry 1302. The memory 1304 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and / or other instructions capable of being executed by the processing circuitry 1302 and utilized by the network node 1300. The memory 1304 may be used to store any calculations made by the processing circuitry 1302 and / or any data received via the communication interface 1306. In some embodiments, the processing circuitry 1302 and memory 1304 is integrated.

[0155] The communication interface 1306 is used in wired or wireless communication of signaling and / or data between a network node, access network, and / or UE. As illustrated, the communication interface 1306 comprises port(s) / terminal(s) 1316 to send and receive data, for example to and from a network over a wired connection. The communication interface 1306 also includes radio front-end circuitry 1318 that may be coupled to, or in certain embodiments a part of, the antenna 1310. Radio front-end circuitry 1318 comprises filters 1320 and amplifiers 1322. The radio front-end circuitry 1318 may be connected to an antenna 1310 and processing circuitry 1302. The radio front- end circuitry may be configured to condition signals communicated between antenna 1310 and processing circuitry 1302. The radio front-end circuitry 1318 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection. The radio front-end circuitry 1318 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 1320 and / or amplifiers 1322. The radio signal may then be transmitted via the antenna 1310. Similarly, when receiving data, the antenna 1310 may collect radio signals which are then converted into digital data by the radio front-end circuitry 1318. The digital data may be passed to the processing circuitry 1302. In other embodiments, the communication interface may comprise different components and / or different combinations of components.

[0156] In certain alternative embodiments, the network node 1300 does not include separate radio front-end circuitry 1318, instead, the processing circuitry 1302 includes radio front-end circuitry and is connected to the antenna 1310. Similarly, in some embodiments, all or some of the RF transceiver circuitry 1312 is part of the communication interface 1306. In still other embodiments, the communication interface 1306 includes one or more ports or terminals 1316, the radio front-end circuitry 1318, and the RF transceiver circuitry 1312, as part of a radio unit (not shown), and the communication interface 1306 communicates with the baseband processing circuitry 1314, which is part of a digital unit (not shown).

[0157] The antenna 1310 may include one or more antennas, or antenna arrays, configured to send and / or receive wireless signals. The antenna 1310 may be coupled to the radio front-end circuitry 1318 and may be any type of antenna capable of transmitting and receiving data and / or signals wirelessly. In certain embodiments, the antenna 1310 is separate from the network node 1300 and connectable to the network node 1300 through an interface or port.

[0158] The antenna 1310, communication interface 1306, and / or the processing circuitry 1302 may be configured to perform any receiving operations and / or certain obtaining operations described herein as being performed by the network node. Any information, data and / or signals may be received from a LIE, another network node and / or any other network equipment. Similarly, the antenna 1310, the communication interface 1306, and / or the processing circuitry 1302 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and / or signals may be transmitted to a UE, another network node and / or any other network equipment.

[0159] The power source 1308 provides power to the various components of network node 1300 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). The power source 1308 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 1300 with power for performing the functionality described herein. For example, the network node 1300 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 1308. As a further example, the power source 1308 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.

[0160] Embodiments of the network node 1300 may include additional components beyond those shown in FIG. 36 for providing certain aspects of the network node’s functionality, including any of the functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, the network node 1300 may include user interface equipment to allow input of information into the network node 1300 and to allow output of information from the network node 1300. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 1300. In some embodiments providing a core network node, such as core network node 108 of FIG. 1 1 , some components, such as the radio front-end circuitry 1318 and the RF transceiver circuitry 1312 may be omitted.

[0161] FIG. 37 is a block diagram illustrating a virtualization environment 1400 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1400 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, LIE, core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized. In some embodiments, the virtualization environment 1400 includes components defined by the O-RAN Alliance, such as an O-Cloud environment orchestrated by a Service Management and Orchestration Framework via an O-2 interface. Virtualization may facilitate distributed implementations of a network node, LIE, core network node, or host.

[0162] Applications 1402 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment Q400 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein. Hardware 1404 includes processing circuitry, memory that stores software and / or instructions executable by hardware processing circuitry, and / or other hardware devices as described herein, such as a network interface, input / output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1406 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1408a and 1408b (one or more of which may be generally referred to as VMs 1408), and / or perform any of the functions, features and / or benefits described in relation with some embodiments described herein. The virtualization layer 1406 may present a virtual operating platform that appears like networking hardware to the VMs 1408.

[0163] The VMs 1408 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 1406. Different embodiments of the instance of a virtual appliance 1402 may be implemented on one or more of VMs 1408, and the implementations may be made in different ways. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.

[0164] In the context of NFV, a VM 1408 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine. Each of the VMs 1408, and that part of hardware 1404 that executes that VM, be it hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 1408 on top of the hardware 1404 and corresponds to the application 1402.

[0165] Hardware 1404 may be implemented in a standalone network node with generic or specific components. Hardware 1404 may implement some functions via virtualization. Alternatively, hardware 1404 may be part of a larger cluster of hardware (e.g. such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 1410, which, among others, oversees lifecycle management of applications 1402. In some embodiments, hardware 1404 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 1412 which may alternatively be used for communication between hardware nodes and radio units.

[0166] Although the computing devices described herein (e.g., UEs, network nodes) may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to be understood that these computing devices may comprise any suitable combination of hardware and / or software needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and / or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and / or the functionality of the components may be partitioned between the processing circuitry and the communication interface. In another example, non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.

[0167] In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer-readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and / or by end users and a wireless network generally.

Claims

CLAIMS1 . A method (400) of marking packets performed by a network node (700) of a network (100), the method comprising: receiving (410) one or more packet flows corresponding to a user of the network (100); and determining (420) one or more marking thresholds based on a rate allocation corresponding to the user; marking (430) a field of a packet of the one or more packet flows with one of a plurality of predefined drop precedence values based on a comparison between a random value and the one or more marking thresholds.

2. The method of claim 1 , wherein the field is a Differentiated Services Code Point field, Multiprotocol Label Switching experimental bits field, or Explicit Congestion Notification field.

3. The method of any one of the preceding claims, wherein determining the one or more marking thresholds based on the rate allocation comprises setting a first marking threshold to the rate allocation.

4. The method of any one of the preceding claims, wherein determining the one or more marking thresholds based on the rate allocation comprises setting a second marking threshold to the rate allocation adjusted by a constant factor.

5. The method of any one of the preceding claims, wherein: the one or more marking thresholds define a plurality of ranges, each range having a respective length; and the length of each range controls a probability of marking the packet with a respective one of the plurality of predefined drop precedence values.

6. The method of any one of the preceding claims, further comprising forwarding the packet comprising the marked field to a further network node.

7. The method of any one of the preceding claims, further comprising measuring a rate of the one or more packet flows.

8. The method of claim 7, further comprising generating the random value from a range limited by the rate of the one or more packet flows.

9. The method of any one of claims 6-8, further comprising: sending the rate of the one or more packet flows to a further network node that is outside of a data plane of the one or more packet flows; and receiving the rate allocation from the further network node.

10. The method of any one of claims 1 -8, wherein the one or more marking thresholds define a plurality of Random Early Detection, RED, algorithm ranges, each of the RED ranges corresponding to a respective one of the predefined drop precedence values.1 1. A network node (700) configured to: receive one or more packet flows corresponding to a user of the network (100); determine one or more marking thresholds based on a rate allocation corresponding to the user; and mark a field of a packet of the one or more packet flows with one of a plurality of predefined drop precedence values based on a comparison between a random value and the one or more marking thresholds.

12. The network node of the preceding claim, further configured to perform the method of any one of claims 2-10.

13. A network node (700) comprising: processing circuitry (710) and a memory (720), the memory (720) containing instructions executable by the processing circuitry (710) whereby the network node (700) is configured to: receive one or more packet flows corresponding to a user of the network (100); determine one or more marking thresholds based on a rate allocation corresponding to the user; and mark a field of a packet of the one or more packet flows with one of a plurality of predefined drop precedence values based on a comparison between a random value and the one or more marking thresholds.

14. The network node of the preceding claim, further configured to perform the method of any one of claims 2-10.

15. A computer program (740), comprising instructions which, when executed on processing circuitry (710) of a network node (700), cause the processing circuitry (710) to carry out the method according to any one of claims 1 -10.

16. A carrier containing the computer program of the preceding claim, wherein the carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

17. A method (500) of resource allocation performed by a network node (700) of a network (100), the method comprising: obtaining (510), from a plurality of other network nodes in the network (100), rate measurements of a plurality of packet flows for a plurality of users of the network (100); estimating (520) throughput demand for each user based on the rate measurements; determining (530) a rate allocation at each of the other network nodes for each user based on the estimated throughput demands; sending (540) the rate allocations to the plurality of other network nodes.

18. The method of claim 17, wherein sending the rate allocation to the plurality of other network nodes controls drop precedence packet marking performed by the other network nodes.

19. The method of any one of claims 17-18, further comprising periodically re-estimating the throughput demand for each user based on periodically updated rate measurements.

20. The method of any one of claims 17-19, wherein estimating the throughput demand for each user is further based on resource availability in the network (100).21 . The method of any one of claims 17-20, further comprising determining a traffic aggregate for each user, wherein estimating the throughput demand for each user is further based on a resource sharing target between the traffic aggregates.

22. The method of claim 21 , further comprising determining the resource sharing target between the traffic aggregates based on a number of the users, traffic volume, and a resource sharing policy.

23. The method of claim 22, wherein the resource sharing policy indicates a sharing target between different tenants of the network.

24. The method of claim 23, wherein the resource sharing policy indicates a sharing target between users of the different tenants of the network.

25. The method of any one of claims 20-24, wherein the resource sharing policy is represented within the network node as a graph of rate transformers.

26. The method of claim 25, wherein each rate transformer maps an outgoing aggregated rate to incoming rates of different traffic groups taking part in an aggregate traffic of the plurality of packet flows.

27. The method of any one of claims 17-26, wherein the network node is outside of a data plane of the plurality of packet flows.

28. The method of any one of claims 17-27, wherein estimating the throughput demand for each user is further based on a resource sharing target between the traffic aggregates.

29. The method of any one of claims 17-28, wherein determining the rate allocations at each of the other network nodes comprises determining the rate allocations at each of the other network nodes in descending congestion order from most congested other network node to least congested other network node.

30. A network node (700) configured to: obtain, from a plurality of other network nodes in the network (100), rate measurements of a plurality of packet flows for a plurality of users of the network (100); estimate throughput demand for each user based on the rate measurements; determine a rate allocation at each of the other network nodes for each user based on the estimated throughput demands; send the rate allocations to the plurality of other network nodes.31 . The network node of the preceding claim, further configured to perform the method of any one of claims 18-29.

32. A network node (700) comprising: processing circuitry (710) and a memory (720), the memory (720) containing instructions executable by the processing circuitry (710) whereby the network node (700) is configured to: obtain, from a plurality of other network nodes in the network (100), rate measurements of a plurality of packet flows for a plurality of users of the network (100); estimate throughput demand for each user based on the rate measurements; determine a rate allocation at each of the other network nodes for each user based on the estimated throughput demands; send the rate allocations to the plurality of other network nodes.

33. The network node of the preceding claim, further configured to perform the method of any one of claims 18-29.

34. A computer program (740), comprising instructions which, when executed on processing circuitry (710) of a network node (700), cause the processing circuitry (710) to carry out the method according to any one of claims 17-29.

35. A carrier containing the computer program of the preceding claim, wherein the carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

Images