Traffic differentiation in an orchestration platform

Abstract

An apparatus (200; 700; 1100), a corresponding method (300; 400), computer program (820) and computer program product (810) for providing traffic differentiation for an orchestration platform are disclosed. The apparatus (200; 700; 1100) comprises containers (225) and one of the containers (230) comprises a container application (280). The method comprises determining (305) a traffic differentiation flow for one or more data packets (285) associated with the at least one container (230) and configuring (310) an indicator associated with the container, wherein the indicator indicates a traffic differentiation flow for the one or more data packets (285) associated with the container (230).

Description

[0001] TRAFFIC DIFFERENTIATION IN AN ORCHESTRATION PLATFORM

[0002] TECHNICAL FIELD

[0003] The invention relates to methods and apparatus for providing traffic differentiation in an orchestration platform.

[0004] BACKGROUND

[0005] The emergence and evolution of new generations of communication technologies, such as 3rdGeneration Partnership Project (3GPP) Fifth Generation (5G) and Sixth Generation (6G), have enabled the development of applications such as Augmented Reality (AR) and Virtual Reality (VR), and the development of numerous applications for end devices. The new generations of communication technologies have improved communication capability while enhancing fields such as traditional robotics with the introduction of cloud robotics as well. One of the key enablers leading the improved communication capability lies in that an access network (e.g., 5G, and WiFi, etc.) can provide traffic differentiation such as different quality of service (QoS) and quality of experience (QoE) to applications catering to different requirements to satisfy application performance requirements while distributing network resources in an efficient manner. For instance, 3GPP TS 23.501 V18.1.0 specifies a User Equipment (UE) can establish multiple QoS flows in the same Protocol Data Unit (PDU) session to a 5G network, according to different 5G QoS Identifier (5QI) values that are pre-set for a given Internet Protocol (IP) flow.

[0006] Cloud native technologies, for example, a Docker container as defined by Docker by Docker Inc. and Kubernetes (K8s) as defined by the Cloud Native Computing Foundation have been extended from cloud to manage edge devices and devices that are not present at the edge to realize the envisioned device-edge-cloud continuum. Efforts to convert internet of things (loT) applications and robotics to cloud-based loT applications and cloud robotics have been witnessed in the industry and research community. It is anticipated that workloads and / or applications can be containerized, and the deployment and maintenance of the workloads and / or applications will be scheduled by a uniform container orchestration platform, for example., K8s or K8s variants such as Lightweight Kubernetes (K3s) as defined by the Linux Foundation. A container orchestration platform can manage a cluster of nodes that span different networks. Considering the network heterogeneity and to simplify communication within the cluster, an overlay network may be established to enable communication among application containers (e.g. pods in a Kubernetes deployment). Typical network backends such as flannel and calico are all able to create overlay networks using VXLAN or I PI P protocols.

[0007] US 10212093 B2 discloses a sub-service flow User-plane interface between an Enforcement Point and a network control system (NCS). Relation and coordination of the sub-service flow User-plane interface associated with sub-service flow management actions on a Control-plane interface is defined. In-band packet marking is created to dynamically assist the identification of sub-service flow identities and receive corresponding QoS or QoE treatments.

[0008] US 9722948 B2 discloses receiving a traffic flow of packets from a virtual machine and identifying a container from which the traffic flow originates based on content of the received traffic flow of packets, retrieving a QoS policy configured for the identified container and forwarding the received traffic flow of packets according to the QoS policy.

[0009] Container Network Interface (CNI) documentation by Cloud Native Computing Foundation project provides a method to provide network QoS by configuring a bandwidth plugin on both ingress and egress traffic for a container which results in the traffic being shaped when reading / writing contents in / to the container.

[0010] SUMMARY

[0011] An object of the invention is to improve management of resources in an orchestration platform.

[0012] According to a first aspect of the invention, a method for providing traffic differentiation for an orchestration platform is provided. The orchestration platform comprises one or more containers, wherein at least one of the containers comprises a container application. The method comprises determining a traffic differentiation flow for one or more data packets associated with the at least one container and configuring an indicator associated with the container, wherein the indicator indicates a traffic differentiation flow for the data packet associated with the container. A technical advantage of the embodiment is that the embodiment provides a capability to enable traffic priority or traffic differentiation configuration for an orchestration platform wherein the traffic differentiation flow is visible or decipherable at a container level as opposed to requiring packet header inspection or separate packet-deciphering / packet- decapsulating protocols or any other data packet associated control protocols. Another technical advantage of the embodiment is that there is no requirement to change a header of the data packet header or a payload of the data packet to embed information related to traffic differentiation. In some examples, the data packet header may already contain the same or similar information but to be used at a separate layer in the communication system / process.

[0013] According to an embodiment, configuring the indicator comprises setting a data packet mark value to the one or more data packets associated with the container. According to an embodiment, setting the data packet mark value comprises applying one or more of linux iprules; linux iptables; linux nftables; and an extended Berkeley Packet Filter (eBPF).

[0014] According to an embodiment, the orchestration platform is connected to a communication network. The method comprises mapping the indicator to a performance metric of the communication network. According to an embodiment, requesting resources from a communication network node in the communication network for mapping the indicator corresponding to the determined traffic differentiation flow. According to an embodiment, the performance metric of the communication network comprises one or more of: a Quality of Service (QoS) flow; a Grade of Service (GoS) flow; a Quality of Experience (QoE) flow; an IEEE 802. Ip class of service value; and an Internet Protocol (IP) differentiated services code point (DSCP) value. A technical advantage of the embodiment is that the embodiment provides the capability for a container to configure traffic priority or traffic differentiation for an orchestration platform in a communication network. Another technical advantage of the embodiment herein is that the method enables mapping of the traffic priority configuration or the indicator to a performance metric such as QoS or QoE or DSCP value of a communication network.

[0015] According to an embodiment, the method comprises obtaining traffic configuration information for the container and setting a node-level traffic priority parameter based on the obtained traffic configuration information. According to an embodiment, the parameter comprises one or more of: a bandwidth; a latency; errorRate; or maxBurst.

[0016] According to an embodiment, the method comprises establishing a session for the container, wherein the session provides a means for communication for the container with the communication network.

[0017] According to an embodiment, the orchestration platform comprises a second container and the second container comprises a second container application. The method comprises determining a second traffic differentiation flow for a second one or more data packets associated with the second container and configuring a second indicator associated with the second container, wherein the second indicator indicates a second traffic differentiation flow for the second one or more data packets associated with the second container. The method comprises setting a filter for egress traffic of the container based on the determined traffic differentiation flow for the one or more data packets associated with the container and setting a filter for egress traffic of the second container based on the determined second traffic differentiation flow for the second one or more data packets associated with the second container. According to an embodiment, the container application and the second container application are the same. According to an embodiment, the container application and the second container application are different.

[0018] According to an embodiment, the orchestration platform comprises an underlay network and an overlay network, wherein the one or more data packets associated with the at least one container comprises one or more data packets of the overlay network and configuring an indicator for the traffic differentiation flow associated with the container comprises configuring an indicator for the traffic differentiation flow for the data packets of the overlay network. The method comprises providing the indicator for the traffic differentiation flow associated with the data packets of the overlay network to the underlay network without decapsulating or decrypting the data packets of the overlay network.

[0019] According to an embodiment, the method comprises setting a container tag for the container, wherein the container tag is used for setting (315) the data packet mark value when encapsulating the one or more data packets associated with the container. According to an embodiment, the method is implemented by a user equipment

[0020] (UE).

[0021] According to a second aspect of the invention, an apparatus for providing traffic differentiation for an orchestration platform is provided. The apparatus comprises one or more containers and at least one of the containers comprises a container application. The apparatus is configured to determine a traffic differentiation flow for one or more data packets associated with the at least one container and configure an indicator associated with the container, wherein the indicator indicates a traffic differentiation flow for the one or more data packets associated with the container.

[0022] According to an embodiment, configuring the indicator comprises setting a data packet mark value to the one or more data packets associated with the container. According to an embodiment, setting the data packet mark value comprises applying one or more of: linux iprules; linux iptables; linux nftables; and an eBPF.

[0023] According to an embodiment, the apparatus is configured to set a container tag for the container, wherein the container tag is used for setting (315) the data packet mark value when encapsulating the data packets associated with the container.

[0024] According to an embodiment, the orchestration platform is connected to a communication network. The apparatus is configured to map the indicator to a performance metric of the communication network. According to an embodiment, the apparatus is configured to request resources from a communication network node in the communication network for mapping the indicator corresponding to the determined traffic differentiation flow. According to an embodiment, the performance metric of the communication network comprises one or more of: a QoS flow; a GoS flow; a QoE flow; an IEEE 802. Ip class of service value; and an IP DSCP value.

[0025] According to an embodiment, the apparatus is configured to obtain traffic configuration information for the container and set a node-level traffic priority parameter based on the obtained traffic configuration information. According to an embodiment, the parameter comprises one or more of: a bandwidth; a latency; errorRate; or maxBurst.

[0026] According to an embodiment, the apparatus is configured to establish a session for the container, wherein the session provides a means for communication for the container with the communication network. According to an embodiment, the apparatus comprises a second container and the second container comprises a second container application. The apparatus is configured to determine a second traffic differentiation flow for a second one or more data packets associated with the second container and configure a second indicator associated with the second container, wherein the indicator indicates a second traffic differentiation flow for the second one or more data packets associated with the second container. The apparatus is configured to set a filter for egress traffic of the container based on the determined traffic differentiation flow for the one or more data packets associated with the container and set a filter for egress traffic of the second container based on the determined second traffic differentiation flow for the second one or more data packets associated with the second container.

[0027] According to an embodiment, the apparatus comprises an underlay network and an overlay network, wherein the one or more data packets associated with the at least one container comprises one or more data packets of the overlay network and configuring an indicator associated with the container, wherein the indicator indicates a traffic differentiation flow comprises configuring an indicator for the traffic differentiation flow for the one or more data packets of the overlay network. The apparatus is configured to provide the indicator for the traffic differentiation flow associated with the one or more data packets of the overlay network to the underlay network without decapsulating or decrypting the data packets of the overlay network.

[0028] According to an embodiment, the apparatus comprises a UE.

[0029] According to an embodiment, the apparatus comprises one or more of: a wireless communication device, a physical computing machine and a virtual computing machine.

[0030] According to a third aspect, an apparatus for providing traffic differentiation for an orchestration platform is provided. The apparatus comprises one or more containers and at least one of the containers comprises a container application. The apparatus comprises a processing circuitry and a memory, wherein the memory contains instructions that when executed by the processing circuitry make the apparatus operative to perform the method according to one or more embodiments of the first aspect.

[0031] According to a fourth aspect, a computer program is provided. The computer program comprises instructions which when executed by a processing circuitry of an apparatus, causes the apparatus to carry out the methods according to one or more embodiments of the first aspect.

[0032] According to a fifth aspect, a computer program product is provided. The computer program comprises a non-transitory computer readable medium on which a computer program according to the fourth aspect is stored.

[0033] Some embodiments provide an application placed in a container or a "pod" running in an orchestration platform with the capability to configure traffic differentiation in an access network. Some embodiments herein disclose mapping traffic differentiation to QoS of the communication network such as an access network. The embodiments herein provide an application placed in a container or a pod running in an orchestration platform hosted in an apparatus with the capability to configure traffic differentiation in an access network or any other communication network without the need to change the packet header or payload to embed QoS information. Some of the embodiments herein enable transparently providing traffic differentiation tags or annotations from packets in an inner network such as a container and / or a pod to an outer network such as VXLAN, IPIP and even encrypted networks such as Internet Engineering Task Force (IETF) Request For Comments (RFC) 6071 Internet Protocol Security (IPSec). In some embodiments, the bandwidth limit feature of a Container Networking Interface (CNI) plugin for an orchestration platform such as Kubernetes (K8s) is enhanced by extending the bandwidth limit scope from the network stack on the node to both the node and the access network. A technical advantage of some embodiments of the disclosure presented herein is that a CNI plugin is the native approach for integration of the method into an existing orchestration platform such as Kubernetes, thus the complexity of implementing a traffic differentiation for each container is reduced.

[0034] BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above, as well as additional objects, features and advantages of the invention, will be better understood through the following illustrative and non-limiting detailed description of embodiments of the invention, with reference to the appended drawings, in which: Figure 1 illustrates an example packet structure of a Virtual Extensible LAN (VXLAN) network packet.

[0036] Figure 2 illustrates an example of an orchestration platform hosted in an apparatus, in accordance with one or more embodiments of the invention.

[0037] Figure 3 illustrates an example method, in accordance with one or more embodiments of the invention.

[0038] Figure 4 illustrates an example method, in accordance with one or more embodiments of the invention.

[0039] Figure 5 illustrates an example signaling flow, in accordance with one or more embodiments of the invention.

[0040] Figure 6 illustrates an example signaling flow, in accordance with one or more embodiments of the invention.

[0041] Figure 7 shows a user equipment, in accordance with one or more embodiments of the invention.

[0042] Figure 8 illustrates a computer program product, in accordance with one or more embodiments of the invention.

[0043] Figure 9 shows an example of a communication network, in accordance with one or more embodiments of the invention.

[0044] Figure 10 shows a communication diagram of a host communicating via a network node with a UE over a partially wireless connection, in accordance with one or more embodiments of the invention.

[0045] Figure 11 shows a network node, in accordance with one or more embodiments of the invention.

[0046] All the figures are schematic, not necessarily to scale, and generally only show parts which are necessary in order to elucidate the invention, wherein other parts may be omitted or merely suggested.

[0047] DETAILED DESCRIPTION

[0048] Embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art. There currently exist many challenges in the existing solutions. Technical terms of the application are further explained at the end of the description. The disclosure herein discloses a computing node, a method performed by the computing node, a corresponding computer program and a computer program product to provide traffic differentiation in an orchestration platform. An object of the disclosure is to provide traffic differentiation for containers and / or pods in an orchestration platform.

[0049] Figure 1 illustrates an example packet structure of a Virtual Extensible LAN (VXLAN) network packet. An overlay network allows network devices to communicate across an underlying network (referred to as the underlay) without the underlay network having any knowledge of the devices connected to the overlay network. The overlay network is a network to which the devices are in communication with or are connected to. Overlay networks may use different protocols for communication. In an example, overlay networks share the same characteristic of obtaining a network packet, referred to as an inner network packet belonging to an underlay network, and encapsulating the inner network packet inside an outer network packet belonging to an overlay network. In this way, the underlay network observes or reads the outer network packets without the requirement or necessity to determine handling of the inner network packets. The overlay network handles the transmittal of the inner network packets and the transmittal may vary by overlay network type and the protocols used by the overlay network. Similarly, wrapping or encapsulation of the inner network packet varies between different overlay network types. In the case of VXLAN for example, the inner network packet is wrapped and sent via User Datagram Protocol (UDP) in the outer network packet. An overlay network can simplify addressing and routing between a container and / or a pod as defined by Kubernetes within a cluster. However, due to the nature of an overlay network that IP packets generated by a container and / or pod are encapsulated inside an underlay network packet, which indicates the Quality of Service (QoS), Quality of Experience (QoE) or Terms of service (ToS) information are hidden in the overlay network. Internet Protocol (IP) packets are invisible to underlay network interface or network filters unless a decapsulation is performed on the packets. Also, this would require user-defined or customized manipulation on the overlay network packets including inserting QoS information to the overlay packet header or payload. This manipulation, decapsulation and reencapsulation procedure can greatly increase the overhead and downgrade the network communication performance.

[0050] The prior art only provides traffic shaping on the node, locally. Similar approaches in the prior art relate to shaping the traffic locally on the node. Embodiments of the invention herein are aimed at providing improvements over the known solutions and addressing the aforementioned problems. Some of the embodiments herein avoid examining or adapting the packet headers, which reduces the time and process complexity. Some of the embodiments herein leave the packet headers unchanged or intact, which reduces the processing overhead. An object of the invention is to enable configuration of traffic priority for each container in an orchestration platform. Another object of the invention is to enable traffic differentiation for each container in an orchestration platform.

[0051] In some embodiments, traffic differentiation such as QoS, QoE and ToS is provided to a container using Linux s / / networking structure (found on github.com / torvalds / linux / blob / master / include / linux / skbuff.h) by a mark classifier such as firewall mark classifier (fwmark) and security mark classifier (secmark) as defined in Linux. The fwmark filter enables classification of packets based on a previously set fwmark by Linux iptables or any other Netfilter framework. If the fwmark set by Linux iptables is identical to the fwmark handle, the matching process leads to a success. Linux iptables allows to mark single packets with the marktarget, or whole connections using connmark. A benefit of using such a mark instead of using Traffic Control (TC) is that by not using TC5the embodiments herein may provide a more time and / or effort-efficient means to differentiate traffic for a container. Some of the embodiments herein disclose tagging or setting an indicator to a container or a pod at Linux kernel packet structure level (by means of, for example, iprules, iptables, nfbtables, eBPF or other alternatives provided by Linux).

[0052] Some of the embodiments herein enable identification of both the container using an indicator and enable identification of the specific flow using an indicator or alternatively, by setting of the mark value to one or more data packets associated with the container. Some of the embodiments herein provide traffic priority mapping of the container to a communication network QoS (e.g. 5G QI parameters). In some examples, the container may specify the indicator or the mark value, previously configured, and the mark value or the indicator may be mapped to a performance metric such as QoS, QoE or ToS in the communication network. In some of the embodiments herein, the indicator may be related to the QoS on an external communication network such as 3rdGeneration Partnership Project (3GPP) networks such as 4G, 5G, and 6G or non-3GPP networks such as WiFi. In some examples, an indicator associated with a container or a pod is provided wherein the indicator indicates a traffic differentiation or a traffic priority flow for one or more data packets associated with the container. In some examples, the one or more data packets associated with the container or the pod may be within the container or the pod. In other words, the one or more data packets associated with the container or the pod refers to when the container or the pod comprise the one or more data packets. Thus, in some examples, the indicator may be at a data packet level. In some examples, the indicator may be at the container or the pod level. In some examples, the indicator may be at a data packets level or at a session level, wherein the session comprises the one or more data packets. In some examples, the indicator may be at a container-level encapsulation.

[0053] Some embodiments of the solution presented herein enable network QoS configuration for applications that are deployed on a container in an orchestration platform, beyond a node wherein the container and / or orchestration platform hosted in an apparatus is placed. An IP flow from a given application in a container may be configured according to the given application's performance demands or metrics in an access network (such as 5G, WiFi and 6G). In some examples, the application's performance demands or metrics may include a required QoS, a required QoE and maintenance of a service-level-agreement (SLA). Some of the embodiments presented herein may be applied to existing container orchestration platform such as Kubernetes (K8s) by means of a Container Network Interface (CNI) plugin in a native and nonintrusive manner without modification to the existing container orchestration platform.

[0054] Some embodiments of the disclosure herein enable network QoS configuration for applications placed in containers and / or pods running on an orchestration platform such as a container orchestration platform. In some examples, the traffic differentiation is provided from overlay network to underlay network by setting up a data packet mark value without manipulating a data packet in the application, which reduces the overhead to a large extent. In other words, some of the embodiments herein avoid examining or adapting the data packet headers and leave the data packet headers unchanged or intact. In some examples, the packet mark value is associated with the packet in Linux kernel and valid within the network stack in a computing node such as a physical hardware device (e.g. a user equipment (UE), a laptop, an edge device) and a software implementation (e.g.: a virtual machine, a cloud-based software). Additionally, some embodiments of the invention enable traffic differentiation for applications placed in containers and / or pods by extending a CNI with a plugin that enables prioritization of traffic such as QoS, QoE, ToS and a service level agreement (SLA) in a communication network either by interfacing an external interface (e.g. Network Exposure Function), or interfacing the network stack (e.g. 5G control plane, WiFi QoS) with the container and / or pod.

[0055] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings.

[0056] Figure 2 illustrates an example of an orchestration platform 202 hosted in an apparatus 200 according to one or more embodiments disclosed herein. The apparatus 200 comprises one or more containers 230. The apparatus 200 comprises a computing node 220, a node agent 213, a set of CNI plugins 240 and a container runtime component 250. The computing node 220 further comprises the containers 225. In some examples, the apparatus 200 may comprise entire part of the orchestration platform 202 and in other examples, the apparatus 200 may comprise a part of the orchestration platform which is not the entire orchestration platform. In cases wherein the apparatus 200 comprises a part of the orchestration platform, the computing node 200 may be controlled by an external node in a communication network or an internal component / node in the apparatus 200. In cases wherein the apparatus 200 comprises the entire orchestration platform 202, the computing node 200 may be controlled by an internal component / node in the apparatus 200 or an internal node in the apparatus 200. In some examples, the apparatus 200 may comprise a controller manager node 207. The computing node 220 may further comprise one or more of: a traffic priority CNI plugin 260 placed in the set of CNI plugins 240 and a traffic priority CNI daemon 270 connected to the traffic priority CNI plugin 260. At least one of the containers 225, container 230, comprises a container application 280 and the container application 280 being capable of processing one or more data packets 285. The communication network may be a 3GPP network such as a 5G network, a 6G network and a 4G network or a non-3GPP network such as WiFi and Low Power Wide area Access Network (LPWAN). The apparatus 200 may comprise the entire parts of the orchestration platform 202 or a part of the orchestration platform 202. In some examples, the apparatus 200 may be a user equipment (UE) in a communication network. In some examples, the apparatus 200 may be a network node in a communication network.

[0057] In some examples, the apparatus 200 comprises a control node 205 comprising an application program interface (API) server 209 and a controller manager node 207, one or more computing nodes, wherein one of the computing nodes is a computing node 220 comprising one or more containers 225. The computing node 220 comprises a node agent 213, a set of CNI plugins 240 and a container runtime component 250. The computing node 220 may further comprise a traffic priority CNI plugin 260 placed in the set of CNI plugins 240 and a traffic priority CNI daemon 270 connected to the traffic priority CNI plugin 260. The set of CNI plugins 240 comprises other CNI plugins 243 and the traffic priority plugin 260. In some examples, the computing node 220 is connected to a communication network 900. The computing node 220 may comprise one or more pods 235 and the one or more containers. Each of the pods 235 comprise at least one of the containers. In an embodiment, a pod 237 of the pods 235 comprises a container 230 which is one of the containers 225. The apparatus 200 may be connected to the communication network 900. In some embodiments, the apparatus 200 is configured to deploy a container application 280 in the computing node 220 through the API server 209. In an embodiment, the apparatus 200 is configured to deploy the container application 280 in the computing node 220 via any other means. In some embodiments, wherein the container application is deployed in the computing node 220, the apparatus 200 or the control node 205 is configured to instruct the node agent 213 and / or the container runtime 250 residing in the computing node 220 to create the one or more containers 225, wherein the container 230 comprises the application 280. The apparatus 200 or the node agent 213 is further configured to invoke a series of CNI plugins from the CNI plugins 240 in a chain to set up a network for the containers 225. The traffic priority CNI plugin 260 is configured to be triggered by the invocation from the control node manager 230 or the orchestration platform 202. The traffic priority CNI plugin 260 or the apparatus 200 is configured to obtain intended network traffic priority of the application 280, wherein the intended network traffic priority comprises information related to requirements of the container 230 or performance characteristics of the container 230. In an example, the network traffic priority comprises performance characteristics such as latency requirements, maximum error rate, bandwidth and maximum burst traffic rate. In other words, the traffic priority CNI plugin 260 or the apparatus 200 is configured to obtain traffic configuration information for the containers 225. In some examples, the apparatus 200 or the traffic priority CNI plugin 260 is configured to trigger the traffic priority CNI daemon 270, which is a daemon process that can be either deployed as a container or a standalone program, to request an underlying network interface to establish a corresponding traffic differentiation flow such as a QoS flow or a QoE flow to the communication network 900. The traffic differentiation flow is established from the computing node 220 or the apparatus 200 to the communication network 900. The traffic priority CNI plugin 260 or the apparatus 200 is further configured to set a packet data mark value to data packets that are generated or processed by the deployed container application 280. In other words, the apparatus 200 is configured to set a data packet mark value to one or more data packets 285 associated with the container 230. The data packet value mark may be set by applying ip-rules, iptables, nftables, eBPF, or other alternative approaches defined by Linux to the data packets. In an example, the packet data mark value may be a Linux fwmark or a Linux secmark, and the CNI daemon 270 will use one of the approaches such as ip-rules, iptables, nftables, eBPF, or other alternative approaches to set a fwmark value or a secmark value to every data packet of the one or more data packets 285 that is generated and / or processed and / or stored by the container application 280 or to each data packet associated to the container 230. The apparatus 200 or the CNI daemon 270 is configured to set a filter for egress traffic of the computing node 220 to distinguish data packets with data packet mark values so that they can be redirected to use the corresponding traffic differentiation flow such as a QoS flow, QoE flow and a Grade of Service (GoS) flow.

[0058] In an example embodiment, the orchestration platform 202 hosted in the apparatus 200 is a platform such as K8s, Amazon Elastic Kubernetes Service (Amazon EKS), Google Kubernetes Engine (GKE) and Azure Kubernetes Service (AKS). Furthermore, the control node 205 is a kube-controller-manager or any control plane node and the node agent 213 is a kubelet or any node agent. In an example, wherein the orchestration platform 202 is Kubernetes, a command-line tool, kubectl, enables to run commands on Kubernetes clusters. Kubectl may be used to deploy the container application 280, inspect and manage cluster resources, and view logs. The features described in figure 3 may be implemented or performed by the computing node 220 or the apparatus 200. The apparatus 200 or the computing node 220 may be a UE 700, a wireless communication device, a physical computing machine, a virtual computing machine, a network node 1100 or any other device capable of computation.

[0059] Figure 3 illustrates an example method 300 according to embodiments disclosed herein. The method comprises determining 305 a traffic differentiation flow for one or more data packets associated with at least one container, for example, the one or more data packets 285 associated with container 230 as depicted in Figure 2. The method comprises configuring 310 an indicator associated with the container, wherein the indicator provides an indication for a traffic differentiation flow for one or more data packets associated with the container 230. In some examples, the method comprises setting 315 a container tag for the container, wherein the container tag is used for setting (315) the data packet mark value when / during encapsulating the one or more data packets associated with the container. In some examples, configuring an indicator for the traffic differentiation flow associated with the container 230 comprises setting a data packet mark value to the one or more data packets associated 285 with the container. In an example embodiment, the data packet mark value may refer to a Linux fwmark filter or a Linux secmark filter which are applied outside an IP packet header. In some examples, the setting of the data packet mark value to the one or more data packets 285 comprises applying to the one or more data packets 285 associated with the container one or more of linux iprules, linux iptables, linux nftables and an extended Berkeley Packet Filter (eBPF). In some examples, the method comprises mapping 317 the indicator for the traffic differentiation flow for the one or more data packets 285 associated with the container to a performance metric of the communication network. In an example embodiment, the mapping may correspond to mapping the indicator for the traffic differentiation flow for the one or more data packets 285 to a Quality of Service, QoS, flow of the communication network. The method may comprise requesting 320 resources from an external node in the communication network for mapping the indicator corresponding to the determined traffic differentiation flow. In an example embodiment, requesting 320 resources may correspond to assigning an IP address to the container. In some embodiments, the performance metric of the communication network comprises one or more of: a Quality of Service, QoS, flow; a Grade of Service, GoS, flow; a QoE flow; an IEEE 802. Ip class of service value; and an Internet Protocol, IP, differentiated services code point, DSCP, value.

[0060] In some examples, the method may comprise obtaining 325 traffic configuration information for the container 230 and setting 330 a node-level traffic priority parameter based on the obtained traffic configuration information. In an example embodiment, the obtained traffic configuration information may comprise requirements such as desired latency, acceptable error rate, allowable maximum burst of data packets from the container and feasible maximum bandwidth allowed in the communication network. The node-level traffic priority parameter based on the obtained traffic configuration information may comprise one or more of a bandwidth, a latency, errorRate and maxBurst. The method may comprise establishing 335 a session for the container, wherein the session provides a means for communication for the container with the communication network.

[0061] In some embodiments, wherein the apparatus 200 comprises a second container 236 and the second container 236 comprises a second container application 290 capable of processing a second one or more data packets 295. The method may comprise determining 340 a second traffic differentiation flow for the second one or more data packets 295 associated with the second container 236. The method may further comprise configuring 345 a second indicator associated with the second container 236, wherein the second indicator indicates a second traffic differentiation flow for the second one or more data packets 295 associated with the second container 236. In some examples, the method comprises setting 350 a filter for egress traffic of the container based on the determined traffic differentiation flow for the one or more data packets associated with the container. In some examples, the method comprises setting 355 a filter for egress traffic of the second container based on the determined second traffic differentiation flow for the second one or more data packets associated with the second container. In an embodiment, the container application 280 and the second container application 290 are the same. In some embodiments, the apparatus 200 comprises an underlay network and an overlay network. In some examples, the orchestration platform 202 hosted in the apparatus 200 comprises an underlay network and an overlay network. The overlay network comprises an overlay network data packet. The underlay network comprises an underlay network data packet. In some examples, determining a traffic differentiation flow for the one or more data packets associated with the at least one container comprises determining a traffic differentiation flow for the overlay network data packet. In some examples, configuring an indicator for the traffic differentiation flow associated with the container comprises configuring an indicator for the traffic differentiation flow associated with the overlay network data packet. Further, the method may comprise, providing 360 the indicator for the traffic differentiation flow associated with the overlay network data packet to the underlay network without decapsulating or decrypting the overlay network data packet.

[0062] Figure 4 illustrates an example method 400 according to embodiments disclosed herein. The method is performed by the apparatus 200 connected to the communication network 900 for providing traffic differentiation in the orchestration platform 202. The apparatus 200 comprises the computing node 220. The method may comprise providing 403 instructions to the control node 205 to create or deploy the container application 280. In an embodiment, the instructions may be provided to the control node 205 via the node agent 213. In some examples, the method comprises deploying or creating 405 a container application 280 in the computing node 220. In some examples, the method comprises providing 410 instructions to the container runtime component 250 to create a container 230 comprising the container application 280 and providing instructions to the container runtime component to set up a network including a network interface for the container 230. In some examples, the method comprises creating 415 a container 230 comprising the container application 280 and setting up the network including the network interface for the container 230. The method may comprise obtaining 420 information about traffic priority for the container 230 or the container application 280. In other words, obtaining intended network traffic priority of the application 280, wherein the intended network traffic priority comprises information related to requirements of the container 230 or application 280, or information related to performance characteristics of the container 230 or the application 280. In an embodiment, the information about traffic priority or the intended network traffic priority is obtained by the CNI plugin 260. In some examples, the method further comprises requesting 425 the set up of a network interface for establishing a traffic differentiation flow such as a QoS flow, QoE flow, ToS flow and GoS flow to the communication network 900 based on the obtained traffic priority information. In some examples, the method comprises establishing 428 the traffic differentiation flow to the communication network 900. The method comprises determining 430 the traffic differentiation flow for the container 230. The traffic differentiation flow is determined by configuring 435 an indicator for the determined traffic differentiation flow. In some examples, the method comprises setting 438 a data packet mark value to packets processed and / or generated by the container 230 comprising the container application 280. In other words, the method comprises setting 438 a data packet mark value to the one or more data packets 285 associated with the container 230. In some examples, the method comprises setting 440 the data packet mark value by means of methods such as iprules, iptables and nftables. In other words, the method comprises applying to the data packet associated with the container one or more of linux iprules; linux iptables; linux nftables; and an eBPF. In some examples, the method comprises setting 445 up a filter for egress traffic of the computing node 220 to distinguish data packet value marked packets in the computing node 220 from other packets in the computing node 220. In some examples, the method comprises mapping 450 the data packet mark value (for example, from the QoS / QoE / ToS flow) to the communication network's 900 performance metrics such as QoS, QoE, ToS and GoS flow. The apparatus 200 or the computing node 220 is configured to perform the method 400.

[0063] Figure 5 illustrates a signaling flow 500 according to embodiments disclosed herein. The signaling flow may be performed by one or more of a computing node 220, a UE 700 comprising an orchestration platform 202 or an apparatus 200 comprising an orchestration platform 202 or a network node 1100 comprising an orchestration platform 202. The first step comprises installing a traffic priority CNI plugin 260 and deploying a CNI daemon 270 in the orchestration platform 202. The pods 235 may be placed in a cluster. In some examples a configuration file wherein the CNI plugin chain is defined and a computing node 220 traffic priority to be applied to the containers 225 and / or the pods 235 may be specified. The embodiment comprises instructing 505 the container runtime component 250 to create the containers 225 and / or pods 235 via the node agent 213. In an example embodiment, the instructions to the container runtime component 250 may comprise a set of resource definitions that are specified by the orchestration platform's API server 209. The configurations relating to the computing node-level traffic priority may apply to all the containers 225 and / or pods 235 deployed (the configurations have been further exemplified below). The embodiment comprises instructing 510 the CNI plugins 240, by the container runtime 250, to setup a network for the container 230 or the pod 237 including creating a container network interface for establishing a connecting to an existing cluster network and the communication network 900. A CNI configuration defines the set of CNI plugins 240 and specifies which CNI plugins are to be used by the orchestration platform 202. The embodiment comprises invoking 515 the traffic priority CNI plugin 260 to setup the traffic priority for the container 230 or the pod 237. In an example, the traffic priority CNI plugin 260 is invoked according to a user-specified traffic priority information / configuration or information relating to required network needs that are passed in from a deployment file or the CNI plugin configuration file. The embodiment comprises triggering 520 the traffic priority CNI daemon 270 by the CNI plugin 260 with the provided traffic priority information. The embodiment further comprises performing the method of features 305 and 310 or the features 430 and 435. The embodiment further comprises mapping 525 the determined traffic differentiation flow to a performance metric of the communication network 900. In an embodiment, the mapping is performed by the CNI daemon 270, the communication network 900 is an access network for 5G and the performance metric is QoS. The embodiment further comprises interfacing 530 the traffic priority CNI daemon 270 with the access network or establishing a session with the communication network 900. In an example, when the communication network is 5G, the interfacing is performed by a network exposure functions (NEF) 570 using standardized interfaces such as 3GPP SA6 Service Enabler Architecture Layer for Verticals (SEAL) Protocol and a modem host interface 575. In another example, mapping of the traffic differentiation flow is performed against the performance metric of the communication network traffic to a WiFi network, wherein the communication network 900 is a WiFi network by using a layer 2 (e.g.: IEEE 802. Ip marking) and layer 3 (e.g.: differentiated services code point (DSCP) marking) marking / filtering system. In another embodiment, the traffic priority CNI daemon may interact directly with a modem stack of the computing node 220, and therefore a QoS request will be performed using the control node 205 of the orchestration platform 202. For example, when the communication network 900 is 5G, the traffic differentiation request will be sent from an access network towards Access and Mobility Management Function (AMF) and Session Management Function (SMF). The embodiment comprises processing 535a the traffic differentiation request such as a QoS, QoE and a ToS flow and replying with a success notification if sufficient resources are present in the communication network which can be allocated to allocate the traffic differentiation request otherwise 535b a failure notification is sent to the traffic priority CNI daemon 270. The embodiment comprises if the traffic differentiation flow is successfully established, configuring 540 an indicator associated with the container 230 wherein the indicator indicates either the determined traffic differentiation flow or another traffic differentiation flow for one or more data packets associated with the container 230. The feature of configuring the indicator comprises setting a data packet mark value to the data packets associated with the container, generating a pair comprising the determined traffic differentiation flow or a performance metric of the container 230 and the data packet mark value (for example, fwmark since fwmark has 32 bits which might have some bits free against the QoS) and recording 545 the pair mapping in a table. The type of performance metric recorded in the table reflects the traffic priority configuration for communication towards the communication network 900. In an example, the performance metric may be 5G QoS flow identifier (5QI) when the communication network is a 5G network. In another example, the performance metric may be a 802. Ip class of service values or differentiated services code point when the communication network 900 is WiFi network. The performance metric can also be any identifier that can be mapped to the traffic priority configuration for the communication network. The table below exemplifies mapping of the indicator or the data packet mark value to traffic differentiation identifiers in different communication networks. The data packet mark value, for example, an fwmark is a 32-bit field which can be customized according to needs of the user and / or the orchestration platform 202. The indicator or the data packet mark value is not set within the actual packet but it is a value that is associated within the Linux kernel packet structure. Therefore, no manipulation to the original packet is needed, which eliminates the necessity and overhead of decapsulation and reencapsulation processes when trying to decipher which traffic differentiation is required by the container 230 or the data packets associated with the container 230. The indicator or the data packet mark value should be consistent with the bits available in the fwmark because some bits are reserved. E.g., Calico uses bits 16-31 (mark mask 0xFFFF0000), Kubernetes uses bits 14-15 (mark mask 0x0000C000), and CNI Portmap uses bit 13 (mark mask 0x00002000). In an example, if we reserve bits 0-12 for traffic priority configuration or traffic priority information, a corresponding packet value mark mask 0X00001FFF may be set. Some example of pairs corresponding to the mapping as mentioned above are provided in the table below. The table only provides examples of how the indicator or the data packet value mark may be mapped to traffic differentiation classes / identifiers such as QoS classes / identifiers in different access networks, however, the table does not indicate any association between the QoS classes of the exemplified identifiers (5QI, IP DSCP and 802. Ip). The pair mapping in practice may be implementation-specific.

[0064] The indicator or the data packet mark value and corresponding performance metric pair table shall be kept consistent at least per device such as a physical machine or a virtual machine to avoid any conflicting use of the data packet mark values or the indicator between containers. For example, in the case that multiple cluster computing nodes reside on the same physical host, in some embodiments, the consistency can be achieved by synchronizing the pair mapping tables between all traffic priority CNI daemons that are deployed in each computing node of the cluster of computing nodes. In another embodiment, the traffic priority CNI daemon 270 is deployed directly on the physical host or a virtual host so that a single unified pair mapping table can be utilized. Also, when the container 230 or the pod 237 is stopped, the data packet mark value or the indicator entry is removed or purged from the table, to make the indicator available for re-use. In an embodiment, the traffic priority CNI daemon 270 configuring 550 the data packet mark value by means of but not limited to iptables, ip rules, nftables and eBPF. For example, when using iptables, a pre-routing chain in a mangle table can be used to mark the data packets generated from the container 230 and / or the pod 237 with the data packet mark value or the indicator as described above. Using a Command Line Interface (CLI) command, in an example, the configuration would take similar effect as executing "sudo iptables -t mangle -A PREROUNTING -j MARK — set-xmark 0xll / 0x00001FFF -s 192.168.1.4 -sport 1234 -p tcp -i cni0".

[0065] The embodiment comprises configuring or setting 555 an egress filter using a utility such as TC, which directs IP packets with a specific data packet mark value into a given queue, wherein the queue is then configured to use the determined traffic differentiation flow established particularly for the IP flow. Using the CLI command, in an example, the filter can be configured in this way: "sudo tc filter add dev ens2f0 parent 1:0 u32 \ match mark 0x110X00001FFF flowid 1:2".

[0066] The embodiment comprises responding 560 via the traffic priority CNI daemon 270 to the traffic priority CNI plugin 260 with a success notification upon establishment of the determined traffic differentiation flow. In case, the traffic differentiation flow cannot be established for some reasons, the embodiment comprises sending 565 a failure notification to the traffic priority CNI plugin 260 by the CNI daemon 270, which may be logged thereafter.

[0067] Figure 6 illustrates a signaling flow 600 in accordance with one or more embodiments disclosed herein. The embodiment discloses on alternate, generalized, embodiment for requesting the traffic differentiation applicable for any communication network such as a 3GPP network or a non-3GPP network. The embodiment comprises the features as defined for method 500 without the modem host interface 575 and the network exposure function 570.

[0068] Figure 7 shows a UE 700 in accordance with some embodiments. The UE 700 may comprise the apparatus 200. As used herein, a UE refers to a device capable, configured, arranged and / or operable to communicate wirelessly with network nodes and / or other UEs. Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle, vehicle-mounted or vehicle embedded / integrated wireless device, etc. Other examples include any UE identified by the 3rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB-IoT) UE, a machine type communication (MTC) UE, and / or an enhanced MTC (eMTC) UE.

[0069] The UE 700 enables provision of traffic differentiation for an orchestration platform, wherein the UE 700 comprises one or more containers 225 and at least one of the containers 230 comprises a container application 280. The UE 700 is configured to determine a traffic differentiation flow for one or more data packets 285 associated with the at least one container 230 and configure an indicator associated with the container, wherein the indicator indicates a traffic differentiation flow for the data packets associated with the container. A processing circuitry 702 is adapted / configured / operable to cause the UE 700 to perform a set of operations, or for example, steps, of the methods 300, 400, 500 and 600 as disclosed above, e.g., by executing instructions stored in memory 704. The processing circuitry 702 may comprise one or more of a microprocessor, a controller, a microcontroller, a central a processing unit, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other components of the UE 700, such as the memory 704, in order to provide relevant functionality. The processing circuitry 702 in this regard may implement certain functional means, units, or modules. Memory 704 may include one or more non-volatile storage medium and / or one or more volatile storage medium or a cloud-based storage medium. A computer program product 810 may be provided in the apparatus 200, the computing node 220 or the UE 700. Such computer program product is described in relation to figure 8. The memory 704 may store any suitable instructions, data, or information, including software, an application including one or more of logic, rules, code, tables, and / or other instructions / computer program code capable of being executed by the processing circuitry 702 and utilized by the UE 700. The memory 704 may further be used to store any calculations made by the processing circuitry 702 and / or any data received via the communication interface 712, such as input from the UE 700. In some embodiments, the processing circuitry 702 and memory 704 are integrated.

[0070] A UE may support device-to-device (D2D) communication, for example by implementing a 3GPP standard for sidelink communication. Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and / or operates the relevant device. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent a device that is not intended for sale to, or operation by, an end user but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).

[0071] The UE 700 includes processing circuitry 702 that is operatively coupled via a bus 722 to an input / output interface 706, a power source 708, a memory 704, a communication interface 712, and / or any other component, or any combination thereof. Certain UEs may utilize all or a subset of the components shown in Figure 7. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0072] The processing circuitry 702 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored os machine-readable computer programs in the memory 704. The processing circuitry 702 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 702 may include multiple central processing units (CPUs).

[0073] In the example, the input / output interface 706 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and / or output devices. Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof. An input device may allow a user to capture information into the UE 700. Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like. The presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user. A sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.

[0074] In some embodiments, the power source 708 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used. The power source 708 may further include power circuitry for delivering power from the power source 708 itself, and / or an external power source, to the various parts of the UE 700 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 708. Power circuitry may perform any formatting, converting, or other modification to the power from the power source 708 to make the power suitable for the respective components of the UE 700 to which power is supplied. The memory 704 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 704 includes one or more application programs 714, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 716. The memory 704 may store, for use by the UE 700, any of a variety of various operating systems or combinations of operating systems.

[0075] The memory 704 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUICC), integrated UICC (iUICC) or a removable UICC commonly known as 'SIM card.' The memory 704 may allow the UE 700 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 704, which may be or comprise a device-readable storage medium.

[0076] The processing circuitry 702 may be configured to communicate with an access network or other network using the communication interface 712. The communication interface 712 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 710. The communication interface 712 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a network node in an access network). Each transceiver may include a transmitter 718 and / or a receiver 720 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, the transmitter 718 and receiver 720 may be coupled to one or more antennas (e.g., antenna 710) and may share circuit components, software or firmware, or alternatively be implemented separately.

[0077] In the illustrated embodiment, communication functions of the communication interface 712 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, near-field communication, locationbased communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may be implemented in according to one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol / internet protocol (TCP / IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext T ransfer Protocol (HTTP), and so forth.

[0078] Regardless of the type of sensor, a UE 700 may provide an output of data captured by its sensors, through its communication interface 712, via a wireless connection to a network node. Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE. The output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).

[0079] As another example, a UE 700 comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection. In response to the received wireless input the states of the actuator, the motor, or the switch may change. For example, the UE 700 may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input. A UE 700, when in the form of on Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare. Non-limiting examples of such an loT device are a device which is or which is embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door / window sensor, a flood / moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal- or item-tracking device, a sensor for monitoring a plant oranimal, an industrial robot, an Unmanned Aerial Vehicle (UAV), and any kind of medical device, like a heart rate monitor or a remote controlled surgical robot. A UE 700 in the form of an loT device comprises circuitry and / or software in dependence of the intended application of the loT device in addition to other components as described in relation to the UE 700 shown in Figure 7.

[0080] As yet another specific example, in an loT scenario, a UE 700 may represent a machine or other device that performs monitoring and / or measurements, and transmits the results of such monitoring and / or measurements to another UE and / or a network node. The UE may in this case be an M2M device, which may in a 3GPP context be referred to as an MTC device. As one particular example, the UE may implement the 3GPP NB-IoT standard. In other scenarios, a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and / or reporting on its operational status or other functions associated with its operation.

[0081] In practice, any number of UEs may be used together with respect to a single use case. For example, a first UE might be or be integrated in a drone and provide the drone's speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone. When the user makes changes from the remote controller, the first UE may adjust the throttle on the drone (e.g. by controlling an actuator) to increase or decrease the drone's speed. The first and / or the second UE can also include more than one of the functionalities described above. For example, a UE 700 might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.

[0082] Figure 8 illustrates one example of a computer program product in accordance with an embodiment of the invention. Computer program product 810 may comprise a computer readable storage medium 830 storing a computer program 820 comprising computer readable instructions. Computer readable medium 830 of the UE 700, may be a non-transitory computer readable medium, such as, magnetic media (e.g., a hard disk), optical media, memory devices (e.g., random access memory, flash memory), and the like. In some embodiments, the computer readable instructions of computer program 820 are configured such that when executed by processing circuitry 702, the computer readable instructions cause the UE 700 to perform any of the method described herein. In other embodiments, the UE 700 may be configured / operable to perform steps described herein without the need for code. That is, for example, processing circuitry 702 may consist merely of one or more ASICs. Hence, the features of the embodiments described herein may be implemented in hardware and / or software.

[0083] Figure 9 shows an example of a communication network 900 in accordance with some embodiments. In the example, the communication network 900 includes a telecommunication network 902 that includes an access network 904, such as a radio access network (RAN), and a core network 906, which includes one or more core network nodes 908. The access network 904 includes one or more access network nodes, such as network nodes 910a and 910b (one or more of which may be generally referred to as network nodes 910), or any other similar 3GPP access nodes or non-3GPP access points. Moreover, as will be appreciated by those of skill in the art, a network node is not necessarily limited to an implementation in which a radio portion and a baseband portion are supplied and integrated by a single vendor. Thus, it will be understood that network nodes include disaggregated implementations or portions thereof. For example, in some embodiments, the telecommunication network 902 includes one or more Open-RAN (ORAN) network nodes. An ORAN network node is a node in the telecommunication network 902 that supports an ORAN specification (e.g., a specification published by the O-RAN Alliance, or any similar organization) and may operate alone or together with other nodes to implement one or more functionalities of any node in the telecommunication network 902, including one or more network nodes 910 and / or core network nodes 908. Examples of 3GPP RAN nodes are a 2G RAN node or a base transceiver substation, a 3G RAN node or a nodeB, a 4G RAN node or a nodeB, a 5G RAN node or a gNodeB, a 6G RAN node or any other RAN node in future communications technologies. Examples of non-3GPP RAN nodes or non-3GPP access node are an loT access node or a vehicular communication access node or any other RAN or access node in other communications technologies.

[0084] Examples of an ORAN network node include an open radio unit (O-RU), an open distributed unit (O-DU), an open central unit (O-CU), including an O-CU control plane (0- CU-CP) or an O-CU user plane (O-CU-UP), a RAN intelligent controller (near-real time or non-real time) hosting software or software plug-ins, such as a near-real time control application (e.g., xApp) or a non-real time control application (e.g., rApp), or any combination thereof (the adjective "open" designating support of an ORAN specification). The network node may support a specification by, for example, supporting an interface defined by the ORAN specification, such as an Al, Fl, Wl, El, E2, X2, Xn interface, an open fronthaul user plane interface, or an open fronthaul management plane interface. Moreover, an ORAN access node may be a logical node in a physical node. Furthermore, an ORAN network node may be implemented in a virtualization environment (described further below) in which one or more network functions are virtualized. For example, the virtualization environment may include an O-Cloud computing platform orchestrated by a Service Management and Orchestration Framework via an 0-2 interface defined by the 0-RAN Alliance or comparable technologies. The network nodes 910 facilitate direct or indirect connection of UE, such as by connecting UEs 912a, 912b, 912c, and 912d (one or more of which may be generally referred to as UEs 912) to the core network 906 over one or more wireless connections.

[0085] Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication network 900 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired or wireless connections. The communication network 900 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0086] The UEs 912 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with the network nodes 910 and other communication devices. Similarly, the network nodes 910 are arranged, capable, configured, and / or operable to communicate directly or indirectly with the UEs 912 and / or with other network nodes or equipment in the telecommunication network 902 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in the telecommunication network 902.

[0087] In the depicted example, the core network 906 connects the network nodes 910 to one or more hosts, such as host 916. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 906 includes one more core network nodes (e.g., core network node 908) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 908. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier Deconcealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0088] The host 916 may be under the ownership or control of a service provider other than an operator or provider of the access network 904 and / or the telecommunication network 902, and may be operated by the service provider or on behalf of the service provider. The host 916 may host a variety of applications to provide one or more service. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0089] As a whole, the communication network 900 of Figure 9 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and / or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0090] In some examples, the telecommunication network 902 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 902 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 902. For example, the telecommunications network 902 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC) / Massive loT services to yet further UEs.

[0091] In some examples, the UEs 912 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 904 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 904. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC). In the example, the hub 914 communicates with the access network 904 to facilitate indirect communication between one or more UEs (e.g., UE 912c and / or 912d) and network nodes (e.g., network node 910b). In some examples, the hub 914 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 914 may be a broadband router enabling access to the core network 906 for the UEs. As another example, the hub 914 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 910, or by executable code, script, process, or other instructions in the hub 914. As another example, the hub 914 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 914 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 914 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 914 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, the hub 914 acts as a proxy server or orchestrator for the UEs, in particular if one or more of the UEs are low energy loT devices.

[0092] The hub 914 may have a constant / persistent or intermittent connection to the network node 910b. The hub 914 may also allow for a different communication scheme and / or schedule between the hub 914 and UEs (e.g., UE 912c and / or 912d), and between the hub 914 and the core network 906. In other examples, the hub 914 is connected to the core network 906 and / or one or more UEs via a wired connection. Moreover, the hub 914 may be configured to connect to an M2M service provider over the access network 904 and / or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with the network nodes 910 while still connected via the hub 914 via a wired or wireless connection. In some embodiments, the hub 914 may be a dedicated hub - that is, a hub whose primary function is to route communications to / from the U Es from / to the network node 910b. In other embodiments, the hub 914 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 910b, but which is additionally capable of operating as a communication start and / or end point for certain data channels. In some examples, the network node 910 may comprise a network node 700. In some examples, the UE 912 may comprise a UE 700.

[0093] Preferably, the communication network 900 comprises an apparatus 200. Preferably, the communication network 900 comprises a computing node 220. Preferably, the communication network 900 comprises a UE 700. Examples of the computing node 220 in the communication network 900 are, but not limited to, a 3GPP network node, a non-3GPP network node, an loT network node, a cloud-native network node, a constrained network node, a vehicular communication network node or any other node in any of the aforementioned network types. In an example, the computing node 220 specified herein may either be an loT device, a constrained device, a cloud-native device, a telecommunications device, or a network device such as a 3GPP Radio Access Network (RAN) node, a core network node, an external 3rd party node or any other non- 3GPP access node. In an embodiment, the communication network 900 may comprise or be a RAN. In an embodiment, the computing node 220 may comprise or be a RAN node.

[0094] The computer program code mentioned above may also be provided, for instance in the form of a data carrier carrying computer program code for performing the embodiments herein when being loaded into the hardware. One such carrier may be in the form of a CD ROM disc. It is however feasible with other data carriers such as a memory stick. The computer program code may furthermore be provided as pure program code on an apparatus 200, a UE 700, a computing node 220, and downloaded to the hardware at production, and / or during software updates.

[0095] Figure 10 shows a communication diagram of a host 1002 communicating via a network node 1004 with a UE 1006 over a partially wireless connection in accordance with some embodiments.

[0096] Embodiments of host 1002 include hardware, such as a communication interface, processing circuitry, and memory. The host 1002 also includes software, which is stored in or accessible by the host 1002 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as the UE 1006 connecting via an over-the-top (OTT) connection 1050 extending between the UE 1006 and host 1002. In providing the service to the remote user, a host application may provide user data which is transmitted using the OTT connection 1050. The host 1002 may comprise one of the apparatus 200, the UE 700 or the computing node 220.

[0097] The network node 1004 includes hardware enabling it to communicate with the host 1002 and UE 1006. The connection 1060 may be direct or pass through a core network and / or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet. The network node 1004 may comprise one of the apparatus 200, the UE 700 or the computing node 220.

[0098] The UE 1006 includes hardware and software, which is stored in or accessible by UE 1006 and executable by the UE's processing circuitry. The software includes a client application, such as a web browser or operator-specific "app" that may be operable to provide a service to a human or non-human user via UE 1006 with the support of the host 1002. In the host 1002, an executing host application may communicate with the executing client application via the OTT connection 1050 terminating at the UE 1006 and host 1002. In providing the service to the user, the UE's client application may receive request data from the host's host application and provide user data in response to the request data. The OTT connection 1050 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through the OTT connection 1050. The UE 1006 may comprise one of the apparatus 200, the UE 700 or the computing node 220.

[0099] The OTT connection 1050 may extend via a connection 1060 between the host 1002 and the network node 1004 and via a wireless connection 1070 between the network node 1004 and the UE 1006 to provide the connection between the host 1002 and the UE 1006. The connection 1060 and wireless connection 1070, over which the OTT connection 1050 may be provided, have been drawn abstractly to illustrate the communication between the host 1002 and the UE 1006 via the network node 1004, without explicit reference to any intermediary devices and the precise routing of messages via these devices.

[0100] As an example of transmitting data via the OTT connection 1050, in step 1008, the host 1002 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interacting with the UE 1006. In other embodiments, the user data isassociated with a UE 1006 that shares data with the host 1002 without explicit human interaction. In step 1010, the host 1002 initiates a transmission carrying the user data towards the UE 1006. The host 1002 may initiate the transmission responsive to a request transmitted by the UE 1006. The request may be caused by human interaction with the UE 1006 or by operation of the client application executing on the UE 1006. The transmission may pass via the network node 1004, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 1012, the network node 1004 transmits to the UE 1006 the user data that was carried in the transmission that the host 1002 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 1014, the UE 1006 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 1006 associated with the host application executed by the host 1002.

[0101] In some examples, the UE 1006 executes a client application which provides user data to the host 1002. The user data may be provided in reaction or response to the data received from the host 1002. Accordingly, in step 1016, the UE 1006 may provide user data, which may be performed by executing the client application. In providing the user data, the client application may further consider user input received from the user via an input / output interface of the UE 1006. Regardless of the specific manner in which the user data was provided, the UE 1006 initiates, in step 1018, transmission of the user data towards the host 1002 via the network node 1004. In step 1020, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 1004 receives user data from the UE 1006 and initiates transmission of the received user data towards the host 1002. In step 1022, the host 1002 receives the user data carried in the transmission initiated by the UE 1006.

[0102] One or more of the various embodiments improve the performance of OTT services provided to the UE 1006 using the OTT connection 1050, in which the wireless connection 1070 forms the last segment. More precisely, the teachings of these embodiments may improve the efficiency, differentiation of traffic within a UE 700 or the apparatus 200, provisioning of different SLAs and thereby provide benefits such as reduced user waiting time, reduced time and resource-complexity while applying traffic differentiation, reduced overhead and optimized resource usage in the communication network. In an example scenario, factory status information may be collected and analyzed by the host 1002. As another example, the host 1002 may process audio and video data which may have been retrieved from a UE for use in creating maps. As another example, the host 1002 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 1002 may store surveillance video uploaded by a UE. As another example, the host 1002 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, the host 1002 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and / or transmitting data.

[0103] In some examples, a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 1050 between the host 1002 and UE 1006, in response to variations in the measurement results. The measurement procedure and / or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of the host 1002 and / or UE 1006. In some embodiments, sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 1050 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 1050 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of the network node 1004. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by the host 1002. The measurements may be implemented in that software causes messages to be transmitted, in particular empty or 'dummy' messages, using the OTT connection 1050 while monitoring propagation times, errors, etc. Although the computing devices described herein (e.g., UEs, network nodes, hosts) may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to be understood that these computing devices may comprise any suitable combination of hardware and / or software needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and / or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and / or the functionality of the components may be partitioned between the processing circuitry and the communication interface. In another example, non- computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.

[0104] In some examples, the orchestration platform 202 may be hosted in the apparatus 200. In some examples, the orchestration platform 202 may be hosted in the UE 700. In some examples, the orchestration platform 202 hosted in the apparatus 200 or the UE 700 may comprise one or more computing nodes. In some examples, the orchestration platform 202 hosted in the apparatus 200 or the UE 700 may comprise one or more pods 235 comprising one or more containers 230. In some examples, the orchestration platform 202 hosted in the apparatus 200 or the UE 700 may comprise one or more clusters, wherein each of the one or more clusters comprises one or more pods 235.

[0105] In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer-readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and / or by end users and a wireless network generally.

[0106] Figure 11 shows a network node 700 in accordance with some embodiments. Figure 11 shows a network node 1100 in accordance with some embodiments. As used herein, network node refers to equipment capable, configured, arranged and / or operable to communicate directly or indirectly with a UE and / or with other network nodes or equipment, in a telecommunication network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations. Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)), O-RAN nodes or components of an O-RAN node (e.g., O-RU, O-DU, O-CU).

[0107] The network node 1100 enables provision of traffic differentiation for an orchestration platform, wherein the network node 1100 comprises one or more containers 225 and at least one of the containers 230 comprises a container application 280. The network node 1100 is configured to determine a traffic differentiation flow for one or more data packets 285 associated with the at least one container 230 and configure an indicator associated with the container, wherein the indicator indicates a traffic differentiation flow for the data packets associated with the container. A processing circuitry 1102 is adapted / configured / operable to cause the network node 1100 to perform a set of operations, or for example, steps, of the methods 300, 400, 500 and 600 as disclosed above, e.g., by executing instructions stored in memory 1104. The processing circuitry 1102 may comprise one or more of a microprocessor, a controller, a microcontroller, a central a processing unit, a digital signal processor, an applicationspecific integrated circuit, a field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other components of the network node 1100, such as the memory 1104, in order to provide relevant functionality. The processing circuitry 1102 in this regard may implement certain functional means, units, or modules. Memory 1104 may include one or more non-volatile storage medium and / or one or more volatile storage medium or a cloud-based storage medium. A computer program product 810 may be provided in the apparatus 200, the computing node 220 or the network node 1100. Such computer program product is described in relation to figure 8. The memory 1104 may store any suitable instructions, data, or information, including software, an application including one or more of logic, rules, code, tables, and / or other instructions / computer program code capable of being executed by the processing circuitry 1102 and utilized by the network node 1100. The memory 1104 may further be used to store any calculations made by the processing circuitry 1102 and / or any data received via the communication interface 1106, such as input from the network node 1100. In some embodiments, the processing circuitry 1102 and memory 1104 are integrated.

[0108] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units, distributed units (e.g., in an O-RAN access node) and / or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0109] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi- cell / multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes. Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g.. Evolved Serving Mobile Location Centers (E-SMLCs)), and / or Minimization of Drive Tests (MDTs). The network node 1100 includes a processing circuitry 1102, a memory 1104, a communication interface 1106, and a power source 1108. The network node 1100 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which the network node 1100 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, the network node 1100 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 1104 for different RATs) and some components may be reused (e.g., a same antenna 1110 may be shared by different RATs). The network node 1100 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 1100, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 1100.

[0110] The processing circuitry 1102 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other network node 1100 components, such as the memory 1104, to provide network node 1100 functionality.

[0111] In some embodiments, the processing circuitry 1102 includes a system on a chip (SOC). In some embodiments, the processing circuitry 1102 includes one or more of radio frequency (RF) transceiver circuitry 1112 and baseband processing circuitry 1114. In some embodiments, the radio frequency (RF) transceiver circuitry 1112 and the baseband processing circuitry 1114 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 1112 and baseband processing circuitry 1114 may be on the same chip or set of chips, boards, or units.

[0112] The memory 1104 may comprise any form of volatile or non-volatile computer- readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and / or any other volatile or non-volatile, non-transitory device- readable and / or computer-executable memory devices that store information, data, and / or instructions that may be used by the processing circuitry 1102. The memory 1104 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and / or other instructions capable of being executed by the processing circuitry 1102 and utilized by the network node 1100. The memory 1104 may be used to store any calculations made by the processing circuitry 1102 and / or any data received via the communication interface 1106. In some embodiments, the processing circuitry 1102 and memory 1104 is integrated.

[0113] The communication interface 1106 is used in wired or wireless communication of signaling and / or data between a network node, access network, and / or UE. As illustrated, the communication interface 1106 comprises port(s) / terminal(s) 1116 to send and receive data, for example to and from a network over a wired connection. The communication interface 1106 also includes radio front-end circuitry 1118 that may be coupled to, or in certain embodiments a part of, the antenna 1110. Radio front-end circuitry 1118 comprises filters 1120 and amplifiers 1122. The radio front-end circuitry 1118 may be connected to an antenna 1110 and processing circuitry 1102. The radio front-end circuitry may be configured to condition signals communicated between antenna 1110 and processing circuitry 1102. The radio front-end circuitry 1118 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection. The radio front-end circuitry 1118 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 1120 and / or amplifiers 1122. The radio signal may then be transmitted via the antenna 1110. Similarly, when receiving data, the antenna 1110 may collect radio signals which are then converted into digital data by the radio front-end circuitry 1118. The digital data may be passed to the processing circuitry 1102. In other embodiments, the communication interface may comprise different components and / or different combinations of components.

[0114] In certain alternative embodiments, the network node 1100 does not include separate radio front-end circuitry 1118, instead, the processing circuitry 1102 includes radio front-end circuitry and is connected to the antenna 1110. Similarly, in some embodiments, all or some of the RF transceiver circuitry 1112 is part of the communication interface 1106. In still other embodiments, the communication interface 1106 includes one or more ports or terminals 1116, the radio front-end circuitry 1118, and the RF transceiver circuitry 1112, as part of a radio unit (not shown), and the communication interface 1106 communicates with the baseband processing circuitry 1114, which is part of a digital unit (not shown).

[0115] The antenna 1110 may include one or more antennas, or antenna arrays, configured to send and / or receive wireless signals. The antenna 1110 may be coupled to the radio front-end circuitry 1118 and may be any type of antenna capable of transmitting and receiving data and / or signals wirelessly. In certain embodiments, the antenna 1110 is separate from the network node 1100 and connectable to the network node 1100 through an interface or port.

[0116] The antenna 1110, communication interface 1106, and / or the processing circuitry 1102 may be configured to perform any receiving operations and / or certain obtaining operations described herein as being performed by the network node. Any information, data and / or signals may be received from a UE, another network node and / or any other network equipment. Similarly, the antenna 1110, the communication interface 1106, and / or the processing circuitry 1102 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and / or signals may be transmitted to a UE, another network node and / or any other network equipment.

[0117] The power source 1108 provides power to the various components of network node 1100 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). The power source 1108 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 1100 with power for performing the functionality described herein. For example, the network node 1100 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 1108. As a further example, the power source 1108 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.

[0118] Embodiments of the network node 1100 may include additional components beyond those shown in Figure 11 for providing certain aspects of the network node's functionality, including any of the functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, the network node 1100 may include user interface equipment to allow input of information into the network node 1100 and to allow output of information from the network node 1100. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 1100.

[0119] The person skilled in the art will appreciate that the blocks in the circuit diagram of the computing node 220 may refer to a combination of analog and digital circuits, and / or one or more controllers, configured with software and / or firmware, e.g. stored in one or more local storage units, that when executed by the apparatus 200, the UE 700, the network node 700 or the computing node 220 perform the steps of the method 300, 400, 500 and 600 as described above. An apparatus 200 can be comprised in a network node 1100. An apparatus 200 can be comprised in a UE 700. One or more of the computing node 220 or the apparatus 200, the UE 700, as well as any other combination of analog and digital circuits, may be included in a single application-specific integrated circuitry (ASIC), or several controllers and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a system-on-a-chip (SoC). The one or more computing node 220 or the apparatus 200, the UE 700 may be any one of, or a combination of a central processing unit (CPU), graphical processing unit (GPU), programmable logic array (PAL) or any other similar type of circuit or logical arrangement. The teachings of these embodiments presented herein may improve the availability and reliability of the electronic device and thereby provide benefits such as higher throughput and / or lower latency.

[0120] The following definitions may assist the reader in understanding the various terms used in the application. Some of the definitions herein may be found on the official Kubernetes website, kubnetes.io.

[0121] Linux is an open-source, Unix clone. Linux is just the kernel and supports more file systems and interfaces than Unix.

[0122] Kernel is a computer program at the core of a computer's operating system and generally has complete control over everything in the system. A kernel is the portion of the operating system code that is always resident in memory and facilitates interactions between hardware and software components.

[0123] A CNI is a common interface between the network plugins and container execution.

[0124] Kubernetes runs your workload by placing containers into pods to run on nodes. A node may be a virtual or physical machine, depending on the cluster. Each node is managed by the control plane and contains the services necessary to run pods. A node may be a computing node 220 or an apparatus 200 or a UE 700 or a network node 1100. Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications. Typically, a cluster comprises several nodes; in a learning or resource-limited environment, a cluster may comprise only one node. The components on a node include the kubelet and a container runtime.

[0125] Pods are the smallest deployable units of computing that can be created / deployed and managed in Kubernetes and other orchestration platforms. A pod contains one or more containers. Containers within one pod share the same resources and network.

[0126] A container image is a ready-to-run software package, containing everything needed to run an application such as the code and any runtime the container image requires, application and system libraries, and default values for any essential settings. Each node in a K8s cluster is configured to run containers that form pods assigned to that node. Containers in a pod are co-located and co-scheduled to run on the same node. Container orchestration is the automation of much of the operational effort required to run containerized workloads and services. Container orchestration includes a wide range of things software teams need to manage a container's lifecycle, including provisioning, deployment, scaling (up and down), networking, load balancing and more.

[0127] A CNI plugin enables usage of user-generated logic in an orchestration platform. Each CNI plugin is executed / called similar to how a function is called (for example, a binary program).

[0128] DaemonSet is a workload object that manages groups of replicated pods. However, DaemonSets attempt to adhere to a one-pod-per-node model, either across the entire cluster or a subset of nodes. As you add nodes to a node pool, DaemonSets automatically add pods to the new nodes as needed.

[0129] A cluster is a set of node machines for running containerized applications.

[0130] For example, in K8s, using flannel and a traffic priority CNI plugin, the configuration will look like:

[0131] {

[0132] "name": "cbr0",

[0133] "cniVersion": "0.3.1",

[0134] "plugins": [

[0135] {

[0136] "type": "flannel",

[0137] "delegate": {

[0138] "hairpinMode": true,

[0139] "forceAddress": true,

[0140] "isDefaultGateway": true

[0141] }

[0142] },

[0143] {

[0144] "type": "portmap",

[0145] "capabilities": {

[0146] "portMappings": true

[0147] }

[0148] }, {

[0149] "type": "traffic_priority",

[0150] "NodePriority": [

[0151] {

[0152] "request": {

[0153] "latency": "100ms",

[0154] "errorRate": "0.01",

[0155] "maxBurst": "None"

[0156] }

[0157] },

[0158] {

[0159] "guaranteed": {

[0160] "latency":"200ms",

[0161] "errorRate":"0.01",

[0162] "maxBurst":"None"

[0163] }

[0164] },

[0165] {

[0166] "limit": {

[0167] "latency":"10ms",

[0168] "errorRate":"0.000001",

[0169] "maxBurst":"None"

[0170] }

[0171] }

[0172] ]

[0173] }

[0174] ]

[0175] }

[0176] The configurations defined in spec / container / ports / trafficPriority would take effect for the specific port. The priority of the configurations from high to low is port level configurations, container or pod level configurations, and node level configurations. The resource definition in a yaml file looks as follows: / / deployment.ya ml apiVersion: vl kind: Pod metadata: name: example annotations: kubernetes.io / requestLatency: "100ms" kubernetes.io / requestErrorRate: "0.001" kubernetes.io / requestMaxBurst: "None" kubernetes.io / guaranteedLatency: "150ms" kubernetes.io / guaranteedErrorRate: "0.01" kubernetes.io / guaranteedMaxBurst: "None" kubernetes.io / limitLatency: "75ms" kubernetes.io / limitErrorRate: "0.0001" kubernetes.io / limitMaxBurst: "None" spec: containers:

[0177] - name: example image: example-image ports:

[0178] - containerPort: 1234 name: server trafficPriority: request: latency: "50ms" errorRate: "0.0001" maxBurst: "255byte" guaranteed: latency: "100ms" errorRate: "0.001" maxBurst: "None" limit: latency: "10ms" errorRate: "0.0001 maxBurst: "None"

[0179] The traffic priority configuration exemplified above is one example of what type of parameters can be configured. However, configurable parameters are not limited to "latency", "errorRate" and "maxBurst". Other parameters such as "bandwidth" or any parameters that are related to traffic priority in a specific access network (e.g., WiFi) may be supported.

[0180] Provision of consumable / usable hardware and software resources from the apparatus 200 or the UE 700 for different users as per each users' requirement may be provided in an orchestration platform such as a container orchestration platform. For example, the low-level parameters may be abstracted by using classes such as "time- critical", "high-bandwidth", "time-and-bandwidth", which are considered as high-level classifications and that can be provided from the resources. At least in some embodiments, the bandwidth can be specified and configured so that the bandwidth limit feature of bandwidth CNI plugin can be extended from only being applied on the cluster node to both on the node and the access network to keep a consistent bandwidth limit.

Claims

CLAIMS1. A method (300) for providing traffic differentiation for an orchestration platform (202), wherein the orchestration platform comprises one or more containers (225) and at least one of the containers (230) comprises a container application (280), the method comprising: determining (305) a traffic differentiation flow for one or more data packets (285) associated with the at least one container (230); and configuring (310) an indicator associated with the container, wherein the indicator indicates a traffic differentiation flow for the one or more data packets (285) associated with the container (230).

2. The method according to claim 1, wherein configuring (310) the indicator comprises setting (315) a data packet mark value to the one or more data packets (285) associated with the container (230).

3. The method according to claim 2, wherein setting (315) the data packet mark value comprises applying one or more of: linux iprules; linux iptables; linux nftables; and an extended Berkeley Packet Filter, eBPF.

4. The method according to claims 1-3, wherein the orchestration platform is comprised in a communication network (900), the method comprising: mapping (317) the indicator to a performance metric of the communication network (900).

5. The method according to claim 4, comprises requesting (320) resources from a communication network node in the communication network for mapping (317) the indicator corresponding to the determined traffic differentiation flow.

6. The method according to claims 4 or 5, wherein the performance metric of the communication network (900) comprises one or more of: a Quality of Service, QoS, flow; a Grade of Service, GoS, flow; a Quality of Experience, QoE, flow; an IEEE 802. Ipclass of service value; and an Internet Protocol, IP, differentiated services code point, DSCP, value.

7. The method according to one or more of claims 1-6, comprising: obtaining (325) traffic configuration information for the container (230); and setting (330) a node-level traffic priority parameter based on the obtained traffic configuration information.

8. The method according to claim 7, wherein the parameter comprises one or more of: a bandwidth; a latency; errorRate; or maxBurst.

9. The method according to one or more of claims 1-8, comprising: establishing (335) a session for the container, wherein the session provides a means for communication for the container (230) with the communication network (900).

10. The method according to one or more of claims 1 to 9, wherein the orchestration platform comprises a second container (236) and the second container (236) comprises a second container application (290), the method comprising: determining (340) a second traffic differentiation flow for a second one or more data packets (295) associated with the second container (236); and configuring (345) a second indicator associated with the second container, wherein the second indicator indicates a second traffic differentiation flow for the second one or more data packets (295) associated with the second container (236); setting (350) a filter for egress traffic of the container based on the determined traffic differentiation flow for the one or more data packets (285) associated with the container (230); setting (355) a filter for egress traffic of the second container (236) based on the determined second traffic differentiation flow for the second one or more data packets (295) associated with the second container (236).

11. The method according to claim 10, wherein the container application (280) and the second container application (290) are the same.

12. The method according to one or more of claims 1 to 11, wherein the orchestration platform comprises an underlay network and an overlay network, wherein the one or more data packets (285) associated with the at least one container (230) comprises one or more data packets of the overlay network and configuring an indicator for the traffic differentiation flow associated with the container comprises: configuring an indicator associated with the container, wherein the indicator indicates a traffic differentiation flow for the data packets of the overlay network, the method further comprising: providing (360) the indicator for the traffic differentiation flow associated with the one or more data packets of the overlay network data packets to the underlay network without decapsulating or decrypting the data packets of the overlay network.

13. The method according to claim 2, comprising: setting a container tag for the container, wherein the container tag is used for setting (315) the data packet mark value when encapsulating the data packets associated with the container.

14. The method according to one or more of claims 1-13, wherein the method is implemented by a user equipment, UE.

15. An apparatus (200; 700; 1100) for providing traffic differentiation for an orchestration platform (202), wherein the apparatus comprises one or more containers (225) and at least one of the containers (230) comprises a container application (280), the apparatus configured to: determine a traffic differentiation flow for one or more data packets (285) associated with the at least one container (230); and configure an indicator associated with the container, wherein the indicator indicates a traffic differentiation flow for the one or more data packets (285) associated with the container (230).

16. The apparatus according to claim 15, wherein configuring the indicator comprises setting a data packet mark value to the one or more data packets (285) associated with the container (230).

17. The apparatus according to claim 16, wherein setting the data packet mark value comprises applying one or more of: linux iprules; linux iptables; linux nftables; and an extended Berkeley Packet Filter, eBPF.

18. The apparatus according to one or more of claims 15-17, configured to: set a container tag for the container, wherein the container tag is used for setting (315) the data packet mark value when encapsulating the data packets associated with the container.

19. The apparatus according to claims 15-18, wherein the orchestration platform (202) is comprised in a communication network (900) and the apparatus configured to: map the indicator to a performance metric of the communication network (900).

20. The apparatus according to claim 19, configured to: request resources from a communication network node in the communication network (900) for mapping the indicator corresponding to the determined traffic differentiation flow.

21. The apparatus according to claims 19 or 20, wherein the performance metric of the communication network comprises one or more of: a Quality of Service, QoS, flow; a Grade of Service, GoS, flow; a Quality of Experience, QoE, flow; an IEEE 802. Ip class of service value; and an Internet Protocol, IP, differentiated services code point, DSCP, value.

22. The apparatus according to one or more of claims 15-21, configured to: obtain traffic configuration information for the container (230);set a node-level traffic priority parameter based on the obtained traffic configuration information.

23. The apparatus according to claim 22, wherein the parameter comprises one or more of: a bandwidth; a latency; errorRate; or maxBurst.

24. The apparatus according to one or more of claims 15-23, configured to: establish a session for the container (230), wherein the session provides a means for communication for the container (230) with the communication network (900).

25. The apparatus according to one or more of claims 15 to 24, wherein the apparatus comprises a second container (236) and the second container (236) comprises a second container application (290), configured to: determine (240) a second traffic differentiation flow for a second one or more data packets (295) associated with the second container (236); and configure (245) a second indicator associated with the second container (236), wherein the indicator indicates a second traffic differentiation flow for the second one or more data packets (295) associated with the second container (236); set (250) a filter for egress traffic of the container based on the determined traffic differentiation flow for the one or more data packets (285) associated with the container (230); set (255) a filter for egress traffic of the second container based on the determined second traffic differentiation flow for the second one or more data packets (295) associated with the second container (236).

26. The apparatus according to one or more of claims 15 to 25, wherein the apparatus comprises an underlay network and an overlay network, wherein the data packets (285) associated with the at least one container (230) comprises one or more data packets of the overlay network and configuring an indicator associated with the container (230), wherein the indicator indicates a traffic differentiation flow associatedwith the container (230) comprises: configuring an indicator associated with the container (230), wherein the indicator indicates a traffic differentiation flow for the one or more data packets of the overlay network, the apparatus configured to: provide (260) the indicator for the traffic differentiation flow associated with the data packets of the overlay network to the underlay network without decapsulating or decrypting the data packets of the overlay network.

27. The apparatus according to one or more of claims 15-27, wherein the apparatus comprises a user equipment, UE.

28. The apparatus according to one or more of claims 15-27, wherein the apparatus comprises one of: a wireless communication device; a physical computing machine; and a virtual computing machine.

29. An apparatus (200; 700; 1100) for providing traffic differentiation for an orchestration platform (202), wherein the apparatus comprises one or more containers (225) and at least one of the containers (230) comprises a container application (280), the apparatus comprising a processing circuitry (702; 1102) and a memory (704; 1104), the memory containing instructions that when executed by the processing circuitry cause the apparatus to perform the method according to one or more of claims 1-14.

30. A computer program (820) comprising instructions which, when executed by a processing circuitry (702; 1102) of an apparatus (700; 1100), causes the apparatus to carry out the methods according to one or more of claims 1-14.

31. A computer program product (810) which comprises a non-transitory computer readable medium (830) on which a computer program (820) according to claim 30 is stored.

Images