Micro-service aware framework for web application collaboration

Abstract

The invention relates to a micro-service aware framework for web application collaboration. A method for managing micro-services includes receiving, for each of a plurality of micro-services, a respective requirement for quality of service, where the plurality of micro-services collectively construct an application; receiving a first tag, where the first tag indicates a policy state of a communication system, the communication system communicating data packets from the microservice for the application; receiving an indication of a first quality of experience of the application; determining an importance level for a first micro-service in the micro-services based on a requirement of service quality of the first micro-service in the micro-services, the first label and the first experience quality; marking data packets originating from or assigned to a first one of the microservices and to be transmitted by the communication system with the importance level determined for the first one of the microservices; the data packet tagged with the importance level is provided to a communication system for transmission of the data packet.

Description

Technical Field

[0001] The present disclosure relates to the collaboration of a network and an application composed of microservices.

[0002] abbreviation

[0003] 3GPP: Third Generation Partnership Project

[0004] 5G: Fifth Generation

[0005] ADAS: Advanced Driver Assistance System

[0006] AI: Artificial Intelligence

[0007] API: Application Programming Interface

[0008] AR: Augmented Reality

[0009] CSC: Communications Services Customer

[0010] CSP: Communications Service Provider

[0011] DDPG: Deep Deterministic Policy Gradient

[0012] ECN: Explicit Congestion Notification

[0013] IMO: Importance Tagging Optimizer

[0014] KPI: Key Performance Indicator

[0015] LAN: Local Area Network

[0016] MAC: Media Access Control

[0017] MF: Maximum Minimum Fairness

[0018] ML: Machine Learning

[0019] NSaaS: Network Slicing as a Service

[0020] NSMF: Network Slice Management Function

[0021] NSSMF: Network Slicing Subnet Management Function

[0022] PF: Proportional Fairness

[0023] PHY: Physical layer

[0024] QoE: Quality of Experience

[0025] QoS: Quality of Service

[0026] RAN: Radio Access Network

[0027] RR: Round Robin

[0028] SLA: Service Level Agreement

[0029] TD: Time Domain

[0030] URLLC: Ultra-Reliable Low Latency Communication

[0031] VR: Virtual Reality Background Art

[0032] In 3GPP, a service level agreement (SLA) is a contract between a communication service provider (CSP) and a communication service customer (CSC) that defines the communication service requirements for a defined service, such as 5G LAN and Ultra-Reliable Low Latency Communication (URLLC) [3GPP TS 28.535]. This means that application services can specify the minimum acceptable standards for network services, and the CSP provides the required network services. Examples of application services include the streaming service Netflix, the cloud gaming service GeForce NOW, or the autonomous driving service TomTom's driver assistance system (ADAS).

[0033] Today's applications are typically composed of a large number of microservices managed through virtual machines or container environments. Companies such as Amazon, Netflix, and Uber have adopted microservices architecture to achieve scalability and adaptability.

[0034] There are several solutions for managing the microservices architecture of cloud-native applications. The most successful platform for designing and deploying cloud applications is Kubernetes, which provides a wide range of solutions for optimal placement, selection, and execution of the microservices that make up an application. Summary of the Invention

[0035] The purpose of the present disclosure is to improve upon the prior art.

[0036] According to a first aspect, there is provided an apparatus comprising:

[0037] a component for receiving, for each of a plurality of microservices, a corresponding quality of service requirement, wherein the plurality of microservices together constitute an application;

[0038] means for receiving a first tag, wherein the first tag indicates a policy state of a communication system that transmits data packets from the microservice for the application;

[0039] means for receiving an indication of a first quality of experience of an application;

[0040] a component for determining an importance level for a first microservice among the microservices based on a quality of service requirement of the first microservice, the first label, and the first quality of experience;

[0041] means for marking a data packet originating from or destined for a first one of the microservices and to be transmitted by the communication system with an importance level determined for the first one of the microservices;

[0042] Means for providing data packets marked with an importance level to a communication system for transmitting the data packets.

[0043] The determining component may be configured to determine a corresponding importance level for each of the microservices based on the quality of service requirement of the corresponding microservice, the first label, and the first quality of experience; and

[0044] The marking component may be configured to mark, for each of the microservices, data packets originating from or destined for the respective microservice and to be transmitted by the communication system using the importance level determined for the respective microservice.

[0045] The determining component may be configured to determine the inter-microservice importance level based on a quality of service requirement of a first microservice among the microservices, a quality of service requirement of a second microservice among the microservices, the first label, and the first quality of experience; and

[0046] The marking component may be configured to mark an inter-microservice data packet with an inter-microservice importance level, wherein the inter-microservice data packet is to be transmitted between a first microservice and a second microservice among the microservices by the communication system.

[0047] The importance level determined for a first one of the microservices may be included in a header of a data packet originating from or destined for the first one of the microservices and to be transmitted by the communication system.

[0048] Data packets originating from or destined for a first one of the microservices and to be transmitted by the communication system may belong to a traffic flow, and the traffic flow may be marked with an importance level.

[0049] The device may also include:

[0050] means for receiving, in response to providing the data packet marked with the importance level to the communication system, from the communication system a second tag indicating a policy state of the communication system after the communication system evaluated the importance level;

[0051] means for receiving an indication of a second quality of experience for the application in response to providing the data packets marked with the importance level to the communication system;

[0052] Means for adapting the determination for optimizing the quality of experience of the application by means for determining based on the first label, the second label, the quality of service requirement of each of the microservices, the first quality of experience, and the second quality of experience.

[0053] The adapting component may be configured to adapt the above determination by a component for determining by machine learning. The machine learning may include at least one of reinforcement learning and supervised learning.

[0054] According to a second aspect, there is provided an apparatus comprising:

[0055] means for receiving data packets from the optimizer for transmitting the data packets by the communication system, wherein the data packets are marked with an importance level;

[0056] means for modifying a policy state of a communication system based on the importance level;

[0057] A component for notifying the optimizer of the policy state of the communication system through the tag after the policy state of the communication system is modified by the modification component.

[0058] The device may also include:

[0059] Means for mapping a policy state of a communication system to tags, wherein the number of available tags is less than the number of available policy states of the communication system.

[0060] The policy state of the communication system may include at least one of the policy state of the physical layer of the communication system, the policy state of the media access control layer of the communication system, the policy state of the transport layer of the communication system, the edge computing policy state of the communication system, or the computing resource allocation policy state of the communication system.

[0061] According to a third aspect, there is provided a method comprising:

[0062] Receive quality of service requirements for each of a plurality of microservices that together comprise an application;

[0063] receiving a first tag, wherein the first tag indicates a policy state of a communication system that transmits a data packet from a microservice for an application;

[0064] receiving an indication of a first quality of experience of the application;

[0065] Determine an importance level for the first microservice among the microservices based on the quality of service requirement of the first microservice among the microservices, the first label, and the first quality of experience;

[0066] marking a data packet originating from or destined for a first one of the microservices and to be transmitted by the communication system using an importance level determined for the first one of the microservices;

[0067] The data packets marked with the importance level are provided to a communication system for transmitting the data packets.

[0068] For each microservice in the microservices, the corresponding importance level may be determined based on the service quality requirement of the corresponding microservice, the first label, and the first experience quality.

[0069] For each of the microservices, data packets originating from or destined for the respective microservice and to be transmitted by the communication system may be marked with an importance level determined for the respective microservice.

[0070] The inter-microservice importance level may be determined based on a quality of service requirement of a first microservice among the microservices, a quality of service requirement of a second microservice among the microservices, the first label, and the first quality of experience; and

[0071] Inter-microservice data packets may be tagged with an inter-microservice importance level, wherein the inter-microservice data packets are to be transmitted by the communication system between a first one of the microservices and a second one of the microservices.

[0072] The importance level determined for a first one of the microservices may be included in a header of a data packet originating from or destined for the first one of the microservices and to be transmitted by the communication system.

[0073] Data packets originating from or destined for a first one of the microservices and to be transmitted by the communication system may belong to a traffic flow, and the traffic flow may be marked with an importance level.

[0074] The method may further include:

[0075] receiving, in response to providing the data packet tagged with the importance level to the communication system, a second tag from the communication system indicating a policy state of the communication system after the communication system evaluated the importance level;

[0076] receiving an indication of a second quality of experience for the application in response to providing the data packets marked with the importance level to the communication system;

[0077] Determination of the importance level is adapted based on the first label, the second label, the quality of service requirement of each of the microservices, the first quality of experience, and the second quality of experience to optimize the quality of experience of the application.

[0078] The determination of the importance level may be adapted through machine learning, which may include at least one of reinforcement learning and supervised learning.

[0079] According to a fourth aspect, there is provided a method comprising:

[0080] receiving data packets from an optimizer for transmission of the data packets by a communication system, wherein the data packets are marked with an importance level;

[0081] modifying a policy state of a communication system based on the importance level;

[0082] The optimizer is informed of the policy state of the communication system through the tag after the policy state of the communication system is modified.

[0083] The method may further include:

[0084] A policy state of the communication system is mapped to tags, wherein a number of available tags is less than a number of available policy states of the communication system.

[0085] The policy state of the communication system may include at least one of the policy state of the physical layer of the communication system, the policy state of the media access control layer of the communication system, the policy state of the transport layer of the communication system, the edge computing policy state of the communication system, or the computing resource allocation policy state of the communication system.

[0086] Each of the methods of the third aspect and the fourth aspect may be a collaborative method between a network and an application.

[0087] According to a fifth aspect, a computer program product is provided, the computer program product comprising an instruction set which, when executed on a device, is configured to cause the device to perform the method according to any one of the third and fourth aspects. The computer program product may be embodied as a computer-readable medium or directly loadable into a computer.

[0088] According to some example embodiments, at least one of the following advantages may be achieved:

[0089] The network can provide resources and capacity based on the importance of microservices to QoE;

[0090] Resources and energy can be used efficiently;

[0091] Oversupply can be avoided or reduced;

[0092] • The privacy of at least one of the application and the network (or both) can be protected from the other. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Further details, features, objects and advantages are apparent from the following detailed description of preferred example embodiments taken in conjunction with the accompanying drawings, in which:

[0094] Figure 1 shows a framework according to some example embodiments;

[0095] Figure 2 (including Figure 2A and Figure 2B ) shows an action flow according to some example embodiments;

[0096] Figure 3 shows importance tag assignment extended to inter-microservice communications according to some example embodiments;

[0097] Figure 4 shows a basic actor-critic architecture for optimizing importance level assignments according to some example embodiments;

[0098] Figure 5 It shows how the example embodiments can be applied to a streaming media application (e.g., Netflix) as an example;

[0099] Figure 6 An apparatus according to an example embodiment is shown;

[0100] Figure 7 A method according to an example embodiment is shown;

[0101] Figure 8 An apparatus according to an example embodiment is shown;

[0102] Figure 9 A method according to an example embodiment is shown; and

[0103] Figure 10 An apparatus according to an example embodiment is shown. DETAILED DESCRIPTION

[0104] Below, with reference to the accompanying drawings, certain exemplary embodiments are described in detail, wherein, unless otherwise specified, the features of the exemplary embodiments can be freely combined with each other. However, it should be clearly understood that the description of certain exemplary embodiments is given only as an example and is by no means intended to limit the present disclosure to the disclosed details.

[0105] Furthermore, it should be understood that the apparatus is configured to perform the corresponding method, although in some cases only the apparatus or only the method is described.

[0106] SLA is essentially a one-way framework, where the CSC specifies the requirements first and the CSP then provides the network services. This approach can lead to the following challenges:

[0107] 1) Widely used applications today are typically built as a collection of microservices that operate independently and communicate via well-defined interfaces (typically using lightweight APIs). Each of these microservices has its own network service requirements and contributes to the end-user experience (QoE) at varying levels of importance. SLAs based on network slicing cannot meet the diverse requirements and importance of multiple microservices in an application.

[0108] 2) The unidirectional framework assumes that the customer / application knows exactly how to specify its requirements to provide the best user experience. However, in reality, due to limited interaction between the CSP and the CSC, neither the application nor the network knows how network QoS maps to the user's QoE. This often leads to overprovisioning of communication resources, which can compromise network performance in multi-user, resource-scarce scenarios such as cloud gaming and AR / VR.

[0109] Decomposing applications into microservices brings some challenges to network services:

[0110] 1) The operations of different microservices can impose different requirements on the communication system (network). For example, for Uber, the microservices for trip management and billing have higher latency and higher security requirements than passenger and driver management (ratings and reviews).

[0111] 2) Communication between microservices can introduce different network requirements. Some microservices require synchronous communication, while others allow asynchronous or event-driven communication. For example, at Netflix, user authentication requires synchronous communication between the authentication microservice and the user management microservice, while rating and recommendation microservices allow asynchronous communication.

[0112] 3) To achieve a good user experience, each microservice must execute correctly and in a timely manner. However, not all microservices contribute equally to the user experience. For example, a delay in rating or reviewing may not have as significant an impact on the user experience as a delay in ordering or paying. Differentiating the importance of different microservices can help the network provide dedicated and prioritized performance for the microservices most critical to the user experience.

[0113] Therefore, it is useful to introduce a microservice-aware communication service solution that targets a goal-oriented user experience. As an additional requirement, it would be beneficial if the interaction between the network and the application is lightweight. Furthermore, it is recommended to protect the privacy of both parties (the application (CSC) and the communication service (CSP)).

[0114] Management solutions for microservices-based applications (such as Kubernetes) typically only consider the cloud infrastructure, performing operations based on the availability and performance of compute and storage nodes. They provide customizable ways to handle microservices by defining both QoS and priority, which are used by the scheduler to optimize resource allocation in the cloud infrastructure. These solutions often lack knowledge of the network side. This lack of knowledge can lead to overprovisioning on the network side, which is very expensive to ensure low latency and ultra-reliability, as is the case for industrial applications. Other services provide network monitoring as a service, but they cannot take action at the network level, but rather respond to any network impairments.

[0115] Some example embodiments consider both the inherent microservices architecture underlying an application and how different microservices can be valued / prioritized differently based on the application's intended goal (i.e., the QoE for the application's users). To ensure good (optimized) QoE, at least some of the packets transmitted by the communication system for the application (i.e., to / from the microservices that comprise the application, and between microservices) are tagged with an importance level. The communication system (sometimes referred to as the network) considers the importance level when processing the packets and adapts its state (network policy) accordingly.

[0116] The traditional "one SLA per application / service" approach fails to consider that microservices can have varying QoS requirements. This ignorance can limit the scalability and profitability of network services. Furthermore, the one-way framework of traditional SLAs (i.e., "the application specifies the requirements, and the network provides the service") often leads to overprovisioning of communication resources and inefficient use of energy. This is especially true in resource-scarce, multi-user scenarios like cloud gaming or AR / VR, where the network can easily become congested due to the overprovisioning of numerous applications.

[0117] Some exemplary embodiments provide a lightweight, microservice-aware, two-sided privacy-preserving web application collaboration framework to improve the QoE of goal-oriented users. In particular, the following problems are addressed:

[0118] Achieving user goals: How can we autonomously learn the importance levels that reflect the different QoS requirements of multiple microservices and their different contributions to the final QoE?

[0119] Application-network information exchange: How to facilitate the exchange of microservice-specific information between application providers and network providers, while preferably protecting the privacy of both parties?

[0120] Lightweight application network collaboration framework:

[0121] From the application side, how do you assign importance levels to microservices and communicate this to the network efficiently?

[0122] From the network side, how to enable importance-level-aware network optimization policies and efficiently communicate them to applications?

[0123] Figure 1 A framework according to some example embodiments is shown in FIG. Figure 1 In the figure, the numbers in the circles indicate where the actions shown in Figure 2 (described further below) occur. The framework is composed as follows:

[0124] On the application / service side:

[0125] Each application / service assigns a different importance level to its microservices and embeds this "importance tag" in the header of network packets and / or traffic flows. Alternatively, the application can signal the importance tag to the network orchestration and management functions to identify flows based on their importance tags. Thus, all packets of a flow are tagged with the flow's importance tag. Note that importance tags can change over time to adapt to the application's current goals.

[0126] Importance Labeling Optimizer (IMO) uses techniques such as reinforcement learning to assign importance levels to each microservice based on the following information:

[0127] Goal-oriented rewards: QoE of the observed user

[0128] Microservice-specific state: Microservice-specific QoS requirements

[0129] • Extracted network policy state: "Network Label" (the extracted label of the network's policy, as provided by the network).

[0130] The network does not need to know any details of the microservices, nor does it participate in the application's decision to assign importance levels to microservices. This protects the privacy of the application from the network.

[0131] On the communication system side:

[0132] The network has a set of optional policies for allocating resources based on "importance marking", such as channel allocation, retransmission probability control, radio resource scheduling, and other resource allocation policies, including both communication and computing resources (for example, in the case of mobile edge computing). The differentiation of importance levels of different microservices helps the network provide dedicated and prioritized performance for the microservices that are most critical to the user experience.

[0133] The network maps its chosen policy (e.g., radio scheduling, congestion control, retransmission policy, etc.) to a “network tag,” e.g., by embedding a classification, and provides it to the application. The network tag may also correspond to a specific network slice identifier, where the network slice embeds the E2E network policy for serving flows with the same importance tag.

[0134] In some example embodiments, the tag may indicate in detail the state of the communication system (network), such as the state of the physical layer of the communication system, the state of the media access control layer of the communication system, the state of the transport layer of the communication system, the state of the edge computing policy of the communication system, or the state of the computing resource allocation policy of the communication system. However, in some example embodiments, the tag shows the aggregate state of the network. That is, generally, the number of available tags is less than the number of available states of the network. This protects the privacy of the network for applications.

[0135] Importance tags and labels enable ultra-lightweight information exchange between applications and the network, as the network can only provide high-level abstract labels representing its policy set (i.e., network configuration) to avoid heavyweight and frequent feedback on its fine-grained state (such as channel status or buffer size).

[0136] In some example embodiments, the framework is extended to inter-microservice communications (i.e., between different microservices of an application) communicated by a communication system. The importance marking assigned to packets in inter-microservice communications can depend on the QoS requirements of the two microservices.

[0137] In some example embodiments, the importance tag can be calculated by the IMO. The IMO can be implemented as an add-on to the API gateway. As another option, the IMO can be implemented as a standalone application or microservice. It helps the network adapt its strategy to achieve the application's goals (i.e., good or even optimized QoE) by providing dedicated or prioritized performance to the microservices that are most critical to the user's QoE.

[0138] Figure 2 (including Figure 2A and Figure 2B ) shows the action flow according to some example embodiments. The numbers of the actions in FIG2 are also Figure 1 Shown in (circled). Figure 1 , numbers indicate where in the framework the corresponding actions occur according to some example embodiments. The actions are as follows:

[0139] 1. The network (e.g., the network management and orchestration function or the network slice management function (NSMF)) defines its policy combinations across different layers (e.g., channel allocation policy at the PHY layer, packet scheduling policy at the MAC layer, etc.) (1.a), and typically maps them to network labels (1.b) through encoding techniques, such as classification embedding. The details of the "mapping from policy combinations to network labels" scheme will be further explained below. At least some of these policies (if not all) are aware of importance tags. For example, according to the channel allocation policy, packets with higher importance tags will be assigned better channels; according to the scheduling policy, packets with lower importance tags will be assigned fewer resources and are considered to have higher delay tolerance. An example design of importance tag-aware network policies is further provided below.

[0140] 2. The network (e.g., Network Management and Orchestration Function or Network Slice Management Function (NSMF)) sends the extracted network label (indicating its policy status) to the IMO on the application side. The IMO receives this message from the network.

[0141] 3. The application collects the QoS requirements of the microservices that make up the application (3.a), for example, via an API gateway. The application provides information about the QoS requirements to the IMO for importance tag assignment (3.b).

[0142] 4. The application collects user feedback and calculates rewards based on the goal-oriented user experience (QoE). Below, we will use Netflix as an example application to describe how to determine QoE. The determination of QoE is application-specific.

[0143] 5. Based on the collected network labels, QoS requirements of microservices and observed user feedback (QoE) data, IMO calculates the status (5.a), reward (5.b), and decides the action (5.c) to assign importance level to each microservice. For example, Figure 1 In the example, microservices 3, 2, and 1 are assigned different importance tags 1, 3, and 2 respectively. Note that although Figure 1 In

[15] , ranked positive integers are used as importance tags, but other values ​​can also be used as importance tags, such as positive floating point numbers. The optimized microservice-specific importance tags are then provided to the API gateway (5.d).

[0144] The importance label assignment IMO can be trained using AI / ML techniques, such as reinforcement learning. An example of reinforcement learning-based IMO is provided below.

[0145] As another option, the IMO may include a lookup table in which an importance tag is indicated for each combination of network tags, QoS requirements of microservices, and QoE. In some example embodiments, the IMO may derive an intermediate value from the lookup table based on a subset of input parameters: network tags, QoS requirements of microservices, and QoE, and determine the importance tag using a trained neural network, into which the intermediate value and the remaining input parameters are input.

[0146] 6. The API gateway embeds the importance tag determined by the IMO into the headers of packets or business flows from different microservices. These packets and business flows are sent and received through the communication system.

[0147] 7. If the packet or traffic flow has an importance tag, the network activates the importance tag-aware policy and applies it to the packet or traffic flow. Furthermore, the network adapts its policy (such as communication and / or computing resource allocation decisions) for the packet or traffic flow based on the importance tag. An example is further described below.

[0148] 8. In some example embodiments, the importance tagging determined by the IMO is static. However, according to some example embodiments, using samples continuously collected in the IMO (QoS requirements of microservices, network labels, and QoE), the IMO can train / update / fine-tune its model based on the preferences of the application. Here are some options:

[0149] a. Regularly fine-tune the model with newly collected samples,

[0150] b. Continuously update the model through continuous learning, or

[0151] c. Activate model retraining reactively or predictively by observing or predicting degradation in user QoE.

[0152] Not only can different microservices have different network requirements, but inter-microservice communication can also have different network requirements. For example, at Netflix, user authentication requires synchronous communication with stricter latency constraints between the authentication microservice and the user management microservice, while rating and recommendation allow asynchronous communication with higher latency tolerance.

[0153] like Figure 3 As shown in , importance tag assignment can not only tag packets from microservices to entities outside the application / from entities outside the application to microservices, but also tag communication messages between microservices in the application. Figure 3 In the example, microservices 3, 2, and 1 are assigned importance tags 1, 3, and 2, respectively. Furthermore, the communication message between microservices 2 and 3 is assigned importance tag 2, while the communication message between microservices 1 and 2 is assigned importance tag 1. Therefore, the resource and energy efficiency of inter-microservice communication is also improved.

[0154] For inter-microservice communication, except actions 3, 5, and 6, all other steps are the same as Figure 1 Same as explained in Figure 2. Modifications to actions 3, 5 and 6 are as follows:

[0155] Action 3: The application collects not only the QoS requirements for operating microservices, but also the QoS requirements for inter-microservice communication.

[0156] Action 5: IMO also includes the QoS requirements for inter-microservice communication in the input characteristics, and in the output operations, decides the importance level of inter-microservice communication.

[0157] Action 6: Assign optimized importance tags to the operation groups / flows of the microservices and the messages for inter-microservice communication.

[0158] From the network side, these actions are related to Figure 1Same as shown in Figure 2, in that the network policy adapts the priority or weight of packets / flows based on the assigned importance tags, whether for operating microservices or for inter-microservice communication.

[0159] In the following, examples of network label extraction (action 1.b), label-aware policy (action 7) and IMO implementation will be described in more detail.

[0160] Network label extraction

[0161] Next, an example of network label extraction (action 1 in Figure 2; i.e., how to derive a network label based on the actual policy state of the network) will be described. In some example embodiments, labels will be created and managed at the network management and orchestration function level. As another option in the context of 5G slicing, label management occurs in the NSMF (Network Slice Management Function), which relies on different NSSMFs (Network Slice Subnet Management Functions) at the access, transport, and core network levels to implement network configuration and optimization policies for the slice. The NSMF and NSSMF can be enhanced to create and maintain network labels.

[0162] The network can be configured with a large number of network optimization strategies consisting of selectable strategies at each layer. For example, the RAN scheduler's options at the MAC layer include Round Robin (RR), Proportional Fair (PF), Max-Min Fair (MF), etc. The network label mapping function can map the network's operating strategies to short codes as "network labels", such as Figure 1 As shown in block A in .

[0163] Assume that the network has a set of N strategies implemented at different layers, represented by the set Each strategy All have K n Different options To activate to operate the network. For example, the policy set Including the beamforming strategy of the PHY layer, the scheduling strategy of the MAC layer, etc., and the scheduling strategy options include RR, PF, and MF. Operate on a set of selected options, where Representation Strategy Therefore, the network label mapping problem is to convert the classification information Projection to the code alphabet This is an encoding problem, that is, to find an encoder:

[0164]

[0165] because Contains classification information, so known classification embedding techniques can be used.

[0166] Tag-aware policies

[0167] At least a portion of the network policy may include importance tag awareness to prioritize packets / flows from microservices with higher importance levels, as indicated by the corresponding importance tags. There may be different strategies to prioritize or weight network performance based on importance tags and these strategies may be introduced into the options of the network policy, such as Figure 1 As shown in module B in

[15] , in this way, the information of the selected strategy that depends on the importance tag is also embedded in the network label and provided to the application to better optimize the importance level assignment.

[0168] In one example, the RAN scheduling policy is extended with importance tagging awareness. Other policies can be easily extended similarly. There are two traditional options for RAN scheduling policies: Round Robin (RR) and Proportional Fairness (PF). RR processes all packets / flows without priority, also known as round-robin execution. In order to introduce the priority of microservices with higher importance levels into RR, they can be given a fixed priority based on the importance level. For example, importance levels 3, 2, and 1 are defined with priority 1, 2, and 3 respectively. For traditional PF, the channel is based on the priority function P=T α / R β Scheduling is performed using where T represents the achievable data rate in the current time slot, R is the historical average data rate, and α and β are tuning factors used to adjust fairness. This formula balances serving the best terminals with the best channel conditions with serving terminals with poorer performance, enabling acceptable performance levels. To introduce importance tagging awareness, we can define a modified priority P' = g(P, I), which monotonically increases with the importance level I for any fixed priority level P. Various functions can be defined, such as the affine function, where P' = aP + b and a and b are non-negative real numbers.

[0169] IMO implementation

[0170] The definition and use of IMO are described with respect to actions 3, 5, and 8. IMO can be implemented as a plug-in attachment in the API gateway. For example, the provider of the attachment can be a CSP. As described above, IMO serves as a lightweight interface between the application API and the network system. On the one hand, it receives feedback from the network through the extracted network tags; on the other hand, it receives information about microservices and user feedback from the API gateway. Since it is an attachment in the API gateway, the application is responsible for its own assignment of importance levels to microservices, and the network only reacts passively to this, which helps to protect the privacy of the application. The network also benefits from the embedded importance tagging of packets / flows because it allows for more efficient resource and energy utilization to meet the user's QoE.

[0171] As an example embodiment, an IMO for importance label assignment can be designed using a reinforcement learning agent, such as Figure 1 As shown in module C in .

[0172] • The state consists of 1) the network label indicating the network policy and 2) the QoS requirements of the microservice. For the second embodiment, we can add 3) the QoS requirements for inter-microservice communication.

[0173] The action is a specific importance tag for each microservice. For inter-microservice communication, the action also includes the importance tag for the inter-microservice communication.

[0174] Rewards are defined as the QoE of goal-oriented users.

[0175] Various reinforcement learning methods can be used, such as the actor-critic algorithm. The actor-critic algorithm combines policy-based methods (actor) and value-based methods (critic). Specifically, the actor learns a policy for making decisions about actions based on the observed states, and the critic evaluates the actions taken by the actor in a given state by estimating a value function. Figure 4 An example embodiment of an actor-critic deep reinforcement learning model architecture for optimizing importance level assignment is shown in FIG.

[0176] Some example embodiments provide at least one of the following advantages:

[0177] Goal-oriented and microservice-aware: Microservice-specific importance tagging helps the network adapt its policies based on the application’s goals by providing dedicated or prioritized performance to the microservices most critical to user QoE.

[0178] Efficient resource and energy utilization: The network adapts its optimization strategy based on importance tags to prevent over-provisioning of resources and encourage energy conservation. For example, less critical microservices can be allocated fewer resources and require less frequent retransmissions without affecting user QoE.

[0179] Two-Sided Privacy: 1) Applications optimize and assign importance tags to their microservices without the network directly intervening. 2) The network maps its policies to tags, and applications are unaware of the details of the network configuration.

[0180] Lightweight network application interaction: The information exchanged only includes high-level abstractions of network labels and importance tags embedded in packets / flows. In addition, the interaction is achieved via IMO as a lightweight plug-in attachment to the API gateway.

[0181] Some example embodiments may be applied to a wide range of applications built from multiple microservices with different QoS requirements, such as:

[0182] Cloud Gaming: The microservice responsible for capturing the player's physical interactions with the game controller will process real-time, high-frequency, small-size data, while the microservice responsible for computing the game visualization will need access to specialized hardware, and the microservice streaming the game's visual state will require high bandwidth.

[0183] AR / VR Applications: VR applications will share many similarities with cloud gaming applications. The different microservices handling control, visualization, and streaming will have very different requirements. AR applications can be more versatile in terms of operating modes and the level of added information. However, their different subcomponents will inevitably be very unbalanced in terms of upstream and downstream network requirements.

[0184] Vertical industrial applications: Some microservices will collect large amounts of heterogeneous data from physical sensors (resulting in a wide variety of reported data frequencies and sizes), while automation microservices will apply control loop algorithms and issue control commands to physical actuators. Some controls need to be real-time (e.g., safety-related controls), while others can have longer time windows (e.g., small performance configuration adjustments).

[0185] As an example, it is shown how the framework according to some example embodiments may be applied to a streaming application, such as Netflix.

[0186] Netflix's services are decomposed into many microservices. Table 1 shows some of these microservices, along with their descriptions and QoS requirements.

[0187]

[0188] Table 1: Example microservices for a streaming application and their QoS requirements.

[0189] User QoE can be calculated based on the following metrics:

[0190] Rebuffer Rate: Measures how often playback is temporarily interrupted while more data is downloaded from the server to replenish the local buffer on the client device.

[0191] Bitrate: The quality of the picture provided / viewed, for example, a low bitrate will produce a blurry picture.

[0192] Other intrinsic metrics related to the viewing experience.

[0193] Direct feedback or ratings from users.

[0194] Other analytics, including active subscribers, subscriber growth rate, churn rate, hours of content streamed, and more.

[0195] like Figure 1As shown in Figure 2, the IMO add-on can be added to the Netflix system so that importance tags are automatically learned and assigned to microservices based on the extracted network tags provided by the network, the QoS requirements of the microservices known to the API gateway, and the user QoE calculated based on the application's interest. Figure 5 Note that for the sake of brevity, the details of the network system side are omitted, but the architecture, functions and message flows are similar to those shown in Figure 2, and the IMO algorithm can have the same Figure 4 Similar design as described above.

[0196] Figure 6 An apparatus according to an example embodiment is shown. The apparatus may be an optimizer (such as an IMO) or an element thereof. Figure 7 A method according to an example embodiment is shown. Figure 6 The device can perform Figure 7 method, but not limited to this method. Figure 7 The method can be Figure 6 The present invention is performed by a device, but is not limited to being performed by the device.

[0197] The apparatus includes a first receiving component 110, a second receiving component 120, a third receiving component 130, a determining component 140, a marking component 150, and a providing component 160. The first receiving component 110, the second receiving component 120, the third receiving component 130, the determining component 140, the marking component 150, and the providing component 160 may be a first receiving component, a second receiving component, a third receiving component, a determining component, a marking component, and a providing component, respectively. The first receiving component 110, the second receiving component 120, the third receiving component 130, the determining component 140, the marking component 150, and the providing component 160 may be a first receiver, a second receiver, a third receiver, a determining component, a marking component, and a providing component, respectively. The first receiving component 110, the second receiving component 120, the third receiving component 130, the determining component 140, the marking component 150, and the providing component 160 may be a first receiving processor, a second receiving processor, a third receiving processor, a determining processor, a marking processor, and a providing processor, respectively.

[0198] The first receiving component 110 receives a corresponding service quality requirement for each of the plurality of microservices (S110). The plurality of microservices jointly construct an application.

[0199] The second receiving component 120 receives a tag (S120). The tag indicates a policy state of a communication system that transmits data packets from a microservice to an application.

[0200] The third receiving component 130 receives an indication of the experience quality of the application (S130).

[0201] The determining component 140 determines an importance level for a first microservice among the microservices ( S140 ), the determination being based on the quality of service requirement, label, and quality of experience of the first microservice among the microservices.

[0202] The marking component 150 marks a data packet using an importance level determined for a first microservice among the microservices ( S150 ). The data packet originates from or is designated for the first microservice among the microservices and is to be transmitted by the communication system.

[0203] The providing component 160 provides the data packets marked with the importance level to the communication system for transmitting the data packets (S160).

[0204] Figure 8 An apparatus according to an example embodiment is shown. The apparatus may be a communication system (such as a network) or an element thereof (eg a NSMF or NSSMF, or an element of one of these network functions). Figure 9 A method according to an example embodiment is shown. Figure 8 The device can perform Figure 9 method, but not limited to this method. Figure 9 The method can be Figure 8 The present invention is performed by a device, but is not limited to being performed by the device.

[0205] The apparatus includes a receiving component 210, a modifying component 220, and a notifying component 230. The receiving component 210, the modifying component 220, and the notifying component 230 may be a receiving component, a modifying component, and a notifying component, respectively. The receiving component 210, the modifying component 220, and the notifying component 230 may be a receiver, a modifier, and a notifier, respectively. The receiving component 210, the modifying component 220, and the notifying component 230 may be a receiving processor, a modifying processor, and a notification processor, respectively.

[0206] The receiving part 210 receives data packets from the optimizer to transmit the data packets by the communication system (S210).The data packets are marked with importance levels.

[0207] The modifying component 220 modifies the policy state of the communication system based on the importance level of S210 (S220).

[0208] The notification component 230 notifies the optimizer of the policy state of the communication system through the tag after the policy state of the communication system is modified by the modification component 220 (S230).

[0209] Figure 10An apparatus according to an example embodiment is shown. The apparatus comprises at least one processor 810, at least one memory 820 storing instructions that, when executed by the at least one processor 810, cause the apparatus to perform at least one of the methods according to the following figures and associated descriptions: Figure 7 or Figure 9 .

[0210] In some example embodiments, packets from / to all microservices are marked with corresponding importance tags. However, in some example embodiments, only packets from / to a subset of microservices are marked with corresponding importance tags. For packets not marked with corresponding importance tags, the network may apply a default policy. For example, the default policy may be low priority treatment (e.g., high latency, low throughput, etc.). The application provider may decide that some microservices are very low in importance to QoE, so the network should always treat them with low priority. For these microservices, the IMO may not determine any importance tags, and their packets may still have no importance tags.

[0211] Some example embodiments are explained for 6G. However, other example embodiments may be adopted in other 3GPP generations, such as 4G, 5G, 7G, etc., or in other environments where a data block is transmitted as a whole in a corresponding transport block.

[0212] A piece of information can be sent from one entity to another in one or more messages. Each of these messages can include other (different) information.

[0213] The names of network elements, network functions, protocols, and methods are based on current standards or proposals. These names are not limiting. For example, in other versions or other technologies, the names of these network elements and / or network functions and / or protocols and / or methods may be different, as long as they provide the corresponding functionality. This also applies to terminals.

[0214] Unless otherwise stated or clear from the context, a statement that two entities are different indicates that they perform different functions. This does not necessarily mean that they are based on different hardware. That is, each entity described in this specification can be based on different hardware, or some or all entities can be based on the same hardware. This does not necessarily mean that they are based on different software. That is, each entity described in this specification can be based on different software, or some or all entities can be based on the same software. Each entity described in this specification can be deployed in the cloud.

[0215] Based on the above description, it should be clear that the example embodiments provide, for example, an optimizer (such as an importance tag optimizer) or an element thereof (which may or may not be actually integrated into the optimizer), an apparatus embodying the optimizer or an element thereof, a method for controlling and / or operating the optimizer or an element thereof, a computer program for controlling and / or operating the optimizer or an element thereof, and a medium carrying such computer program(s) and forming such computer program product(s). Based on the above description, it should be clear that the example embodiments provide, for example, a communication system (such as a 3GPP system) or an element thereof (which may or may not be actually integrated into the communication system), an apparatus embodying the system or an element thereof, a method for controlling and / or operating the system or an element thereof, a computer program for controlling and / or operating the system or an element thereof, and a medium carrying such computer program(s) and forming such computer program product(s).

[0216] As non-limiting examples, implementations of any of the above blocks, devices, systems, techniques, or methods may include hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof. Each entity described in this specification may be implemented in the cloud.

[0217] It should be understood that the foregoing is what is presently considered to be the preferred exemplary embodiment. However, it should be noted that the description of the preferred exemplary embodiment is given by way of example only and that various modifications may be made without departing from the scope of the disclosure as defined in the appended claims.

[0218] Unless otherwise stated, the terms "first X" and "second X" include the following options: "first X" is the same as "second X", and "first X" is different from "second X". As used herein, "at least one of the following: " and "at least one item in a list of two or more elements>" and similar expressions (wherein a list of two or more elements is connected by "and" or "or") refer to at least any one of these elements, or at least any two or more of these elements, or at least all of these elements. The word "or" refers to a non-exclusive "or" unless otherwise stated (for example, using "otherwise" or "or in the alternative").

Claims

1. A device for managing microservices, comprising: at least one processor; At least one memory storing instructions, which, when executed by the at least one processor, cause the apparatus to at least perform the following: receiving, for each microservice in a plurality of microservices, a corresponding quality of service requirement, wherein the plurality of microservices together constitute an application; receiving a first tag, wherein the first tag indicates a policy state of a communication system that transmits data packets from the microservice for the application; receiving an indication of a first quality of experience of the application; determining an importance level for a first microservice among the microservices based on the quality of service requirement of the first microservice among the microservices, the first label, and the first quality of experience; marking data packets originating from or destined for the first of the microservices and to be transmitted by the communication system with the importance level determined for the first of the microservices; as well as The data packets marked with the importance level are provided to the communication system for transmitting the data packets.

2. The device according to claim 1, wherein: The determining includes determining a corresponding importance level for each of the microservices based on the service quality requirement of the corresponding microservice, the first label, and the first quality of experience, and The marking includes marking, for each of the microservices, data packets originating from or destined for the corresponding microservice and to be transmitted by the communication system with the importance level determined for the corresponding microservice.

3. The device according to any one of claims 1 and 2, wherein: The determining includes: determining an inter-microservice importance level based on the service quality requirement of the first microservice among the microservices, the service quality requirement of the second microservice among the microservices, the first label, and the first quality of experience; and The marking includes marking an inter-microservice data packet using the inter-microservice importance level, wherein the inter-microservice data packet is to be transmitted by the communication system between the first microservice among the microservices and the second microservice among the microservices.

4. The apparatus according to any one of claims 1 to 2, wherein the importance level determined for the first of the microservices is included in a header of the data packet originating from or designated for the first of the microservices and to be transmitted by the communication system.

5. The apparatus according to any one of claims 1 to 2, wherein the data packets originating from or destined for the first of the microservices and to be transmitted by the communication system belong to a traffic flow, and the traffic flow is marked with the importance level.

6. The device according to any one of claims 1 to 2, further comprising: in response to providing the data packet tagged with the importance level to the communication system, receiving from the communication system a second tag indicating a policy state of the communication system after the communication system evaluated the importance level; receiving an indication of a second quality of experience for the application in response to providing the data packets marked with the importance level to the communication system; The determination is adapted by means for determining based on the first label, the second label, the quality of service requirement of each of the microservices, the first quality of experience, and the second quality of experience to optimize the quality of experience of the application.

7. The apparatus of claim 6, wherein the adapting comprises: The determination is adapted by machine learning.

8. The apparatus of claim 7, wherein the machine learning comprises: At least one of reinforcement learning and supervised learning.

9. A method for managing microservices, comprising: receiving, for each microservice in a plurality of microservices, a respective quality of service requirement, wherein the plurality of microservices together constitute an application; receiving a first tag, wherein the first tag indicates a policy state of a communication system that transmits data packets from the microservice for the application; receiving an indication of a first quality of experience of the application; determining an importance level for a first microservice among the microservices based on the quality of service requirement of the first microservice among the microservices, the first label, and the first quality of experience; marking data packets originating from or destined for the first of the microservices and to be transmitted by the communication system with the importance level determined for the first of the microservices; as well as The data packets marked with the importance level are provided to the communication system for transmitting the data packets.

10. The method according to claim 9, wherein: For each of the microservices, a corresponding importance level is determined based on the quality of service requirement of the corresponding microservice, the first label, and the first quality of experience, and For each of the microservices, data packets originating from or destined for the respective microservice and to be transmitted by the communication system are marked with the importance level determined for the respective microservice.

11. The method according to any one of claims 9 and 10, wherein: The inter-microservice importance level is determined based on the service quality requirement of the first microservice among the microservices, the service quality requirement of the second microservice among the microservices, the first label, and the first quality of experience; and An inter-microservice data packet is marked with the inter-microservice importance level, wherein the inter-microservice data packet is to be transmitted by the communication system between the first microservice among the microservices and the second microservice among the microservices.

12. The method according to any one of claims 9 to 10, wherein the importance level determined for the first of the microservices is included in a header of the data packet originating from or designated for the first of the microservices and to be transmitted by the communication system.

13. The method according to any one of claims 9 to 10, wherein the data packets originating from or destined for the first of the microservices and to be transmitted by the communication system belong to a traffic flow, and the traffic flow is marked with the importance level.

14. The method according to any one of claims 9 to 10, further comprising in response to providing the data packet tagged with the importance level to the communication system, receiving from the communication system a second tag indicating a policy state of the communication system after the communication system evaluated the importance level; receiving an indication of a second quality of experience for the application in response to providing the data packets marked with the importance level to the communication system; Adapting determination of the importance level based on the first label, the second label, the quality of service requirement of each of the microservices, the first quality of experience, and the second quality of experience to optimize the quality of experience of the application. The method of claim 14 , wherein the determination of the importance level is adapted by machine learning.

16. A computer-readable medium comprising a set of instructions which, when executed on a device, are configured to cause the device to perform the method according to any one of claims 9 to 10.

Images