Power management aware vertical scaling in cloud native environment

Abstract

Methods and systems are described to enable vertical auto-scaling in Kubernetes (or other container orchestration systems) to interact with power management capabilities (e.g., DVSF) of e.g., nodes in a cluster in order to reduce any unnecessary scaling action (e.g., evicting and re- creating Pods) and achieve a balance of performance and energy consumption.

Description

POWER MANAGEMENT AWARE VERTICAL SCALING IN CLOUD NATIVE ENVIRONMENTTECHNICAL FIELD

[0001] The present disclosure generally relates to power management techniques for cloud environments.BACKGROUND

[0002] In Kubernetes and other cloud native environments (e.g., Amazon Web Service), vertical scaling is a common method to provide the resource elasticity for those cloud native applications. Vertical scaling usually refers to adding more resources like CPU (central processing unit), memory to an existing entity (e.g., server, microservice). It can improve utilization of resources, while minimizing the risk of containers or servers running out of memory or getting CPU starved. Compared to another common scaling method Horizontal scaling, it is less complex as it need not change the current number of the replicates. To provide better performance and resource utilization, the cloud native environment usually will provide auto-scaling vertically, i.e., the size of the allocated resource will be automatically adjusted according to some dynamic performance metrics like CPU load.

[0003] Kubernetes based environments (e.g., Google Kubernetes Engine) provide Vertical Pod Auto-Scaler (VP A) which constantly monitors the consumption of resources by Pods and changes the allocated resources to the actual consumption of resources and can both increase the value of the requests and decrease it, thus automatically adjusting the needs of a Pod.

[0004] The general architecture of a Kubernetes VP A 10 is shown in Figure 1.Further documentation can be found available at: https: / / github.com / kubernetes / design-proposals- archive / blob / main / autoscaling / vertical-pod-autoscaler.md.

[0005] VPA 10 has introduced three main components: recommender 32, admission controller 34, and updater 36. Recommender 32 is responsible for computing recommended resources. On startup the recommender 32 fetches historical resource utilization of all Pods 40 together with the history of Pod events (e.g., OOM - Out of Memory) from the historystorage 15. VPA recommender 32 watches all Pods, keeps calculating fresh recommended resources for them and stores the recommendations in the VPA objects. Additionally, the recommender 32 exposes a synchronous API that takes a Pod description and returns recommended resources. VPA admission controller 34 intercepts Pod creation requests. If the Pod 40 is matched by a VPA configuration with mode not set to “off’, the admission controller 34 rewrites the request by applying recommended resources to the Pod spec. Otherwise it leaves the Pod spec unchanged. If the recommender 32 is not available, it falls back to the recommendation cached in the VPA object. VPA updater 36 is a component responsible for applying recommended resources to existing Pods 40. It monitors all VPA objects 42 and Pods 40 in the cluster, periodically fetching recommendations for the Pods 40 that are controlled by VPA 10 by calling the recommender API. When recommended resources significantly diverge from configured resources, the updater 36 may decide to update a Pod 40. History storage 15 is a component that consumes utilization signals and OOMs (same data as the recommender 32) from the API server 22 and stores it persistently. It is used by the recommender 32 to initialize its state on startup.

[0006] VPA 10 controls the request (memory and CPU) of containers. The request is calculated based on analysis of the current and previous runs of the container and other containers with similar properties (name, image, command, args). The recommendation model (MVP) assumes that the memory and CPU consumption are independent random variables with distribution equal to the one observed in the last N days (recommended value is N=8 to capture weekly peaks). A more advanced model in future could attempt to detect trends, periodicity and other time-related patterns.

[0007] For a CPU the objective is to keep the fraction of time when the container usage exceeds a high percentage (e.g. 95%) of request below a certain threshold (e.g. 1% of time). In this model the "CPU usage" is defined as mean usage measured over a short interval. The shorter the measurement interval, the better the quality of recommendations for spiky, latency sensitive workloads. Minimum reasonable resolution is 1 / min, recommended is 1 / sec.

[0008] Below is one procedure flow to use the VPA 10 in Kubernetes:• The user configures VPA 10.• VPA recommender 32 reads the VPA configuration and the resource utilization metrics from the metric server 24.• VPA recommender 32 provides Pod 40 resource recommendations.• VPA updater 36 reads the Pod 40 resource recommendations.• VPA updater 36 initiates the pod termination.• The deployment realizes the pod 40 was terminated and will recreate the pod 40 to match its replica configuration.• When the pod 40 is in the recreation process, the VPA admission controller 34 gets the pod resource recommendation. Since Kubernetes does not support dynamically changing the resource limits of a running pod 40, VPA 10 cannot update existing pods 40 with new limits. It terminates pods 40 that are using outdated limits. When the pod’s 40 controller requests the replacement from the Kubernetes API service, the VPA admission controller 34 injects the updated resource request and limit values into the new pod’s 40 specification.• Finally, the VPA admission controller 34 overwrites the recommendations to the pod 40.

[0009] VPA 10 can provide the several benefits:• Setting the right resource requests and limits for the workloads improves stability and cost efficiency. If the Pod resource sizes are smaller than given workloads require, an application can either be throttled or it can fail due to out-of- memory errors.• Cluster nodes are used efficiently because pods use exactly what they need.• Reduced maintenance time because the VPA can adjust CPU and memory requests over time without any manual action.

[0010] In Kubernates, the container or Pod usually specifies two kinds of resource requirements, CPU and memory. The CPU resource is measured in CPU units. One CPU could be equivalent to 1 vCore in AWS, GCP, etc. or 1 core in a bare-metal processor. Fractional values are allowed. A Container that requests 0.5 CPU is guaranteed half as much CPU as a Container that requests 1 CPU. It’s possible to use the suffix m to mean milli. For example, 100m CPU, 100 milliCPU, and 0.1 CPU are all the same. The VPA will make new recommendations for these resource requests. For example, can adjust the CPU request from CPU 100m to CPU 200m. But it is to be noted that in Kubernates and VPA, the CPU is only measured in CPU units, it is impossibleto specify the parameters like frequency or CPU model. Therefore, the CPU capability may vary significantly among different nodes in the cluster.

[0011] On the other hand, almost all modern processor architectures (e.g., Intel Xeon and AMD Opteron) can provide a power management technique called Dynamic voltage frequency scaling (DVFS). It can dynamically scale the voltage and frequency (v-f) settings of the CPU to provide “just-enough” speed to process the system workload. Scaling down of voltage and frequency levels can result in a reduction in CPUs dynamic power consumption while scaling up can result in higher performance of the running application.

[0012] Nowadays, operating systems can communicate with different hardware power managers through the open standard ACPI (Advanced Configuration and Power Interface). For example, Intel has implemented the ACPI specification and provides the so-called P-state which is the capability to run the processor at different voltage and / or frequency levels. Generally, P0 is the highest state resulting in maximum performance, while Pl, P2, and so on, will save power but at some penalty to CPU performance. Since Haswell, Intel cores allow independent per-core P-state. Many operating systems have also implemented the ACPI specification to manage these power management capabilities. For example, Intel P-states can be managed by various power governors implemented by Linux kernel. Therefore, the P-States and CPU frequency scaling can be done in response to system load, ACPI events, or be manually changed by user space programs.SUMMARY

[0013] One embodiment under the present disclosure comprises a computer implemented method for scaling resources in a software deployment system comprising one or more pods in one or more nodes in one or more clusters. The method includes receiving, at a VP A, a configuration for the one or more pods; and monitoring, by the VP A, one or more metrics in the one or more pods. The method also includes detecting, by the VP A, that a recommendation is needed for the one or more pods according to the configuration; and communicating with a PM component in the one or more clusters to confirm if the PM component can adjust a performance level for the one or more pods according to the recommendation. The method further includes, if the PM component can satisfy the recommendation, then triggering by the PM component a PM action according to one or more conditions; or if the PM component cannot satisfy therecommendation, then continuing by the VPA to revise the resource request according to the recommendation.

[0014] Another possible embodiment under the present disclosure is a method performed by a PM component for scaling resources in a software deployment system comprising one or more pods in one or more nodes in one or more clusters. The method includes receiving, by the PM component in the one or more nodes from a VPA a request to adjust a performance level according to a recommendation comprising a configuration for the one or more pods; and transmitting to the VPA at least one of a confirmation or a rejection of the request. The method also includes, if the PM component transmits the confirmation, then triggering by the PM component a PM action according to one or more conditions.

[0015] Another embodiment under the present disclosure is a system for scaling resources in a software deployment system comprising one or more pods in one or more nodes in one or more clusters. The system includes processing circuitry; and a memory, the memory containing instructions executable by the processing circuitry. The system is operative to receive, at a VPA, a configuration for the one or more pods; monitor, by the VPA, one or more metrics in the one or more pods; detect, by the VPA, that a recommendation is needed for the one or more pods according to the configuration; and communicate with a PM component in the one or more nodes in the one or more clusters to confirm if the PM component can adjust a performance level for the one or more pods according to the recommendation. The system is further operative to, if the PM component can satisfy the recommendation, then trigger by the PM component a PM action according to one or more conditions; and if the PM component cannot satisfy the recommendation, then continue by the VPA to revise the resource request according to the recommendation.

[0016] Another embodiment under the present disclosure is a system for scaling resources in a software deployment system comprising one or more pods in one or more nodes in one or more clusters. The system comprises a PMA-VPA, a PM Agent, PM Controller, a MAS; and a CSA. The PMA-VPA is configured to: detect a need to adjust a current resource recommendation for a pod; calculate a target performance adjustment for the pod based at least in part on a difference between a new resource recommendation and the current resource recommendation; and formulate a request containing at least one of an identifier of the pod and the target performance adjustment. The PM Agent is deployed on each of one or more nodes and configured to: manipulate one or more PM features of each respective one or more nodes, whereinthe one or more PM features comprise available power management capabilities; report PM feature availability. The PM Controller is configured to; receive the request from the PMA-VPA; receive PM feature availability from the PM Agent; evaluate whether the request can be fulfilled by the one or more PM features; manage the PM Agent deployed in each of the one or more nodes in one or more clusters; and communicate PM feature availability to at least one of the PMA-VPA or a Metric and Analytics Service. The MAS is configured to provide the PMA-VPA with one or more performance metrics on the one or more pods and one or more PM feature preferences for the one or more pods. The CSA is configured to provide the PMA-VPA with one or more application programming interfaces, APIs, to manipulate the one or more clusters.

[0017] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an indication of the scope of the claimed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0018] For a more complete understanding of the present disclosure, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

[0019] Fig. 1 illustrates an example of general architecture of a Kubernetes VPA;

[0020] Fig. 2 illustrates a flow-chart of a possible method embodiment under the present disclosure;

[0021] Fig. 3 illustrates a system embodiment under the present disclosure;

[0022] Fig. 4 illustrates a flow chart of a method embodiment under the present disclosure;

[0023] Fig. 5 illustrates a flow-chart of a method embodiment under the present disclosure;

[0024] Fig. 6 illustrates a flow-chart of a method embodiment under the present disclosure;

[0025] Fig. 7 shows a schematic of a communication system embodiment under the present disclosure;

[0026] Fig. 8 shows a schematic of a user equipment embodiment under the present disclosure;

[0027] Fig. 9 shows a schematic of a network node embodiment under the present disclosure;

[0028] Fig. 10 shows a schematic of a host embodiment under the present disclosure;

[0029] Fig. 11 shows a schematic of a virtualization environment embodiment under the present disclosure; and

[0030] Fig. 12 shows a schematic representation of an embodiment of communication amongst nodes, hosts, and user equipment under the present disclosure.DETAILED DESCRIPTION

[0031] Before describing various embodiments of the present disclosure in detail, it is to be understood that this disclosure is not limited to the parameters of the particularly exemplified systems, methods, apparatus, products, processes, and / or kits, which may, of course, vary. Thus, while certain embodiments of the present disclosure will be described in detail, with reference to specific configurations, parameters, components, elements, etc., the descriptions are illustrative and are not to be construed as limiting the scope of the claimed embodiments. In addition, the terminology used herein is for the purpose of describing the embodiments and is not necessarily intended to limit the scope of the claimed embodiments.

[0032] In the previous section, vertical autoscaling was discussed in regard to, e.g., Kubernetes, including the Vertical Pod Auto-scaler (VP A) which will revise the resource (e.g., CPU, Memory) request of the Pods according to performance metrics like CPU usage. One main disadvantage of VP A is that changing the size of the resource assigned to the Pod(s) will result in stopping the current Pod and starting a new one. Thus, it will introduce the interruption and additional overhead to restart the Pod. Additionally, as we mentioned above, Kubernetes and VPA can only specify the CPU request with CPU unit which doesn’t consider the dynamic compute capability of the CPU processor and the utilization of the power management capabilities (e.g., DVFS based Intel P-state, CPU frequency tuning) provided by the underlying system. For example, one CPU core’s frequency can be set within a range from IGhz to 2Ghz, which meansthe CPU’s compute performance could be doubled by just adjusting the CPU frequency. However, such dynamic power management of the CPU currently is invisible to the VPA.

[0033] IBM has implemented an open-source project “CLEVER - Container Level Energy-efficient VPA Recommender for Kubernetes” (see htt s: / / github ..com / su.stainabje-It provides a customized VPA recommender considering the current CPU frequency and target performance objective (https; / 7g ithub.com sustajnable-conIt will monitor the CPU frequency of the nodes in the cluster. When the frequency is changed, the customized VPA recommender will trigger to re-compute the recommendation of CPU / GPU requests for the Pods managed by the VPA based on updated CPU / GPU frequencies - basically making the CPU / GPU requests adapted to the frequencies of the nodes. However, it doesn’t utilize the Power Management features in the node to dynamically adjust the performance and power usage of the Pods (e.g., setting of the CPU frequency of the cores).

[0034] There are also some previous works that can support proactive autoscaling including vertical scaling by using machine learning techniques to predict future demands to determine the resource requests, for example. See, for example, F. Rossi, M. Nardelli, and V. Cardellini, “Horizontal and Vertical Scaling of Container-Based Applications Using Reinforcement Learning,” in 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), Milan, Italy, Jul. 2019, pp. 329-338. doi: 10.1109 / CLOUD.2019.00061; andK. Rzadca et al., “Autopilot: workload autoscaling at Google,” in Proceedings of the Fifteenth European Conference on Computer Systems, Heraklion Greece, Apr. 2020, pp. 1-16. doi: 10.1145 / 3342195.3387524. They can help to automatically and timely decide to scale up or down according to the forecast. However, they cannot avoid the disruption caused by the vertical scaling in Kubernetes. And they also don’t utilize the power management capabilities provided by the system either to save energy or boost performance.

[0035] Certain aspects of the disclosure and their embodiments may provide solutions to these or other challenges. Certain embodiments include systems and methods to enable the vertical auto-scaling in Kubernetes to interact with the power management capabilities (e.g., DVSF) of the nodes in the cluster in order to reduce the unnecessary scaling action (e.g., evicting and re-creating Pods) and achieve the balance of performance and energy consumption. Some of the embodiments contemplated herein will now be described more fully with reference to theaccompanying drawings. Embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art.

[0036] One possible method embodiment 200 is shown in Figure 2, After the VP A is configured for the given Pod at step 210, the VPA will monitor and analyze the related performance metrics at step 220, for example, CPU usage of the Pod. If VPA has detected at step 230 that a recommendation for the resource request is needed for the Pod according to the predefined recommendation model, it should communicate with the Power Management (PM) component in the cluster to check (step 240) if the PM could adjust the performance and power level for the Pod according to the VPA’s request (step 270). If the feedback is yes, the PM could trigger a Power Management action (e.g., to change the Intel P-state to a higher performance one) according to preference or other conditions, at step 260. For example, when there is need to scale up the Pod (i.e., to increase the resource requested by the Pod), the CPU frequency could be increased thus the performance will be improved. When there is need to scale down the Pod (i.e., to reduce the resource requested by the Pod), the CPU frequency could be decreased, and the power consumption will be reduced. In such case, the VPA shall skip the current planned new recommendation and doesn’t modify the resource request of the Pod and restart it. If the feedback is no, the VPA will continue to revise the resource request according to the recommendation model, at step 250. It should be noted that the embodiment of Figure 2 can also be applied to other types of scaling, for example, horizontal auto-scaling in Kubernetes. Certain references in the current disclosure are made in reference to Kubernetes. But the current disclosure can be implemented in a variety of scenarios, such as, Amazon Web Services Fargate; Azure Container Instances; Google Cloud Run; Google Kubernetes Engine; Amazon Elastic Kubernetes Service; Openshift Container Platform; Rancher; Docker Swarm; or Nomad.

[0037] Certain embodiments may provide one or more of the following technical advantages. Main advantages include:• Utilizing the power management benefits provided by the nodes in the cluster, so that either to change the CPU performance or energy efficiency level accordingly when making vertical scaling recommendations; and• Avoiding unnecessary VPA actions, thus the service disruption and additional overhead introduced by the VPA is minimized.

[0038] One possible system embodiment 400 is illustrated in Figure 3. Certain components include: the Power Management Aware Vertical Pod Auto-scaler (PMA-VPA) 630, the Power Management Controller (PM Controller) 650, the Power Management Agent (PM Agent) 682, 664, the Power Management Features (PM Features) 666, 684, the Metric and Analytics Server (MAS) 620, and Cluster API Server (CAS) 660 for e.g., cluster 665 with nodes 662, 680 and pods 668, 669, 686, 688.

[0039] The PMA-VPA 630 can have similar functions to the normal VPA but the PMA-VPA 630 could interact with the PM Controller 650 when the PMA-VPA 630 is going to make a potential change of request recommendation. When the PMA-VPA 630 sees the need to adjust the recommendation for a Pod 668, 669, 688, 686, it formulates a request to the PM Controller 650, which contains information like the Pod 668, 669, 688, 686 name, the target performance adjustment on the Pod 668, 669, 688, 686, etc. The target performance adjustment is calculated by the PMA-VPA 630 according to the difference between the new potential resource (CPU) recommendation and the current resource recommendation. For example, if the current CPU request recommendation is 500m and the new recommendation is 800m, the target performance adjustment could be set to +60%. There could be other ways to calculate the target performance adjustment and the adjustment could be either to increase the performance or decrease the performance.

[0040] The PM Controller 650 is responsible for managing the PM Agent 664, 682 deployed in each node 662, 680 in the cluster, discovering the available PM features 666, 684, exposing the PM features 666, 684, and communicate with VPA or another component like MAS 620 if needed. The PM Controller 650 should also be able to evaluate the request coming from the PMA-VPA 630 to determine whether the request (i.e., the target performance adjustment) could be fulfilled by using the PM feature 666, 684.

[0041] The PM Agent 664, 682 is preferably deployed in each node whose system has support for the power management features or capabilities, e.g., the CPU frequency tuning or P-state. According to the instruction from the PM controller 650, it calls the mechanism(s) available in the underlying system (e.g., the command line tool provided by the operating system, the library provided by the vendor like Intel) to control or manage the PM features 666, 684 for the Pods 668, 669, 688, 686. It also preferably reports the available PM features 666, 684 to the PM controller 650. PM components can comprise for example, any computing device orcomponents or logical division thereof, including UEs, network nodes, servers, databases, Internet of Things components, logical partition or component of e.g. a server, or a variety of power consuming elements or devices.

[0042] The PM Features 666, 684 are the power management capabilities available in the node 662, 680. For example, the CPU frequency tuning or P-state. Such capabilities can be called by PM Agent 664, 682. For example, in Linux, there is one kernel subsystem called CPUFreq which provides the kernel infrastructure and user spaces for all platforms supporting CPU performance scaling.

[0043] MAS 620 is the component to provide the collected metrics like performance and analytics results (e.g., predicted performance data) of the Pods 668, 669, 688, 686. The MAS 620 could also give suggestions on whether utilizing PM feature 666, 684 is preferred for the given Pod 668, 669, 688, 686. CSA 660 is the component to provide the APIs to access the resources or objects of the cluster’s cloud platform, e.g., Kubernetes. The PM Controller 650 or other components could interact with the CSA 660 to query the information of the Pod 668, 669, 688, 686 (e.g., the node 662, 680 that the Pod 668, 669, 688, 686 is running), or other information.

[0044] One process flow of a method embodiment 800 under the present disclosure is shown in Figure 4. Step 801 is to start the PM Controller 880 in one node in the cluster, e.g., the master node. The PM Controller 880 should also provide the interface which other components like the VPA 875 can access. At step 802, PM Agent 890 is deployed in each node in the cluster and shall register with the PM controller 880. At 803, PM Agent 890 shall discover the PM features 895 in the current node which are available for it to operate in the current node, e.g., P-state or CPU frequency tuning. At step 804, PM Agents 890 shall report the supported PM features 895 and other information in the node to the PM controller 880. The report can include information such as (but not limited to), e.g., the Agent ID, the Node ID, CPU information, PM feature description, etc. PM Controller 880 shall store all such information reported from PM agents 890 either locally or remotely for later use.

[0045] Here is one example of the report from the PM agent 890:{"nodelD": "nodel","agentID": "nodel_agent","cpuVendor": "Intel",cpuModel": "Sandy Bridge", PMFeatures": ["featureType" : "frequency Tuning"},{"featureType": "SST-CF"

[0046] At step 805, VPA 875 shall be configured for the given Pod 897, which is similar to the normal VPA. Additionally, here the VPA 875 can set up an option to indicate whether VPA 875 shall interact with PM controller 880 for the given Pod 897 when it is making a new VPA recommendation. Because sometimes the application may prefer to scale Pod 897 than utilizing the PM feature 895 to adjust the performance. The option can be set manually when the application is deployed or is automatically set by MAS 870 according to the analytics results.

[0047] At step 806, MAS 870 shall collect the related performance metrics (e.g., resource usage) of the Pods 897 from the cluster. At step 807 MAS 870 shall process the collected metrics, analyze, and store them into storage. At step 808, VPA 875 periodically queries MAS 870 to get the latest metrics data and analytics results. At 809, VPA 875 checks the performance data and analytics results and determines whether a new request recommendation is needed for the given Pod 897. At 810, if VPA 875 determines that the performance of the Pod 897 has changed and needs either a new request recommendation or to utilize PM features 895 available for Pod 897, it then checks the option to see if utilizing PM features 895 and interacting with PM controller 880 is wanted. If yes, it shall send a request to PM controller 880 to check if PM controller 880 can satisfy the request. In the request, the VPA 875 could specify the ID of the Pod 897, and the expected performance change. Below is one example of the request:{"PMRquest": {"PodID": "podl","PerformanceDifP: "20%", "Performancechange": "up"

[0048] At step 811, upon the request being received, PM controller 880 communicates with Cluster API server 885 to get related Pod 897 information and identify thenode where Pod 897 is running. At 812, PM Controller 880 can inspect the information reported from the PM agent 890 running in the identified node and determine if any PM feature 895 exists in that node (i.e., Pod 897). From here the process might proceed to step 813 or step 816.

[0049] At 813, if there is not any PM Feature 895 available for the specified Pod 897, the PM controller 880 sends back a reply to the VPA 875 to indicate that. At step 814, VPA 875 shall make a new VPA request recommendation according to its model. At 815, VPA 875 restarts the specified Pod 897 with the new request recommendation.

[0050] At 816, if PM controller 880 determines that there is potential PM feature 895 that can be utilized to meet the VPA request for the Pod 897, it shall send a corresponding request (similar to the request sent from the VPA 875) to the PM Agent 890 in the node where the Pod 897 is running. At step 817, PM Agent 890 shall communicate with the PM Features 895 and the system to get the related information of the Pod 897, e.g., the current assigned core, the current frequency of the CPU core, the min and max CPU frequency. At step 818, PM Agent 890 will use this information to determine whether the expected performance tuning of the Pod 897 could be fulfilled. The process may proceed to step 819 or step 820.

[0051] At step 819, if PM Agent 890 decides that the tuning cannot be met, it shall send a response to the PM controller 880 to indicate the tuning is unsuccessful.

[0052] At step 820, if PM agent 890 decides that the tuning could be fulfilled, it shall communicate with the PM Feature 895 and request a performance tuning via the specified PM feature 895, for example, “tune the CPU frequency to 2GHz for podl.” At 821, PM Feature 895 shall execute the tuning according to the request from the PM Agent 890. At 822, PM Feature 895 sends a reply to the PM Agent 890 to indicate the tuning result (either “successful” or “unsuccessful”). At step 823, PM Agent 890 shall forward the result to the PM Controller 880. At step 824, PM Controller 880 shall formulate a response according to the response received at step 819 and 823 and send it to the VPA 875. The process may then proceed to step 825 or 826.

[0053] At step 825, if PM Controller 880 indicates that the PM tuning is successful, the VPA 875 shall not make any new recommendation.

[0054] At step 826, if PM Controller 880 indicates that the PM tuning is unsuccessful, the VPA 875 shall create a new scaling recommendation. At step 827, VPA 875 shall restart Pod 897 and apply the new resource recommendation to Pod 897.

[0055] At step 828, MAS 870 shall continuously monitor the performance metrics of the Pod 897, and VPA 875 shall check the data from the MAS 870 and repeat from step 808.Additional Embodiments

[0056] Another possible method embodiment under the present disclosure is shown in Figure 5. Method 1200 comprises a computer implemented method for scaling resources in a software deployment system comprising one or more pods in one or more nodes in one or more clusters. Step 1210 comprises receiving, at a VPA, a configuration for the one or more pods. Step 1220 is monitoring, by the VPA, one or more metrics in the one or more pods. Step 1230 is detecting, by the VPA, that a recommendation is needed for the one or more pods according to the configuration. Step 1240 comprises communicating with a PM component in the one or more clusters to confirm if the PM component can adjust a performance level for the one or more pods according to the recommendation. Step 1250 is, if the PM component can satisfy the recommendation, then triggering by the PM component a PM action according to one or more conditions. Alternatively, at 1260, if the PM component cannot satisfy the recommendation, then continuing by the VPA to revise the resource request according to the recommendation. Method 1200 can comprise multiple alternative embodiments with additional or alternative steps.

[0057] Another possible method embodiment under the present disclosure is shown in Figure 6. Method 1400 comprises a method performed by a PM component for scaling resources in a software deployment system comprising one or more pods in one or more nodes in one or more clusters. Step 1410 is receiving, by the PM component in the one or more nodes from a VPA a request to adjust a performance level according to a recommendation comprising a configuration for the one or more pods. Step 1420 is transmitting to the VPA at least one of a confirmation or a rejection of the request. Step 1430 is, if the PM component transmits the confirmation, then triggering by the PM component a PM action according to one or more conditions. Method 1400 can comprise multiple alternative embodiments with additional or alternative steps.

[0058] Figure 7 shows an example of a communication system 2100 in accordance with some embodiments. In the example, the communication system 2100 includes a telecommunication network 2102 that includes an access network 2104, such as a RAN, and a core network 2106, which includes one or more core network nodes 2108. The access network 2104 includes one or more access network nodes, such as network nodes 2110a and 2110b (one or moreof which may be generally referred to as network nodes 2110), or any other similar 3rd Generation Partnership Project (3GPP) access node or non-3GPP access point. The network nodes 2110 facilitate direct or indirect connection of UE, such as by connecting UEs 2112a, 2112b, 2112c, and 2112d (one or more of which may be generally referred to as UEs 2112) to the core network 2106 over one or more wireless connections.

[0059] Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 1100 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired or wireless connections. The communication system 2100 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0060] The UEs 2112 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with the network nodes 2110 and other communication devices. Similarly, the network nodes 2110 are arranged, capable, configured, and / or operable to communicate directly or indirectly with the UEs 2112 and / or with other network nodes or equipment in the telecommunication network 2102 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in the telecommunication network 2102.

[0061] In the depicted example, the core network 2106 connects the network nodes 2110 to one or more hosts, such as host 2116. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 2106 includes one more core network nodes (e.g., core network node 2108) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 2108. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), SessionManagement Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0062] The host 2116 may be under the ownership or control of a service provider other than an operator or provider of the access network 2104 and / or the telecommunication network 2102, and may be operated by the service provider or on behalf of the service provider. The host 2116 may host a variety of applications to provide one or more service. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0063] As a whole, the communication system 2100 of Figure 7 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z- Wave, Near Field Communication (NFC) ZigBee, LiFi, and / or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0064] In some examples, the telecommunication network 2102 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 2102 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 2102. For example, the telecommunications network 2102 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC)ZMassive loT services to yet further UEs.

[0065] In some examples, the UEs 2112 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 2104 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 2104. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).

[0066] In the example, the hub 2114 communicates with the access network 2104 to facilitate indirect communication between one or more UEs (e.g., UE 2112c and / or 2112d) and network nodes (e.g., network node 2110b). In some examples, the hub 2114 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 2114 may be a broadband router enabling access to the core network 2106 for the UEs. As another example, the hub 2114 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 2110, or by executable code, script, process, or other instructions in the hub 2114. As another example, the hub 2114 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 2114 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 2114 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 2114 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, the hub 2114 acts as a proxy server or orchestrator for the UEs, in particular in if one or more of the UEs are low energy loT devices.

[0067] The hub 2114 may have a constant / persistent or intermittent connection to the network node 2110b. The hub 2114 may also allow for a different communication scheme and / or schedule between the hub 2114 and UEs (e.g., UE 2112c and / or 2112d), and between the hub 2114 and the core network 2106. In other examples, the hub 2114 is connected to the core network 2106 and / or one or more UEs via a wired connection. Moreover, the hub 2114 may be configured to connect to an M2M service provider over the access network 1104 and / or to anotherUE over a direct connection. In some scenarios, UEs may establish a wireless connection with the network nodes 2110 while still connected via the hub 2114 via a wired or wireless connection. In some embodiments, the hub 2114 may be a dedicated hub - that is, a hub whose primary function is to route communications to / from the UEs from / to the network node 2110b. In other embodiments, the hub 2114 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 2110b, but which is additionally capable of operating as a communication start and / or end point for certain data channels.

[0068] Figure 8 shows a UE 2200 in accordance with some embodiments. As used herein, a UE refers to a device capable, configured, arranged and / or operable to communicate wirelessly with network nodes and / or other UEs. Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle-mounted or vehicle embedded / integrated wireless device, etc. Other examples include any UE identified by the 3rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB-IoT) UE, a machine type communication (MTC) UE, and / or an enhanced MTC (eMTC) UE.

[0069] A UE may support device-to-device (D2D) communication, for example by implementing a 3 GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to- everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and / or operates the relevant device. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent a device that is not intended for sale to, or operation by, an end user but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).

[0070] The UE 2200 includes processing circuitry 2202 that is operatively coupled via a bus 2204 to an input / output interface 2206, a power source 2208, a memory 2210, a communication interface 2212, and / or any other component, or any combination thereof. CertainUEs may utilize all or a subset of the components shown in Figure 10. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0071] The processing circuitry 2202 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine- readable computer programs in the memory 2210. The processing circuitry 2202 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 2202 may include multiple central processing units (CPUs).

[0072] In the example, the input / output interface 2206 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and / or output devices. Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof. An input device may allow a user to capture information into the UE 2200. Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like. The presencesensitive display may include a capacitive or resistive touch sensor to sense input from a user. A sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.

[0073] In some embodiments, the power source 2208 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used. The power source 2208 may further include power circuitry for delivering power from the power source 2208 itself, and / or an external power source, to the various parts of the UE 2200 via input circuitry or an interface such as anelectrical power cable. Delivering power may be, for example, for charging of the power source 2208. Power circuitry may perform any formatting, converting, or other modification to the power from the power source 2208 to make the power suitable for the respective components of the UE 2200 to which power is supplied.

[0074] The memory 2210 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 2210 includes one or more application programs 2214, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 2216. The memory 2210 may store, for use by the UE 2200, any of a variety of various operating systems or combinations of operating systems.

[0075] The memory 2210 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD- DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUICC), integrated UICC (iUICC) or a removable UICC commonly known as ‘SIM card.’ The memory 2210 may allow the UE 2200 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 2210, which may be or comprise a device-readable storage medium.

[0076] The processing circuitry 2202 may be configured to communicate with an access network or other network using the communication interface 2212. The communication interface 2212 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 2222. The communication interface 2212 may include oneor more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a network node in an access network). Each transceiver may include a transmitter 2218 and / or a receiver 2220 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, the transmitter 2218 and receiver 2220 may be coupled to one or more antennas (e.g., antenna 2222) and may share circuit components, software or firmware, or alternatively be implemented separately.

[0077] In the illustrated embodiment, communication functions of the communication interface 2212 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, near-field communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may be implemented in according to one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol / internet protocol (TCP / IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.

[0078] Regardless of the type of sensor, a UE may provide an output of data captured by its sensors, through its communication interface 2212, via a wireless connection to a network node. Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE. The output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).

[0079] As another example, a UE comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection. In response to the received wireless input the states of the actuator, the motor, or the switch may change. For example, the UE may comprise a motor that adjusts thecontrol surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input.

[0080] A UE, when in the form of an Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare. Non-limiting examples of such an loT device are a device which is or which is embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door / window sensor, a flood / moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal- or item-tracking device, a sensor for monitoring a plant or animal, an industrial robot, an Unmanned Aerial Vehicle (UAV), and any kind of medical device, like a heart rate monitor or a remote controlled surgical robot. A UE in the form of an loT device comprises circuitry and / or software in dependence of the intended application of the loT device in addition to other components as described in relation to the UE 2200 shown in Figure 10.

[0081] As yet another specific example, in an loT scenario, a UE may represent a machine or other device that performs monitoring and / or measurements, and transmits the results of such monitoring and / or measurements to another UE and / or a network node. The UE may in this case be an M2M device, which may in a 3 GPP context be referred to as an MTC device. As one particular example, the UE may implement the 3 GPP NB-IoT standard. In other scenarios, a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and / or reporting on its operational status or other functions associated with its operation.

[0082] In practice, any number of UEs may be used together with respect to a single use case. For example, a first UE might be or be integrated in a drone and provide the drone’s speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone. When the user makes changes from the remote controller, the first UE may adjust the throttle on the drone (e.g. by controlling an actuator) to increase or decrease the drone’s 1speed. The first and / or the second UE can also include more than one of the functionalities described above. For example, a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.

[0083] Figure 9 shows a network node 3300 in accordance with some embodiments. As used herein, network node refers to equipment capable, configured, arranged and / or operable to communicate directly or indirectly with a UE and / or with other network nodes or equipment, in a telecommunication network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)).

[0084] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units and / or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0085] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSRBSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi-cell / multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and / or Minimization of Drive Tests (MDTs).

[0086] The network node 3300 includes a processing circuitry 3302, a memory 3304, a communication interface 3306, and a power source 3308. The network node 3300 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which the network node 3300 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate componentsmay be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, the network node 1300 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 3304 for different RATs) and some components may be reused (e.g., a same antenna 3310 may be shared by different RATs). The network node 3300 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 3300, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 1300.

[0087] The processing circuitry 3302 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other network node 3300 components, such as the memory 3304, to provide network node 3300 functionality.

[0088] In some embodiments, the processing circuitry 3302 includes a system on a chip (SOC). In some embodiments, the processing circuitry 3302 includes one or more of radio frequency (RF) transceiver circuitry 3312 and baseband processing circuitry 3314. In some embodiments, the radio frequency (RF) transceiver circuitry 3312 and the baseband processing circuitry 3314 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 3312 and baseband processing circuitry 3314 may be on the same chip or set of chips, boards, or units.

[0089] The memory 3304 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), readonly memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory devices that store information, data, and / or instructions that may be used by the processingcircuitry 3302. The memory 3304 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and / or other instructions capable of being executed by the processing circuitry 3302 and utilized by the network node 3300. The memory 3304 may be used to store any calculations made by the processing circuitry 3302 and / or any data received via the communication interface 3306. In some embodiments, the processing circuitry 3302 and memory 3304 is integrated.

[0090] The communication interface 3306 is used in wired or wireless communication of signaling and / or data between a network node, access network, and / or UE. As illustrated, the communication interface 3306 comprises port(s) / terminal(s) 3316 to send and receive data, for example to and from a network over a wired connection. The communication interface 3306 also includes radio front-end circuitry 3318 that may be coupled to, or in certain embodiments a part of, the antenna 3310. Radio front-end circuitry 3318 comprises filters 3320 and amplifiers 3322. The radio front-end circuitry 3318 may be connected to an antenna 3310 and processing circuitry 3302. The radio front-end circuitry may be configured to condition signals communicated between antenna 3310 and processing circuitry 3302. The radio front-end circuitry 3318 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection. The radio front-end circuitry 3318 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 3320 and / or amplifiers 3322. The radio signal may then be transmitted via the antenna 3310. Similarly, when receiving data, the antenna 3310 may collect radio signals which are then converted into digital data by the radio front-end circuitry 3318. The digital data may be passed to the processing circuitry 3302. In other embodiments, the communication interface may comprise different components and / or different combinations of components.

[0091] In certain alternative embodiments, the network node 3300 does not include separate radio front-end circuitry 3318, instead, the processing circuitry 3302 includes radio frontend circuitry and is connected to the antenna 3310. Similarly, in some embodiments, all or some of the RF transceiver circuitry 3312 is part of the communication interface 3306. In still other embodiments, the communication interface 3306 includes one or more ports or terminals 3316, the radio front-end circuitry 3318, and the RF transceiver circuitry 3312, as part of a radio unit (not shown), and the communication interface 3306 communicates with the baseband processing circuitry 3314, which is part of a digital unit (not shown).

[0092] The antenna 3310 may include one or more antennas, or antenna arrays, configured to send and / or receive wireless signals. The antenna 3310 may be coupled to the radio front-end circuitry 3318 and may be any type of antenna capable of transmitting and receiving data and / or signals wirelessly. In certain embodiments, the antenna 3310 is separate from the network node 3300 and connectable to the network node 3300 through an interface or port.

[0093] The antenna 3310, communication interface 3306, and / or the processing circuitry 3302 may be configured to perform any receiving operations and / or certain obtaining operations described herein as being performed by the network node. Any information, data and / or signals may be received from a UE, another network node and / or any other network equipment. Similarly, the antenna 3310, the communication interface 3306, and / or the processing circuitry 3302 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and / or signals may be transmitted to a UE, another network node and / or any other network equipment.

[0094] The power source 3308 provides power to the various components of network node 3300 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). The power source 3308 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 3300 with power for performing the functionality described herein. For example, the network node 3300 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 3308. As a further example, the power source 3308 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.

[0095] Embodiments of the network node 3300 may include additional components beyond those shown in Figure 9 for providing certain aspects of the network node’s functionality, including any of the functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, the network node 3300 may include user interface equipment to allow input of information into the network node 3300 and to allow output of information from the network node 3300. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 3300.

[0096] Figure 10 is a block diagram of a host 4400, which may be an embodiment of the host 2116 of Figure 7, in accordance with various aspects described herein. As used herein, the host 4400 may be or comprise various combinations hardware and / or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm. The host 4400 may provide one or more services to one or more UEs.

[0097] The host 4400 includes processing circuitry 4402 that is operatively coupled via a bus 4404 to an input / output interface 4406, a network interface 4408, a power source 4410, and a memory 4412. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as Figures 8 and 9, such that the descriptions thereof are generally applicable to the corresponding components of host 4400.

[0098] The memory 4412 may include one or more computer programs including one or more host application programs 4414 and data 4416, which may include user data, e.g., data generated by a UE for the host 4400 or data generated by the host 4400 for a UE. Embodiments of the host 4400 may utilize only a subset or all of the components shown. The host application programs 4414 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (WC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application programs 4414 may also provide for user authentication and licensing checks and may periodically report health, routes, and content availability to a central node, such as a device in or on the edge of a core network. Accordingly, the host 4400 may select and / or indicate a different host for over-the-top services for a UE. The host application programs 4414 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.

[0099] Figure 11 is a block diagram illustrating a virtualization environment 5500 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may includevirtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 5500 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, UE, core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized. Container 5575 (such as a Kubernetes container, or other container, e.g., Amazon Web Services Fargate; Azure Container Instances; Google Cloud Run; Google Kubernetes Engine; Amazon Elastic Kubernetes Service; Openshift Container Platform; Rancher; Docker Swarm; Nomad) can be coupled to virtualization environment 5500. Container 5575 can be a cloud-based container system. In certain environments, container 5575 can comprise or manage virtualization environment 5500.[000100] Applications 5502 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment 5500 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.[000101] Hardware 5504 includes processing circuitry, memory that stores software and / or instructions executable by hardware processing circuitry, and / or other hardware devices as described herein, such as a network interface, input / output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 5506 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 5508a and 5508b (one or more of which may be generally referred to as VMs 5508), and / or perform any of the functions, features and / or benefits described in relation with some embodiments described herein. The virtualization layer 5506 may present a virtual operating platform that appears like networking hardware to the VMs 5508.[000102] The VMs 5508 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 5506. Different embodiments of the instance of a virtual appliance 5502 may be implemented on one or more of VMs 5508, and the implementations may be made in different ways. Virtualizationof the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.[000103] In the context of NFV, a VM 5508 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine. Each of the VMs 5508, and that part of hardware 5504 that executes that VM, be it hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 5508 on top of the hardware 5504 and corresponds to the application 5502.[000104] Hardware 5504 may be implemented in a standalone network node with generic or specific components. Hardware 5504 may implement some functions via virtualization. Alternatively, hardware 5504 may be part of a larger cluster of hardware (e.g., such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 5510, which, among others, oversees lifecycle management of applications 5502. In some embodiments, hardware 5504 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 5512 which may alternatively be used for communication between hardware nodes and radio units.[000105] Figure 12 shows a communication diagram of a host 6602 communicating via a network node 6604 with a UE 6606 over a partially wireless connection in accordance with some embodiments. Example implementations, in accordance with various embodiments, of the UE (such as a UE 2112a of Figure 7 and / or UE 2200 of Figure 8), network node (such as network node 2110a of Figure 7 and / or network node 3300 of Figure 9), and host (such as host 2116 of Figure 7 and / or host 4400 of Figure 10) discussed in the preceding paragraphs will now be described with reference to Figure 12.[000106] Like host 4400, embodiments of host 6602 include hardware, such as a communication interface, processing circuitry, and memory. The host 6602 also includes software, which is stored in or accessible by the host 6602 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as the UE 6606 connecting via an over-the-top (OTT) connection 6650 extending between the UE 6606 and host 6602. In providing the service to the remote user, a host application may provide user data which is transmitted using the OTT connection 6650.[000107] The network node 6604 includes hardware enabling it to communicate with the host 6602 and UE 6606. The connection 6660 may be direct or pass through a core network (like core network 2106 of Figure 7) and / or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet.[000108] The UE 6606 includes hardware and software, which is stored in or accessible by UE 6606 and executable by the UE’s processing circuitry. The software includes a client application, such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 6606 with the support of the host 6602. In the host 6602, an executing host application may communicate with the executing client application via the OTT connection 6650 terminating at the UE 6606 and host 6602. In providing the service to the user, the UE's client application may receive request data from the host's host application and provide user data in response to the request data. The OTT connection 6650 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through the OTT connection 6650.[000109] The OTT connection 6650 may extend via a connection 6660 between the host 6602 and the network node 6604 and via a wireless connection 6670 between the network node 6604 and the UE 6606 to provide the connection between the host 6602 and the UE 6606. The connection 6660 and wireless connection 6670, over which the OTT connection 6650 may be provided, have been drawn abstractly to illustrate the communication between the host 6602 and the UE 1606 via the network node 6604, without explicit reference to any intermediary devices and the precise routing of messages via these devices.[000110] As an example of transmitting data via the OTT connection 6650, in step 6608, the host 6602 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interacting with the UE 6606. In other embodiments, the user data is associated with a UE 6606 that shares data with the host 6602 without explicit human interaction. In step 6610, the host 6602 initiates a transmission carrying the user data towards the UE 6606. The host 6602 may initiate the transmission responsive to a request transmitted by the UE 6606. The request may be caused by human interaction with the UE 6606 or by operation of the client application executing on the UE 6606. The transmission may pass via the network node 6604, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 6612, the network node 6604 transmits to the UE 6606 the user data that was carried in the transmission that the host 6602 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 6614, the UE 6606 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 6606 associated with the host application executed by the host 6602.[000111] In some examples, the UE 6606 executes a client application which provides user data to the host 6602. The user data may be provided in reaction or response to the data received from the host 6602. Accordingly, in step 6616, the UE 6606 may provide user data, which may be performed by executing the client application. In providing the user data, the client application may further consider user input received from the user via an input / output interface of the UE 6606. Regardless of the specific manner in which the user data was provided, the UE 6606 initiates, in step 6618, transmission of the user data towards the host 6602 via the network node 6604. In step 6620, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 6604 receives user data from the UE 6606 and initiates transmission of the received user data towards the host 6602. In step 6622, the host 6602 receives the user data carried in the transmission initiated by the UE 6606.[000112] One or more of the various embodiments improve the performance of OTT services provided to the UE 6606 using the OTT connection 6650, in which the wireless connection 6670 forms the last segment. More precisely, the teachings of these embodiments may improve the data rate, latency, and / or power consumption and thereby provide benefits such as reduced userwaiting time, relaxed restriction on file size, improved content resolution, better responsiveness, and / or extended battery lifetime.[000113] In an example scenario, factory status information may be collected and analyzed by the host 6602. As another example, the host 6602 may process audio and video data which may have been retrieved from a UE for use in creating maps. As another example, the host 6602 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 6602 may store surveillance video uploaded by a UE. As another example, the host 6602 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, the host 6602 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and / or transmitting data.[000114] In some examples, a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 6650 between the host 6602 and UE 6606, in response to variations in the measurement results. The measurement procedure and / or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of the host 6602 and / or UE 6606. In some embodiments, sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 6650 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 6650 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of the network node 6604. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by the host 6602. The measurements may be implemented in that software causes messages to be transmitted, in particular empty or ‘dummy’ messages, using the OTT connection 6650 while monitoring propagation times, errors, etc.[000115] Although the computing devices described herein (e.g., UEs, network nodes, hosts) may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to be understood that these computing devices may comprise any suitable combination of hardware and / or software needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and / or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and / or the functionality of the components may be partitioned between the processing circuitry and the communication interface. In another example, non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.[000116] In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer- readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and / or by end users and a wireless network generally.[000117] It will be appreciated that computer systems are increasingly taking a wide variety of forms. In this description and in the claims, the terms “controller,” “computer system,” or “computing system” are defined broadly as including any device or system — or combination thereof — that includes at least one physical and tangible processor and a physical and tangible memory capable of having thereon computer-executable instructions that may be executed by a processor. By way of example, not limitation, the term “computer system” or “computing system,” as used herein is intended to include personal computers, desktop computers, laptop computers, tablets, hand-held devices (e.g., mobile telephones, PDAs, pagers), microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, multi-processor systems, network PCs, distributed computing systems, datacenters, message processors, routers, switches, and even devices that conventionally have not been considered a computing system, such as wearables (e.g., glasses).[000118] The computing system also has thereon multiple structures often referred to as an “executable component.” For instance, the memory of a computing system can include an executable component. The term “executable component” is the name for a structure that is well understood to one of ordinary skill in the art in the field of computing as being a structure that can be software, hardware, or a combination thereof. For instance, when implemented in software, one of ordinary skill in the art would understand that the structure of an executable component may include software objects, routines, methods, and so forth, that may be executed by one or more processors on the computing system, whether such an executable component exists in the heap of a computing system, or whether the executable component exists on computer-readable storage media. The structure of the executable component exists on a computer-readable medium in such a form that it is operable, when executed by one or more processors of the computing system, to cause the computing system to perform one or more functions, such as the functions and methods described herein. Such a structure may be computer-readable directly by a processor — as is the case if the executable component were binary. Alternatively, the structure may be structured to be interpretable and / or compiled — whether in a single stage or in multiple stages — so as to generate such binary that is directly interpretable by a processor.[000119] The terms “component,” “service,” “engine,” “module,” “control,” “generator,” or the like may also be used in this description. As used in this description and in this case, these terms — whether expressed with or without a modifying clause — are also intended tobe synonymous with the term “executable component” and thus also have a structure that is well understood by those of ordinary skill in the art of computing.[000120] In terms of computer implementation, a computer is generally understood to comprise one or more processors or one or more controllers, and the terms computer, processor, and controller may be employed interchangeably. When provided by a computer, processor, or controller, the functions may be provided by a single dedicated computer or processor or controller, by a single shared computer or processor or controller, or by a plurality of individual computers or processors or controllers, some of which may be shared or distributed. Moreover, the term “processor” or “controller” also refers to other hardware capable of performing such functions and / or executing software, such as the example hardware recited above.[000121] In general, the various exemplary embodiments may be implemented in hardware or special purpose chips, circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor, or other computing device, although the disclosure is not limited thereto. While various aspects of the exemplary embodiments of this disclosure may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques, or methods described herein may be implemented in, as nonlimiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.[000122] While not all computing systems require a user interface, in some embodiments a computing system includes a user interface for use in communicating information from / to a user. The user interface may include output mechanisms as well as input mechanisms. The principles described herein are not limited to the precise output mechanisms or input mechanisms as such will depend on the nature of the device. However, output mechanisms might include, for instance, speakers, displays, tactile output, projections, holograms, and so forth. Examples of input mechanisms might include, for instance, microphones, touchscreens, projections, holograms, cameras, keyboards, stylus, mouse, or other pointer input, sensors of any type, and so forth.Conclusion[000123] The present disclosure includes any novel feature or combination of features disclosed herein either explicitly or any generalization thereof. Various modifications and adaptations to the foregoing exemplary embodiments of this disclosure may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings. However, any and all modifications will still fall within the scope of the non-limiting and exemplary embodiments of this disclosure.[000124] It is understood that for any given component or embodiment described herein, any of the possible candidates or alternatives listed for that component may generally be used individually or in combination with one another, unless implicitly or explicitly understood or stated otherwise. Additionally, it will be understood that any list of such candidates or alternatives is merely illustrative, not limiting, unless implicitly or explicitly understood or stated otherwise.[000125] In addition, unless otherwise indicated, numbers expressing quantities, constituents, distances, or other measurements used in the specification and claims are to be understood as being modified by the term “about,” as that term is defined herein. Accordingly, unless indicated to the contrary, the numerical parameters set forth in the specification and attached claims are approximations that may vary depending upon the desired properties sought to be obtained by the subject matter presented herein. At the very least, and not as an attempt to limit the application of the doctrine of equivalents to the scope of the claims, each numerical parameter should at least be construed in light of the number of reported significant digits and by applying ordinary rounding techniques. Notwithstanding that the numerical ranges and parameters setting forth the broad scope of the subject matter presented herein are approximations, the numerical values set forth in the specific examples are reported as precisely as possible. Any numerical values, however, inherently contain certain errors necessarily resulting from the standard deviation found in their respective testing measurements.[000126] Any headings and subheadings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. The terms and expressions which have been employed herein are used as terms of description and not of limitation, and there is no intention in the use of such terms and expressions of excluding any equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the present disclosure. Thus, it should be understood that although the present disclosure has been specifically disclosed in part by certain embodiments,and optional features, modification and variation of the concepts herein disclosed may be resorted to by those skilled in the art, and such modifications and variations are considered to be within the scope of this present description.[000127] It will also be appreciated that systems, devices, products, kits, methods, and / or processes, according to certain embodiments of the present disclosure may include, incorporate, or otherwise comprise properties or features (e.g., components, members, elements, parts, and / or portions) described in other embodiments disclosed and / or described herein. Accordingly, the various features of certain embodiments can be compatible with, combined with, included in, and / or incorporated into other embodiments of the present disclosure. Thus, disclosure of certain features relative to a specific embodiment of the present disclosure should not be construed as limiting application or inclusion of said features to the specific embodiment. Rather, it will be appreciated that other embodiments can also include said features, members, elements, parts, and / or portions without necessarily departing from the scope of the present disclosure.[000128] Moreover, unless a feature is described as requiring another feature in combination therewith, any feature herein may be combined with any other feature of a same or different embodiment disclosed herein. Furthermore, various well-known aspects of illustrative systems, methods, apparatus, and the like are not described herein in particular detail in order to avoid obscuring aspects of the example embodiments. Such aspects are, however, also contemplated herein.[000129] It will be apparent to one of ordinary skill in the art that methods, devices, device elements, materials, procedures, and techniques other than those specifically described herein can be applied to the practice of the described embodiments as broadly disclosed herein without resort to undue experimentation. All art-known functional equivalents of methods, devices, device elements, materials, procedures, and techniques specifically described herein are intended to be encompassed by this present disclosure.[000130] When a group of materials, compositions, components, or compounds is disclosed herein, it is understood that all individual members of those groups and all subgroups thereof are disclosed separately. When a Markush group or other grouping is used herein, all individual members of the group and all combinations and sub-combinations possible of the group are intended to be individually included in the disclosure.[000131] The above-described embodiments are examples only. Alterations, modifications, and variations may be effected to the particular embodiments by those of skill in the art without departing from the scope of the description, which is defined solely by the appended claims.

Claims

CLAIMSWhat is claimed is:

1. A computer implemented method for scaling resources in a software deployment system (600) comprising one or more pods (668, 669, 686, 688) in one or more nodes (662, 680) in one or more clusters (665), comprising: receiving (1210), at a Vertical Pod Auto-scaler, VPA (630), a configuration for the one or more pods; monitoring (1220), by the VPA, one or more metrics in the one or more pods; detecting (1230), by the VPA, that a recommendation is needed for the one or more pods according to the configuration; communicating (1240) with a Power Management, PM, component (2200, 3300) in the one or more nodes in the one or more clusters to confirm if the PM component can adjust a performance level for the one or more pods according to the recommendation; and if the PM component can satisfy the recommendation, then triggering (1250) by the PM component a PM action according to one or more conditions; and if the PM component cannot satisfy the recommendation, then continuing (1260) by the VPA to revise the resource request according to the recommendation.

2. The method of claim 2, wherein the software deployment system comprises at least one of: Kubernetes; Amazon Web Services Fargate; Azure Container Instances; Google Cloud Run; Google Kubernetes Engine; Amazon Elastic Kubernetes Service; Openshift Container Platform; Rancher; Docker Swarm; Nomad.

3. The method of claim 2 or 3, wherein the method is performed by one or more of: one or more user equipment; one or more network nodes; one or more hosts; one or more cloud computing components.

4. The method of any of claims 1 to 3, wherein if the recommendation comprises a request to scale up the one or more pods, then the method further comprises increasing, by the PM component, a central processing unit, CPU, frequency.

5. The method of any of claims 1 to 3, wherein if the recommendation comprises a request to scale down the one or more pods, then the method further comprises skipping, by the VP A, the recommendation.

6. The method of any of claims 1 to 5, wherein the PM action comprises at least one of: scaling up at least one of the one or more pods; scaling down at least one of the one or more pods; increasing a Central Processing Unit, CPU, frequency; decreasing a CPU frequency.

7. The method of any of claims 1 to 6, wherein the request comprises at least one of: one or more pod names; a target performance adjustment to one or more pods.

8. A method performed by a Power Management, PM, component (2200, 3300) for scaling resources in a software deployment system (600) comprising one or more pods (668, 669, 686, 688) in one or more nodes (662, 680) in one or more clusters (665), the method comprising: receiving (1410), by the PM component in the one or more nodes from a Vertical Pod Auto-Scaler, VPA (630, 875), a request to adjust a performance level according to a recommendation comprising a configuration for the one or more pods; transmitting (1420) to the VPA at least one of a confirmation or a rejection of the request; and if the PM component transmits the confirmation, then triggering (1430) by the PM component a PM action according to one or more conditions.

9. The method of claim 8, wherein if the recommendation comprises a request to scale up the one or more pods, then the method further comprises increasing, by the PM component, a central processing unit, CPU, frequency.

10. The method of claim 8 or 9, wherein the PM action comprises at least one of: scaling up at least one of the one or more pods; scaling down at least one of the one or more pods; increasing a Central Processing Unit, CPU, frequency; decreasing a CPU frequency.

11. The method of any of claims 8 to 10, wherein the request comprises at least one of: one or more pod names; a target performance adjustment to one or more pods.

12. The method of any of claims 8 to 11, further comprising, if the PM component transmits a rejection, then receiving a revised request from the VPA to adjust the performance level according to the recommendation.

13. A system (10) for scaling resources in a software deployment system (600) comprising one or more pods (668, 669, 686, 688) in one or more nodes (662, 680) in one or more clusters (665), comprising: processing circuitry (2202, 3302); and a memory (2210, 3304), the memory containing instructions executable by the processing circuitry whereby the system is operative to: receive (1210), at a Vertical Pod Auto-scaler, VPA (630, 875), a configuration for the one or more pods; monitor (1220), by the VPA, one or more metrics in the one or more pods; detect (1230), by the VPA, that a recommendation is needed for the one or more pods according to the configuration; communicate (1240) with a Power Management, PM, component (2200, 3300) in the one or more nodes in the one or more clusters to confirm if the PM component can adjust a performance level for the one or more pods according to the recommendation; and if the PM component can satisfy the recommendation, then trigger (1250) by the PM component a PM action according to one or more conditions; and if the PM component cannot satisfy the recommendation, then continue (1260) by the VPA to revise the resource request according to the recommendation.

14. The system of claim 13, wherein the system comprises one or more of: one or more user equipment; one or more network nodes; one or more hosts; one or more cloud computing components.

15. A system (600) for scaling resources in a software deployment system (600) comprising one or more pods (668, 669, 686, 688) in one or more nodes (662, 680) in one or more clusters (665), comprising: a Power Management Aware Vertical Pod Auto-scaler, PMA-VPA (630, 875), configured to: detect (809) a need to adjust a current resource recommendation for a pod; calculate a target performance adjustment for the pod based at least in part on a difference between a new resource recommendation and the current resource recommendation; and formulate a request (810) containing at least one of an identifier of the pod and the target performance adjustment; a Power Management Agent, PM Agent (664, 682, 890), deployed on each of one or more nodes and configured to; manipulate (820) one or more PM features (666, 684) of each respective one or more nodes, wherein the one or more PM features comprise available power management capabilities; report PM feature availability (804); and a Power Management Controller, PM Controller (650, 880), configured to; receive (810) the request from the PMA-VPA; receive (804) PM feature availability from the PM Agent; evaluate (812) whether the request can be fulfilled by the one or more PM features; manage (802, 816) the PM Agent deployed in each of the one or more nodes in one or more clusters; communicate (813, 824) PM feature availability to at least one of the PMA-VPA or a Metric and Analytics Service; a Metric and Analytics Server, MAS (620, 870), configured to provide (808) the PMA- VPA with one or more performance metrics on the one or more pods and one or more PM feature preferences for the one or more pods; and a Cluster API Server, CAS (660), configured to provide (811) the PMA-VPA with one or more application programming interfaces, APIs, to manipulate the one or more clusters.

16. The system of claim 15, wherein the system comprises one or more of: one or more user equipment; one or more network nodes; one or more hosts; one or more cloud computing components.

17. The system of claim 15 or 16, wherein the one or more PM features comprise at least one of: a central processing unit, CPU, frequency tuning; P-state; a kernel subsystem for Linux called CPUFreq.

18. A user equipment, UE (2200), for scaling resources in a software deployment system (10, 600) comprising one or more pods (668, 669, 686, 688) in one or more nodes (662, 680) in one or more clusters (665), comprising: processing circuitry (2202) configured to perform any of the steps of any of claims 1-10; and power supply circuitry (2208) configured to supply power to the processing circuitry.

19. A network node (3300) for scaling resources in a software deployment system (10, 600) comprising one or more pods (668, 669, 686, 688) in one or more nodes (662, 680) in one or more clusters (665), the network node comprising: processing circuitry (3302) configured to perform any of the steps of any of claims 1-10; and power supply circuitry (3308) configured to supply power to the processing circuitry.

20. A host node (4400) for scaling resources in a software deployment system (10, 600) comprising one or more pods (668, 669, 686, 688) in one or more nodes (662, 680) in one or more clusters (665), the network node comprising: processing circuitry (4402) configured to perform any of the steps of any of claims 1-10; and power supply circuitry (4408) configured to supply power to the processing circuitry.

Images