Resource configuration method, device, equipment and medium

Abstract

The present application discloses a resource configuration method, apparatus, device, and medium. The method includes: obtaining the historical idle index of a target resource in a cluster and the historical query index of a configuration resource for applying for the target resource; predicting the idle prediction value of the target resource based on the historical idle index; and predicting the query prediction value of the configuration resource based on the historical query index; and increasing or decreasing the number of replicas of the configuration resource based on the idle prediction value and the query prediction value. The embodiments of the present application improve the utilization of cluster GPU resources by collecting the predicted values ​​of the query rate per second of the target resource GPU and the configuration resource Pod.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a resource configuration method, device, equipment and medium. Background Art

[0002] With the maturity of container technology (Docker) and container orchestration technology (Kubernetes), more and more companies are moving their machine learning training processes into containers. Container technology solves the problem of inconsistencies between development and operations environments, enabling process isolation and allowing developers to package applications into containers. Containers combine application code, operating system libraries, and required dependencies, allowing them to be deployed and run on any popular system. Container orchestration technology can be used to manage container clusters, enabling the startup, monitoring, and recovery of containers running on a group of servers while ensuring their healthy operation.

[0003] Using container orchestration technology (K8S) to aggregate GPU (Graphics Processing Unit) resources into a resource pool for unified management, and using Docker to deliver the runtime environment for deep learning, has become an important branch of K8S cluster business development.

[0004] However, Kubernetes currently only offers a Horizontal Pod Autoscaler (HPA) solution based on CPU utilization, and supports implementing HPA using custom metrics. Existing HPA relies on CPU and memory utilization to control autoscaling, which cannot meet the multi-metric scaling and autoscaling stability requirements of cluster services. Summary of the Invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, it is desired to provide a resource configuration method, apparatus, device and medium to solve the problem that the existing HPA cannot effectively improve the efficiency of cluster GPU utilization when cluster services require GPU resources.

[0006] In a first aspect, an embodiment of the present invention provides a resource configuration method, the method comprising:

[0007] Obtain the historical idle indicators of the target resources in the cluster and the historical query indicators of the configured resources for applying for the target resources;

[0008] Predicting an idle prediction value of a target resource based on historical idle indicators; and predicting a query prediction value of the configured resource based on historical query indicators;

[0009] Increase or decrease the number of replicas of configured resources based on the idle prediction value and query prediction value.

[0010] In a second aspect, an embodiment of the present invention further provides a resource configuration device, which includes at least an indicator collection module, an indicator calculation module and a configuration module, wherein:

[0011] The indicator collection module is used to obtain the historical idle indicators of the target resources in the cluster and the historical query indicators of the configured resources that use the target resources;

[0012] An indicator calculation module is used to predict the idle prediction value of the target resource based on the historical idle indicators; and to predict the query prediction value of the configured resource based on the historical query indicators;

[0013] The configuration module is used to increase or decrease the number of replicas of the configuration resource based on the idle prediction value and the query prediction value.

[0014] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the embodiment of the present invention when executing the program.

[0015] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in the embodiment of the present invention when the computer program is executed by a processor.

[0016] The beneficial effects brought about by the technical solution provided by the present invention are:

[0017] The present application provides a resource configuration method, apparatus, device and medium, which obtains the historical idle index of the target resource in the cluster and the historical query index of the configuration resource for applying for the target resource; then, predicts the idle prediction value of the target resource based on the historical idle index; and predicts the query prediction value of the configuration resource based on the historical query index. Finally, the number of copies of the configuration resource is increased or decreased according to the idle prediction value and the query prediction value. Based on the idle prediction value and the query prediction value of the target resource and the configuration resource, the method more accurately identifies the utilization rate of the configuration resource using the target resource, and can release the target resource when the utilization rate of the target resource is low, thereby reducing the number of copies of the configuration resource, thereby effectively improving the utilization efficiency of the target resource in the cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic diagram showing an application scenario of the resource scaling control method provided by an embodiment of the present invention is shown;

[0020] Figure 2 A schematic diagram of the K8S framework structure provided by an embodiment of the present invention is shown;

[0021] Figure 3 A schematic diagram showing a flow chart of a resource configuration method provided by an embodiment of the present invention is shown;

[0022] Figure 4 A schematic diagram showing a flow chart of a resource configuration method provided by an embodiment of the present invention is shown;

[0023] Figure 5 A schematic diagram showing a flow chart of a resource configuration method provided by another embodiment of the present invention is shown;

[0024] Figure 6 A schematic diagram showing the principle of implementing resource configuration using a K8S aggregation API according to another embodiment of the present invention is shown;

[0025] Figure 7 A schematic diagram of the structure of a resource configuration device provided in an embodiment of the present application is shown;

[0026] Figure 8 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] To make the objectives, technical solutions, and advantages of the present invention more apparent, embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the invention are shown in the accompanying drawings.

[0028] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0029] Please refer to Figure 1 , Figure 1 A schematic diagram of an application scenario of the resource scaling control method provided in an embodiment of the present application is shown.

[0030] exist Figure 1In the illustrated application scenario, one or more terminal devices 101 access resources from one or more servers 103 deployed in different regions via a public network 102. Terminal devices 101 may include, but are not limited to, mobile devices such as smartphones, tablets, and smart glasses, as well as electronic devices such as desktop computers. Applications for performing service access may be pre-installed on terminal devices 101.

[0031] The public network 102 is used to support the terminal device 101 in accessing service resources in various network situations. Optionally, the public network can include a wireless network or a wired network, using standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network, or any combination of a virtual private network.

[0032] Server 103 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0033] In the above application scenario, server 103 can be configured to meet the requirements of K8S cloud platform deployment. K8S, the full name of which is Kubernetes, is a portable container orchestration and management tool based on container services. Figure 2 As shown, a K8S cluster consists of one or more Master nodes and multiple Node (Worker) nodes. The Master node refers to the cluster control node, which manages and controls the entire cluster. Nodes other than the Master node are called Node or Worker nodes. Each Node node is assigned some work by the Master node, which maintains the running Pods and provides the Kubernetes operating environment. Containers are run in Pods. Pods are the basic unit of K8S management. Master nodes include but are not limited to the gateway API Server, and all command requests need to be forwarded through the API Server. Node nodes include but are not limited to one or more Pods, and each Pod can include one or more containers.

[0034] When Kubernetes-based clusters face issues like low load or fluctuating business volumes, scaling Pod resources can effectively ensure system stability. With the growing popularity of machine learning and deep learning across various industries, moving machine learning training processes into containers can allow for the use of GPU resources for inference calculations of AI algorithm models to meet business development needs. In scenarios where some cluster services utilize GPU resources, service providers will be concerned about GPU utilization. How to improve GPU utilization in this scenario? Relying solely on existing scaling strategies that control the number of Pod replicas based on CPU utilization clearly cannot effectively guarantee GPU utilization.

[0035] Therefore, based on the above problems, the present invention proposes a resource configuration method, which implements HPA in this scenario by monitoring the usage of GPU resources, thereby ensuring the stability of services running on Pods and improving the GPU usage efficiency in the cluster.

[0036] In order to more clearly understand the inventive concept provided by the present invention, Figure 3-6 The resource configuration method proposed in the present invention is described.

[0037] Please refer to Figure 3 , Figure 3 The present invention provides a resource configuration method, which can be executed by a master node configured in an electronic device. Specifically, the method can be implemented by creating a new functional module in the master node or adding a functional submodule to an existing functional module. The method includes the following steps:

[0038] Step 301: Obtain historical idle indicators of target resources in the cluster and historical query indicators of configured resources for applying for target resources;

[0039] Step 302: predicting the target resource's idle prediction value based on historical idle indicators; and predicting the configured resource's query prediction value based on historical query indicators;

[0040] Step 303: Increase or decrease the number of copies of the configured resource according to the idle prediction value and the query prediction value.

[0041] In the above steps, target resources refer to the GPU resources included in the cluster. Configuration resources refer to the Pod resources included in the cluster. The historical idle index refers to the GPU idle rate corresponding to the GPU resources in the cluster under different monitoring cycles. The historical idle index includes the first historical idle index corresponding to the first monitoring cycle, the second historical idle index corresponding to the second monitoring cycle, the third historical idle index corresponding to the third monitoring cycle, and the current idle index corresponding to the current time. The current time can be the current time corresponding to the system each time the timer used to monitor auto-scaling runs. For example, the timer executes every 5 minutes. Assuming the timer runs at 6:00 PM on November 16, 2021, the current time is 6:00 PM on November 16, 2021, and the next execution time is 6:05 PM on November 16, 2021. The current time is updated and assigned when the timer executes.

[0042] The current time may be determined by responding to a start signal of a timer for monitoring resource configuration and determining the current system clock signal as the current time, or responding to a request for cluster resource configuration and determining the clock signal corresponding to the request as the current time.

[0043] The first monitoring period, the second monitoring period, and the third monitoring period correspond to time periods of different lengths. The time periods can be days, months, or years. For example, the first monitoring period can be 24 hours, the second monitoring period can be 30 or 31 days, or a calendar month, and the third monitoring period can be 365 or 366 days.

[0044] When the predicted time is determined to be t1, the first historical idle index corresponding to the first monitoring period is the GPU idle rate corresponding to the same time as the determined predicted time within the first monitoring period before the determined predicted time. For example, if the predicted time is 18:10 on November 16, 2021, the first historical idle index corresponding to the first monitoring period can be the GPU idle rate corresponding to 18:10 on November 15, 2021.

[0045] Optionally, when the predicted time is determined to be t1, the second historical idle index corresponding to the second monitoring period refers to the GPU idle rate corresponding to the same time as the determined predicted time within a second monitoring period before the determined predicted time. For example, if the predicted time is 18:10 on November 16, 2021, the second historical idle index corresponding to the second monitoring period may be the GPU idle rate corresponding to 18:10 on October 16, 2021.

[0046] Optionally, when the predicted time is determined to be t1, the third historical idle index corresponding to the third monitoring period refers to the GPU idle rate corresponding to the same time as the determined predicted time within a third monitoring period before the determined predicted time. For example, if the predicted time is 18:10 on November 16, 2021, the third historical idle index corresponding to the third monitoring period may be the GPU idle rate corresponding to 18:10 on November 16, 2020.

[0047] The above obtains the historical idle indicators of the target resources in the cluster, including:

[0048] Get the current idle indicator corresponding to the current moment;

[0049] A first historical idle indicator corresponding to the first monitoring period, a second historical idle indicator corresponding to the second monitoring period, and a third historical idle indicator corresponding to the third monitoring period are obtained.

[0050] The process of obtaining the current idle indicator corresponding to the current moment includes:

[0051] Get the total number of target resources in the cluster at the current moment;

[0052] Get the statistical number of unused target resources in the cluster at the current moment;

[0053] Calculate the historical idle index based on the total number and statistical number.

[0054] Add the Custom-Metrics Server module (i.e., the metric calculation module), the Prometheus Client module, and the Prometheus Exporter module (the Prometheus Client and Prometheus Exporter modules are also the metric collection modules) to the Master node. The metric collection modules collect GPU data and QPS data, respectively. GPU data can be obtained using the Prometheus Exporter module developed with the Client-Go toolkit, while QPS (Queries Per Second) data can be obtained using the Prometheus Client module.

[0055] For Pod resources that apply for GPU resources, the current status of the Pod resource may be Evicted. In this status, the Pod resource also applies for GPU resources. Pod resources in this status cannot actually work properly. Therefore, it is necessary to exclude the situation where GPU resources are applied in this status. This can effectively improve the accuracy of statistical solutions and thus increase the utilization of GPU resources.

[0056] To obtain the current number of unused target resources in the cluster, perform the following steps:

[0057] Obtain a status tag of each configuration resource included in the cluster; identify unused configuration resources according to the status tags; and accumulate the statistical quantity according to the unused configuration resources.

[0058] A status tag is a tag used to indicate the resource status of a configuration resource. For example, the status can be Evicted, and a binary or other numeric identifier can be used.

[0059] Get the current number of unused target resources in the cluster. This can also include:

[0060] After determining the total number of target resources in the cluster at the current moment, determine the status tag of each configuration resource contained in the cluster, and then use the total number to subtract the number of GPUs requested by the configuration resources in an abnormal state; and then subtract the number of GPUs already in use in the cluster.

[0061] The current idle index corresponding to the current moment, the first historical idle index corresponding to the first monitoring period, the second historical idle index corresponding to the second monitoring period, and the third historical idle index corresponding to the third monitoring period all represent the GPU idle rate, which can be determined by the following formula:

[0062] GPU idle rate = number of unused GPUs in the cluster / total number of GPUs

[0063] For example, the current idle index corresponding to the current moment refers to the GPU idle rate corresponding to the current moment. Similarly, the first historical idle index corresponding to the first monitoring period refers to the GPU idle rate corresponding to the first monitoring period. The only difference is the statistical time; the calculation method for the GPU idle rate is the same.

[0064] In step 302, predicting the idle prediction value of the target resource based on the historical idle index may include:

[0065] The idle prediction value of the target resource is calculated according to the first historical idle indicator corresponding to the first monitoring period, the second historical idle indicator corresponding to the second monitoring period, the third historical idle indicator corresponding to the third monitoring period and the current idle indicator corresponding to the current moment.

[0066] The idle prediction value refers to the GPU idle rate corresponding to the predicted time after the current time is determined. For example, if the current time is determined to be t0, the predicted time is t1. The predicted time can be reasonably determined based on the accuracy requirements of the algorithm model.

[0067] In the above step 301, the historical query indicators include the first query prediction value corresponding to the first monitoring period, the second query prediction value corresponding to the second monitoring period, the third query prediction value corresponding to the third monitoring period and the current query prediction value corresponding to the current moment.

[0068] The historical query indicator refers to the QPS corresponding to the configured resources of the target resource application at the historical moment. QPS is the "query rate per second", which refers to the number of queries that a server can respond to per second. It is a measure of the amount of traffic handled by a specific query server within a specified time. For each Pod resource that applies to use GPU resources, the corresponding QPS can be understood as the QPS in each time interval of the running Pod according to the periodic statistics. If the statistics are counted every 10 minutes, the total number of requests received by the Pod resources counted within 10 minutes is divided by 10 minutes * 60 seconds, which represents the QPS corresponding to the historical moment. For example, historical QPS (January 1, 2021, 12:00-10) = total number of requests / (10 minutes * 60 seconds).

[0069] You can also calculate the query prediction value by collecting the QPS of the current interval. The QPS of the current interval can be calculated by calculating the QPS of one cycle forward from the current moment; for example, the QPS of the current interval = (current moment t0 - cycle value) total number of requests / (10 minutes * 60).

[0070] Optionally, in the above steps, predicting the query prediction value of the configuration resource based on the historical query indicator includes:

[0071] The query prediction value of the configured resources is calculated based on the first query prediction value corresponding to the first monitoring period, the second query prediction value corresponding to the second monitoring period, the third query prediction value corresponding to the third monitoring period, and the current query prediction value corresponding to the current moment.

[0072] When the predicted time is determined to be t1, the first query prediction value corresponding to the first monitoring period refers to the historical QPS corresponding to the same time as the determined prediction time within the first monitoring period before the determined prediction time. For example, if the predicted time is 18:10 on November 16, 2021, the first query prediction value corresponding to the first monitoring period can be the historical QPS corresponding to 18:10-18:00 on November 15, 2021.

[0073] When the predicted time is determined to be t1, the second query prediction value corresponding to the second monitoring period refers to the historical QPS corresponding to the same time as the determined prediction time within the second monitoring period before the determined prediction time. For example, if the predicted time is 18:10 on November 16, 2021, the second query prediction value corresponding to the second monitoring period can be the historical QPS corresponding to 18:10-18:00 on October 16, 2021.

[0074] When the predicted time is determined to be t1, the third query prediction value corresponding to the third monitoring period refers to the historical QPS corresponding to the same time as the determined prediction time within the third monitoring period before the determined prediction time. For example, if the predicted time is 18:10 on November 16, 2021, the third query prediction value corresponding to the third monitoring period can be the historical QPS corresponding to 18:10-18:00 on November 16, 2020.

[0075] The resource configuration method proposed in the present invention obtains the historical usage of GPU resources, and then predicts the idle rate of GPU resources and the QPS of Pod resources at future moments based on the historical usage. The number of Pod copies in the cluster is controlled based on the prediction results. It can accurately understand the usage of GPU resources and precisely control the number of Pod copies, thereby improving the utilization of GPU resources.

[0076] Based on the above embodiment, Figure 4 As shown, Figure 4 An exemplary flow chart of another resource configuration method provided by an embodiment of the present application is shown. This method can be executed by a master node (Master node) configured in an electronic device. Specifically, a new functional module can be created in the Master node or a functional submodule can be added to the original functional module to implement the method. The method includes:

[0077] Step 401: Obtain historical idle indicators of target resources in the cluster and historical query indicators of configured resources for applying for target resources;

[0078] Step 402: predicting the idle prediction value of the target resource based on the historical idle index; and predicting the query prediction value of the configured resource based on the historical query index.

[0079] Step 403: Determine a scaling control strategy identifier based on the idle prediction value and the query prediction value.

[0080] Step 404: Increase or decrease the number of replicas of the configured resource based on the scaling control policy identifier.

[0081] In the above steps, the scaling control policy identifier refers to an indication value used to indicate an increase or decrease in the number of replicas of the configured resources. This indication value can use binary identifiers or other symbols for indication.

[0082] Among them, determining the scaling control policy identifier according to the idle prediction value and the query prediction value includes:

[0083] When the idle prediction value is greater than the first threshold and the query prediction value is greater than the second threshold, determine that the scaling control policy identifier is the first control identifier;

[0084] When the idle prediction value is between the first threshold and the third threshold, and the query prediction value is greater than the second threshold, and the service priority of the configured resource is the first flag value, determine that the scaling control policy identifier is the first control identifier;

[0085] When the idle prediction value is less than the first threshold, and the query prediction value is less than the fourth prediction value, and the service priority of the configured resource is the second flag value, then determine that the scaling control policy identifier is the second control identifier;

[0086] Among them, the third threshold is less than the first threshold.

[0087] Assume that the maximum QPS value that a Pod resource can support is QPS_max. Usually, stress testing is performed before deploying the Pod resource to obtain the maximum QPS value in advance. The service priority of the Pod resource can be input when creating the Pod. QPS_t1 represents the query prediction value corresponding to time t1, and GPUAvail_t1 represents the idle prediction value corresponding to time t1.

[0088] When QPS_t1 > QPS_max * 80% and GPUAvail_t1 > 20%, it is determined that scaling up is required, and the first control identifier can be set. For example, the first control identifier is 1.

[0089] When QPS_t1 > QPS_max * 80% and 0 < GPUAvail_t1 < 20% and the priority of the Pod resource is equal to 1, it is determined that scaling up is required, and the first control identifier can be set. For example, the first control identifier is 1. That is, increase the number of replicas of the Pod according to the first control identifier.

[0090] When QPS_t1 < QPS_max * 20% and GPUAvail_t1 < 20% and the priority of the Pod resource is 0, it is determined that scaling down is required, and the second control identifier can be set. For example, the second control identifier is 0. That is, reduce the number of replicas of the Pod according to the second control identifier.

[0091] When QPS_t1 and GPUAvail_t1 cannot simultaneously meet the above conditions, no scaling control policy is required for the Pod resources in the cluster.

[0092] The resource configuration method proposed in the present invention obtains the usage of GPU resources, and then predicts the idle rate of GPU resources and the QPS of Pod resources at future moments based on the usage. The number of Pod copies in the cluster is controlled based on the control identifier corresponding to the prediction result, which can save overhead and improve the utilization of GPU resources.

[0093] Based on the above embodiment, Figure 5 As shown, Figure 5 An exemplary flow chart of another resource configuration method provided by an embodiment of the present application is shown. This method can be executed by a master node (Master node) configured in an electronic device. Specifically, a new functional module can be created in the Master node or a functional submodule can be added to the original functional module to implement the method. The method includes:

[0094] Step 501: Obtain historical idle indicators of target resources in the cluster and historical query indicators of configured resources for applying for target resources;

[0095] Step 502: predicting the idle prediction value of the target resource based on the historical idle index; and predicting the query prediction value of the configured resource based on the historical query index.

[0096] Step 503: Increase or decrease the number of copies of the configured resource according to the idle prediction value and the query prediction value.

[0097] Step 504 : Formatting the idle prediction value, the query prediction value, the maximum query value corresponding to the configured resource, and the scaling control policy identifier corresponding to increasing or decreasing the number of replicas of the configured resource.

[0098] In the preceding steps, formatting HPA Metrics converts QPS_t1, GPUAvail_t1, QPS_max, and the decision result (capacity expansion or contraction) in the indicator calculation module into a data type (Prometheus Metrics format) that HPA Metrics can recognize. This data can then be provided to the HPA controller in the Master node for collection and use.

[0099] The resource configuration method proposed in the present invention, based on the above-mentioned embodiment, can improve the data processing speed and simultaneously enhance the utilization rate of GPU resources by formatting the predicted value and the control strategy identifier.

[0100] In order to better understand the present invention, the following takes the K8S container orchestration system aggregation API to implement the Custom-MetricsServer module as an example. Figure 6 The resource configuration method proposed in the present invention is described in detail.

[0101] Through the Kube-Aggregator component, that is, the API aggregator, the K8S native API Server and Custom-Metrics Server modules are combined, so that the GPU resource data can be directly obtained from the API Server of the K8S cluster without having to go through other channels. In order to implement the resource configuration method proposed in the present invention, it can at least include an indicator collection module for collecting information such as GPU, Pod QPS, Pod priority, and an indicator calculation module for predicting the GPU idle rate and the QPS of Pod resources, as well as a configuration module for making decisions based on the prediction results, and the prediction results are calculated by the indicator calculation module. Among them, the indicator collection module can be implemented using the Goland language. The indicator calculation module and the configuration module can be implemented in Python.

[0102] The metrics collection module can include at least: a Prometheus Exporter submodule for collecting GPU idle rate and a Prometheus Client module for collecting Pod resource QPS. GPU data is obtained by using a Prometheus Exporter module developed with the Client-GO toolkit. The specific implementation process is as follows:

[0103] a) Pre-label the GPU machines in the Kubernetes cluster with the tag hwtype="NVIDIAGPU" and use the K8SClient-Go toolkit to count the GPU data on these tagged GPU machines. The accumulated data is used to obtain the total number of GPUs available for Pod resources. This data is then exposed as a Prometheus metric called "AllocatableGPU," which indicates the number of allocatable GPUs in the cluster.

[0104] b) Communicate with the K8S API-Server through the K8S Client-Go toolkit to collect the total number of GPUs requested by Pod resources in normal status in the K8S cluster and expose it as a Prometheus metric called "AllocatedGPU", which indicates the number of allocated GPUs in the cluster.

[0105] c) Persist the above data in the MySQL database.

[0106] The indicator collection module may further include: a submodule for collecting QPS data, which may be implemented using a Prometheus Client module.

[0107] The Prometheus Client module can periodically collect the QPS values ​​of Pod resources that have applied for GPU resources from the API Server within a certain period of time, as well as the QPS values ​​of Pod resources at the current time. Based on the collected QPS at the current time, it calculates the QPS of the previous period and persists this data in the MySQL database.

[0108] The Prometheus Client module can also periodically obtain the priority of Pod resources that apply for GPU resources in the K8S cluster and persist this data in a MySQL database.

[0109] The indicator calculation module retrieves historical data from the MySQL database and real-time data from the indicator collection module. Based on this historical data and the current real-time data, the indicator calculation module predicts the GPU idle rate and QPS of Pod resources corresponding to the next moment.

[0110] Assuming the current time is t0 and the predicted time is t1, the QPS prediction value corresponding to the predicted time t1 can be obtained according to the following formula:

[0111]

[0112] Among them, QPS yest-t1 is the QPS value at time t1 yesterday; QPS mon-t1 (Month-on-month) is the QPS value at time t1 on the same day of the previous month; QPS year-t1 (Year-on-year) is the QPS value at time t1 of the same month and day of the previous year; QPS cur is the QPS value at the current time t0.

[0113] Assuming the current time is t0 and the predicted time is t1, the predicted value of the GPU idle rate corresponding to the predicted time t1 can be obtained according to the following formula:

[0114]

[0115] Among them, GPUAvail yest-t1 GPUAvail is the GPU idle rate at time t1 yesterday; mon-t1 (Month-on-month) is the GPU idle rate at time t1 on the same day of the previous month; GPUAvail year-t1 (Year-on-year) is the GPU idle rate at time t1 on the same day of the same month of the previous year; GPUAvailcur is the GPU idle rate at the current moment t0.

[0116] The configuration module can obtain relevant prediction results from the metric calculation module via HTTP, such as the predicted value of the GPU idle rate and the predicted value of the QPS of the Pod, as well as the maximum query value corresponding to the configured resource and the scaling control policy identifier corresponding to increasing or decreasing the replica number of the configured resource, and convert the formats of these predicted values and scaling control policy identifiers into data types recognizable by HPA Metrics. The configuration module can also provide a Restful HTTP service externally through the Flask framework.

[0117] Assume that the maximum QPS value that a Pod resource can support is QPS_max.

[0118] The configuration module at QPS t1 > QPS_max * 80% and GPUAvail_t1 > 20%, determines that scaling out is required. Set the first control identifier, for example, the first control identifier is 1; QPS t1 represents the predicted QPS value corresponding to the moment t1, and GPUAvail_t1 represents the predicted idle value corresponding to the moment t1.

[0119] The configuration module at QPS t1 > QPS_max * 80% and 0 < GPUAvail_t1 < 20% and the priority of the Pod resource is equal to 1, determines that scaling out is required. Set the first control identifier, for example, the first control identifier is 1;

[0120] The configuration module at QPS t1 < QPS_max * 20% and GPUAvail_t1 < 20% and the priority of the Pod resource is 0, determines that scaling in is required. Set the second control identifier, for example, the second control identifier is 0.

[0121] The configuration module at QPS t1 and GPUAvail_t1 do not satisfy the above conditions simultaneously, no scaling control policy needs to be adopted for the Pod resources in the cluster.

[0122] The resource configuration method proposed by the present invention can integrate external services into the current Master node through an API aggregation component (i.e., API Aggregator), which effectively improves the resource configuration efficiency.

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] Please refer to Figure 7 , Figure 7 The following is a schematic diagram showing the structure of a resource configuration device provided in an embodiment of the present application. The device includes:

[0125] An indicator acquisition module 601 is used to obtain historical idle indicators of target resources in the cluster and historical query indicators of configured resources using the target resources;

[0126] An indicator calculation module 602 is configured to predict the idle prediction value of the target resource based on the historical idle indicators; and to predict the query prediction value of the configured resource based on the historical query indicators;

[0127] The configuration module 603 is configured to increase or decrease the number of copies of the configuration resource according to the idle prediction value and the query prediction value.

[0128] The indicator acquisition module 601 may further include:

[0129] A submodule for obtaining the current idle indicator corresponding to the current moment;

[0130] A submodule for obtaining a first historical idle indicator corresponding to a first monitoring period, a second historical idle indicator corresponding to a second monitoring period, and a third historical idle indicator corresponding to a third monitoring period.

[0131] The submodule used to obtain the current idle indicator corresponding to the current moment is also used to:

[0132] Obtain the total number of the target resources in the cluster corresponding to the current moment;

[0133] Obtaining a statistical number of the target resources in an unused state in the cluster corresponding to the current moment;

[0134] The current idle index is calculated according to the total number and the statistical number.

[0135] The indicator calculation module 602 may further include:

[0136] The first calculation submodule is used to calculate the idle prediction value of the target resource based on the first historical idle indicator corresponding to the first monitoring period, the second historical idle indicator corresponding to the second monitoring period, the third historical idle indicator corresponding to the third monitoring period and the current idle indicator corresponding to the current moment.

[0137] The second calculation submodule is used to calculate the query prediction value of the configuration resource based on the first query prediction value corresponding to the first monitoring period, the second query prediction value corresponding to the second monitoring period, the third query prediction value corresponding to the third monitoring period and the current query prediction value corresponding to the current moment.

[0138] The configuration module 603 may further include:

[0139] a determination submodule, configured to determine a scaling control strategy identifier according to the idle prediction value and the query prediction value;

[0140] The control submodule is configured to increase or decrease the number of replicas of the configuration resource based on the scaling control policy identifier.

[0141] The determination submodule is further configured to: determine that the scaling control strategy identifier is a first control identifier when the idle prediction value is greater than a first threshold and the query prediction value is greater than a second threshold;

[0142] When the idle prediction value is between the first threshold and the third threshold, the query prediction value is greater than the second threshold, and the service priority of the configured resource is a first flag value, determining that the scaling control policy identifier is a first control identifier;

[0143] When the idle prediction value is less than the first threshold, the query prediction value is less than the fourth prediction value, and the service priority of the configured resource is the second flag value, determining that the scaling control policy identifier is the second control identifier;

[0144] The third threshold is smaller than the first threshold.

[0145] The above-mentioned device may further include a formatting processing module, which is used to format and process the idle prediction value, the query prediction value, the maximum query value corresponding to the configuration resource, and the scaling control policy identifier corresponding to increasing or decreasing the number of replicas of the configuration resource.

[0146] Reference below Figure 8 , Figure 8 The electronic device provided in the embodiment of the present application is shown in the structural diagram. The electronic device can be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, other mobile internet devices (MID), a PAD, and other terminals. Figure 7 It does not limit the structure of electronic equipment. Figure 7 As shown, the electronic device at least includes a memory 701 and a processor 702. For example, the electronic device may also include Figure 7 More or fewer components (such as network interfaces, display devices, etc.) may be shown.

[0147] In particular, according to the embodiment provided by the present invention, the above reference flow chart Figure 3 The described process can be implemented as a computer software program. For example, embodiments provided by the present invention include a computer program product comprising a computer program carried on a machine-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication portion, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the system of the present application are performed.

[0148] It should be noted that the computer-readable medium shown in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and computer program products described in accordance with the various embodiments provided by the present invention. Each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the aforementioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0150] The units or modules involved in the embodiments of the present application may be implemented in software or in hardware. The units or modules described may also be provided in a processor. For example, they may be described as: a processor including an indicator acquisition module, an indicator calculation module, and a configuration module. Among them, the names of these units or modules do not, in some cases, constitute a limitation on the units or modules themselves. For example, the service subscription module may also be described as a "module for obtaining historical idle indicators of target resources in a cluster and historical query indicators of configuration resources using target resources."

[0151] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments, or may exist independently and not be incorporated into the electronic device. The computer-readable storage medium stores one or more programs, which, when used by one or more processors, execute the service addressing access method described in the present application.

[0152] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention herein is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features having similar functions disclosed in this application.

Claims

1. A resource configuration method, characterized in that: The method includes: Obtain the historical idle indicators of the target resources in the cluster and the historical query indicators of the configured resources for applying for the target resources; Predicting an idle prediction value of the target resource based on the historical idle indicator; and predicting a query prediction value of the configuration resource based on the historical query indicator; increasing or decreasing the number of copies of the configuration resource according to the idle prediction value and the query prediction value; The increasing or decreasing the number of copies of the configuration resource according to the idle prediction value and the query prediction value includes: determining a scaling control strategy identifier according to the idle prediction value and the query prediction value; increasing or decreasing the number of replicas of the configuration resource based on the scaling control policy identifier; The determining of the scaling control strategy identifier according to the idle prediction value and the query prediction value includes: When the idle prediction value is greater than a first threshold and the query prediction value is greater than a second threshold, determining that the scaling control strategy identifier is a first control identifier; When the idle prediction value is between the first threshold and the third threshold, the query prediction value is greater than the second threshold, and the service priority of the configured resource is a first flag value, determining that the scaling control policy identifier is a first control identifier; When the idle prediction value is less than the first threshold, the query prediction value is less than the fourth prediction value, and the service priority of the configured resource is the second flag value, determining that the scaling control policy identifier is the second control identifier; The third threshold is smaller than the first threshold.

2. The method according to claim 1, characterized in that The step of obtaining the historical idle index of the target resource in the cluster includes: Get the current idle indicator corresponding to the current moment; A first historical idle indicator corresponding to the first monitoring period, a second historical idle indicator corresponding to the second monitoring period, and a third historical idle indicator corresponding to the third monitoring period are obtained.

3. The method according to claim 2, characterized in that The obtaining of the current idle indicator corresponding to the current moment includes: Obtain the total number of the target resources in the cluster corresponding to the current moment; Obtaining a statistical number of the target resources in an unused state in the cluster corresponding to the current moment; The current idle index is calculated according to the total number and the statistical number.

4. The method according to claim 1, wherein The historical idle indicators include a first historical idle indicator corresponding to a first monitoring period, a second historical idle indicator corresponding to a second monitoring period, a third historical idle indicator corresponding to a third monitoring period, and a current idle indicator corresponding to a current moment. Then, predicting the idle prediction value of the target resource based on the historical idle indicators includes: The idle prediction value of the target resource is calculated according to the first historical idle indicator corresponding to the first monitoring period, the second historical idle indicator corresponding to the second monitoring period, the third historical idle indicator corresponding to the third monitoring period and the current idle indicator corresponding to the current moment.

5. The method according to claim 1, wherein The historical query indicators include a first query prediction value corresponding to a first monitoring period, a second query prediction value corresponding to a second monitoring period, a third query prediction value corresponding to a third monitoring period, and a current query prediction value corresponding to a current moment. Then, predicting the query prediction value of the configuration resource based on the historical query indicators includes: The query prediction value of the configured resource is calculated based on the first query prediction value corresponding to the first monitoring period, the second query prediction value corresponding to the second monitoring period, the third query prediction value corresponding to the third monitoring period and the current query prediction value corresponding to the current moment.

6. The method according to claim 1, characterized in that The method further includes: The idle prediction value, the query prediction value, the maximum query value corresponding to the configuration resource, and the scaling control policy identifier corresponding to increasing or decreasing the number of replicas of the configuration resource are formatted.

7. A resource allocation device, characterized in that: The device at least includes an indicator collection module, an indicator calculation module and a configuration module, wherein: The indicator collection module is used to obtain historical idle indicators of target resources in the cluster and historical query indicators of configuration resources using the target resources; The indicator calculation module is used to predict the idle prediction value of the target resource based on the historical idle indicator; and predict the query prediction value of the configuration resource based on the historical query indicator; The configuration module is configured to increase or decrease the number of copies of the configuration resource according to the idle prediction value and the query prediction value; The determining of the scaling control strategy identifier according to the idle prediction value and the query prediction value includes: When the idle prediction value is greater than a first threshold and the query prediction value is greater than a second threshold, determining that the scaling control strategy identifier is a first control identifier; When the idle prediction value is between the first threshold and the third threshold, the query prediction value is greater than the second threshold, and the service priority of the configured resource is a first flag value, determining that the scaling control policy identifier is a first control identifier; When the idle prediction value is less than the first threshold, the query prediction value is less than the fourth prediction value, and the service priority of the configured resource is the second flag value, determining that the scaling control policy identifier is the second control identifier; The third threshold is smaller than the first threshold.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 6 when executed by a processor.

Images