Computing resource scheduling method, device and system based on bare metal or container

Abstract

The present application provides a method, device and system for scheduling computing resources based on bare metal or containers, the method comprising: predicting the first computing power resource data through a pre-trained safety waterline prediction model to obtain the inventory safety waterline corresponding to the current period; performing system initialization deployment, and monitoring the usage data of the target computing power resources and system resources in real time, and converting them into inventory information; when a new user resource creation request is detected, parsing to obtain the target priority label; if the target priority label is a high priority label, and the ratio of the number of idle groups to the total number of groups is judged based on the inventory information, and is not greater than the threshold corresponding to the preset safety waterline, identifying the target computing power resource group corresponding to the low priority label and performing an instance release operation; based on the resources after the instance release operation, responding to the user resource creation request. The present application can perform reasonable resource scheduling, ensure the stability of key tasks, and improve service quality and user experience.

Description

Technical Field

[0001] The present application relates to the technical field of scheduling computing resources, and in particular to a method, device and system for scheduling computing resources based on bare metal or containers. Background Art

[0002] As the core force driving the development of the digital economy and a solid foundation supporting the development of the digital economy, computing power plays an important role in promoting scientific and technological progress, promoting the digital transformation of industries, and economic and social development. With the diversification of business scenarios, the demand for computing power resources has increased significantly.

[0003] However, due to the differences in the demand for computing resources in various business scenarios, it is difficult to accurately and specifically adjust the corresponding computing resources when faced with computing resource adjustments, resulting in the inability to reasonably allocate resources, causing idle resources to be underutilized or unable to ensure the stability of key tasks, resulting in a high resource idle rate and wasting resources or reducing user experience. For example, due to the relatively fixed resource allocation method of bare metal servers, when the demand for new resources is greater than its remaining resources, there is a lack of corresponding resource scheduling strategies to solve the problem of insufficient resources, resulting in delayed or unresponsive key tasks, unable to ensure the stability of key businesses and poor user experience. For another example, most container resource management methods are based on rules or manual processing, and the processing logic for requests of different priorities is not flexible enough, which can easily cause low-priority container instance resources to be occupied for a long time, resulting in high-priority requests not being met in a timely manner; not only is it easy to waste computing resources, but it also cannot ensure that there are enough reserved resources for key tasks, and cannot ensure the stability of key tasks, reducing service quality and user experience. Summary of the invention

[0004] The purpose of this application is to provide a computing resource scheduling method, device and system based on bare metal or containers. Through a pre-trained safety water level prediction model, the real-time operation status of the target computing resource group is predicted, the current inventory safety water level is determined, and then deployment is performed based on the water level. The monitoring task is started, and when a user request is monitored, it is determined whether the instance needs to be released based on the real-time inventory information and the corresponding threshold of the preset safety water level, so as to perform reasonable resource scheduling, ensure the stability of key tasks, and improve service quality and user experience.

[0005] In a first aspect, the present application provides a computing resource scheduling method based on bare metal or containers, the method comprising: obtaining first computing resource data; the first computing resource data comprising: real-time resource demand data, real-time resource supply data and first priority label data of a target computing resource group; predicting the first computing resource data through a pre-trained safety waterline prediction model to obtain an inventory safety waterline corresponding to the current period; the inventory safety waterline is used to characterize the minimum value of the resource demand of a high-priority target computing resource group instance; performing system initialization deployment according to the inventory safety waterline, starting a monitoring task, and real-time monitoring of the use of target computing resources and system resources. The data is collected and converted into inventory information; the inventory information at least includes: the total number of groups and the number of idle groups corresponding to the target computing resource group; when a new user resource creation request is detected, the user resource creation request is parsed to obtain a target priority tag; if the target priority tag is a high priority tag, and the ratio of the number of idle groups to the total number of groups is judged based on the inventory information and is not greater than the threshold corresponding to the preset safety waterline, then the priority tags corresponding to all running target computing resource groups are traversed, the target computing resource group corresponding to the low priority tag is identified, and the instance release operation is performed; based on the resources after the instance release operation, the user resource creation request is responded to.

[0006] Furthermore, the training process of the above-mentioned safety water level prediction model is as follows: obtain the second computing power resource data; the second computing power resource data includes: historical resource demand data of the target computing power resource group and second priority label data; clean, filter, and format the second computing power resource data to obtain a labeled data set; based on the labeled data set, determine the characteristic value sequence corresponding to each computing power resource, and the multivariate time series corresponding to multiple computing power resources; the characteristic value sequence includes the time-axis-based demand average value corresponding to the computing power resource; based on the machine learning model, use the characteristic value sequences corresponding to all computing power resources as model input values, and the multivariate time series as model output values ​​for training to obtain the safety water level prediction model.

[0007] Furthermore, the above-mentioned steps of determining the characteristic value sequence corresponding to each computing power resource and the multivariate time series corresponding to multiple computing power resources based on the annotated data set include: extracting key features from the annotated data of high-priority tags in the annotated data set to obtain the characteristic value sequence corresponding to each computing power resource; processing the characteristic value sequence corresponding to each computing power resource through an autoregressive model to obtain a prediction sequence corresponding to each computing power resource; the prediction sequence includes: a sequence that reflects long-term trend characteristics, periodic characteristics and load pattern characteristics based on the time axis; integrating the prediction sequences corresponding to all computing power resources corresponding to the same time point on the time axis to obtain the multivariate time series corresponding to multiple computing power resources.

[0008] Furthermore, the above-mentioned steps of identifying the target computing power resource group corresponding to the low priority tag and performing the instance release operation include: obtaining the priority tag data corresponding to all the running target computing power resource groups, and forming the target computing power resource groups corresponding to all the low priority tags into a to-be-processed set; for each target computing power resource group in the to-be-processed set, obtaining the current resource operation data and preset weight value corresponding to the target computing power resource group; based on the current resource operation data and the preset weight value, calculating the resource weighted value corresponding to the target computing power resource group; judging whether the resource weighted value is less than a first threshold, the first threshold is the minimum weighted value required for the target computing power resource group to achieve effective resource utilization; if so, determining the instance corresponding to the target computing power resource group as the to-be-released instance; based on the size of the preset weight values ​​in all the to-be-released instances, arranging the to-be-released instances to obtain a target release instance sequence; releasing the instance resources of the target computing power resource group based on the target release instance sequence and the preset quantity, and updating the inventory information based on the released resources.

[0009] Furthermore, the step of calculating the resource weighted value corresponding to the target computing power resource group based on the current resource operation data and the preset weight value includes: calculating the resource weighted value corresponding to the target computing power resource group according to the following specified formula:

[0010] ;

[0011] Among them, A represents the resource weighted value corresponding to the target computing resource group; △t is the single continuous use duration of the target computing resource group; k is the idle impact assessment value corresponding to different application scenarios at the current time; is the weight corresponding to the i-th computing resource operation indicator in the current period; is the actual operation data corresponding to the i-th computing resource operation indicator; m is the total number of computing resource operation indicators.

[0012] Furthermore, after the above-mentioned step of updating the inventory information based on the released resources, it also includes: judging whether the updated inventory information is lower than or equal to the threshold value corresponding to the preset safety water level; if so, determining the resource index to be adjusted based on the updated inventory information, the resource index to be adjusted is an indicator of current resource shortage determined after comparing the updated inventory information with the inventory safety water level; calculating based on the preset weight value corresponding to the resource index to be adjusted, to obtain the updated preset weight value corresponding to each instance to be released; if not, continuing to execute the step of responding to the user resource creation request based on the resources after the instance release operation.

[0013] Furthermore, the above-mentioned step of responding to the user resource creation request based on the resources after the instance release operation includes: determining the configuration parameters corresponding to the target node scheduling resources based on the resource demand parameters carried in the user resource creation request and the dynamic resource scheduling algorithm of the maximum resource utilization of the resource weight, calling the target computing power resource group operation interface based on the configuration parameters, and creating the target computing power resource group instance; returning the target computing power resource group instance details and access information to the user, and updating the inventory information at the same time.

[0014] In a second aspect, the present application further provides a bare metal or container-based computing resource scheduling device, which includes multiple modules for executing the steps of the bare metal or container-based computing resource scheduling method described in the first aspect, and the multiple modules include a data acquisition module, a water level prediction module, a task monitoring module, a request parsing module, an instance release module and a request response module, wherein: the data acquisition module is used to acquire first computing power resource data; the first computing power resource data includes: real-time resource demand data, real-time resource supply data and first priority label data of the target computing power resource group; the water level prediction module is used to predict the first computing power resource data through a pre-trained safety water level prediction model to obtain the inventory safety water level corresponding to the current time period; the inventory safety water level is used to characterize the minimum value of the instance resource demand of the high-priority target computing power resource group; the task monitoring module, It is used to perform system initialization deployment according to the inventory safety waterline, start the monitoring task, monitor the usage data of the target computing resources and system resources in real time, and convert it into inventory information; the inventory information at least includes: the total number of groups and the number of idle groups corresponding to the target computing resource group; the request parsing module is used to parse the user resource creation request to obtain the target priority tag when monitoring the new user resource creation request; the instance release module is used to traverse the priority tags corresponding to each running target computing resource group if the target priority tag is a high priority tag, and the ratio of the number of idle groups to the total number of groups is judged based on the inventory information and is not greater than the threshold corresponding to the preset safety waterline, identify the target computing resource group corresponding to the low priority tag and perform the instance release operation; the request response module is used to respond to the user resource creation request based on the resources after the instance release operation.

[0015] In a third aspect, the present application also provides a bare metal or container-based computing resource scheduling system, comprising a processor and a memory, the memory storing computer executable instructions that can be executed by the processor, and the processor executing the computer executable instructions to implement the method described in the first aspect above.

[0016] In a fourth aspect, the present application further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method described in the first aspect above.

[0017] In the bare metal or container-based computing resource scheduling method, device and system provided in the present application, the real-time operation status of the target computing power resource group is predicted through a pre-trained water level prediction model to determine the current inventory safety water level. Then, based on the dynamic inventory safety water level and the actual operation status of the target computing power resource group instance in the current system, it is determined whether to trigger the mechanism of automatically releasing low-priority instances, thereby realizing flexible scheduling of the target computing power resource group instance resources, ensuring that there are sufficient reserved resources for key tasks, ensuring the stability of key tasks, and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A flowchart of a bare metal or container-based computing resource scheduling method provided in an embodiment of the present application;

[0020] Figure 2 A flowchart of a model training process in a bare metal or container-based computing resource scheduling method provided in an embodiment of the present application;

[0021] Figure 3 A flowchart of an instance release process in a bare metal or container-based computing resource scheduling method provided in an embodiment of the present application;

[0022] Figure 4 A structural block diagram of a bare metal or container-based computing resource scheduling device provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of the structure of a bare metal or container-based computing resource scheduling system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solution of the present application will be described clearly and completely in conjunction with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0025] In response to the technical problems in the prior art that computing resources are not used reasonably and the stability of key tasks is poor, the embodiments of the present application provide a computing resource scheduling method, device and system based on bare metal or containers. Through a pre-trained water level prediction model, the real-time operation status of the target computing resource group is predicted, the current inventory safety water level is determined, and then deployment is performed based on the water level, and the monitoring task is started. When a user request is monitored, it is determined whether the instance needs to be released based on the real-time inventory information and the corresponding threshold of the safety water level, so as to perform reasonable resource scheduling, ensure the stability of key tasks, and improve service quality and user experience.

[0026] To facilitate understanding of this embodiment, a bare metal or container-based computing resource scheduling method disclosed in an embodiment of the present application is first introduced in detail.

[0027] Figure 1 A flowchart of a bare metal or container-based computing resource scheduling method provided in an embodiment of the present application, the method comprising the following steps:

[0028] Step S102, obtaining first computing power resource data; the first computing power resource data includes: real-time resource demand data, real-time resource supply data and first priority tag data of the target computing power resource group;

[0029] In the embodiment of the present application, the target computing resource group includes container instance resources or bare metal server instance resources; the real-time resource demand data of the target computing resource group includes the current usage of the target computing resource group, specifically including the current number of instances, the CPU usage rate of each instance or the overall CPU usage, memory usage, disk IO, network traffic and other indicators real-time data. These real-time data are recorded in time series and can reflect the current situation of resource usage.

[0030] The real-time resource supply data of the above target computing resource group: including the target computing resource group resources that the cloud platform can currently provide. Including the container instance CPU quantity, container instance memory quantity, container instance disk capacity, container instance GPU card number, and the available quantity of each specification of bare metal instance. For example, the cloud platform can currently create a container instance with 16vCPU, 64GB memory, 100GB disk, 2 GPU cards, and 2 bare metal units of PSTD001 specification.

[0031] The above-mentioned target computing power resource group corresponds to the first priority label data: including the priority label of each resource in the resource group. The priority label can be a level label of the target computing power resource group created based on different business scenarios or user service priority levels. When resources are limited, high-priority resource groups can obtain resource allocation first to ensure the smooth execution of critical tasks. Priority labels can also be determined by the cloud platform based on user levels and resource business types. For example, for more important large customers, the resources created under their projects will be automatically tagged with high priority labels; for some users with critical businesses, users will label resources, and the platform will classify the labels.

[0032] Step S104, predicting the first computing power resource data by using a pre-trained safety waterline prediction model to obtain an inventory safety waterline corresponding to the current period; the inventory safety waterline is used to characterize the minimum value of the resource demand of the high-priority target computing power resource group instance;

[0033] The above-mentioned safety water level prediction model is based on the second computing power resource data (including the historical resource demand data of the target computing power resource group and the second priority label data) to perform data preprocessing and feature extraction to obtain the characteristic value sequence of each computing power resource and the multivariate time series corresponding to multiple computing power resources, and then the model is trained based on the characteristic value sequence of each computing power resource and the multivariate time series corresponding to multiple computing power resources.

[0034] Step S106, perform system initialization deployment according to the inventory safety waterline, start the monitoring task, monitor the usage data of the target computing resources and system resources in real time, and convert it into inventory information; the inventory information at least includes: the total number of groups and the number of idle groups corresponding to the target computing resource group;

[0035] The above inventory information may also include: the number of allocated groups corresponding to the target computing resource group, the priority distribution corresponding to the allocated group, and the user resource demand group data; wherein, the total number of target computing resource groups = the number of allocated groups + the number of free groups. The priority distribution corresponding to the allocated group includes the priority label corresponding to each allocated target computing resource group, which can be divided into two categories according to the priority level, one is the allocated target computing resource group with high priority, and the other is the allocated target computing resource group with low priority.

[0036] In an embodiment of the present application, the usage data of each container or bare metal instance resource and system resource is monitored in real time based on the monitoring task, and the usage data is converted and updated into the inventory information, so that the timing of releasing the low-priority target computing resource group can be decided in a timely manner according to the inventory information to ensure the demand of the high-priority target computing resource group.

[0037] Step S108, when a new user resource creation request is detected, the user resource creation request is parsed to obtain a target priority tag;

[0038] Step S110: If the target priority tag is a high priority tag, and the ratio of the number of free groups to the total number of groups is determined based on the inventory information and is not greater than the threshold corresponding to the preset safety waterline, then the priority tags corresponding to all running target computing resource groups are traversed, the target computing resource groups corresponding to the low priority tags are identified, and the instance release operation is performed;

[0039] The threshold corresponding to the preset safety waterline is a preset value. In the early stage of system deployment, the operation and maintenance personnel complete a series of basic configurations through the command line. The configuration of the inventory safety waterline is based on the comprehensive assessment of the historical resource demand data of all containers or bare metal instances, and the lowest inventory safety waterline is set. This threshold is used to determine when to trigger the mechanism of automatically releasing low-priority instances to ensure that there are sufficient resource reserves when high-priority demands arrive.

[0040] In this embodiment, the preset safety waterline corresponds to a threshold value, which is a preset percentage value to ensure that there are sufficient resource reserves when high-priority demands arrive.

[0041] The above judgment logic is: if the number of idle groups of the target computing power resource group / the total number of groups of the target computing power resource group >= the threshold corresponding to the preset safety waterline, it is higher than the safety waterline, and the target computing power resource can be directly created.

[0042] If the number of idle groups of the target computing power resource group / the total number of groups of the target computing power resource group < the threshold corresponding to the preset safety waterline, it is lower than the safety waterline. It is necessary to automatically release low-priority resources while creating the target computing power according to the distribution of target computing power resources of each priority level.

[0043] In other embodiments, if the target priority tag is a non-high priority tag, the number of creation requests, waiting response time, number of existing resources and continuous use time within the preset time at the current time of the user are obtained; based on the number of creation requests or the waiting response time, it is determined whether it is greater than the first preset parameter, and the first preset parameter is used to adjust the target priority tag to the minimum number of requests or the maximum value of the waiting response time corresponding to the high priority; if the number of creation requests is greater than the first preset parameter, the priority tag of the current creation request is converted to a high priority tag until the current task is completed and then changed back to the original low priority tag; if the number of creation requests is less than or equal to the first preset parameter, it is determined based on the number of existing resources and the continuous use time whether it is greater than the corresponding second preset parameter, and the second preset parameter is used to adjust the target priority tag to the minimum number of existing resources and the continuous use time corresponding to the high priority, where the continuous use time is the cumulative time that the user uses computing resources at the current time; if the number of existing resources and the continuous use time are greater than the corresponding second preset parameters, the priority tag of the current creation request is converted to a high priority tag until the current task is completed and then changed back to the original low priority tag. If the existing resource quantity and continuous usage duration are less than or equal to the corresponding second preset parameters, the priority tag of the current creation request is maintained as a low priority tag.

[0044] In this application, for low-priority creation requests, in order to avoid the loss of old customers and reduce user waiting time, the priority label of the creation request will be dynamically adjusted by comprehensively considering factors such as the frequency of initiating requests within the preset time, the waiting response time, the number of existing resources and the continuous usage time, so as to avoid ignoring the response time of low-priority tasks and the waiting time of old users in order to ensure the stability of critical tasks. The current task creation request will be temporarily changed to a high priority, and the need for resource release will be determined based on the number of resources in the user resource creation request and the preset safety waterline. If so, the resource release will be carried out according to the corresponding strategy of this application based on the running low-priority target computing power resource group, so as to reduce user waiting time and improve user experience.

[0045] Step S112: respond to the user resource creation request based on the resources after the instance release operation.

[0046] In the computing resource scheduling method based on bare metal or container provided in the embodiment of the present application, the first resource data collected in real time is input into the safety waterline prediction model to predict the corresponding safety waterline at the current time, so as to achieve the effect of real-time dynamic adjustment of the inventory safety waterline according to the actual operation of the computing resources and system resources of the target computing resource group in the current system. Then, based on the dynamic inventory safety waterline and the actual operation of the corresponding resources of the target computing resource group in the current system, it is determined whether to trigger the mechanism of automatically releasing the low-priority target computing resource group, so as to realize flexible scheduling of computing resources, ensure that there are sufficient resource reserves when high-priority demands arrive in different time periods (peak or non-peak), improve resource scheduling efficiency of resource utilization, ensure the stability of key tasks, and improve user experience.

[0047] The following is a detailed description of the training process of the safety water level prediction model:

[0048] See also Figure 2 As shown, the training process specifically includes the following steps:

[0049] Step S202, obtaining second computing power resource data; the second computing power resource data includes: historical resource demand data of the target computing power resource group and second priority tag data;

[0050] The historical resource demand data of the target computing resource group includes the usage of the target computing resource group in the past specified time period, including the number of instances, historical data of CPU usage of each instance or the total, memory usage, disk IO, network traffic and other indicators. These data are recorded in time series and can reflect the trend and pattern of resource usage. For example, on a cloud platform, how many container instances there are at a certain point in time, and the CPU usage and memory usage at that time.

[0051] The second priority tag data corresponding to the above-mentioned target computing resource group includes the priority tag corresponding to each computing resource; the priority tag is used to identify the importance level of different resource groups or services. When resources are limited, high-priority resource groups can obtain resource allocation first to ensure the smooth execution of critical tasks. Priority tags are determined by the cloud platform based on user level and resource business type. For example, for more important large customers, the resources created under their projects will be automatically tagged with high priority tags; for some users with critical business, users will tag resources, and the platform will classify the tags.

[0052] The second priority tag data may be the same as or different from the first priority tag data. Generally, the second priority tag data may be adaptively adjusted according to actual resource usage.

[0053] Step S204, cleaning, filtering, and formatting the second computing power resource data to obtain a labeled data set;

[0054] Step S206: Based on the annotated data set, determine the characteristic value sequence corresponding to each computing resource and the multivariate time series corresponding to the multiple computing resources; the characteristic value sequence includes the time axis-based demand average value corresponding to the computing resource; the specific implementation process of this step is as follows:

[0055] (1) Extract key features from the labeled data with high priority tags in the labeled data set to obtain a feature value sequence corresponding to each computing resource; the feature value sequence is the average demand value of a resource based on the time axis.

[0056] In this embodiment, after normalization processing is performed on the data categories corresponding to the annotated data, the normalized operation data is processed using a moving average algorithm for the same type of resources to obtain a time-based average value series corresponding to each resource. For example, the average value time series of CPU utilization and the average value time series of memory utilization, etc., from which the smooth short-term fluctuations of different resources and the trend characteristics of long-term resource demand can be seen.

[0057] (2) The characteristic value sequence corresponding to each computing resource is processed through an autoregressive model to obtain a prediction sequence corresponding to each computing resource; the prediction sequence includes: a sequence based on the time axis reflecting long-term trend characteristics, periodic characteristics, and load pattern characteristics;

[0058] In this embodiment, the autoregressive model is an ARIMA model. In other embodiments, the autoregressive model may also be a VAR model, an ARMA model, etc.

[0059] (3) The prediction sequences corresponding to all computing resources at the same time point on the time axis are integrated to obtain a multivariate time series corresponding to multiple computing resources.

[0060] For example, for the same time node, such as 10:00 on December 31, 2022, the characteristic values ​​of the prediction sequence corresponding to the resources such as CPU utilization, memory utilization, disk IO, network traffic, etc. at the time node are used as column vectors, and they are integrated as row vectors based on the time axis to obtain a multivariate time series.

[0061] Step S208: Based on the machine learning model, the characteristic value sequence corresponding to all computing resources is used as the model input value, and the multivariate time series is used as the model output value for training to obtain a safety water level prediction model.

[0062] In an embodiment of the present application, the standardized data corresponding to the high-priority label is first screened out, and then data mining is performed on it to extract the characteristic values ​​corresponding to the peak demand for computing resources and system resources of the target computing resource group in the high-priority task in different time periods, which are then used to train the prediction model. Deep learning is used to learn and simulate the complex mathematical relationship between the characteristic values ​​to obtain a safety water level prediction model.

[0063] The following describes in detail the implementation process of the step of "identifying the target computing resource group corresponding to the low priority tag and performing the instance release operation", see Figure 3 As shown:

[0064] Step S302, obtaining priority tag data corresponding to all running target computing resource groups, and grouping all target computing resource groups corresponding to low priority tags into a to-be-processed set;

[0065] Step S304: for each target computing resource group in the to-be-processed set, obtain the current resource operation data and preset weight value corresponding to the target computing resource group; calculate the resource weight value corresponding to the target computing resource group based on the current resource operation data and the preset weight value; determine whether the resource weight value is less than a first threshold, where the first threshold is the minimum weight value required for the target computing resource group to achieve effective resource utilization; if so, determine the instance corresponding to the target computing resource group as an instance to be released;

[0066] The above current resource operation data at least includes: CPU utilization, memory utilization, disk IO, network traffic, business request volume and timestamp; the preset weight value is based on the weight value corresponding to each indicator (CPU utilization, memory utilization, disk IO, network traffic, business request volume) in the current resource operation data at different times; for example, the CPU utilization of the container instance is 80%, and the corresponding preset weight value for the current period is 0.5, then the CPU utilization resource weight value is 0.4; the memory utilization is 90%, and the corresponding preset weight value for the current period is 0.7, then the memory utilization resource weight value is 0.63, and so on.

[0067] The above step of calculating the resource weighted value corresponding to the target computing power resource group based on the current resource operation data and the preset weight value includes: calculating the resource weighted value corresponding to the target computing power resource group according to the following specified formula:

[0068] ;

[0069] Among them, A represents the resource weighted value corresponding to the target computing resource group; △t is the single continuous use duration of the target computing resource group; k is the idle impact assessment value corresponding to different application scenarios at the current time; is the weight corresponding to the i-th computing resource operation indicator in the current period; is the actual operation data corresponding to the i-th computing resource operation indicator; m is the total number of computing resource operation indicators.

[0070] Step S306, arranging the instances to be released based on the preset weight values ​​of all the instances to be released to obtain a target release instance sequence;

[0071] Step S308, releasing instance resources of the target computing resource group based on the target release instance sequence and the preset quantity, and updating the inventory information based on the released resources.

[0072] In this embodiment, based on the factors of volatility of demand for different resources in different tenses, the computing resource utilization of the low-priority target computing resource group in the current period is evaluated with different emphasis dimensions, and the corresponding computing resource weight values ​​are calculated based on the preset weights that are dynamically adjusted in different tenses, and then the underutilized target computing resource group is determined based on the resource weight values ​​and used as the instance resource of the target computing resource group to be released. Based on the corresponding dynamic weights of each resource operation indicator, the demand changes for different resources in different tenses can be known, and the resource release of all target computing resource groups to be released is reconfirmed through the size of the dynamic preset weight values ​​to ensure that the target computing resource group with the least amount of resources is effectively released in the current period to meet the demand of high-level instances for the instance resources of the target computing resource group, and to minimize the preemption of resources of the low-level target computing resource group.

[0073] In another embodiment, after the above step of updating the inventory information based on the released resources, the method further includes:

[0074] Step S310, determining whether the updated inventory information is lower than or equal to a threshold corresponding to a preset safety waterline;

[0075] Step S312: if yes, determine the resource index to be adjusted based on the updated inventory information, where the resource index to be adjusted is an index of current resource shortage determined after comparing the updated inventory information with the inventory safety waterline; calculate based on the preset weight value corresponding to the resource index to be adjusted to obtain the updated preset weight value corresponding to each instance to be released;

[0076] If not, continue to execute step S112, a step of responding to the user resource creation request based on the resources after the instance release operation.

[0077] Furthermore, the step of responding to a user resource creation request based on the resources after the instance release operation includes:

[0078] (1) Based on the resource demand parameters carried in the user resource creation request and the dynamic resource scheduling algorithm with the maximum resource utilization rate of resource weight, the corresponding configuration parameters of the target node scheduling resources are determined, and the target computing resource group operation interface is called based on the configuration parameters to create the target computing resource group instance;

[0079] In this embodiment, based on the specific resource requirements such as CPU, memory, disk size, etc. in the user creation request, the relevant parameters of the resources currently running on the cloud platform, and the relevant parameters corresponding to the safety waterline, it is brought into the dynamic resource scheduling algorithm with the maximum resource utilization of resource weight for processing, and the corresponding configuration parameters of the target node scheduling resources are obtained. The dynamic resource scheduling algorithm based on the maximum resource utilization of resource weight in this application adopts the Filter Scheduler algorithm. In other embodiments, it can also be a Load Balancing algorithm, a Priority Scheduling algorithm, etc.

[0080] (2) Return the instance details and access information of the target computing resource group to the user, and update the inventory information at the same time.

[0081] In this embodiment, the load of the container or bare metal server instance is predicted based on the high priority tag, and whether the container or bare metal instance needs to be scaled based on the dynamically adjusted inventory safety waterline while ensuring high-quality service, ensuring that the scaling timing is not too early to waste resources, nor too late to reduce service quality. Then, based on the combination of the container or bare metal load prediction results and the dynamic resource scheduling algorithm, the scheduling of the container or bare metal is guided to improve resource utilization.

[0082] In the computing resource scheduling method based on bare metal or container provided in the embodiment of the present application, firstly, the historical resource demand data of the target computing resource group is extracted based on the high priority label, and the corresponding characteristic values ​​of the resource demand in different time periods are used to train the prediction model using the machine model, and the complex mathematical relationship between the characteristic values ​​is learned and simulated using deep learning to obtain the safety water level prediction model. Then, the safety water level prediction model is used to predict based on the real-time collected target computing resource group operation data, so as to achieve the effect of real-time dynamic adjustment of the inventory safety water level according to the actual operation of the target computing resource group in the current system. Then, based on the dynamic inventory safety water level and the actual operation of the target computing resource group in the current system, it is determined whether to trigger the mechanism of automatically releasing low-priority instances, so as to realize the flexible scheduling of the target computing resource group, ensure that the scaling time is not too early to waste resources, and not too late to reduce the service quality, and also ensure that there are sufficient resource reserves when high-priority demands arrive in different time periods (peak or non-peak), so as to ensure the stability of key tasks and improve the user experience. Finally, the dynamic resource scheduling algorithm is combined to guide the target computing resource group to perform resource scheduling and improve the utilization of resources.

[0083] Furthermore, based on the factors of volatility of demand for different resources in different time states, the resource utilization of the low-priority target computing resource group in the current period is evaluated with different emphasis dimensions, and the corresponding resource weight values ​​are calculated based on the preset weights that are dynamically adjusted in different time states, and the underutilized target computing resource group is determined based on the resource weight values ​​and used as the instance to be released. Based on the corresponding dynamic weights of each resource operation indicator, the demand changes for different resources in different time states can be known, and the resource release of all instances to be released is reconfirmed through the size of the dynamic preset weight value to ensure that the minimum instance resources are effectively released in the current period to meet the resource needs of high-level instances and minimize the preemption of low-level instance resources.

[0084] Based on the above method embodiment, the present application embodiment also provides a computing resource scheduling device based on bare metal or container, see Figure 4 As shown, the device includes multiple modules for executing the steps of the bare metal or container-based computing resource scheduling method described in the above embodiment, and the multiple modules include a data acquisition module 402, a water level prediction module 404, a task monitoring module 406, a request parsing module 408, an instance release module 410 and a request response module 412, wherein:

[0085] The data acquisition module 402 is used to obtain the first computing power resource data; the first computing power resource data includes: the real-time resource demand data of the target computing power resource group, the real-time resource supply data and the first priority label data; the water level prediction module 404 is used to predict the first computing power resource data through the pre-trained water level prediction model to obtain the inventory safety water level corresponding to the current period; the inventory safety water level is used to characterize the minimum value of the resource demand of the high-priority target computing power resource group instance; the task monitoring module 406 is used to perform system initialization deployment according to the inventory safety water level, start the monitoring task, so as to monitor the usage data of the target computing power resources and system resources in real time, and convert it into inventory information; the inventory information to At least include: the total number of groups and the number of idle groups corresponding to the target computing power resource group; a request parsing module 408, which is used to parse the user resource creation request to obtain the target priority tag when monitoring a new user resource creation request; an instance release module 410, which is used to traverse the priority tags corresponding to all running target computing power resource groups if the target priority tag is a high priority tag, and the ratio of the number of idle groups to the total number of groups is judged based on inventory information and is not greater than the threshold corresponding to the preset safety waterline, identify the target computing power resource group corresponding to the low priority tag and perform an instance release operation; a request response module 412, which is used to respond to the user resource creation request based on the resources after the instance release operation.

[0086] Furthermore, the above-mentioned device also includes: a model training module, which is used to execute the following training process of the safety water level prediction model: obtaining second computing power resource data; the second computing power resource data includes: historical resource demand data of the target computing power resource group and second priority label data; cleaning, filtering, and formatting the second computing power resource data to obtain an annotated data set; based on the annotated data set, determining the characteristic value sequence corresponding to each computing power resource, and the multivariate time series corresponding to multiple computing power resources; the characteristic value sequence includes the time-axis-based demand average value corresponding to the computing power resource; based on the machine learning model, the characteristic value sequences corresponding to all computing power resources are used as model input values, and the multivariate time series are used as model output values ​​for training to obtain the safety water level prediction model.

[0087] Furthermore, the above-mentioned model training module is used to extract key features of the labeled data with high priority labels in the labeled data set to obtain the characteristic value sequence corresponding to each computing power resource; the characteristic value sequence corresponding to each computing power resource is processed through the autoregressive model to obtain the prediction sequence corresponding to each computing power resource; the prediction sequence includes: a sequence that reflects long-term trend characteristics, periodic characteristics and load pattern characteristics based on the time axis; the prediction sequences corresponding to all computing power resources corresponding to the same time point on the time axis are integrated to obtain a multivariate time series corresponding to multiple computing power resources.

[0088] Furthermore, the instance release module 410 is used to: obtain priority label data corresponding to all target computing power resource groups that are in operation, and group the target computing power resource groups corresponding to all low-priority labels into a to-be-processed set; for each target computing power resource group in the to-be-processed set, obtain the current resource operation data and preset weight value corresponding to the target computing power resource group; based on the current resource operation data and the preset weight value, calculate the resource weight value corresponding to the target computing power resource group; determine whether the resource weight value is less than a first threshold, and the first threshold is the minimum weight value required for the target computing power resource group to achieve effective resource utilization; if so, determine the instance corresponding to the target computing power resource group as the to-be-released instance; based on the size of the preset weight values ​​in all to-be-released instances, arrange the to-be-released instances to obtain a target release instance sequence; release the instance resources of the target computing power resource group based on the target release instance sequence and the preset quantity, and update the inventory information based on the released resources.

[0089] Furthermore, the example release module 410 is used to calculate the resource weight value corresponding to the target computing power resource group according to the following specified formula:

[0090] ;

[0091] Among them, A represents the resource weighted value corresponding to the target computing resource group; △t is the single continuous use duration of the target computing resource group; k is the idle impact assessment value corresponding to different application scenarios at the current time; is the weight corresponding to the i-th computing resource operation indicator in the current period; is the actual operation data corresponding to the i-th computing resource operation indicator; m is the total number of computing resource operation indicators.

[0092] Furthermore, the instance release module 410 is used to determine whether the updated inventory information is lower than or equal to the threshold corresponding to the preset safety waterline; if so, determine the resource indicator to be adjusted based on the updated inventory information, and the resource indicator to be adjusted is an indicator of current resource shortage determined after comparing the updated inventory information with the inventory safety waterline; calculate based on the preset weight value corresponding to the resource indicator to be adjusted, and obtain the updated preset weight value corresponding to each instance to be released; if not, continue to execute the step of responding to the user resource creation request based on the resources after the instance release operation.

[0093] Furthermore, the above-mentioned request response module 412 is used to: determine the configuration parameters corresponding to the target node scheduling resources based on the resource demand parameters carried in the user resource creation request and the dynamic resource scheduling algorithm of resource weight maximum resource utilization, call the target computing power resource group operation interface based on the configuration parameters, and create a target computing power resource group instance; return the target computing power resource group instance details and access information to the user, and update the inventory information at the same time.

[0094] The device provided in the embodiment of the present application has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the embodiment of the device, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0095] The present application also provides a bare metal or container-based computing resource scheduling system. Figure 5 As shown, it is a schematic diagram of the structure of the system, wherein the system includes a processor 51 and a memory 50, the memory 50 stores computer executable instructions that can be executed by the processor 51, and the processor 51 executes the computer executable instructions to implement the above method.

[0096] exist Figure 5 In the illustrated embodiment, the system further includes a bus 52 and a communication interface 53 , wherein the processor 51 , the communication interface 53 and the memory 50 are connected via the bus 52 .

[0097] Among them, the memory 50 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 53 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 52 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 52 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0098] The processor 51 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 51. The above processor 51 can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor 51 reads the information in the memory and completes the steps of the method of the above embodiment in combination with its hardware.

[0099] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above method. The specific implementation can be found in the aforementioned method embodiment, which will not be repeated here.

[0100] The computer program products of the methods, devices, and systems provided in the embodiments of the present application include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0101] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0102] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0103] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0104] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A bare metal or container-based computing resource scheduling method, characterized in that: The method comprises: Acquire first computing power resource data; the first computing power resource data includes: real-time resource demand data, real-time resource supply data and first priority label data of the target computing power resource group; The first computing power resource data is predicted by a pre-trained safety waterline prediction model to obtain an inventory safety waterline corresponding to the current period; the inventory safety waterline is used to characterize the minimum value of the resource demand of the high-priority target computing power resource group instance; Perform system initialization deployment according to the inventory safety waterline, start the monitoring task, monitor the usage data of the target computing resources and system resources in real time, and convert it into inventory information; the inventory information at least includes: the total number of groups and the number of idle groups corresponding to the target computing resource group; When a new user resource creation request is detected, the user resource creation request is parsed to obtain a target priority tag; If the target priority tag is a high priority tag, and the ratio of the number of idle groups to the total number of groups is determined based on the inventory information and is not greater than the threshold corresponding to the preset safety waterline, then the priority tags corresponding to all running target computing resource groups are traversed, the target computing resource groups corresponding to the low priority tags are identified, and the instance release operation is performed; Respond to user resource creation requests based on the resources after instance release operations; The steps of identifying the target computing power resource group corresponding to the low priority tag and performing the instance release operation include: obtaining the priority tag data corresponding to all the running target computing power resource groups, and forming the target computing power resource groups corresponding to all the low priority tags into a to-be-processed set; for each target computing power resource group in the to-be-processed set, obtaining the current resource operation data and preset weight value corresponding to the target computing power resource group; based on the current resource operation data and the preset weight value, calculating the resource weight value corresponding to the target computing power resource group; judging whether the resource weight value is less than a first threshold value, which is the minimum weight value required for the target computing power resource group to achieve effective resource utilization; if so, determining the instance corresponding to the target computing power resource group as the to-be-released instance; based on the size of the preset weight value in all the to-be-released instances, arranging the to-be-released instances to obtain a target release instance sequence; releasing the instance resources of the target computing power resource group based on the target release instance sequence and the preset quantity, and updating the inventory information based on the released resources.

2. The method according to claim 1, characterized in that The training process of the safety water level prediction model is as follows: Acquire second computing power resource data; the second computing power resource data includes: historical resource demand data of the target computing power resource group and second priority tag data; Cleaning, filtering, and formatting the second computing power resource data to obtain a labeled data set; Based on the annotated data set, determine the characteristic value sequence corresponding to each computing resource and the multivariate time series corresponding to the multiple computing resources; the characteristic value sequence includes the time axis-based demand average value corresponding to the computing resource; Based on the machine learning model, the characteristic value sequence corresponding to all computing resources is used as the model input value, and the multivariate time series is used as the model output value for training to obtain the safety water level prediction model.

3. The method according to claim 2, characterized in that Based on the annotated data set, the steps of determining the characteristic value sequence corresponding to each computing resource and the multivariate time series corresponding to multiple computing resources include: Extract key features of the labeled data of the high-priority tags in the labeled data set to obtain a feature value sequence corresponding to each computing resource; The characteristic value sequence corresponding to each computing resource is processed by an autoregressive model to obtain a prediction sequence corresponding to each computing resource; the prediction sequence includes: a sequence based on a time axis reflecting long-term trend characteristics, periodic characteristics, and load pattern characteristics; The prediction sequences corresponding to all computing resources at the same time point on the time axis are integrated to obtain a multivariate time series corresponding to multiple computing resources.

4. The method according to claim 1, characterized in that: The step of calculating the resource weighted value corresponding to the target computing power resource group based on the current resource operation data and the preset weight value includes: Calculate the resource weighted value corresponding to the target computing power resource group according to the following specified formula: ； Wherein, A represents the resource weighted value corresponding to the target computing resource group; △t is the single continuous use duration of the target computing resource group; k is the idle impact assessment value corresponding to different application scenarios at the current time; is the weight corresponding to the i-th computing resource operation indicator in the current period; is the actual operation data corresponding to the i-th computing resource operation indicator; m is the total number of computing resource operation indicators.

5. The method according to claim 1, characterized in that After the step of updating the inventory information based on the released resources, the method further includes: Determine whether the updated inventory information is lower than or equal to the threshold corresponding to the preset safety waterline; If yes, determine the resource index to be adjusted based on the updated inventory information, where the resource index to be adjusted is an index of current resource shortage determined after comparing the updated inventory information with the inventory safety waterline; calculate based on the preset weight value corresponding to the resource index to be adjusted to obtain the updated preset weight value corresponding to each instance to be released; If not, continue to execute the step of responding to the user resource creation request based on the resources after the instance release operation.

6. The method according to claim 5, characterized in that The steps of responding to the user's resource creation request based on the resources after the instance release operation include: Based on the resource demand parameters carried in the user resource creation request and the dynamic resource scheduling algorithm of the maximum resource utilization of the resource weight, the configuration parameters corresponding to the target node scheduling resources are determined, and the target computing resource group operation interface is called based on the configuration parameters to create a target computing resource group instance; The target computing resource group instance details and access information are returned to the user, and the inventory information is updated at the same time.

7. A bare metal or container-based computing resource scheduling device, characterized in that: The device includes multiple modules for executing the steps of the bare metal or container-based computing resource scheduling method according to any one of claims 1 to 6, wherein the multiple modules include a data acquisition module, a watermark prediction module, a task monitoring module, a request parsing module, an instance release module, and a request response module, wherein: The data acquisition module is used to acquire first computing power resource data; the first computing power resource data includes: real-time resource demand data, real-time resource supply data and first priority label data of the target computing power resource group; The water level prediction module is used to predict the first computing power resource data through a pre-trained safety water level prediction model to obtain the inventory safety water level corresponding to the current period; the inventory safety water level is used to represent the minimum value of the resource demand of the high-priority target computing power resource group instance; The task monitoring module is used to perform system initialization deployment according to the inventory safety waterline, start the monitoring task, monitor the usage data of the target computing resources and system resources in real time, and convert it into inventory information; the inventory information at least includes: the total number of groups and the number of idle groups corresponding to the target computing resource group; The request parsing module is used to parse the user resource creation request to obtain a target priority tag when a new user resource creation request is detected; The instance release module is used to traverse the priority labels corresponding to each running target computing resource group if the target priority label is a high priority label and the ratio of the number of free groups to the total number of groups is determined based on the inventory information and is not greater than the threshold corresponding to the preset safety waterline, identify the target computing resource group corresponding to the low priority label and perform the instance release operation; The request response module is used to respond to the user resource creation request based on the resources after the instance release operation; The instance release module is also used to obtain priority label data corresponding to all running target computing power resource groups, and group the target computing power resource groups corresponding to all low-priority labels into a to-be-processed set; for each target computing power resource group in the to-be-processed set, obtain the current resource operation data and preset weight value corresponding to the target computing power resource group; based on the current resource operation data and the preset weight value, calculate the resource weight value corresponding to the target computing power resource group; determine whether the resource weight value is less than a first threshold, and the first threshold is the minimum weight value required for the target computing power resource group to achieve effective resource utilization; if so, determine the instance corresponding to the target computing power resource group as the to-be-released instance; based on the size of the preset weight value in all to-be-released instances, arrange the to-be-released instances to obtain a target release instance sequence; release the target computing power resource group instance resources based on the target release instance sequence and the preset number, and update the inventory information based on the released resources.

8. A bare metal or container-based computing resource scheduling system, characterized in that: The system includes a processor and a memory, the memory storing computer executable instructions that can be executed by the processor, and the processor executing the computer executable instructions to implement the bare metal or container-based computing resource scheduling method described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to implement the bare metal or container-based computing resource scheduling method described in any one of claims 1 to 6.

Images