Service scheduling method, device, equipment, medium and program product

Abstract

The invention provides a service scheduling method which can be applied to the technical field of cloud computing. The service scheduling method comprises the steps that in response to a target service request, the service type of the target service request is input into a demand prediction model for prediction, a prediction result is obtained, the prediction result comprises resources needed when the target service request is processed, and the resources needed when the target service request is processed are stored in the demand prediction model; the demand prediction model is obtained by pre-training a preprocessing data set based on a linear regression algorithm; a service scheduling decision is output based on the prediction result and a pre-constructed service scheduling model, and the service scheduling model is pre-constructed based on a double-Q learning algorithm; and scheduling the service according to the service scheduling decision. The invention further provides a service scheduling device, equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present application relates to the field of cloud computing, specifically to service scheduling, and more specifically to a service scheduling method, apparatus, device, medium, and program product. Background Art

[0002] With the rapid development of information technology, cloud services play a vital role in numerous fields. They provide users with various resources via the internet, enabling efficient resource sharing. Cloud network resources are occupied by service requests, resulting in an uneven distribution of resources. Many services have both preemptive and non-preemptive connections to resources, which can prevent some services from accessing required resources. Therefore, during batch service scheduling, it is crucial to ensure that services respond within a reasonable timeframe and that the load on each host or server remains within a reasonable range. Summary of the Invention

[0003] In view of the above problems, the present application provides a service scheduling method, apparatus, device, medium and program product for improving the quality and efficiency of service scheduling.

[0004] According to the first aspect of the present application, a service scheduling method is provided, comprising: in response to a target service request, inputting the service type of the target service request into a demand prediction model for prediction to obtain a prediction result, wherein the prediction result includes the resources required to process the target service request, and the demand prediction model is based on a linear regression algorithm and is pre-trained through a preprocessed data set; based on the prediction result and a pre-built service scheduling model, outputting a service scheduling decision, wherein the service scheduling model is pre-built based on a dual-Q learning algorithm; and scheduling the service according to the service scheduling decision.

[0005] According to an embodiment of the present application, based on the prediction results and a pre-built service scheduling model, a service scheduling decision is output, including: setting the state space, action space and reward function of the dual-Q learning algorithm, setting the state space to the prediction results, setting the action space to the scheduling operation of the target service request, and setting the reward function to a comprehensive reward value, wherein the comprehensive reward value includes resource utilization and load balancing; iterating based on the dual-Q learning algorithm, simulating multiple scheduling schemes until the termination condition is met, and outputting the optimal scheduling scheme among the multiple scheduling schemes.

[0006] According to an embodiment of the present application, the steps of setting the reward function include: calculating resource utilization and calculating the load balancing degree based on the resource utilization; setting a weight factor, which is used to adjust the proportional relationship between resource utilization and load balancing degree; and setting the reward function of the service scheduling model based on the weight factor, the resource utilization and the load balancing degree.

[0007] According to an embodiment of the present application, satisfying the termination condition includes: the calculated resource utilization satisfies a preset first threshold; and the calculated average load of the physical resource satisfies a preset second threshold.

[0008] According to an embodiment of the present application, the resources required for processing the target service request include the time required for the service and the computing resources required for the service; the demand prediction model includes a service time prediction model and a computing resource prediction model, wherein the service time prediction model is used to predict the time required for the service, and the computing resource prediction model is used to predict the computing resources required for the service.

[0009] According to an embodiment of the present application, defining the state space as the prediction result includes: setting a time grouping threshold and a time grouping label; setting a resource grouping threshold and a resource grouping label; grouping the time required for the service according to the time grouping threshold to obtain a corresponding time grouping label; grouping the computing resources required for the service according to the resource grouping threshold to obtain a corresponding resource grouping label; setting the corresponding time grouping label and the corresponding resource grouping label to the state space.

[0010] According to an embodiment of the present application, pre-training the demand forecasting model based on a linear regression algorithm and through a pre-processed data set includes: obtaining the operating history data of the server resources, and pre-processing the historical data to obtain a pre-processed data set; and fitting a linear regression algorithm based on the pre-processed data set to establish a demand forecasting model.

[0011] The second aspect of the present application provides a service scheduling device, including: a resource prediction module, which is used to respond to a target service request, input the service type of the target service request into a demand prediction model for prediction, and obtain a prediction result, wherein the prediction result includes the resources required to process the target service request, and the demand prediction model is based on a linear regression algorithm and is pre-trained through a preprocessed data set; a strategy determination module, which is used to output a service scheduling decision based on the prediction result and a pre-built service scheduling model, wherein the service scheduling model is pre-built based on a dual Q learning algorithm; and a scheduling module, which is used to schedule the service according to the service scheduling decision.

[0012] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0013] The fourth aspect of the present application further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.

[0014] The fifth aspect of the present application further provides a computer program product, comprising a computer program or instructions, which implement the steps of the above method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above contents and other objects, features and advantages of the present application will become more apparent through the following description of the embodiments of the present application with reference to the accompanying drawings, in which:

[0016] Figure 1 Schematically illustrates an application scenario diagram of a service scheduling method, apparatus, device, medium, and program product according to an embodiment of the present application;

[0017] Figure 2 The following schematically shows a flow chart of a service scheduling method according to an embodiment of the present application;

[0018] Figure 3 The following schematically shows a flow chart for obtaining prediction results according to an embodiment of the present application;

[0019] Figure 4 Schematically shows a flow chart of an output scheduling solution according to an embodiment of the present application;

[0020] Figure 5 The following schematically shows a flow chart of an overall output scheduling solution according to an embodiment of the present application;

[0021] Figure 6 Schematically shows a flow chart of setting a state space according to an embodiment of the present application;

[0022] Figure 7 A structural block diagram of a service scheduling device according to an embodiment of the present application is schematically shown; and

[0023] Figure 8 The block diagram schematically shows an electronic device suitable for implementing the service scheduling method according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present application. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present application.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit this application. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0028] An embodiment of the present application provides a service scheduling method, which includes: in response to a target service request, inputting the service type of the target service request into a demand prediction model for prediction to obtain a prediction result, wherein the prediction result includes the resources required to process the target service request, and the demand prediction model is based on a linear regression algorithm and is pre-trained through a preprocessed data set; based on the prediction result and a pre-built service scheduling model, outputting a service scheduling decision, wherein the service scheduling model is pre-built based on a dual Q learning algorithm; and scheduling the service according to the service scheduling decision.

[0029] The service scheduling method provided by the embodiments of the present disclosure can give full play to the advantages of the two algorithms. The linear model calculation process is relatively simple. On the basis of the resource prediction it quickly provides, the dual-Q learning algorithm continuously adapts to the complex environment of service scheduling to make dynamic decisions, thereby outputting the optimal service scheduling solution. This method can improve the accuracy of scheduling decisions while ensuring service response time. It is suitable for complex scenarios that require efficient and accurate predictions. It can balance cloud service loads and ensure the quality of cloud services. In addition, this method can reduce the impact of manual intervention and subjective judgment. The method can automatically learn and adapt to changes in the system, continuously optimize scheduling strategies, and improve scheduling efficiency.

[0030] Figure 1 The application scenario diagram of the service scheduling method, apparatus, device, medium and program product according to the embodiments of the present application is schematically shown.

[0031] like Figure 1 As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or optical fiber cables.

[0032] A user may use a first terminal device 101, a second terminal device 102, or a third terminal device 103 to interact with a server 105 via a network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, or the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0033] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0034] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal devices.

[0035] It should be noted that the service scheduling method provided in the embodiments of the present application can generally be executed by the server 105. Accordingly, the service scheduling device provided in the embodiments of the present application can generally be set in the server 105. The service scheduling method provided in the embodiments of the present application can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the service scheduling device provided in the embodiments of the present application can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0036] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0037] The following will be based on Figure 1 The scene described by Figures 2 to 6 The service scheduling method according to the embodiment of the present application is described in detail.

[0038] Figure 2 The flowchart of the service scheduling method according to the embodiment of the present application is schematically shown.

[0039] like Figure 2 As shown, the service scheduling method of this embodiment includes operations S210 to S230, and the service scheduling method can be executed by the server 105.

[0040] In operation S210 , in response to a target service request, the service type of the target service request is input into a demand prediction model for prediction to obtain a prediction result.

[0041] In an embodiment of the present application, the prediction result includes the resources required to process the target service request. The demand prediction model is based on a linear regression algorithm and is pre-trained using a pre-processed data set.

[0042] The linear regression algorithm is a supervised machine learning algorithm used to predict continuous output variables based on input variables. Its basic concept is to minimize the error between the predicted and actual values by building a linear model. The linear regression algorithm's computational process is relatively simple, enabling rapid prediction and model fitting. This maintains high computational efficiency while meeting the real-time requirements of service scheduling.

[0043] In operation S220 , a service scheduling decision is output based on the prediction result and a pre-built service scheduling model.

[0044] The Q-learning algorithm is an algorithm in the field of reinforcement learning, primarily used to solve the problem of estimating optimal policies. Q-learning is a classic model-free reinforcement learning algorithm that selects the optimal action by estimating the Q-value of a state-action pair (i.e., the expected reward obtained from taking a certain action in a certain state). However, Q-learning can suffer from overestimation in certain situations, overestimating the Q-value of certain actions, thus affecting the optimization effect of service scheduling policies. To alleviate this problem, the dual Q-learning algorithm was proposed.

[0045] The Double Q-learning (Double Q-Learning) algorithm introduces two independent Q-value estimators, or Q-tables, to calculate the Q-values for the current state and the next state, respectively. Specifically, Double Q-learning selects a Q-function at each time step to determine the action and estimates the action based on another Q-function. This approach effectively reduces Q-value overestimation and improves the robustness and stability of the algorithm. The application of the Double Q-Learning algorithm in service scheduling is primarily reflected in its adaptability to environmental changes, demonstrating improved adaptive performance and scheduling effectiveness under varying conditions.

[0046] In an embodiment of the present application, the service scheduling model is pre-built based on a double-Q learning algorithm.

[0047] In operation S230 , the service is scheduled according to the service scheduling decision.

[0048] According to the service scheduling method of the disclosed embodiment, a linear regression model is used to perform feature analysis to quickly predict the time and resources required for a service, which is then used as the state set for the Dual Q-Learning model. Then, leveraging the adaptive learning, model-free, and long-term optimization capabilities of Dual Q-Learning, a service scheduling model with an optimal service scheduling strategy is constructed based on the Dual Q-Learning algorithm. This further balances the cloud service load and maximizes service stability and efficiency.

[0049] Figure 3 The flowchart for obtaining prediction results according to an embodiment of the present application is schematically shown.

[0050] like Figure 3 As shown, this embodiment is one of the embodiments of operation S210, including operations S211 to S214.

[0051] In operation S211 , operation history data of server resources is acquired.

[0052] In an embodiment of the present application, historical data related to service scheduling is collected, including information such as the time, type, and required resource amount of the service request.

[0053] Optionally, historical data can be obtained for the past year. The more data obtained, the more conducive it is to model training. The specific time range for obtaining data can be determined according to actual needs and is not limited here.

[0054] In operation S212 , the historical data is cleaned to obtain a preprocessed data set.

[0055] In the embodiments of this application, the collected historical data is cleaned to remove outliers and missing values, and feature information useful for prediction, such as time features and service type features, is extracted. The cleaned historical data is vectorized and organized into a vector matrix to obtain a preprocessed dataset.

[0056] It should be noted that when performing demand forecasting, the extracted feature information is closely related to the forecast results. Specifically, if the goal is to predict the processing time of the service through the demand forecasting model, then it is necessary to extract time-related feature information, such as historical processing time, response time, etc. On the contrary, if the goal is to predict the computing resources required during the service processing, then it is necessary to extract feature information related to computing resources, such as the server's CPU usage, memory usage, network bandwidth, etc. In short, the extraction of feature information is determined according to the specific forecasting goals to ensure that the model can accurately output the required forecast results. The extracted feature information can be determined according to actual needs and is not limited here.

[0057] In operation S213 , a linear regression algorithm is fitted based on the preprocessed data set to establish a demand forecasting model.

[0058] The multiple linear regression algorithm is used to predict a dependent variable through multiple independent variables. Its calculation process is relatively simple, as shown in formula (1):

[0059]

[0060] Where y is the dependent variable, , ,…, is the independent variable, , ,…, is the regression coefficient, is the error term.

[0061] In an embodiment of the present application, the obtained preprocessed data set is segmented to obtain a training data set and a test data set. Optionally, the training data set and the test data set are segmented in a ratio of 7:3. The specific segmentation ratio can be determined according to actual needs and is not limited here. The training data set is input into a multivariate linear regression algorithm for training to obtain a demand forecasting model; the model is evaluated using the test data set, and indicators such as prediction error and determination coefficient are calculated to measure the model's predictive performance. The trained linear regression model is then applied to actual service scheduling scenarios.

[0062] In an embodiment of the present application, the demand prediction model includes a service time prediction model and a computing resource prediction model, wherein the service time prediction model is used to predict the time required for the service, that is, the task duration, and the computing resource prediction model is used to predict the computing resources required for the service. For example, computing resources may include CPU, memory, storage, and network bandwidth, etc.

[0063] It should be noted that both the service time prediction model and the computing resource prediction model are models obtained by fitting, training, and evaluating a linear regression algorithm based on a preprocessed data set.

[0064] In operation S214, the service type of the target service request is input into a demand prediction model for prediction to obtain a prediction result.

[0065] In the embodiments of this application, the service type of the target service request includes, for example, AI training, web services, and data processing. The service type of the target service request is input into the demand forecasting model to obtain a forecast result. The resource predicted by the computing resource forecasting model in this application is memory. The forecast result output in this application is the service time and memory requirements required for the service, for example, 5.5 hours for the service time and 9.0 GB for the memory. The service type can be determined based on actual needs and is not limited here.

[0066] In this application, the demand forecasting model can quickly predict key parameters such as the time and resources required for service, providing scheduling prior experience for the dual Q-Learning algorithm.

[0067] Figure 4 The flowchart of the output scheduling solution according to the embodiment of the present application is schematically shown.

[0068] like Figure 4 As shown, this embodiment is one of the embodiments of operation S220, including operations S221 to S225.

[0069] In operation S221, input and output are defined, and two Q tables are initialized.

[0070] The Dual Q-Learning algorithm uses two independent Q-tables, commonly referred to as Q1 and Q2. These tables store the estimated values for different state-action pairs. These two Q-tables correspond to two Q-update functions. During each execution cycle, the agent first observes the current state and then selects and executes the appropriate action based on the updated Q-tables. The Dual Q-Learning model uses these two Q-tables to enhance itself during the update phase, thereby improving the accuracy of action predictions.

[0071] In an embodiment of the present application, the dual-Q learning algorithm, the initialization operation includes:

[0072] Define input, let V={ , ,…, }, where V is The set of m is the cloud network The total number of each Each has its own resources (such as CPU and RAM), with different definitions of usage costs and computing power; The i-th virtual machine is shown.

[0073] Let T = { , ,…, }, where T is the set of assigned tasks and n is the number of independent tasks executed on the VM (n). Represents the jth task. Each task submission includes the number of instructions, required memory, required CPU, etc.

[0074] Define outputs, optimize scheduling, map n tasks and m virtual machines (resources) to minimize resource (CPU and memory) utilization and improve load balancing of resource utilization.

[0075] Initialize two Q tables and create two independent Q tables ( and ), each table is the size of the number of states times the number of actions. These Q-tables will be used to estimate the Q-values respectively. The values in the Q-tables are initialized to 0.

[0076] Define states and actions. The state can be set to the predicted result, that is, the time and resources required for the service. The action can be expressed as a service request for allocation, scheduling or management.

[0077] In an embodiment of the present application, the service time and resource requirements of the service predicted by the demand forecasting model are used as the state set of the dual Q-Learning model to guide the action selection of the dual Q-Learning model in a specific state.

[0078] Optionally, the prediction results can be grouped before setting them into the state space. Figure 6Schematically shows a flow chart of setting a state space according to an embodiment of the present application;

[0079] like Figure 6 As shown, this embodiment is one of the embodiments of operation S221, including operations S2221 to S2215.

[0080] In operation S2221, a time grouping threshold and a time grouping label are set.

[0081] In an embodiment of the present application, the time grouping label may include short, medium, and long. The grouping label for the time in the interval [0h, 3h] is defined as "short", the grouping label for the time in the interval (3h, 6h] is defined as "medium", and the grouping label for the time greater than 6h is defined as "long". The time grouping threshold and time grouping label can be determined according to actual needs and are not limited here.

[0082] In operation S2222 , a resource grouping threshold and a resource grouping label are set.

[0083] In an embodiment of the present application, the resource predicted by the computing resource prediction model is memory demand. The memory grouping label can be set to: low, medium, or high. For example, the grouping label for memory demand in the range [0GB, 5GB] is defined as "low", the grouping label for memory demand in the range (5GB, 10GB] is defined as "medium", and the grouping label for memory demand greater than 10GB is defined as "high". The resource grouping threshold and resource grouping label can be determined according to actual needs and are not limited here.

[0084] In operation S2223 , the time required for the service is grouped according to a time grouping threshold to obtain a corresponding time grouping label.

[0085] In operation S2224, computing resources required for the service are grouped according to a resource grouping threshold, and corresponding resource grouping labels are obtained.

[0086] In operation S2225 , the corresponding time grouping tag and the corresponding resource grouping tag are set as a state space.

[0087] In an embodiment of the present application, the service time and resource requirements of the service predicted by the demand forecasting model are grouped according to the time grouping threshold and the resource grouping threshold to obtain corresponding time grouping labels and resource grouping labels.

[0088] For example, the division and setting of group labels are shown in Table 1.

[0089] Table 1

[0090] State (time, memory) Action 1 (high priority) Action 2 (Normal) Action 3 (low priority) (short, low) 2.1 1.5 0.3 (short, medium) 1.8 1.2 0.1 (middle, middle) 3.5 2.7 -0.2

[0091] In cloud environments, task queues receive and manage task requests and assign tasks to different virtual machines based on their priority. For example, a priority classifier can categorize tasks into high, normal, and low priorities. High-priority tasks are placed directly in the high-priority waiting queue, while low-priority tasks are placed in the low-priority waiting queue. The scheduler schedules tasks based on their priority and assigns them to the appropriate virtual machines, ensuring that high-priority tasks are processed first.

[0092] According to the division and setting of the group labels in Table 1, the possible scheduling actions in Table 1 are: action 1 is assigned to the high-priority queue (fast processing), action 2 is assigned to the normal queue (balanced processing), and action 3 is assigned to the low-priority queue (delayed processing).

[0093] For example, the state corresponding to (5h, 9GB) is (middle, middle). In this state, according to the Q table, the Q value of action 1 is the highest, so this action is selected.

[0094] In the embodiments of this application, when faced with a large number of service requests, a grouping approach can effectively improve scheduling efficiency. By categorizing and grouping the time and resources required by the requests according to specific rules, the characteristics and requirements of each group of requests can be quickly identified, allowing for the rapid development of targeted scheduling strategies based on the needs of each group of requests. This approach not only optimizes resource allocation but also ensures that service requests are processed promptly, improving overall service quality and response speed.

[0095] In operation S222 , hyperparameters are set: hyperparameters such as learning rate, discount factor, and exploration rate are determined.

[0096] In the embodiments of this application, the learning rate (alpha) determines the extent to which new information covers old information. The initial value of the learning rate generally ranges from 0.1 to 0.3, such as 0.2. A moderate learning rate allows the algorithm to quickly adapt to environmental changes early on, and then decreases to stabilize the policy later. If resources are highly dynamic (e.g., tasks arrive frequently), a slightly higher learning rate (e.g., 0.3) can be maintained. The discount factor (gamma) determines the discounted value of future rewards. The discount factor generally ranges from 0.9 to 0.95. Cloud scheduling needs to consider the impact of task allocation on future resources, such as avoiding hotspots. A higher gamma can capture long-term effects. If tasks are independent and have no subsequent dependencies, a lower value (e.g., 0.85) can be used. The exploration rate (epsilon) determines the balance between exploring new actions and leveraging known actions. It generally ranges from 0.9 to 1.0, with 1.0 for example, indicating completely random initial exploration. A high initial exploration rate covers a wide range of the action space, gradually shifting to leveraging the known policy. This decay prevents excessive randomness later on, and a lower bound ensures continuous exploration of the dynamic environment.

[0097] It should be noted that the values of the learning rate, discount factor, and exploration rate can be determined according to actual needs and are not limited here.

[0098] In operation S223 , the dual-Q learning algorithm is iterated to simulate multiple scheduling schemes.

[0099] In the embodiment of the present application, the double Q learning algorithm simulates multiple scheduling schemes by iteration. In each training round (episode), an initial state is randomly selected. In each state, a random action or a random action is selected based on the current exploration rate epsilon. The table selects the action with the highest Q-value, executes the selected action, and observes the new state and reward.

[0100] Randomly update one of the Q tables to update For example, The table update formula is as follows:

[0101] ] (2)

[0102] ‌ : Indicates the Q value of taking action a in state s (using function).

[0103] ‌ : Learning rate, which controls the degree of influence of new information on old information during each update.

[0104] ‌ : The immediate reward obtained by taking action a in state s.

[0105] ‌ : A discount factor used to measure the importance of future rewards.

[0106] :Indicates The function selects the action a' that maximizes the Q value.

[0107] ‌: Indicates the next state Next, use Function selection action (Right now ) after The Q value in the function.

[0108] Random use and The table is used to select actions and update Q-values to reduce the overestimation of Q-values.

[0109] Evaluate and optimize the model by simulating the effects of service scheduling. After each training round, evaluate the performance of the current scheduling solution and adjust the hyperparameters based on the evaluation results to optimize the model performance.

[0110] When building a model using the dual Q-learning algorithm, the most important aspect is the design of the immediate reward (reward function). The reward function directly determines the model's learning direction and optimization path, guiding the model to make decisions in complex and changing environments. In this application, the reward function is set as a comprehensive reward value, taking into account not only resource utilization but also the load balancing of virtual resources. The reward function is established based on these two indicators: resource utilization and load balancing, as shown in the following formula:

[0111]

[0112]

[0113]

[0114] in, is the CPU utilization, is the memory utilization, The amount of CPU used. is the total amount of CPU, is the amount of memory used, is the total amount of memory, For resource utilization.

[0115] Load balancing is the standard deviation of CPU utilization of all VMs. Load balancing has a negative contribution to the reward. The larger the load balancing value, the greater the penalty.

[0116] Comprehensive rewards (instant rewards):

[0117] It should be noted that 0.6 and 0.4 are weight factors, which are used to adjust the proportional relationship between resource utilization and load balancing and the value of the exploration rate. The setting of the weight factor can be determined according to actual needs and is not limited here.

[0118] Using resource utilization and load balance as the reward function of Dual-Q Learning enables the Dual-Q Learning algorithm to take into account both resource utilization efficiency and load balance for various simulated scheduling schemes throughout the entire iterative process, thereby ensuring that the optimal scheduling scheme among the multiple scheduling schemes outputted also takes into account both resource utilization efficiency and load balance.

[0119] In operation S224, a termination condition of the iterative process is set.

[0120] In the embodiment of the present application, the termination condition is reaching a certain threshold of resource (CPU and memory) utilization of different numbers of processing units in the preset cloud network, for example The memory load calculation formula is:

[0121]

[0122] in, Is to execute the task on the i-th VM Previous memory usage, is the memory containing the jth task request, is the total memory available on the ith virtual machine. The CPU load calculation formula is:

[0123]

[0124] in, The task is executed at the i-th VM Previous CPU usage, is the CPU requested by the jth task, is the total CPU available at the i-th virtual machine.

[0125] The formula for calculating VM utilization is as follows:

[0126]

[0127] in and are the weights of CPU and memory respectively. In this application, given , Both are equal to 0.5, because CPU and memory are equally important. The specific weights can be determined according to actual needs and are not restricted here.

[0128] The total load (LH) on k hosts, where k is the total number of hosts in the system, is calculated as follows:

[0129]

[0130] The average load of all physical machines is calculated as follows:

[0131]

[0132] Where p is the number of hosts in the cloud network.

[0133] In the embodiment of the present application, the resource utilization rate calculated is The iteration process ends when the first threshold is met, for example, the CPU threshold can be set to 75% and the memory threshold can be set to 80%; and the calculated average load of the physical resource meets the second threshold. If the average load approaches the preset target range (such as 0.6~0.7), the system is considered to have achieved load balancing. For example, the second threshold can be set to The threshold value can be set based on actual needs and is not limited here.

[0134] In the embodiments of the present application, first, resource utilization can measure the degree to which system resources are effectively utilized. Using it as one of the termination conditions can ensure that the dual-Q learning algorithm can continuously optimize resource allocation during the learning process to avoid waste or excessive occupation of resources. When resource utilization tends to be stable and reaches a high level, it indicates that the system has found a more reasonable resource allocation strategy. Secondly, the average load reflects the overall pressure of the system during operation. Including the average load in the termination condition can enable the dual-Q learning algorithm to fully consider the load balance of the processing unit during the optimization process. When the average load reaches a relatively stable and reasonable state, it means that the distribution between each node or task is relatively uniform, which can effectively avoid local overload or idleness. Using resource utilization and average load as the iterative termination conditions of dual-Q learning can enable the output scheduling strategy to take into account both resource utilization efficiency and load balance, thereby achieving a more efficient and optimized decision-making strategy.

[0135] In operation S225 , the dual-Q learning algorithm iterates until a termination condition is met, and outputs the optimal scheduling solution among the solutions.

[0136] In an embodiment of the present application, after the termination condition is met, the optimal action in each state is output according to the Q table to form an optimal strategy for service scheduling.

[0137] Figure 5 The flowchart of the overall output scheduling solution according to an embodiment of the present application is schematically shown.

[0138] like Figure 5 As shown, the present application can take advantage of the advantages of both algorithms. The linear model calculation process is relatively simple. Based on the resource prediction it provides quickly, the dual-Q learning algorithm continuously adapts to the complex environment of service scheduling to make dynamic decisions, thereby outputting the optimal service scheduling solution. This method can improve the accuracy of scheduling decisions while ensuring service response time. It is suitable for complex scenarios that require efficient and accurate predictions, can balance cloud service loads, and ensure the quality of cloud services. In addition, this method can reduce the impact of manual intervention and subjective judgment. This method can automatically learn and adapt to changes in the system, continuously optimize scheduling strategies, and improve scheduling efficiency.

[0139] Based on the above service scheduling method, this application also provides a service scheduling device. Figure 7 The device is described in detail.

[0140] Figure 7 The structural block diagram of the service scheduling device according to an embodiment of the present application is schematically shown.

[0141] like Figure 7 As shown, the service scheduling device 700 of this embodiment includes a resource prediction module 710 , a policy determination module 720 and a scheduling module 730 .

[0142] Resource prediction module 710 is configured to respond to a target service request by inputting the service type of the target service request into a demand prediction model for prediction, thereby obtaining a prediction result. The prediction result includes the resources required to process the target service request. The demand prediction model is pre-trained using a preprocessed dataset and is based on a linear regression algorithm. In one embodiment, resource prediction module 710 can be used to perform operation S210 described above and will not be further described here.

[0143] The strategy determination module 720 is used to output a service scheduling decision based on the prediction results and a pre-built service scheduling model, wherein the service scheduling model is pre-built based on the double Q learning algorithm. The strategy determination module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0144] The scheduling module 730 is used to schedule the service according to the service scheduling decision. The scheduling module 730 can be used to perform the operation S230 described above, which will not be described in detail here.

[0145] According to the embodiments of the present application, the advantages of the two algorithms can be fully utilized. The linear model calculation process is relatively simple. On the basis of the resource prediction it quickly provides, the dual-Q learning algorithm continuously adapts to the complex environment of service scheduling to make dynamic decisions, thereby outputting the optimal service scheduling solution. This method can improve the accuracy of scheduling decisions while ensuring service response time. It is suitable for complex scenarios that require efficient and accurate predictions. It can balance cloud service loads and ensure the quality of cloud services. In addition, this method can reduce the impact of manual intervention and subjective judgment. The method can automatically learn and adapt to changes in the system, continuously optimize scheduling strategies, and improve scheduling efficiency.

[0146] According to embodiments of the present application, any multiple modules among the resource prediction module 710, the policy determination module 720, and the scheduling module 730 may be combined into a single module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in a single module. According to embodiments of the present application, at least one of the resource prediction module 710, the policy determination module 720, and the scheduling module 730 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented in hardware or firmware through any other reasonable means of circuit integration or packaging, or may be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of these. Alternatively, at least one of the resource prediction module 710, the policy determination module 720, and the scheduling module 730 may be at least partially implemented as a computer program module that, when executed, performs the corresponding functionality.

[0147] Figure 8 The block diagram schematically shows an electronic device suitable for implementing the service scheduling method according to an embodiment of the present application.

[0148] like Figure 8 As shown, an electronic device 800 according to an embodiment of the present application includes a processor 801, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 802 or programs loaded from a storage unit 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or related chipsets and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for executing different actions of the method flow according to an embodiment of the present application.

[0149] Various programs and data required for the operation of the electronic device 800 are stored in the RAM 803. The processor 801, ROM 802, and RAM 803 are connected to each other via a bus 804. The processor 801 performs various operations of the method flow according to the embodiment of the present application by executing the programs in the ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than the ROM 802 and RAM 803. The processor 801 may also perform various operations of the method flow according to the embodiment of the present application by executing the programs stored in the one or more memories.

[0150] According to an embodiment of the present application, electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to bus 804. Electronic device 800 may also include one or more of the following components connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 808 including a hard disk; and a communication section 809 including a network interface card such as a LAN card or modem. Communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. Removable media 811, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 810 as needed, so that computer programs read from the removable media can be installed into storage section 808 as needed.

[0151] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of this application is implemented.

[0152] According to an embodiment of the present application, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, a computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0153] The embodiments of the present application also include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the service scheduling method provided by the embodiments of the present application.

[0154] The computer program executes the above functions defined in the system / device of the embodiment of the present application when the processor 801 executes the computer program. According to the embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0155] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0156] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 809, and / or installed from a removable medium 811. When the computer program is executed by the processor 801, the above-described functions defined in the system of the embodiment of the present application are performed. According to the embodiment of the present application, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0157] According to an embodiment of the present application, the program code for executing the computer program provided by the embodiment of the present application can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, languages such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0158] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0159] Those skilled in the art will appreciate that the features described in the various embodiments of this application may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in this application. In particular, the features described in the various embodiments of this application may be combined and / or coupled in various ways without departing from the spirit and teachings of this application. All such combinations and / or couplings fall within the scope of this application.

Claims

1. A service scheduling method, characterized in that: The method comprises: In response to a target service request, inputting the service type of the target service request into a demand forecasting model for forecasting to obtain a forecast result, wherein the forecast result includes resources required to process the target service request, the demand forecasting model being pre-trained based on a linear regression algorithm using a pre-processed data set; outputting a service scheduling decision based on the prediction result and a pre-built service scheduling model, wherein the service scheduling model is pre-built based on a double-Q learning algorithm; and The service is scheduled according to the service scheduling decision.

2. The method according to claim 1, characterized in that Outputting a service scheduling decision based on the prediction result and a pre-built service scheduling model includes: Setting a state space, an action space, and a reward function of a double-Q learning algorithm, setting the state space to the prediction result, setting the action space to the scheduling operation of the target service request, and setting the reward function to a comprehensive reward value, wherein the comprehensive reward value includes resource utilization and load balancing; Iterate based on the double-Q learning algorithm, simulate multiple scheduling schemes until the termination condition is met, and output the optimal scheduling scheme among the multiple scheduling schemes.

3. The method according to claim 2, characterized in that The steps of setting the reward function include: Calculating resource utilization, and calculating load balancing based on the resource utilization; Setting a weight factor, which is used to adjust the proportional relationship between resource utilization and load balancing; A reward function of the service scheduling model is set based on the weight factor, the resource utilization rate, and the load balancing degree.

4. The method according to claim 2, characterized in that The termination conditions include: The calculated resource utilization meets a preset first threshold; and the calculated average load of the physical resource meets a preset second threshold.

5. The method according to claim 2, characterized in that The resources required for processing the target service request include the time required for the service and the computing resources required for the service; the demand prediction model includes a service time prediction model and a computing resource prediction model, wherein the service time prediction model is used to predict the time required for the service, and the computing resource prediction model is used to predict the computing resources required for the service.

6. The method according to claim 5, characterized in that Defining the state space as the prediction result includes: Set time grouping thresholds and time grouping labels; Set resource grouping thresholds and resource grouping labels; Grouping the time required for the service according to the time grouping threshold to obtain corresponding time grouping labels; Grouping the computing resources required for the service according to the resource grouping threshold to obtain corresponding resource grouping labels; The corresponding time grouping label and the corresponding resource grouping label are set as the state space.

7. The method according to claim 1, characterized in that The demand forecasting model is pre-trained based on a linear regression algorithm and a pre-processed dataset, including: Acquiring historical operation data of server resources and preprocessing the historical data to obtain a preprocessed data set; and Based on the preprocessed data set, a linear regression algorithm is fitted to establish a demand forecasting model.

8. A service scheduling device, characterized in that: include: a resource prediction module, configured to, in response to a target service request, input the service type of the target service request into a demand prediction model for prediction, thereby obtaining a prediction result, wherein the prediction result includes the resources required to process the target service request, wherein the demand prediction model is pre-trained based on a linear regression algorithm using a pre-processed dataset; a strategy determination module, configured to output a service scheduling decision based on the prediction result and a pre-built service scheduling model, wherein the service scheduling model is pre-built based on a double-Q learning algorithm; The scheduling module is used to schedule services according to the service scheduling decision.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, wherein when the instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images