Scheduling information generation method and device and related equipment

Abstract

The invention provides a scheduling information generation method and apparatus, and a related device. The method comprises the steps of matching a target server corresponding to a to-be-processed task in a plurality of candidate servers; setting a reward function according to the target server and a first deep learning network, and updating the first deep learning network based on the reward function to obtain a second deep learning network; and inputting the to-be-processed task into the second deep learning network for processing, and generating a target scheduling strategy. According to the method and the device, after the target server corresponding to the to-be-processed task is matched, the corresponding first deep learning network is determined according to the target server, so that the first deep learning network is generated after the first deep learning network is updated by setting the reward function; and through the second deep learning network, generating a scheduling strategy for the target server to process the to-be-processed task, thereby improving the resource scheduling efficiency of the server.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of deep learning technology, and in particular, to a method and apparatus for generating scheduling information and related equipment. Background Art

[0002] With the surge in the number of cloud users and the continuous expansion of cloud computing resources, task resource scheduling has become a research hotspot in edge computing technology (Mobile Edge Computing, MEC). Traditional scheduling solutions are too complex. Once there are too many tasks and resources, the processing speed will be slower, and there will be a large delay in task scheduling, which cannot respond in real time. In addition, there is little consideration of server resource types, which can easily cause load imbalance of other important resources, resulting in low server resource scheduling efficiency. Summary of the invention

[0003] The embodiments of the present application provide a method, apparatus and related equipment for generating scheduling information to solve the problem of low efficiency of server resource scheduling in the prior art.

[0004] To solve the above problems, this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for generating scheduling information, the method comprising:

[0006] Matching a target server corresponding to the task to be processed from a plurality of candidate servers, wherein the target server includes a plurality of first virtual machines for processing the task to be processed;

[0007] A reward function is set according to the target server and the first deep learning network, and the first deep learning network is updated based on the reward function to obtain a second deep learning network, wherein the first deep learning network matches the target server, and the first deep learning network is used to generate a scheduling strategy for the target server to process the task to be processed;

[0008] The task to be processed is input into the second deep learning network for processing, and a target scheduling strategy is generated, where the target scheduling strategy is used to allocate at least one of the virtual machines to the task to be processed.

[0009] Optionally, before setting a reward function according to the target server and the first deep learning network and updating the first deep learning network based on the reward function to obtain the second deep learning network, the method further includes:

[0010] Determine an initial deep learning network that matches the target server, wherein the initial deep learning network includes a Critic network and an Actor network;

[0011] Initialize the parameters of the Critic network and initialize the parameters of the Actor network;

[0012] Inputting training sample data into the initial deep learning network for recognition to obtain an output value, wherein the training sample data includes a plurality of sample tasks and training labels, and the training labels are scheduling strategies for processing the plurality of sample tasks;

[0013] Using a loss function to calculate the loss between the output value and the training label to obtain a loss value;

[0014] The Critic network and the Actor network are updated based on the loss value to obtain the first deep learning network.

[0015] Optionally, the virtual machine types of the multiple first virtual machines include K types, and determining an initial deep learning network matching the target server includes:

[0016] A K*(N+2) matrix is ​​set according to the target server, wherein the first column of the K*(N+2) matrix is ​​used to record the K types, the second to Nth columns of the K*(N+2) matrix are used to record the identifiers of the virtual machines of the corresponding types, the N+1th column of the K*(N+2) matrix is ​​used to record the working status of all virtual machines of the corresponding types, the N+2th column of the K*(N+2) matrix is ​​used to record the current queued task information of all virtual machines of the corresponding types, wherein N is a positive integer, and K is a positive integer;

[0017] Generate an input expression and an output expression of the target server based on the K*(N+2) matrix, wherein the input expression is an expression of the first virtual machine pre-assigned to the target server for processing tasks, and the output expression is an expression of the tasks processed by the target server;

[0018] An initial deep learning network matching the target server is determined according to the input expression and the output expression.

[0019] Optionally, inputting the training sample data into the initial deep learning network for recognition to obtain an output value includes:

[0020] Fill the training sample data into the N+2th column of the input expression and the second column of the output expression to generate a target input expression and a target output expression corresponding to the training sample data;

[0021] Determine the Gaussian noise corresponding to the training sample data according to the target input expression;

[0022] Performing feature extraction on the target input expression to obtain target features;

[0023] The target feature and the Gaussian noise are input into the Actor network for recognition to obtain the output value.

[0024] Optionally, the reward function is set according to the target server and the first deep learning network, and the first deep learning network is updated based on the reward function to obtain a second deep learning network:

[0025] Setting a reward function according to the target server and the first deep learning network;

[0026] Putting the reward function, the input expression and the output expression into the experience pool of the first deep learning network;

[0027] When the experience pool of the first deep learning network is full, the first deep learning network is iteratively updated using a gradient descent method to obtain the second deep learning network.

[0028] Optionally, setting a reward function according to the target server and the first deep learning network, and updating the first deep learning network based on the reward function to obtain a second deep learning network includes:

[0029] Setting an objective function according to the attribute information of the target server and the network structure of the first deep learning network;

[0030] Setting the reward function based on the objective function;

[0031] The first deep learning network is updated based on the objective function and the reward function to obtain the second deep learning network.

[0032] Optionally, setting the objective function according to the attribute information of the target server and the network structure of the first deep learning network includes:

[0033] Calculating resource utilization rates of the plurality of first virtual machines to obtain a plurality of first resource utilization rates, wherein the plurality of first resource utilization rates correspond one-to-one to the plurality of first virtual machines;

[0034] Calculating standard deviations of the plurality of first resource utilizations to obtain a standard deviation;

[0035] Calculating load conditions of the multiple first virtual machines to obtain multiple first load values, where the multiple first load values ​​correspond one-to-one to the multiple first virtual machines;

[0036] Calculating an average of the plurality of first load values ​​to obtain an average load value;

[0037] Obtaining the service quality and energy consumption of the target server after the t-th scheduling, where t is a positive integer;

[0038] The objective function is generated according to the standard deviation, the average load value, the service quality of the target server after the tth scheduling, the energy consumption of the target server after the tth scheduling, and the network structure of the first deep learning network. The objective function is used to constrain the scheduling strategy of the target server for the t+1th time.

[0039] Optionally, setting the reward function based on the objective function includes:

[0040] Obtaining the resource upper limit of the target server;

[0041] Calculate the maximum number of the first virtual machines that are allowed to be started simultaneously in the target server according to the resource upper limit of the target server;

[0042] Obtaining the service quality of the target server after the t-th scheduling and the energy consumption after the t-th scheduling according to the objective function;

[0043] The reward function is generated according to the service quality of the target server after the tth scheduling, the energy consumption of the target server after the tth scheduling and the maximum quantity value, and the reward function is used to constrain the scheduling strategy of the target server for the t+1th time.

[0044] In a second aspect, an embodiment of the present application further provides a device for generating scheduling information, including:

[0045] A matching module, used to match a target server corresponding to the task to be processed from a plurality of candidate servers, wherein the target server includes a plurality of first virtual machines for processing the task to be processed;

[0046] An updating module, configured to set a reward function according to the target server and the first deep learning network, and to update the first deep learning network based on the reward function to obtain a second deep learning network, wherein the first deep learning network matches the target server, and the first deep learning network is used to generate a scheduling strategy for the target server to process the task to be processed;

[0047] A generation module is used to input the task to be processed into the second deep learning network for processing, and generate a target scheduling strategy, wherein the target scheduling strategy is used to allocate at least one of the virtual machines to the task to be processed.

[0048] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps of the method described in the first aspect above.

[0049] In a fourth aspect, an embodiment of the present application further provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the method described in the first aspect above.

[0050] In a fifth aspect, an embodiment of the present application further provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps in the method described in the first aspect.

[0051] The present application provides a method, device and related equipment for generating scheduling information, the method comprising: matching a target server corresponding to a task to be processed among multiple candidate servers, the target server comprising multiple first virtual machines for processing the task to be processed; setting a reward function according to the target server and a first deep learning network, and updating the first deep learning network based on the reward function to obtain a second deep learning network, the first deep learning network matches the target server, and the first deep learning network is used to generate a scheduling strategy for the target server to process the task to be processed; inputting the task to be processed into the second deep learning network for processing to generate a target scheduling strategy, the target scheduling strategy is used to allocate at least one of the virtual machines to the task to be processed. The present application determines the corresponding first deep learning network according to the target server after matching the target server corresponding to the task to be processed, thereby generating the first deep learning network after updating the first deep learning network by setting a reward function, and generating the scheduling strategy for the target server to process the task to be processed through the second deep learning network, thereby improving the resource scheduling efficiency of the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0053] Figure 1 A flowchart of a method for generating scheduling information provided in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of the deep learning network structure provided in the embodiment of the present application;

[0055] Figure 3 A schematic diagram of the structure of a device for generating scheduling information provided in an embodiment of the present application;

[0056] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] The terms "first", "second" etc. in the embodiments of the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. In addition, the terms "include" and "have" and any variation thereof are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. In addition, "and / or" is used in the present application to represent at least one of connected objects, such as A and / or B and / or C, representing the inclusion of separate A, separate B, separate C, and A and B all exist, B and C all exist, A and C all exist, and 7 situations that A, B and C all exist.

[0059] See also Figure 1 , Figure 1 It is a flow chart of the method for generating scheduling information provided in an embodiment of the present application. Figure 1 The method for generating scheduling information shown can be executed by an electronic device such as a server.

[0060] like Figure 1 As shown, the method for generating scheduling information may include the following steps:

[0061] Step 101: Match a target server corresponding to a task to be processed from a plurality of candidate servers, wherein the target server includes a plurality of first virtual machines for processing the task to be processed.

[0062] In this embodiment, multiple candidate servers belong to the same cluster, where each candidate server acts as an agent, not only having the Actor and Critic networks of the Multiple Agent Deepdeterministic policy gradient (MADDPG) algorithm, but also introducing Convolutional Neural Networks (CNN) as a front-end feature extractor.

[0063] Specifically, by matching a target server corresponding to the task to be processed from multiple candidate servers, the task type and processing capacity that the target server can handle can meet the requirements of the task to be processed. Specifically, the target server includes multiple first virtual machines for processing the tasks to be processed. The multiple first virtual machines can be of different types and are not specifically limited in this embodiment.

[0064] Step 102: set a reward function according to the target server and the first deep learning network, and update the first deep learning network based on the reward function to obtain a second deep learning network, wherein the first deep learning network matches the target server, and the first deep learning network is used to generate a scheduling strategy for the target server to process the task to be processed.

[0065] In this embodiment, the first deep learning network is used to generate a scheduling strategy. Specifically, when the target server processes a newly added or unfinished task, it first processes the high-dimensional state information through CNN, and then uses the processed features to make decisions and learn, and uses the CNN network to automatically extract key features from the real-time data of each task and server. For example, information such as the CPU usage, memory usage, and network bandwidth utilization of the task will be converted into a structured format that can be processed by CNN, and feature mapping is performed through the convolution layer. The extracted features will be used to construct the state vector of each intelligent agent, which not only includes its own resource utilization, but also takes into account the interaction with other intelligent agents, ensuring the state representation from a global perspective. In the process of each task scheduling, the newly added tasks and unfinished tasks of each server are taken into account. The newly added tasks are the tasks assigned by other servers. After each server adjusts its own state to the optimal state, the state of the entire server can reach the optimal state.

[0066] The reward function is a core concept in reinforcement learning (RL), which is used to evaluate the quality of an action taken by an intelligent agent in a specific state. Specifically, it is used in this application to evaluate the quality of the generated scheduling strategy, that is, to take all resources in the server into account, to minimize latency and improve QoS while taking into account the purpose of reducing energy consumption. Each server in the cluster can set its own reward function according to demand. The first deep learning network is updated by the reward function to obtain the second deep learning network. In this embodiment, the reward function can also be added and divided by the total number of servers in the cluster as the reward function of all neural networks. In the MADDPG framework, each server will also use the features extracted by CNN to combine with the original state to determine its action (i.e., resource allocation decision). The designed reward mechanism takes into account the feature information obtained from CNN, so that the reward not only reflects the immediate task execution effect, but also reflects the long-term utilization efficiency of resources and system load balance. Through the features extracted by CNN and the feedback mechanism of the MADDPG algorithm, the behavior of the server is continuously optimized to find a balance between reducing energy consumption and improving service quality.

[0067] Step 103: input the task to be processed into the second deep learning network for processing, and generate a target scheduling strategy, where the target scheduling strategy is used to allocate at least one of the virtual machines to the task to be processed.

[0068] In this embodiment, by inputting the pending tasks into the second deep learning network, a target scheduling strategy for processing the pending tasks is generated, which includes a specific plan for how to allocate multiple first virtual machines and how to process the pending tasks. Since the performance of virtual machines is different, the energy consumption generated is different. Through reasonable task allocation, the number of virtual machines that can be turned on can be reduced to achieve the purpose of energy saving. This application introduces a convolutional neural network (CNN) to perform in-depth analysis and feature extraction of server operating status and task data. These high-level features extracted by CNN will be combined with the state information of the agent to provide a richer and more accurate state representation for the MADDPG algorithm. On this basis, the MADDPG algorithm optimizes the action decisions of each agent to ensure that each task scheduling can accurately allocate resources according to real-time environmental feedback, significantly improving the response speed and resource utilization efficiency of the entire system.

[0069] This application matches the target server corresponding to the task to be processed, determines the corresponding first deep learning network according to the target server, and then generates the first deep learning network after updating the first deep learning network by setting a reward function, and generates a scheduling strategy for the target server to process the task to be processed through the second deep learning network, thereby improving the resource scheduling efficiency of the server.

[0070] In some optional implementations, optionally, before setting a reward function according to the target server and the first deep learning network and updating the first deep learning network based on the reward function to obtain the second deep learning network, the method further includes:

[0071] Determine an initial deep learning network that matches the target server, wherein the initial deep learning network includes a Critic network and an Actor network;

[0072] Initialize the parameters of the Critic network and initialize the parameters of the Actor network;

[0073] Inputting training sample data into the initial deep learning network for recognition to obtain an output value, wherein the training sample data includes a plurality of sample tasks and training labels, and the training labels are scheduling strategies for processing the plurality of sample tasks;

[0074] Using a loss function to calculate the loss between the output value and the training label to obtain a loss value;

[0075] The Critic network and the Actor network are updated based on the loss value to obtain the first deep learning network.

[0076] In this embodiment, if Figure 2 As shown, Figure 2 This is a schematic diagram of the network structure in an embodiment of the present application, wherein the initial deep learning network that matches the target server includes a Critic network and an Actor network. Specifically, initialize all agent Critic network parameters Q(s,a|ω) and Actor network parameters π(a|θ), and obtain output values ​​by inputting training sample data into the initial deep learning network for identification, wherein the training sample data includes multiple sample tasks and labels corresponding to the sample tasks, and the training labels are scheduling strategies for processing the multiple sample tasks. Thus, the training samples are input into the initial deep learning network for identification, the output values ​​are obtained, and the loss between the output values ​​and the training labels is calculated through the loss function to obtain the loss value. Finally, the Critic network and the Actor network are updated by the loss value to obtain the first deep learning network.

[0077] The training process includes the following steps:

[0078] Data preprocessing:

[0079] X t ' = reshape(X t )

[0080] Among them, is an operation that converts linear time series data into three-dimensional input (height, width, number of channels) that CNN can accept.

[0081] Convolution operation: Apply multiple convolution filters W k and bias b k to extract features.

[0082] F k (X t ')=ReLU(W k *X t '+b k )

[0083] Where * represents the convolution operation, F k is the output of the kth convolutional layer, and ReLU is the activation function used to increase nonlinearity.

[0084] Feature aggregation: Combine the outputs of all convolutional layers into a single feature vector for use by the next layer.

[0085] F(X t )=concatenate({F1,F2,...,F K})

[0086] Where concatenate is an operation that combines the outputs of all convolutional layers into a single feature vector.

[0087] In this embodiment, the processing of input parameters is optimized by introducing the CNN network. Compared with the traditional reinforcement learning method, the technical solution of the present application allows the neural network of each server to focus on processing the key state features extracted by the CNN, thereby significantly reducing the number of input parameters and reducing the difficulty of model convergence, thereby improving training efficiency.

[0088] Optionally, the virtual machine types of the multiple first virtual machines include K types, and determining an initial deep learning network matching the target server includes:

[0089] A K*(N+2) matrix is ​​set according to the target server, wherein the first column of the K*(N+2) matrix is ​​used to record the K types, the second to Nth columns of the K*(N+2) matrix are used to record the identifiers of the virtual machines of the corresponding types, the N+1th column of the K*(N+2) matrix is ​​used to record the working status of all virtual machines of the corresponding types, the N+2th column of the K*(N+2) matrix is ​​used to record the current queued task information of all virtual machines of the corresponding types, wherein N is a positive integer, and K is a positive integer;

[0090] Generate an input expression and an output expression of the target server based on the K*(N+2) matrix, wherein the input expression is an expression of the first virtual machine pre-assigned to the target server for processing tasks, and the output expression is an expression of the tasks processed by the target server;

[0091] An initial deep learning network matching the target server is determined according to the input expression and the output expression.

[0092] In this embodiment, a specific mathematical expression of server resource usage, virtual machine startup status, resources required for newly added tasks and legacy tasks, and the location of virtual machines is designed. In this embodiment, the mathematical expression takes a matrix as an example, wherein a K*(N+2) matrix is ​​set according to the target server, the first column of the K*(N+2) matrix is ​​used to record K types, the second to Nth columns of the K*(N+2) matrix are used to record the identifiers of virtual machines of the corresponding type, the N+1th column of the K*(N+2) matrix is ​​used to record the working status of all virtual machines of the corresponding type, the N+2th column of the K*(N+2) matrix is ​​used to record the current queued task information of all virtual machines of the corresponding type, N is a positive integer, and K is a positive integer.

[0093] Specifically, a virtual machine includes N resources such as CPU, memory, I / O, bandwidth, etc., which can be expressed as (r1, r2.....rN). According to different user needs, the types of virtual machines also vary, such as Windows, Linux, etc. Assume that there are K types of virtual machines in a server, and each type of virtual machine has M, then VM km represents the mth virtual machine among the kth virtual machines on the server. The resources of different virtual machines on the server can be expressed as follows:

[0094]

[0095] The last column indicates the sleep state when it is 0, and starts running when it is 1. The resources required by different users' requests are also different. Assuming that there are P requests each time, the resources required for each request can be expressed as follows:

[0096]

[0097] In addition, during the task scheduling process, we must not only consider the addition of new tasks, but also consider the situation after other tasks are processed. Therefore, the remaining tasks in the server at each task scheduling are expressed as follows:

[0098]

[0099] The last column indicates the location of the unfinished task in the virtual machine, and 1 indicates that the task is being processed and cannot be scheduled.

[0100] Optionally, inputting the training sample data into the initial deep learning network for recognition to obtain an output value includes:

[0101] Fill the training sample data into the N+2th column of the input expression and the second column of the output expression to generate a target input expression and a target output expression corresponding to the training sample data;

[0102] Determine the Gaussian noise corresponding to the training sample data according to the target input expression;

[0103] Performing feature extraction on the target input expression to obtain target features;

[0104] The target feature and the Gaussian noise are input into the Actor network for recognition to obtain the output value.

[0105] In this embodiment, the input state S of each server (agent) is designed for each scheduling. RS(t) That is, input expression and output action A RS(t) That is, the output expression. Assuming that the number of scheduling times in the entire scheduling process is T, then RS (t) Represents the tth scheduling.

[0106] Fill the training sample data into the N+2th column of the input expression and the second column of the output expression to generate a target input expression and a target output expression corresponding to the training sample data;

[0107] The remaining server requests and the newly added requests are used as the input of the neural network, i.e., state S1:

[0108]

[0109] 0 means that the new request has not yet been assigned a virtual machine. In this way, only using the request as input can reduce the dimension of the input state and improve the stability of the neural network. In addition, in order to meet the needs of different scenarios, weight parameters can be provided to affect the server allocation strategy. Combined with the feature value extracted in step 1, the input state S RS(t) It can be expressed as follows:

[0110] R RS(t) =concatenate{F(X t ),S1,α,β,γ}

[0111] The output action A of the neural network is the virtual machine location to which each request (including new requests and remaining requests) is assigned after resource scheduling is completed, as shown below:

[0112]

[0113] Optionally, the reward function is set according to the target server and the first deep learning network, and the first deep learning network is updated based on the reward function to obtain a second deep learning network:

[0114] Setting a reward function according to the target server and the first deep learning network;

[0115] Putting the reward function, the input expression and the output expression into the experience pool of the first deep learning network;

[0116] When the experience pool of the first deep learning network is full, the first deep learning network is iteratively updated using a gradient descent method to obtain the second deep learning network.

[0117] In this embodiment, after setting the reward function, before iterating the first deep network, all agent Critic network parameters Q(s,a|ω) and Actor network parameters π(a|θ) are initialized, and all agent target networks θ'=θ, ω'=ω and experience pools are initialized;

[0118] Each iteration starts with:

[0119] Initialize the states of all agents and input the request state s RS(t) Gaussian noise n N :

[0120] Each time the scheduling starts:

[0121] Each agent (server) executes:

[0122] Apply the pre-trained CNN network to extract features from the state S to obtain the feature representation F(S);

[0123] Take F(S) as input and pass it through the current Actor network and noise to get action a RS(t) ;

[0124] Through the current action a RS(t) Change the server status and get rewards RS(t) ;

[0125] Will (s RS(t) ,a RS(t) ,s RS(t+1) ,r RS(t) ) into the experience pool;

[0126] Calculate and update the Critic network through the loss function;

[0127] Update the Actor network using gradient descent via policy gradients;

[0128] Back-propagate the loss function of the Critic network to update the CNN parameters;

[0129] Update the target network through soft update;

[0130] End the agent loop;

[0131] End the scheduling loop;

[0132] End the iteration loop.

[0133] According to the proposed training scheme, the MADDPG algorithm is used to simulate the real environment through randomly generated requests for training. The CNN network parameters are updated by back-propagating the gradient from the MADDPG network to the CNN to update W and b.

[0134] When randomly generating requests, it is necessary to consider the user's demand for different virtual machine types and virtual machine resources. Based on the original data, the Poisson distribution function of each request for different types of virtual machines is calculated. The virtual machine type of the randomly generated request must obey the Poisson distribution, and the virtual machine resources required for each request are determined in combination with other distributions (such as normal distribution, uniform distribution or binomial distribution).

[0135] In this embodiment, the CNN provided by this application can automatically learn the representation of tasks and resources without manually designing features, which enhances the versatility and adaptability of the model. In addition, the MADDPG algorithm is dynamically adjusted through the real-time features extracted by the CNN, which quickly and accurately responds to environmental changes and improves the real-time nature of decision-making.

[0136] Optionally, setting a reward function according to the target server and the first deep learning network, and updating the first deep learning network based on the reward function to obtain a second deep learning network includes:

[0137] Setting an objective function according to the attribute information of the target server and the network structure of the first deep learning network;

[0138] Setting the reward function based on the objective function;

[0139] The first deep learning network is updated based on the objective function and the reward function to obtain the second deep learning network.

[0140] In this embodiment, the objective function is a function that needs to be maximized or minimized in the optimization problem. In mathematics and operations research, the objective function is usually used to describe the performance or benefit of a system, and its value changes with the change of the decision variable.

[0141] Specifically, under the constraints of physical resources and service-level agreement (SLA), the objective function TF of the RL algorithm is designed for each scheduling. RS(t) and the reward function R RS(t) . Through the objective function TF RS(t) and the reward function R RS(t) The first deep learning network is updated to obtain an updated second deep learning network, thereby generating a scheduling policy for the target server through the second deep learning network.

[0142] This application proposal greatly enhances the strategic flexibility and decision-making efficiency of cloud service providers in task scheduling by introducing convolutional neural networks. While maintaining high service quality, it can also effectively reduce energy consumption, thereby bringing cost savings economically. The addition of CNN enables the system to more accurately process and analyze complex task request data, thereby achieving a deep understanding of the status of cloud computing resources and optimized resource allocation. In addition, cloud service providers can also control the focus of task allocation for each server as needed according to different situations. For example, in a large-scale event, the cloud service provider can directly input the energy-saving weight into 0, and the neural network of each server will automatically change to only consider how to provide better services to users, while ignoring energy consumption issues. Depending on the performance of the server, the server can also actively handle more or less work.

[0143] Optionally, setting the objective function according to the attribute information of the target server and the network structure of the first deep learning network includes:

[0144] Calculating resource utilization rates of the plurality of first virtual machines to obtain a plurality of first resource utilization rates, wherein the plurality of first resource utilization rates correspond one-to-one to the plurality of first virtual machines;

[0145] Calculating standard deviations of the plurality of first resource utilizations to obtain a standard deviation;

[0146] Calculating load conditions of the multiple first virtual machines to obtain multiple first load values, where the multiple first load values ​​correspond one-to-one to the multiple first virtual machines;

[0147] Calculating an average of the plurality of first load values ​​to obtain an average load value;

[0148] Obtaining the service quality and energy consumption of the target server after the t-th scheduling, where t is a positive integer;

[0149] The objective function is generated according to the standard deviation, the average load value, the service quality of the target server after the tth scheduling, the energy consumption of the target server after the tth scheduling, and the network structure of the first deep learning network. The objective function is used to constrain the scheduling strategy of the target server for the t+1th time.

[0150] In this embodiment, in order to reduce delay, improve QoS and minimize energy consumption, the objective function is expressed as follows:

[0151]

[0152] Assuming that the server performs resource scheduling T times during operation, RS (t) Represents the tth scheduling, α+β+γ=1. According to the different values, the neural network will have a biased consideration of which goal is the highest priority for current optimization. The cloud service provider can adjust the parameter size according to the actual situation. Therefore, it is considered to add it to the input of the neural network to participate in the training, so that the trained neural network can adjust the corresponding resource scheduling tasks according to different actual conditions in the future.

[0153] In the objective function, σ L represents the standard deviation of all server resource load utilization, σ L The smaller the value, the closer the load of each server is to each other, and the better the load balancing effect is. L It is expressed as follows:

[0154]

[0155] in Indicates the average load of all servers, as follows:

[0156]

[0157] in If it is not enabled, the value is 0; if it is enabled, the value is 1.

[0158] QoS RS(t) It represents the QoS of the user after the t-th scheduling. The QoS of the user is related to the SLA protocol. The present invention considers the response time as an indicator to measure the QoS. The response time RT RS(t) The shorter the time, the better the QoS for the user. In addition, other criteria must not violate the SLA during each scheduling, because user experience is not only related to response time. Therefore, if the SLA agreement is violated during a scheduling process, a penalty needs to be added. RS(t) , indicating that this scheduling violates the SLA agreement and is an infeasible scheduling attempt. This scheduling should be avoided in future training. The penalty can be expressed as follows:

[0159]

[0160] -p a is a negative value, so QoS RS(t) It can be expressed as follows:

[0161]

[0162] EC RS(t) Indicates the energy consumption after the tth scheduling. In cloud computing, the energy consumption of the server is mainly related to the CPU utilization rate. The higher the CPU utilization rate, the higher the energy consumption. In addition, the proposed algorithm reduces the number of virtual machines enabled through reasonable resource allocation to achieve energy saving. Therefore, the energy consumption of the virtual machine when it is enabled must be taken into account to limit the algorithm from enabling many virtual machines in order to improve QoS. Therefore, EC RS(t) It is expressed as follows:

[0163]

[0164] in Indicates that the CPU occupancy rate of the hth server in the tth scheduling is U cpu(t) Power at time, m t represents the number of virtual machines started in t scheduling, p(VM start ) indicates the power consumed by the virtual machine when it is powered on.

[0165] Optionally, setting the reward function based on the objective function includes:

[0166] Obtaining the resource upper limit of the target server;

[0167] Calculate the maximum number of the first virtual machines that are allowed to be started simultaneously in the target server according to the resource upper limit of the target server;

[0168] Obtaining the service quality of the target server after the t-th scheduling and the energy consumption after the t-th scheduling according to the objective function;

[0169] The reward function is generated according to the service quality of the target server after the tth scheduling, the energy consumption of the target server after the tth scheduling and the maximum quantity value, and the reward function is used to constrain the scheduling strategy of the target server for the t+1th time.

[0170] In this embodiment, we need to consider that the various resources of the server are not infinite during resource scheduling, so the total resources of the virtual machines enabled on a server cannot exceed the upper limit of the physical resources of the server. Although the resource utilization rate is taken into account in the above formula, it cannot guarantee that the total resources of the virtual machines will not exceed the upper limit, so it is also necessary to constrain the number of enabled virtual machines, as shown below:

[0171]

[0172] Crn represents the upper limit of the nth resource in the current server. If the upper limit is exceeded during scheduling, a penalty will be imposed, as shown below:

[0173]

[0174] Based on the above settings, the reward function for each dispatch of the hth server in the algorithm is It is expressed as follows

[0175]

[0176] Therefore, the reward function of each neural network is expressed as follows

[0177]

[0178] In this embodiment, the CNN and MADDPG algorithms are used to not only consider the current tasks, but also optimize the scheduling of historical tasks, thereby improving the user satisfaction of each resource allocation and the load balancing of the cluster, while reducing energy consumption and costs. In addition, by simulating real environment training through randomly generated user requests, this method reduces the possibility of relying on historical data and overfitting, and enhances the adaptability of the model to complex real environments.

[0179] This application matches the target server corresponding to the task to be processed, determines the corresponding first deep learning network according to the target server, and then generates the first deep learning network after updating the first deep learning network by setting a reward function, and generates a scheduling strategy for the target server to process the task to be processed through the second deep learning network, thereby improving the resource scheduling efficiency of the server.

[0180] See also Figure 3 , Figure 3 is a structural diagram of a device for generating scheduling information provided in an embodiment of the present application. Figure 3 As shown, the scheduling information generation device 300 includes:

[0181] A matching module 310, configured to match a target server corresponding to the task to be processed from a plurality of candidate servers, wherein the target server includes a plurality of first virtual machines for processing the task to be processed;

[0182] An updating module 320 is used to set a reward function according to the target server and the first deep learning network, and to update the first deep learning network based on the reward function to obtain a second deep learning network, wherein the first deep learning network matches the target server, and the first deep learning network is used to generate a scheduling strategy for the target server to process the task to be processed;

[0183] The generation module 330 is used to input the task to be processed into the second deep learning network for processing and generate a target scheduling strategy, wherein the target scheduling strategy is used to allocate at least one of the virtual machines to the task to be processed.

[0184] Optionally, also include:

[0185] A network determination module, used to determine an initial deep learning network that matches the target server, wherein the initial deep learning network includes a Critic network and an Actor network;

[0186] A network initialization module, used to initialize the parameters of the Critic network and the Actor network;

[0187] A network identification module, used to input training sample data into the initial deep learning network for identification to obtain an output value, wherein the training sample data includes a plurality of sample tasks and training labels, and the training labels are scheduling strategies for processing the plurality of sample tasks;

[0188] A network calculation module, used to calculate the loss between the output value and the training label using a loss function to obtain a loss value;

[0189] A network update module is used to update the Critic network and the Actor network based on the loss value to obtain the first deep learning network.

[0190] Optionally, the virtual machine types of the multiple first virtual machines include K types, and the network determination module includes:

[0191] A first setting submodule is used to set a K*(N+2) matrix according to the target server, wherein the first column of the K*(N+2) matrix is ​​used to record the K types, the second to Nth columns of the K*(N+2) matrix are used to record the identifiers of the virtual machines of the corresponding types, the N+1th column of the K*(N+2) matrix is ​​used to record the working status of all virtual machines of the corresponding types, the N+2th column of the K*(N+2) matrix is ​​used to record the current queued task information of all virtual machines of the corresponding types, and N is a positive integer and K is a positive integer;

[0192] A first generating submodule is used to generate an input expression and an output expression of the target server based on the K*(N+2) matrix, wherein the input expression is an expression of a first virtual machine pre-assigned to the target server for processing tasks, and the output expression is an expression of tasks processed by the target server;

[0193] The first determination submodule is used to determine an initial deep learning network that matches the target server according to the input expression and the output expression.

[0194] Optionally, the network identification module includes:

[0195] A second generating submodule is used to fill the training sample data into the N+2th column of the input expression and the second column of the output expression to generate a target input expression and a target output expression corresponding to the training sample data;

[0196] A second determination submodule, configured to determine the Gaussian noise corresponding to the training sample data according to the target input expression;

[0197] A feature extraction submodule is used to extract features from the target input expression to obtain target features;

[0198] The recognition submodule is used to input the target feature and the Gaussian noise into the Actor network for recognition to obtain the output value.

[0199] Optionally, the updating module 320 includes:

[0200] A second setting submodule, used to set a reward function according to the target server and the first deep learning network;

[0201] Putting a submodule, used to put the reward function, the input expression and the output expression into the experience pool of the first deep learning network;

[0202] An iterative submodule is used to iteratively update the first deep learning network using a gradient descent method to obtain the second deep learning network when the experience pool of the first deep learning network is full.

[0203] Optionally, the updating module 320 includes:

[0204] A third setting submodule, used to set an objective function according to the attribute information of the target server and the network structure of the first deep learning network;

[0205] A fourth setting submodule, configured to set the reward function based on the objective function;

[0206] An updating submodule is used to update the first deep learning network based on the objective function and the reward function to obtain the second deep learning network.

[0207] Optionally, the third setting submodule includes:

[0208] A first calculation unit, configured to calculate resource utilizations of the plurality of first virtual machines to obtain a plurality of first resource utilizations, wherein the plurality of first resource utilizations correspond one-to-one to the plurality of first virtual machines;

[0209] A second calculation unit, configured to calculate a standard deviation of the plurality of first resource utilizations to obtain a standard deviation;

[0210] A third calculation unit, configured to calculate load conditions of the plurality of first virtual machines to obtain a plurality of first load values, wherein the plurality of first load values ​​correspond one-to-one to the plurality of first virtual machines;

[0211] a fourth calculation unit, configured to calculate an average value of the plurality of first load values ​​to obtain an average load value;

[0212] A first acquisition unit is used to acquire the service quality of the target server after the t-th scheduling and the energy consumption after the t-th scheduling, where t is a positive integer;

[0213] A generating unit is used to generate the objective function according to the standard deviation, the average load value, the service quality of the target server after the t-th scheduling, the energy consumption of the target server after the t-th scheduling and the network structure of the first deep learning network, wherein the objective function is used to constrain the scheduling strategy of the target server for the t+1th time.

[0214] Optionally, the fourth setting submodule includes:

[0215] A second acquisition unit, configured to acquire a resource upper limit of the target server;

[0216] A calculation unit, configured to calculate a maximum number of the first virtual machines that are allowed to be simultaneously started in the target server according to a resource upper limit of the target server;

[0217] A third acquisition unit is used to acquire the service quality of the target server after the t-th scheduling and the energy consumption after the t-th scheduling according to the objective function;

[0218] The reward function is generated according to the service quality of the target server after the tth scheduling, the energy consumption of the target server after the tth scheduling and the maximum quantity value, and the reward function is used to constrain the scheduling strategy of the target server for the t+1th time.

[0219] This application matches the target server corresponding to the task to be processed, determines the corresponding first deep learning network according to the target server, and then generates the first deep learning network after updating the first deep learning network by setting a reward function, and generates a scheduling strategy for the target server to process the task to be processed through the second deep learning network, thereby improving the resource scheduling efficiency of the server.

[0220] The present application also provides an electronic device. Figure 3 , the electronic device may include a processor 301, a memory 302, and a program 3021 stored in the memory 302 and executable on the processor 301.

[0221] When the program 3021 is executed by the processor 301, it can achieve Figure 1 Any step in the corresponding method embodiment:

[0222] Acquire a plurality of first log data to be transmitted;

[0223] Classifying the plurality of first log data to obtain a plurality of classification sets, wherein the classification labels corresponding to all the first log data included in each classification set are the same;

[0224] Based on an arithmetic coding method, the multiple classification sets are respectively compressed to obtain multiple compressed target classification sets;

[0225] The plurality of target classification sets are sent to a target system.

[0226] Optionally, the compressing the multiple classification sets respectively based on the arithmetic coding method to obtain the compressed multiple target classification sets includes:

[0227] Determine, according to the Bayesian inference model and the arithmetic coding method, a plurality of coding intervals corresponding to a first classification set, wherein the first classification set is any one of the plurality of classification sets, and the first classification set includes a plurality of symbols;

[0228] Based on the arithmetic coding method, determining, among the multiple coding intervals, a coding interval corresponding to each symbol in the first classification set;

[0229] Based on the coding interval corresponding to each symbol, the multiple symbols are concatenated into multiple character strings, and the multiple character strings correspond to the multiple coding intervals one by one;

[0230] Convert the multiple character strings into multiple binary sequences, and concatenate the multiple binary sequences to obtain a target binary sequence, wherein the multiple binary sequences correspond one-to-one to the multiple character strings;

[0231] The target binary sequence is divided into a plurality of data blocks of preset lengths, and the plurality of data blocks of preset lengths are converted into a plurality of byte sequences to obtain a target classification set corresponding to the first classification set.

[0232] Optionally, dividing the target binary sequence into a plurality of data blocks of preset lengths, and converting the plurality of data blocks of preset lengths into a plurality of byte sequences includes:

[0233] Setting a weight-optimized hash function and an adaptive matching algorithm, wherein the weight-optimized hash function is used to adjust the rate of dividing data blocks, and the adaptive matching algorithm is used to adjust the length of the data blocks;

[0234] Based on a weight-optimized hash function and an adaptive matching algorithm, the target binary sequence is divided into a plurality of data blocks of preset lengths, and the plurality of data blocks of preset lengths are converted into a plurality of corresponding byte sequences.

[0235] Optionally, the classifying the plurality of first log data to obtain a plurality of classification sets includes:

[0236] At least one first classification label is generated according to the plurality of first log data, wherein the label type included in the at least one first classification label includes at least one of the following: a log type label, an importance level label, and an event type label; the log type label includes at least one of the following: a system log type label, an application log type label, and a security log type label; the event type label includes at least one of the following: an information event type label, a warning event type label, and an error event type label;

[0237] Filtering duplicate content and erroneous content in the plurality of first log data to obtain filtered first log data;

[0238] The filtered first log data is classified according to at least one first classification label to obtain a plurality of classification sets, wherein each classification set corresponds to at least one first classification label.

[0239] Optionally, after respectively compressing the multiple classification sets based on the arithmetic coding method to obtain multiple compressed target classification sets, the method further includes:

[0240] Detecting the multiple target classification sets;

[0241] In the case that there is a target classification set whose data volume exceeds a target threshold value among the multiple target classification sets, a target filtering rule is set to reclassify the multiple first log data.

[0242] This application matches the target server corresponding to the task to be processed, determines the corresponding first deep learning network according to the target server, and then generates the first deep learning network after updating the first deep learning network by setting a reward function, and generates a scheduling strategy for the target server to process the task to be processed through the second deep learning network, thereby improving the resource scheduling efficiency of the server.

[0243] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned method for generating scheduling information is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0244] An embodiment of the present application further provides a computer program product, which is stored in a storage medium. The computer program product is executed by at least one processor to implement the various processes of the above-mentioned generation method based on scheduling information, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0245] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0246] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0247] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for generating scheduling information, characterized in that: The method comprises: Matching a target server corresponding to the task to be processed from a plurality of candidate servers, wherein the target server includes a plurality of first virtual machines for processing the task to be processed; A reward function is set according to the target server and the first deep learning network, and the first deep learning network is updated based on the reward function to obtain a second deep learning network, wherein the first deep learning network matches the target server, and the first deep learning network is used to generate a scheduling strategy for the target server to process the task to be processed; The task to be processed is input into the second deep learning network for processing, and a target scheduling strategy is generated, where the target scheduling strategy is used to allocate at least one of the virtual machines to the task to be processed.

2. The method according to claim 1, characterized in that Before setting the reward function according to the target server and the first deep learning network and updating the first deep learning network based on the reward function to obtain the second deep learning network, the method further includes: Determine an initial deep learning network that matches the target server, wherein the initial deep learning network includes a Critic network and an Actor network; Initialize the parameters of the Critic network and initialize the parameters of the Actor network; Inputting training sample data into the initial deep learning network for recognition to obtain an output value, wherein the training sample data includes a plurality of sample tasks and training labels, and the training labels are scheduling strategies for processing the plurality of sample tasks; Using a loss function to calculate the loss between the output value and the training label to obtain a loss value; The Critic network and the Actor network are updated based on the loss value to obtain the first deep learning network.

3. The method according to claim 2, characterized in that The virtual machine types of the plurality of first virtual machines include K types, and the determining an initial deep learning network matching the target server includes: A K*(N+2) matrix is ​​set according to the target server, wherein the first column of the K*(N+2) matrix is ​​used to record the K types, the second to Nth columns of the K*(N+2) matrix are used to record the identifiers of the virtual machines of the corresponding types, the N+1th column of the K*(N+2) matrix is ​​used to record the working status of all virtual machines of the corresponding types, the N+2th column of the K*(N+2) matrix is ​​used to record the current queued task information of all virtual machines of the corresponding types, wherein N is a positive integer, and K is a positive integer; Generate an input expression and an output expression of the target server based on the K*(N+2) matrix, wherein the input expression is an expression of the first virtual machine pre-assigned to the target server for processing tasks, and the output expression is an expression of the tasks processed by the target server; An initial deep learning network matching the target server is determined according to the input expression and the output expression.

4. The method according to claim 3, characterized in that The step of inputting the training sample data into the initial deep learning network for recognition to obtain an output value includes: Fill the training sample data into the N+2th column of the input expression and the second column of the output expression to generate a target input expression and a target output expression corresponding to the training sample data; Determine the Gaussian noise corresponding to the training sample data according to the target input expression; Performing feature extraction on the target input expression to obtain target features; The target feature and the Gaussian noise are input into the Actor network for recognition to obtain the output value.

5. The method according to claim 3, characterized in that: The reward function is set according to the target server and the first deep learning network, and the first deep learning network is updated based on the reward function to obtain a second deep learning network: Setting a reward function according to the target server and the first deep learning network; Putting the reward function, the input expression and the output expression into the experience pool of the first deep learning network; When the experience pool of the first deep learning network is full, the first deep learning network is iteratively updated using a gradient descent method to obtain the second deep learning network.

6. The method according to claim 1, characterized in that The step of setting a reward function according to the target server and the first deep learning network, and updating the first deep learning network based on the reward function to obtain a second deep learning network includes: Setting an objective function according to the attribute information of the target server and the network structure of the first deep learning network; Setting the reward function based on the objective function; The first deep learning network is updated based on the objective function and the reward function to obtain the second deep learning network.

7. The method according to claim 6, characterized in that The setting of the objective function according to the attribute information of the target server and the network structure of the first deep learning network includes: Calculating resource utilization rates of the plurality of first virtual machines to obtain a plurality of first resource utilization rates, wherein the plurality of first resource utilization rates correspond one-to-one to the plurality of first virtual machines; Calculating standard deviations of the plurality of first resource utilizations to obtain a standard deviation; Calculating load conditions of the multiple first virtual machines to obtain multiple first load values, where the multiple first load values ​​correspond one-to-one to the multiple first virtual machines; Calculating an average of the plurality of first load values ​​to obtain an average load value; Obtaining the service quality and energy consumption of the target server after the t-th scheduling, where t is a positive integer; The objective function is generated according to the standard deviation, the average load value, the service quality of the target server after the tth scheduling, the energy consumption of the target server after the tth scheduling, and the network structure of the first deep learning network. The objective function is used to constrain the scheduling strategy of the target server for the t+1th time.

8. The method according to claim 7, characterized in that The step of setting the reward function based on the objective function comprises: Obtaining the resource upper limit of the target server; Calculate the maximum number of the first virtual machines that are allowed to be started simultaneously in the target server according to the resource upper limit of the target server; Obtaining the service quality of the target server after the t-th scheduling and the energy consumption after the t-th scheduling according to the objective function; The reward function is generated according to the service quality of the target server after the tth scheduling, the energy consumption of the target server after the tth scheduling and the maximum quantity value, and the reward function is used to constrain the scheduling strategy of the target server for the t+1th time.

9. A device for generating scheduling information, characterized in that: The device comprises: A matching module, used to match a target server corresponding to the task to be processed from a plurality of candidate servers, wherein the target server includes a plurality of first virtual machines for processing the task to be processed; An updating module, configured to set a reward function according to the target server and the first deep learning network, and to update the first deep learning network based on the reward function to obtain a second deep learning network, wherein the first deep learning network matches the target server, and the first deep learning network is used to generate a scheduling strategy for the target server to process the task to be processed; A generation module is used to input the task to be processed into the second deep learning network for processing, and generate a target scheduling strategy, wherein the target scheduling strategy is used to allocate at least one of the virtual machines to the task to be processed.

10. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps in the method for generating scheduling information as described in any one of claims 1 to 8.

11. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps in the method for generating scheduling information according to any one of claims 1 to 8 are implemented.

12. A computer program product, characterized in that The computer program product is stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps in the method for generating scheduling information according to any one of claims 1 to 8.

Images