Cloud-edge collaborative resource scheduling method, system, electronic device and readable medium

Abstract

The present invention discloses a cloud-edge collaborative resource scheduling method, system, electronic device and readable medium, the method comprising: establishing energy consumption models for IT equipment, power supply equipment and cooling equipment in the cloud-edge collaborative system respectively; setting the energy consumption optimization target of the cloud-edge collaborative system based on the energy consumption model; and using a reinforcement learning algorithm to generate resource scheduling decisions based on the current resource status, task request and energy consumption of the cloud-edge collaborative system to minimize the energy consumption optimization target. The cloud-edge collaborative resource scheduling method, system, electronic device and readable medium provided by the present invention respond to low-carbon operation and reduce energy consumption in a cloud-edge collaborative resource scheduling scenario for computing-network integration, while taking into account the energy consumption of IT equipment and infrastructure of edge clusters and data centers, and performing multi-dimensional joint optimization.

Description

Technical Field

[0001] The present invention belongs to the technical field of computing resource scheduling, and specifically relates to a cloud-edge collaborative resource scheduling method, system, electronic device and readable medium. Background Art

[0002] With the rapid development of information technology, especially the widespread application of artificial intelligence, cloud computing, 5G and other technologies, the demand for computing resources has shown unprecedented growth. As a computing architecture that combines centralized cloud computing with distributed edge computing, the cloud-edge collaborative system has become an effective solution to meet the current complex computing needs. The system is usually composed of several cloud computing centers with powerful computing capabilities and edge computing clusters widely deployed in various regions, in order to improve the overall computing resource utilization efficiency while ensuring data processing speed and reducing network latency.

[0003] In this context, the scheduling problem of cloud-edge computing power is particularly critical. Reasonable resource scheduling can not only ensure that computing tasks can be efficiently and accurately assigned to the most appropriate processing nodes, but also dynamically adjust resource allocation strategies according to real-time business needs and system status, thereby maximizing the performance of the overall system. However, with the development of technology and the increasing complexity of application scenarios, existing resource scheduling solutions face various challenges. On the one hand, these solutions often fail to fully consider the synergy between cloud computing centers and edge computing clusters, resulting in low resource utilization and inability to meet the growing computing needs; on the other hand, the complexity of the scheduling algorithm itself has also become an important factor restricting its practical application.

[0004] Therefore, it is necessary to provide a new solution to the above technical problems. Summary of the invention

[0005] The purpose of the present invention is to provide a cloud-edge collaborative resource scheduling method, system, electronic device and readable medium, which can reasonably schedule system resources and improve resource utilization.

[0006] To achieve the above purpose, the technical solution provided by the present invention is as follows:

[0007] In a first aspect, the present invention provides a cloud-edge collaborative resource scheduling method, which can be applied to a cloud-edge collaborative system including IT equipment, power supply equipment, and cooling equipment, wherein the IT equipment includes a cloud computing center and an edge device, and the method includes:

[0008] Energy consumption models are established for the IT equipment, power supply equipment and cooling equipment in the cloud-edge collaborative system respectively; the energy consumption optimization target of the cloud-edge collaborative system is set based on the energy consumption model; based on the current resource status, task requests and energy consumption of the cloud-edge collaborative system, a reinforcement learning algorithm is used to generate resource scheduling decisions to minimize the energy consumption optimization target.

[0009] In one or more embodiments, the energy consumption model of the IT equipment is expressed as:

[0010]

[0011] Among them, E(X t ) is the resource scheduling decision X for all IT devices in the cloud-edge collaborative system t The total energy consumption under the condition, i is the number of each IT equipment, t is the time slot index, Calculate the energy consumption of the CPU of IT device i executing tasks locally in time slot t, is the memory storage energy consumption of IT device i executing tasks locally in time slot t, Calculate the energy consumption of the CPU of IT device i to offload all tasks in time slot t, is the overflow of the local task queue of IT device i in time slot t. hour when hour ω is the penalty coefficient after the local task queue overflows.

[0012] In one or more embodiments,

[0013]

[0014]

[0015]

[0016] in, is the total number of task applications generated by IT device i in time slot t, and the jth task generated by IT device i in time slot t is expressed as when When executing locally otherwise is the task data size of the jth task generated by IT device i in time slot t, f i is the CPU frequency of IT device i, ρ i is the energy consumption of CPU calculation per unit time of IT device i, μ represents the calculation density of unit data, is the amount of data accumulated in the task queue of IT device i in time slot t, and σ is the proportional coefficient.

[0017] In one or more embodiments,

[0018]

[0019]

[0020] in, Indicates that the task request is offloaded to the i-th edge server for calculation. For the The CPU frequency of each IT device, For IT equipment The energy consumption required for CPU calculation per unit time, μ represents the calculation density of unit data, Indicates IT equipment The cumulative amount of data that needs to be processed before queuing the current task, Apply for tasks Energy consumption during unloading, Indicates that IT equipment Execute task application The required CPU computing energy consumption.

[0021] In one or more implementations, the energy consumption model of the power supply device is expressed as:

[0022]

[0023]

[0024] in, For IT equipment i in resource scheduling decision X t The power consumption in the next time slot t is: is the scheduling strategy X for IT device i in time slot t t The required computing delay, the function f represents the relationship between the task running time and the energy consumption of the power supply equipment, and R represents the time required to unload unit data.

[0025] In one or more embodiments, the energy consumption model of the cooling device is expressed as:

[0026]

[0027] in, For IT equipment i in resource scheduling decision X t Cooling energy consumption in the next time slot t, is the scheduling strategy X for IT device i in time slot t t The required computing delay, function g represents the relationship between the task running time and the energy consumption of the cooling equipment.

[0028] In one or more embodiments, the energy consumption optimization target is expressed as:

[0029]

[0030] Among them, α1 is the energy consumption ratio coefficient of IT equipment, α2 is the energy consumption ratio coefficient of power supply equipment, α3 is the energy consumption ratio coefficient of cooling equipment, and γ represents the penalty factor for a single device exceeding the energy consumption limit. The degree to which IT equipment exceeds the energy consumption limit of a single device.

[0031] In one or more embodiments,

[0032]

[0033]

[0034]

[0035] in, is the maximum energy consumption limit of IT equipment i in time slot t, The amount of task data offloaded to IT device i by other devices.

[0036] In a second aspect, the present invention provides a cloud-edge collaborative resource scheduling system, which can be applied to a cloud-edge collaborative system including IT equipment, power supply equipment, and cooling equipment, wherein the IT equipment includes a cloud computing center and an edge device, and the system includes:

[0037] The energy consumption module is used to establish energy consumption models for IT equipment, power supply equipment and cooling equipment in the cloud-edge collaborative system respectively; the optimization module is used to set the energy consumption optimization target of the cloud-edge collaborative system based on the energy consumption model; the learning module is used to generate resource scheduling decisions based on the current resource status, task requests and energy consumption of the cloud-edge collaborative system, using a reinforcement learning algorithm to minimize the energy consumption optimization target.

[0038] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the cloud-edge collaborative resource scheduling method as described above is implemented.

[0039] In a fourth aspect, the present invention provides a computer-readable medium, which carries computer execution instructions. When the computer execution instructions are executed by a processor, they are used to implement the cloud-edge collaborative resource scheduling method as described above.

[0040] Compared with the prior art, the cloud-edge collaborative resource scheduling method, system, electronic device and readable medium provided by the present invention have the following advantages:

[0041] (1) In the cloud-edge collaborative resource scheduling scenario for computing-network integration, it responds to low-carbon operation and reduces energy consumption. It also considers the energy consumption of edge clusters and data center IT equipment and infrastructure, and performs multi-dimensional joint optimization. At the same time, it detects the energy consumption limit of a single edge device and controls the energy consumption of each device within a reasonable range. This is an aspect that existing methods have not considered.

[0042] (2) Learning strategies based on reinforcement learning methods, and using reinforcement learning for energy consumption optimization. There is no need to simulate a specific model, but the behavior of the system can be learned through interaction with the environment. Traditional optimization decision algorithms usually need to assume that the environment is static and predictable. Many conditions in cloud-edge collaboration scenarios are nonlinear, non-static, and non-deterministic. We can learn how to make the best decisions in such an environment.

[0043] (3) It is adaptive and can continuously improve its decision-making strategy in the process of continuous interaction with the environment, gradually approaching the optimal strategy. Even if the subsequent environment and rewards change, the allocation strategy can be changed through secondary learning, which has stronger dynamics. The intelligent agents in the system can gradually learn the characteristics and laws of the environment and find excellent strategies.

[0044] (4) The energy consumption behavior of the allocation strategy is evaluated through the positive and negative feedback of the reward signal. The agent adjusts its strategy and behavior according to the different reward signals to increase the chance of obtaining positive feedback. This method of using feedback information effectively learns and makes decisions, so that the predetermined goals can be achieved in different environments, making the resource allocation in the cloud-edge collaborative scheduling system more reasonable. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 This is a flow chart of a cloud-edge collaborative resource scheduling method in one embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a cloud-edge collaboration system in one embodiment of the present invention;

[0048] Figure 3 A schematic diagram of reinforcement learning in a cloud-edge resource scheduling scenario in one embodiment of the present invention;

[0049] Figure 4 Schematic diagram of a cloud-edge collaborative resource scheduling system in one embodiment of the present invention;

[0050] Figure 5 FIG. 4 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0052] In the field of information technology, the combination of cloud computing and edge computing technologies provides powerful data processing capabilities to meet the growing computing needs. However, with the expansion of the application of advanced technologies such as artificial intelligence and 5G, the requirements for cloud-edge computing scheduling have become more stringent. Not only does it need to efficiently process large amounts of data, but it also needs to optimize energy consumption in the process and promote the realization of green computing. Existing technologies have many shortcomings in terms of resource scheduling strategies, resource utilization, and the complexity of scheduling algorithms. These problems directly affect the performance and operating costs of the system, and are contrary to the current emphasis on environmental protection and energy conservation.

[0053] In view of this, the present invention aims to provide a cloud-edge collaborative resource scheduling method, which fully utilizes the synergistic advantages of cloud computing and edge computing through the establishment of a refined energy consumption model and an intelligent resource scheduling strategy, thereby improving the processing efficiency of computing tasks and reducing the energy consumption of the entire system, so as to achieve efficient and energy-saving cloud-edge collaborative operation.

[0054] The technical implementation ideas of the present invention are based on a deep understanding and comprehensive analysis of the shortcomings of the prior art. First, by establishing an accurate energy consumption model for key equipment in the cloud-edge collaborative system (including but not limited to IT equipment, power supply equipment and cooling equipment), the present invention can comprehensively monitor and evaluate the energy consumption during the operation of the system. Then, by setting clear energy consumption optimization goals, energy consumption optimization is used as an important consideration for resource scheduling decisions to ensure that resource scheduling strategies not only meet performance requirements, but also meet the goals of energy conservation and emission reduction. Finally, the present invention uses a reinforcement learning algorithm to dynamically generate optimal resource scheduling decisions based on the current resource status, task requests and energy consumption of the system to minimize the energy consumption optimization goals. The adaptability and intelligent characteristics of the algorithm enable the resource scheduling strategy to be continuously optimized in a constantly changing environment, ensuring that the system optimizes resource usage while meeting computing needs.

[0055] Please refer to Figure 1 The figure is a flow chart of a cloud-edge collaborative resource scheduling method in an embodiment of the present invention, which can be applied to a cloud-edge collaborative system including IT equipment, power supply equipment and cooling equipment, wherein the IT equipment includes a cloud computing center and edge equipment. The cloud-edge collaborative resource scheduling method specifically includes the following steps:

[0056] S101: Establish energy consumption models for IT equipment, power supply equipment and cooling equipment in the cloud-edge collaborative system respectively.

[0057] like Figure 2 As shown in the figure, consider a cloud-edge collaborative resource scenario consisting of a cloud computing center and multiple edge devices. The end device transmits the computing task to the nearest edge device through a wireless network for computing. It can be assumed that the task application is generated directly on the edge device. A total of M different task applications are considered. For each task application on the edge device, it has two operating modes: computing on the local edge device or offloading the task to other edge devices (or cloud centers) for computing. The symbol N is used to represent the number of edge servers, and the symbol n is used to represent the number of edge servers. i (0≤i≤N) represents the i-th IT device, n0 is used to represent the cloud center, and n i (i≠0) is used to represent edge devices.

[0058] The task application calculation in the cloud-edge collaboration scenario runs in a time slot manner. Let t be the time slot index, where t∈{1,2…T}. For all the task applications of the edge device in a time slot, they can be offloaded to other devices for calculation, or they can be calculated locally. represents the total number of task applications generated by the i-th edge device in time slot t, using represents the jth task of the i-th edge device in time slot t. t represents the decision matrix of cloud-edge resource collaboration in time slot t, where represents the scheduling strategy for the jth task request of the i-th edge device in time slot t, Indicates that the task request is offloaded to the i-th edge server for calculation. Indicates offloading to the cloud center for computing.

[0059] In an exemplary embodiment, the energy consumption model of the IT equipment is expressed as:

[0060]

[0061] Among them, E(X t ) is the resource scheduling decision X for all IT devices in the cloud-edge collaborative system t The total energy consumption under the condition, i is the number of each IT equipment, t is the time slot index, Calculate the energy consumption of the CPU of IT device i executing tasks locally in time slot t, is the memory storage energy consumption of IT device i executing tasks locally in time slot t, Calculate the energy consumption of the CPU of IT device i to offload all tasks in time slot t, is the overflow of the local task queue of IT device i in time slot t. hour when hour ω is the penalty coefficient after the local task queue overflows.

[0062] For edge device i, the generated task application in time slot t if It represents the task application Perform calculations locally. Use Represents task application The service type is d m (1≤m≤M) represents the task data size of the mth task application, and the task application data of each task application is set to a fixed size. Therefore, the data size generated and executed locally in time slot t can be expressed as follows:

[0063]

[0064] in Only in When the value is 1, it means that the task on the i-th edge device is executed locally on the i-th edge device. The computational energy consumption of local execution is directly related to the amount of task data executed locally, and the amount of task data in the local time slot t is not only related to the currently generated data. It is also related to the amount of data accumulated in the previous queue. The unit is Bits. Therefore, the CPU computing energy consumption of local execution can be expressed by the following formula:

[0065]

[0066] where f i represents the CPU frequency of the i-th edge device, ρ i represents the energy consumption required for device calculation per unit time, and μ represents the calculation density per unit data. As the time slot t changes, it can be updated by the following formula:

[0067]

[0068] where [x]+ =max(x,0), The capacity of the local computing queue, τ represents the length of a single time slot. Computation tasks that exceed the maximum capacity of the queue will be discarded. The larger the backlog of data in the local computing queue, the greater the local storage energy consumption of the current edge device, so the local execution memory storage energy consumption The amount of backlog data Proportional, σ is used to represent the proportionality coefficient, which can be expressed by the following formula:

[0069]

[0070] For the backlog of data If it exceeds the maximum capacity of the local queue, it means that the local task queue has overflowed, which also proves that some tasks cannot be completed in time, indicating that the current scheduling strategy is imperfect. For this reason, a penalty is defined to define the overflow of the local queue. The formula is as follows:

[0071]

[0072] when It proves that the scheduling strategy of edge device i in time slot t causes the local storage queue to overflow, and the resource scheduling strategy should avoid this situation.

[0073] The energy consumption required for offloading to other nodes or cloud center computing consists of three parts: the energy consumption during task offloading, the energy consumption during task queuing, and the energy consumption required for task computing. For the scheduling strategy for the jth task request of the i-th edge device in time slot t, Indicates that the task request is offloaded to the i-th edge server for calculation.

[0074] First consider the task application Energy consumption during unloading, symbol Represents task application The data size of the corresponding service type, μ represents the computing density of unit data, ε represents the energy consumption required for unloading unit data, and the energy consumption generated by the unloading process can be expressed as follows:

[0075]

[0076] Consider the mission application The energy consumption required in the queuing process belongs to The energy consumption generated by the edge server, not the energy consumption generated by the server that currently generates the task. Similarly, before queuing the current task, you need to process the accumulated data first, you can use The energy consumption required to process the accumulated data is expressed as follows:

[0077]

[0078] in Indicates The CPU frequency of each edge device, represents the energy consumption per unit time required for device calculation, and μ represents the calculation density per unit data. Execute tasks on edge devices The required computing energy consumption can be expressed by the following formula:

[0079]

[0080] For the i-th edge server, the total energy consumption required to offload all tasks on the CPU is:

[0081]

[0082] For the i-th edge server, Represents other edge devices unloading to the first The amount of data on the edge device, the total energy consumption required to offload all tasks in memory is:

[0083]

[0084] Therefore, the total IT equipment energy consumption required in time slot t during cloud-edge collaborative scheduling can be expressed by the following formula:

[0085]

[0086] The first half of the formula represents the total energy consumption required for the normal operation of IT equipment, and the symbol ω in the second half represents the penalty coefficient after the task queue overflows. Only in That is, when the task queue overflows, the value is 1, and in other cases, the value is 0. This means that more energy consumption is required when the task queue overflows.

[0087] In an exemplary embodiment, the energy consumption models of the power supply device and the cooling device are respectively expressed as:

[0088]

[0089]

[0090] in, For IT equipment i in resource scheduling decision X t The power consumption in the next time slot t is: For IT equipment i in resource scheduling decision Xt Cooling energy consumption in the next time slot t, is the scheduling strategy X for IT device i in time slot t t The required calculation delay, function f represents the relationship between the task running time and the energy consumption of the power supply equipment, and function g represents the relationship between the task running time and the energy consumption of the cooling equipment.

[0091] The energy consumption of power supply equipment and cooling equipment is usually positively correlated with the delay of task execution. The longer the task execution time is, the greater the energy consumption of power supply equipment and cooling equipment. t , the computation delay required by the i-th edge server can be expressed as follows:

[0092]

[0093] Where R represents the time required to unload a unit of data, that is, the unloading rate. Therefore, the energy consumption of power supply equipment and cooling equipment can be measured using sensors deployed on the infrastructure, and the relationship between them and the task running time can be fitted, which may be a linear function or a nonlinear function.

[0094] S102: Setting energy consumption optimization targets for the cloud-edge collaborative system based on the energy consumption model.

[0095] For a cloud center device or edge device i, its maximum energy consumption in time slot t is set not to exceed For the task queue generated on edge device i, the energy consumption of local task execution can be expressed as The amount of task data offloaded from other devices to edge device i is expressed as The energy consumption of executing the task can be expressed as

[0096]

[0097] Therefore, the total amount of IT equipment, power supply equipment, and cooling equipment required by edge device i in time slot t can be expressed as

[0098]

[0099] For single device energy consumption exceeding the limit situation, that is, when When the total energy consumption is increased, a penalty is added to tell it that it is not a good scheduling strategy if the energy consumption of a single device exceeds the limit.

[0100] According to the scheduling strategy X t ,E(X t ) represents the energy consumption index of the total IT equipment, Indicates the energy consumption index of power supply equipment in infrastructure. Represents the energy consumption index of cooling equipment in infrastructure, where E(X t ) takes into account the scheduling situation where the tasks in a single edge device accumulate and exceed the maximum task queue.

[0101] In addition, it is necessary to consider the situation where the energy consumption of a single device exceeds the limit. Indicates whether the energy consumption of edge device i in time slot t exceeds its limit and the extent to which the limit is exceeded, The calculation formula is as follows:

[0102]

[0103] Therefore, we can set the optimization target V(X t ):

[0104]

[0105] Among them, α1 is the energy consumption ratio coefficient of IT equipment, α2 is the energy consumption ratio coefficient of power supply equipment, α3 is the energy consumption ratio coefficient of cooling equipment, and γ represents the penalty factor for a single device exceeding the energy consumption limit.

[0106] The optimization goal is to find the scheduling strategy X for each time slot t using strategy π t , so that the energy consumption in the entire scheduling process reaches the minimum value, which can be specifically expressed as:

[0107]

[0108] S103: Based on the current resource status, task requests and energy consumption of the cloud-edge collaborative system, a reinforcement learning algorithm is used to generate resource scheduling decisions to minimize the energy consumption optimization target.

[0109] It should be noted that reinforcement learning (RL) is a machine learning (ML) technique that can make decisions through training to achieve the best results. It mimics the trial-and-error learning process that humans take to achieve their goals. Actions that contribute to the optimization goal are reinforced, while actions that deviate from the goal are ignored. Reinforcement learning algorithms use a reward and punishment model when processing data. These algorithms learn from the feedback of each action and discover the best processing path to achieve the final result on their own. Reinforcement learning is a powerful method that can help intelligent systems achieve the best results in unseen environments. PPO (Proximal Policy Optimization) is an On Policy reinforcement learning algorithm, due to its simple implementation, easy to understand, stable performance, ability to handle both discrete and continuous action space problems, and conducive to large-scale training.

[0110] Reinforcement learning can make corresponding cloud-edge collaborative scheduling decisions based on the status, that is, the resource status, queue status, and task request status in the cloud-edge collaborative scenario at this moment, and interactively learn with the environment, thereby continuously optimizing the cloud-edge collaborative resource scheduling system during the learning process.

[0111] The schematic diagram of reinforcement learning in the cloud-edge resource scheduling scenario in the present invention is as follows: Figure 3 As shown in the figure, the scheduling strategy in the cloud-edge collaborative resource scheduling system is given by the intelligent agent. In the reinforcement learning process, the intelligent agent is based on the current cloud-edge collaborative environment state S t Generate an action A t , which represents the scheduling strategy X in the current state t , then according to action A t The interaction with the environment, the cloud-edge collaboration state is S t Transformed to S t+1 , and returns a reward R t The reward is the environment telling the agent whether the current action has good performance. In the above problem, the reward is the energy consumption optimization target of the current time slot. If action A is executed t After that, the energy consumption is reduced, and the agent can be told to act A t Energy consumption can be optimized, and the next time the state S is encountered t When selecting action A, t The probability of taking action A t After that, the energy consumption increases, so the action A in the current state is reduced. t Probability of occurrence.

[0112] Therefore, the PPO-based cloud-edge collaborative resource scheduling system defines the following state, environment, reward, and action information.

[0113] In the present invention, the action is defined as the task offloading scheduling scheme X t , represents the scheduling allocation for application tasks in time slot t. Among them, for the scheduling strategy of the jth task request for the i-th edge device in time slot t, Indicates that the task request is offloaded to the i-th edge server for calculation, where The value of the action is discrete, ranging from So the action can be defined as A t ={X t}.

[0114] Next, we define the state in PPO. The state mainly includes the task data accumulated in each edge node and the task queue of the cloud data center in each time slot t. Task application request queue for time slot t So the state can be expressed as Backlogged data queue and task application request queue It has a very large state space, so it is difficult to solve with traditional methods.

[0115] The reward in reinforcement learning is defined as minimizing energy consumption. The energy optimization goal of the cloud-edge resource collaborative scheduling system is defined as V(X t ), where α represents the proportional coefficient of energy consumption of IT equipment, power supply equipment and cooling equipment, which can be adjusted according to the actual application situation. Since the optimization target is to be minimized, and reinforcement learning generally maximizes the reward value, the reward is defined as R t =-V(X t ).

[0116] Due to the status of the set cloud-edge collaborative environment The next state is only related to the previous state, but not to the previous state. Therefore, the cloud-edge collaborative environment conforms to the Markov decision process, and can find the optimal decision to maximize the cumulative reward. And reinforcement learning can learn how to make the best decision in such an environment without knowing the environmental model.

[0117] The cloud-edge collaborative resource scheduling system provided by the present invention focuses on the energy consumption of data centers and infrastructure, and can also focus on the energy consumption limits of single-point edge clusters to make reasonable optimization scheduling. And consider reusing the collected data. For the previously made scheduling strategies, i.e., actions, the corresponding states and rewards are saved in the memory bank. After collecting enough data, the parameters are adjusted using gradient descent according to the loss function.

[0118] The cloud-edge collaborative resource scheduling system provided by the present invention has three sets of network parameters and two model architectures, namely, actor network and critic network, i.e., actor-critic architecture. The actor network is responsible for interacting with the state in the environment to generate corresponding actions; while the critic updates the evaluation of the state in a supervised learning manner based on the collected data. The higher the evaluation score of the current state, the more accurate the action taken by the system for the current state, and the better scheduling decision can be made.

[0119] For the resource scheduling scheme in this application scenario, the workflow of the cloud-edge collaborative resource scheduling system provided by the present invention is as follows: In the initial state, when the time slot t is 0, according to the current state Interact with the environment, make corresponding scheduling decisions to generate the next state, and generate corresponding rewards. When the state changes from S0 to S1, repeat the above steps again, generate scheduling decision action A1, and get reward R1. After executing the action in each time slot t, the state S of the current time slot t is t 、Action A t , Reward R t And the subsequent state S t+1 After a certain amount of data is collected, for each data sequence collected, t ,A t ) to make an advantage assessment. In the cloud-edge collaborative resource scheduling system provided by the present invention, a critic model is also introduced to evaluate the current state S t The average expected reward of the current state S t The corresponding reward for all subsequent time slots minus the score evaluated by the critic model is used to evaluate the performance of the algorithm in S t Next, execute action A t Is it reasonable? Is the reward higher? If the action is evaluated well, it will be related to the state S in the future. t In a similar situation, we can make action A t The probability of occurrence increases.

[0120] The cloud-edge collaborative resource scheduling system provided by the present invention can learn how to make the best decision in such an environment in the face of large-scale state space and dynamically changing environment in cloud-edge resource scenarios without knowing the environmental model. PRSS is adaptive and can continuously adjust strategies and behaviors according to environmental changes and feedback signals, realize joint optimization based on considerations of IT equipment and infrastructure, detect and track green and low-carbon indicators of infrastructure equipment, take into account the scheduling of task resources and the load of a single device, and try to control the carbon emissions of each device within a reasonable range.

[0121] Please refer to Figure 4 As shown, based on the same inventive concept as the aforementioned cloud-edge collaborative resource scheduling method, the present invention provides a cloud-edge collaborative resource scheduling system 400, which includes: an energy consumption module 401, an optimization module 402 and a learning module 403.

[0122] The energy consumption module 401 is used to establish energy consumption models for IT equipment, power supply equipment and cooling equipment in the cloud-edge collaborative system. The optimization module 402 is used to set the energy consumption optimization target of the cloud-edge collaborative system based on the energy consumption model. The learning module 403 is used to generate resource scheduling decisions based on the current resource status, task requests and energy consumption of the cloud-edge collaborative system using a reinforcement learning algorithm to minimize the energy consumption optimization target.

[0123] In summary, the cloud-edge collaborative resource scheduling method and system provided by the present invention have the following advantages:

[0124] (1) In the cloud-edge collaborative resource scheduling scenario for computing-network integration, it responds to low-carbon operation and reduces energy consumption. It also considers the energy consumption of edge clusters and data center IT equipment and infrastructure, and performs multi-dimensional joint optimization. At the same time, it detects the energy consumption limit of a single edge device and controls the energy consumption of each device within a reasonable range. This is an aspect that existing methods have not considered.

[0125] (2) Learning strategies based on reinforcement learning methods, and using reinforcement learning for energy consumption optimization. There is no need to simulate a specific model, but the behavior of the system can be learned through interaction with the environment. Traditional optimization decision algorithms usually need to assume that the environment is static and predictable. Many conditions in cloud-edge collaboration scenarios are nonlinear, non-static, and non-deterministic. We can learn how to make the best decisions in such an environment.

[0126] (3) It is adaptive and can continuously improve its decision-making strategy in the process of continuous interaction with the environment, gradually approaching the optimal strategy. Even if the subsequent environment and rewards change, the allocation strategy can be changed through secondary learning, which has stronger dynamics. The intelligent agents in the system can gradually learn the characteristics and laws of the environment and find excellent strategies.

[0127] (4) The energy consumption behavior of the allocation strategy is evaluated through the positive and negative feedback of the reward signal. The agent adjusts its strategy and behavior according to the different reward signals to increase the chance of obtaining positive feedback. This method of using feedback information effectively learns and makes decisions, so that the predetermined goals can be achieved in different environments, making the resource allocation in the cloud-edge collaborative scheduling system more reasonable.

[0128] Please refer to Figure 5 As shown, an embodiment of the present invention further provides an electronic device 500, which includes at least one processor 501, a memory 502 (for example, a non-volatile memory), a memory 503, and a communication interface 504, and the at least one processor 501, the memory 502, the memory 503, and the communication interface 504 are connected together via a bus 505. At least one processor 501 is used to call at least one program instruction stored or encoded in the memory 502, so that the at least one processor 501 performs various operations and functions of the cloud-edge collaborative resource scheduling method described in various embodiments of this specification.

[0129] In the embodiments of the present specification, the electronic device 500 may include, but is not limited to, personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile electronic devices, smart phones, tablet computers, cellular phones, personal digital assistants (PDAs), handheld devices, messaging devices, wearable electronic devices, consumer electronic devices, and the like.

[0130] An embodiment of the present invention also provides a computer-readable medium, which carries computer execution instructions. When the computer execution instructions are executed by a processor, they can be used to implement various operations and functions of the cloud-edge collaborative resource scheduling method described in the various embodiments of this specification.

[0131] The computer-readable medium in the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0132] In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0133] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatuses, systems, and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0136] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A cloud-edge collaborative resource scheduling method, applied to a cloud-edge collaborative system including IT equipment, power supply equipment and cooling equipment, wherein the IT equipment includes a cloud computing center and edge equipment, characterized in that: The method comprises: Establish energy consumption models for IT equipment, power supply equipment, and cooling equipment in the cloud-edge collaborative system respectively; Setting an energy consumption optimization target for the cloud-edge collaborative system based on the energy consumption model; Based on the current resource status, task requests and energy consumption of the cloud-edge collaborative system, a reinforcement learning algorithm is used to generate resource scheduling decisions to minimize the energy consumption optimization goal; The energy consumption model of the IT equipment is expressed as: ； in, Make resource scheduling decisions for all IT devices in the cloud-edge collaborative system The total energy consumption under i is the number of each IT device, t is the time slot indicator, For IT equipment i In time slot t The CPU computing energy consumption of local execution tasks, For IT equipment i In time slot t Memory storage energy consumption of local execution tasks, For IT equipment i In time slot t Unload the CPU computing energy consumption of executing all tasks, For IT equipment i In time slot t In the case of a local task queue overflow, hour ,when hour , It is the penalty coefficient after the local task queue overflows; ； ； ； in, For IT equipment i In time slot The task data size of all task applications executed locally in the task applications generated within; For IT equipment i In time slot Total number of tasks and applications generated within IT equipment i In time slot The internally generated j The task application is represented as ,when When executing locally ,otherwise , For IT equipment i In time slot t The internally generated j Task Application The task data size, For IT equipment i CPU frequency, For IT equipment i The energy consumption required by CPU calculation per unit time, represents the computational density of unit data, For IT equipment i The task queue in the time slot t The amount of data accumulated in the is the proportionality coefficient; ； ； in, Indicates that the task request is offloaded to The computation is performed on edge servers. For the The CPU frequency of each IT device, For IT equipment The energy consumption required for CPU calculation per unit time, represents the computational density of unit data, Indicates IT equipment The cumulative amount of data that needs to be processed before queuing the current task, Apply for tasks Energy consumption during unloading, Indicates that IT equipment Execute task application The required CPU computing energy consumption; The energy consumption model of the power supply equipment is expressed as: ； ； in, For IT equipment i In resource scheduling decisions Next time slot t The power consumption within For IT equipment i For time slot Scheduling strategy within The required computational delay, function Indicates the relationship between task running time and energy consumption of power supply equipment, Indicates the time required to unload unit data; The energy consumption model of the cooling equipment is expressed as: ； in, For IT equipment i In resource scheduling decisions Next time slot t Cooling energy consumption inside For IT equipment i For time slot Scheduling strategy within The required computational delay, function Represents the relationship between task running time and cooling equipment energy consumption; The energy consumption optimization target is expressed as: ； in, is the energy consumption ratio coefficient of IT equipment, is the energy consumption ratio coefficient of the power supply equipment, is the energy consumption proportional coefficient of the cooling equipment, Indicates the penalty factor for a single device exceeding the energy consumption limit, For IT equipment i the extent to which the energy consumption limits of individual devices are exceeded; ； ； ； in, For IT equipment i In time slot t The maximum energy consumption limit within Offload to IT equipment for other devices i The amount of task data on .

2. A cloud-edge collaborative resource scheduling system for executing the cloud-edge collaborative resource scheduling method according to claim 1, applied to a cloud-edge collaborative system including IT equipment, power supply equipment and cooling equipment, wherein the IT equipment includes a cloud computing center and an edge device, characterized in that: The system comprises: The energy consumption module is used to establish energy consumption models for IT equipment, power supply equipment, and cooling equipment in the cloud-edge collaborative system. An optimization module, used to set an energy consumption optimization target of the cloud-edge collaborative system based on the energy consumption model; The learning module is used to generate resource scheduling decisions based on the current resource status, task requests and energy consumption of the cloud-edge collaborative system using a reinforcement learning algorithm to minimize the energy consumption optimization target.

3. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the cloud-edge collaborative resource scheduling method as described in claim 1 is implemented.

4. A computer-readable medium, characterized in that The computer-readable medium carries computer execution instructions, which, when executed by a processor, are used to implement the cloud-edge collaborative resource scheduling method as described in claim 1.

Images