Resource joint scheduling method and system facing edge cloud fusion industrial Internet, and server

Abstract

The invention belongs to the related technical field of wireless communication, and discloses a resource joint scheduling method and system oriented to an edge cloud fusion industrial internet, and a server. The method comprises the steps that a scheduling model in a long-time scheduling period of a current edge server is constructed, the long-time scheduling model comprises T discrete time slot sub-models, the optimization objective of each time slot sub-model is to minimize an objective function under a corresponding time slot, and the objective function comprises an equation # imgabs0 # Q (t) which is the length of an energy consumption virtual queue in a time slot t, the queue of each time slot meets the condition that # imgabs1 # t = 0,..., T-1; zeta (t) is the time delay satisfaction rate of all calculation task requests in the time slot t; and converting all the time slot sub-models into a Markov decision process, and performing optimization by adopting a deep reinforcement learning algorithm to obtain a memory, computing resource and bandwidth resource allocation strategy of each time slot in a long-time scheduling period. According to the method, a long-period resource allocation decision can be made, so that the time delay and the energy consumption of the industrial Internet of Things in a long period can be considered at the same time.

Description

Technical Field

[0001] The present invention belongs to the technical field related to wireless communications, and more specifically, relates to a resource joint scheduling method, system and server for edge-cloud integrated industrial Internet. Background Art

[0002] In the Industry 4.0 era, the number of mobile devices connected to the Industrial Internet of Things (IIoT) is rapidly increasing, bringing with it exponential growth in mobile data traffic. This has placed tremendous pressure on existing network architectures and scarce network resources. Due to the limited battery capacity and processing power of terminals, emerging industrial applications such as quality inspection, fault diagnosis, and remote operation require data to be uploaded to cloud centers for processing. However, real-time data transmission can lead to network congestion and fail to meet the low-latency and high-reliability requirements of the IIoT. To promote the development of high-concurrency, low-latency, and energy-efficient industrial applications, mobile edge computing (MEC) addresses the needs of IIoT applications by moving key network functions (e.g., communication, computing, and caching) to the network edge. Specifically, it offloads compute-intensive tasks to the network edge, where computing resources are relatively abundant. The network edge can pre-store the basic model data or database data required to complete the computing tasks, thus avoiding the high bandwidth and high latency of the cloud backhaul link.

[0003] However, due to the dynamic and time-varying network environment and the random and unpredictable nature of user requests, the mutual influence between caching decisions and bandwidth and computing resource allocation, the trade-off between latency and energy consumption performance indicators, and the dependency between consecutive time slots, it is difficult to perform long-term optimization of resource allocation to simultaneously take into account the latency and energy consumption of the Industrial Internet of Things over a long period of time to meet the strict requirements of the Industrial Internet of Things. Summary of the Invention

[0004] In response to the above-mentioned defects or improvement needs of the existing technology, the present invention provides a resource joint scheduling method, system and server for the edge-cloud integrated industrial Internet, which aims to make long-term resource allocation decisions to take into account the latency and energy consumption of the industrial Internet of Things over a long period of time.

[0005] To achieve the above objectives, according to a first aspect of the present invention, a resource joint scheduling method for edge-cloud integrated industrial Internet is provided, comprising:

[0006] Construct a scheduling model for the current edge server's long-term scheduling cycle, wherein the long-term scheduling cycle includes T time slots, and the long-term scheduling model includes T discrete time slot sub-models, each of which corresponds to a time slot, and each of which includes an optimization objective and constraints under the corresponding time slot. The optimization objective is to minimize the objective function under the corresponding time slot, and the objective function includes the factor Where E(t) is the total energy consumption caused by completing all tasks in time slot t, is the set total energy consumption limit, Q(t) is the energy consumption value of the energy consumption virtual queue in time slot t, and the queue in each time slot satisfies V is a set proportional coefficient, ζ(t) is the delay satisfaction rate of the completed task in time slot t; the constraints include memory constraints, computing resource allocation constraints, and bandwidth resource allocation constraints. The memory constraint indicates that the cache content in the corresponding time slot does not exceed the total memory of the edge server. The computing resource allocation constraint indicates that the computing resources allocated to all requests in the corresponding time slot do not exceed the total computing resources of the edge server. The bandwidth resource allocation constraint ensures that the bandwidth allocated to all requests in the corresponding time slot does not exceed the total bandwidth resources of the edge server.

[0007] All time slot sub-models are converted into Markov decision processes, and a deep reinforcement learning algorithm is used for optimization to obtain the memory, computing resource and bandwidth resource allocation strategy for each time slot in the long-term scheduling cycle.

[0008] Optionally, the expression of the memory constraint is:

[0009]

[0010] c n (t)∈{0,1};

[0011] Where C is the total memory of the edge server itself, c n (t) indicates whether the software data required to execute the nth task is cached in the current edge server in the current time slot t. 0 means it is not stored in the current edge server, and 1 means it is stored in the current edge server. t ={c1(t),...,c N (t)} represents the cache strategy in time slot t, q n Indicates the software data size required to execute the nth type of task, N is the number of task types, It is a set of time slot indexes within the long-term scheduling period.

[0012] Optionally, the expression of the computing resource allocation constraint is:

[0013]

[0014] Where, f k (t) represents the proportion of computing resources allocated to terminal k in the current time slot t, K is the number of terminals within the current edge server coverage, and f t ={f1(t),...,f K (t)} is the computing resource allocation strategy within time slot t, It is a set of time slot indexes within the long-term scheduling period.

[0015] Optionally, the bandwidth resource allocation constraint is expressed as:

[0016]

[0017] Where b k (t) represents the bandwidth ratio allocated to terminal k in the current time slot t, K is the number of terminals within the coverage of the current edge server, and b t ={b1(t),...,b K (t)} is the bandwidth resource allocation strategy within time slot t, It is a set of time slot indexes within the long-term scheduling period.

[0018] Optionally, the optimization objective is expressed as:

[0019]

[0020] Optionally, the expression of the delay satisfaction rate ζ(t) is:

[0021]

[0022] ζ k (t)=1, when

[0023] ζ k (t)=0, when

[0024] Where K is the number of terminals within the coverage of the current edge server, ζ k (t) represents task I k (t) Whether the service delay meets the maximum tolerable delay of the task, I k (t) represents the index of the computing task type generated by terminal k in time slot t, is the task type index set, N is the number of task types, T k (t) is task I k (t) service delay, Represents Task I k (t) maximum tolerable delay;

[0025] Among them, the service delay T k (t) is expressed as:

[0026]

[0027] Where, Indicates execution of task I k(t) Whether the required software data is cached in the current edge server, 0 means it is not stored in the current edge server, and 1 means it is stored in the current edge server. Indicates execution of task I k (t) Task I when the required software data is already cached on the edge server k (t) service delay, To perform mission I k (t) Task I when the required software data is not cached in the edge server k (t) service delay;

[0028] The expression of total energy consumption E(t) is:

[0029]

[0030] Where, E k (t) To complete Task I k (t) the total energy consumption caused, Indicates execution of task I k (t) Task I is completed when the required software data is cached on the edge server k (t) total energy consumption, To perform mission I k (t) Complete Task I when the required software data is not cached on the edge server k (t) total energy consumption.

[0031] Optionally, Including user data upload delay and edge server processing task I k (t) processing delay;

[0032] Including the transmission energy consumption of terminal k transmitting user data to the edge server and the edge server processing task I k (t) calculation energy consumption;

[0033] Including user data upload delay, Task I k (t) The return delay of the required software data from the cloud to the edge server and the edge server processing task I k (t) processing delay;

[0034] Including the transmission energy consumption of terminal k transmitting user data to the edge server, task I k (t) The energy consumption of the return trip of the required software data from the cloud to the edge server and the energy consumption of the edge server processing the computing task I k (t) is the energy consumption of calculation.

[0035] Optionally, the deep reinforcement learning algorithm is a parameterized deep Q-network algorithm.

[0036] According to a second aspect of the present invention, an edge server is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0037] According to a third aspect of the present invention, a resource joint scheduling system for edge-cloud integrated industrial Internet is provided, which includes:

[0038] The terminal layer includes terminals that randomly access the network;

[0039] An edge server layer, comprising an edge server cluster consisting of a plurality of edge servers, wherein the edge servers are the edge servers described above;

[0040] Cloud data centers are used to provide the required software and data for the edge server layer.

[0041] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0042] The resource joint scheduling method mentioned in the present invention quickly obtains resource allocation decisions by modeling and solving the model. When modeling, three basic constraints, memory constraints, computing resource allocation constraints, and bandwidth resource allocation constraints, are taken into account to ensure the normal operation of the edge server. In addition, long-term energy consumption constraints are added and the optimization goal is to maximize the long-term average delay satisfaction rate. This can ensure that the service delay of as many tasks as possible meets their delay requirements on the basis of not exceeding its upper limit of long-term energy consumption, while taking into account the delay and energy consumption of the industrial Internet of Things in the long term. Since the original model built based on the above constraints and optimization goals is a long-term optimization problem, it corresponds to a multi-stage random mixed integer nonlinear programming problem. Therefore, directly solving the above long-term optimization problem requires mastering the state of each time slot. When the number of time slots is too large, the model solution is more difficult. Therefore, in order to reduce the difficulty of model solution, the present invention optimizes and transforms the above long-term scheduling original model, constructs an energy consumption virtual queue, and applies Lyapunov optimization theory to split the long-term constraints, thereby decoupling the optimization problem of the original model into a single-stage deterministic subproblem, that is, obtaining the time slot submodel proposed by the present invention. Subsequently, the constructed model is converted into a Markov decision process, and a deep reinforcement learning algorithm is used for optimization, which can quickly output the resource allocation strategy for each time slot in the long scheduling cycle. This strategy can also take into account the latency and energy consumption of the industrial Internet of Things in the long period. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1It is a schematic diagram of a resource joint scheduling system for edge-cloud integrated industrial Internet;

[0044] Figure 2 is a flowchart of the steps of a resource joint scheduling method in one embodiment of the present invention;

[0045] Figure 3 is a network structure diagram of a PDQN in one embodiment of the present invention;

[0046] Figure 4 is a graph of convergence performance when the PDQN algorithm is used in one embodiment;

[0047] Figure 5 This is a performance comparison chart of the delay satisfaction rate under different solutions in one embodiment;

[0048] Figure 6 This is a performance comparison chart of total system energy consumption under different solutions in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0050] First, the terms involved in the present invention are explained as follows:

[0051] Edge-cloud fusion refers to a technical architecture that integrates cloud data centers, edge servers, and terminals. It aims to optimize resource allocation, improve system efficiency, and meet the needs of diverse scenarios through layered collaboration.

[0052] The Industrial Internet refers to an industrial ecosystem that achieves digitization, networking, and intelligence of the entire industrial production process through the deep integration of new-generation information technologies such as the Internet of Things, big data, and artificial intelligence with industrial systems, thereby improving efficiency, reducing costs, and creating new value.

[0053] Network resources refer to different types of resources allocated to edge servers, including communication resources, computing resources, cache resources, etc.

[0054] Edge computing refers to a distributed computing paradigm that moves computing, storage, and data processing capabilities from traditional centralized cloud computing to data sources close to the source. By processing data locally, reliance on the cloud is reduced, thereby reducing latency, optimizing bandwidth usage, and improving system real-time performance and privacy.

[0055] like Figure 1Figure 1 shows a schematic diagram of a resource joint scheduling system for edge-cloud converged industrial internet. From near to far, the system consists of a terminal, an edge server, and a cloud data center. Within each time slot, a terminal randomly generates a computing task and uploads it to an edge server via a wireless link for processing, essentially issuing a task request to the corresponding edge server. To execute the computing task, the edge server requires corresponding user data and software data. It can collect the required user data from the terminal via a wireless link and download the required software data from the cloud data center. The software data can also be pre-cached on the edge server for reuse. When the software data required for the user-requested computing task is cached on the edge server, it can be retrieved directly from memory, saving the latency and energy consumption required to retrieve the data from the cloud data center. After the terminal issues a task request, the edge server executes the task and returns the result to the terminal. This process is considered task completion. The duration involved is the task's service latency, and the energy consumed by this process is the task's energy consumption. The objective of this invention is to formulate long-term resource allocation decisions for edge servers, ensuring that the service latency of as many tasks as possible meets their latency requirements without exceeding the upper limit of long-term energy consumption, thereby improving the task's latency satisfaction rate.

[0056] Example 1

[0057] This paper proposes a resource joint scheduling method for edge-cloud integrated industrial Internet. Figure 2 FIG1 is a flowchart of the steps of the resource joint scheduling method in one embodiment of the present invention, and each step thereof is introduced below.

[0058] S1. Construct a scheduling model for the current edge server's long-term scheduling period, wherein the long-term scheduling period includes T time slots, and the long-term scheduling model includes T discrete time slot sub-models, each of which corresponds to a time slot, and each of which includes an optimization objective and constraints under the corresponding time slot, wherein the optimization objective is to minimize the objective function under the corresponding time slot, and the objective function includes the factor Where E(t) is the total energy consumption caused by completing all tasks in time slot t, is the set total energy consumption limit, Q(t) is the energy consumption value of the energy consumption virtual queue in time slot t, and the queue in each time slot satisfies V is the set proportional coefficient, ζ(t) is the delay satisfaction rate of completing the task within time slot t; the constraints include memory constraints, computing resource allocation constraints and bandwidth resource allocation constraints.

[0059] Specifically, before modeling, the system parameters need to be determined.

[0060] In one embodiment, it is assumed that there are K users (terminals) within the coverage of the current edge server, and the user index is represented as Taking T time slots as the scheduling period, it is necessary to optimize the scheduling of each time slot within T time slots to obtain the long-term scheduling strategy. Assume that in each time slot, each terminal randomly generates a computing task with probability p, and the task type index is recorded as Assume that the computing tasks generated by different terminals in each time slot are independent of each other, and the N types of tasks follow the Zipf distribution, that is, the probability of any terminal requesting the nth type of task is:

[0061]

[0062] Where α is the Zipf index, which indicates the popularity of the content.

[0063] After the user generates a computing task, he will request the edge server to execute the computing task. The type of task requested by the user is different, and the size of the software data required to complete the task, the size of the user data required to complete the computing task, the number of CPU cycles required to calculate the user data per bit, and the maximum tolerance time of the task may be different, which are respectively denoted as {q n ,e n ,w n ,τ n}, when the task type is clear, the above data can be determined accordingly.

[0064] During modeling, we considered constraints including memory constraints, computing resource allocation constraints, and bandwidth resource allocation constraints. These three types of constraints are all designed to ensure the normal operation of edge servers. Each constraint is described below.

[0065] The memory constraint means that the cached content in the corresponding time slot does not exceed the total memory of the edge server. Specifically, the edge server itself is equipped with a certain cache resource C, which needs to ensure that the cached content of the edge server in each time slot does not exceed its maximum memory.

[0066] In one embodiment, the cache strategy in each time slot t can be defined as c t ={c1(t),...,c N (t)}, where c n (t) indicates whether the software data required to execute the nth type of task is cached in the current edge server. 0 indicates that the software data is not cached in the current edge server, and 1 indicates that the software data is cached in the current edge server. At this time, the memory constraint of the current edge server in time slot t can be expressed as:

[0067]

[0068] The computational resource allocation constraint states that the computational resources allocated to all requests within a time slot must not exceed the total computational resources of the edge server. Specifically, the total computational resources allocated to the edge server are F, and it is necessary to ensure that the computational resources allocated to the edge server to complete the computation task within each time slot do not exceed its total computational resources.

[0069] In one embodiment, the computing resource allocation strategy within each time slot t can be defined as f t ={f1(t),...,f K (t)}, where f k (t)∈[0,1] represents the proportion of computing resources allocated to user k. At this time, the computing resource allocation constraint of the current edge server in time slot t can be expressed as:

[0070]

[0071] The bandwidth resource allocation constraint is characterized by the fact that the bandwidth allocated to all requests in the corresponding time slot does not exceed the total bandwidth resources of the edge server. Specifically, the total bandwidth resources equipped with the edge server are B. Assuming that the terminal uses frequency division multiplexing when uploading user data to the edge server, the bandwidth resource allocation strategy in each time slot is defined as b. t ={b1(t),...,b K (t)}, where b k (t)∈[0,1] represents the bandwidth ratio allocated to terminal k. At this time, the bandwidth resource allocation constraint of the current edge server in time slot t can be expressed as:

[0072]

[0073] Combining the above analysis, we can obtain memory constraints, computing resource allocation constraints, and bandwidth resource allocation constraints.

[0074] In addition to considering the above resource allocation constraints, it is also necessary to consider the energy consumption constraints of the edge server within the total scheduling period T to ensure that the long-term total energy consumption of the edge server does not exceed its set energy consumption upper limit, so as to minimize network latency while meeting the energy consumption constraints.

[0075] The following analyzes the construction of energy consumption and latency.

[0076] Assuming that the transmission between the terminal and the edge server is through an additive white Gaussian noise channel, the transmission rate from terminal k to the edge server is expressed as:

[0077]

[0078] Where p k is the transmission power of terminal k, N0 is the Gaussian white noise power, h k(t) is the channel gain of terminal k in time slot t, which obeys the Rice distribution, and the mean of the Rice distribution is

[0079] Assume that the average channel gain follows the path loss model:

[0080]

[0081] Where A d represents wireless gain, f c Indicates the carrier frequency, d e represents the path loss index, d k represents the distance from terminal k to the edge server.

[0082] Assume that the task set generated by K terminals in time slot t is expressed as I(t) = {I1(t),...,I K (t)}, where I k (t) indicates the index of the computing task type generated by terminal k in time slot t. When the terminal device does not request a computing task, I k (t)=0, otherwise The above task set is the computing task requested to be executed by the edge server in time slot t. It is assumed that before the start of time slot t, the cache data of the edge server has been updated and all cached content has been obtained from the cloud, without considering the required cost.

[0083] For the computing task of terminal k in the task set generated in time slot t, the task is represented as I k (t), the corresponding delay and energy consumption are calculated according to whether the corresponding software data is cached in the edge server. Whether the software data required by different computing tasks is cached in the edge server depends on the cache strategy c in time slot t. t ={c1(t),...,c N Considering that the downlink transmission delay of the calculation result returned to the terminal is extremely short, to simplify the calculation, this embodiment only considers the uplink data transmission delay from the terminal to the edge server and ignores the downlink transmission delay of the calculation result returned to the terminal.

[0084] When executing the computing task I k (t) When the required software data is cached on the edge server, the service delay includes the upload delay of the user data and the delay of the edge server processing the computing task I k The processing delay of (t) can be expressed as:

[0085]

[0086] Where, Indicates the computation of task I in time slot t k(t) The required software data is cached in the edge server to respond to the computing task I k (t) service delay, e Ik(t) Represents computing task I k (t) the size of the user data required, Represents computing task I k Calculate the number of CPU cycles required for unit bit of user data at (t).

[0087] At this time, the response calculation task I k The total energy consumption of (t) includes the transmission energy consumption of terminal k transmitting user data to the edge server and the energy consumption of edge server processing computing task I k The energy consumption of computing (t) can be expressed as:

[0088]

[0089] Where, Indicates the computation of task I in time slot t k (t) The required software data is cached in the edge server to respond to the computing task I k (t) is the total energy consumption, and ε is the energy efficiency coefficient.

[0090] When executing the computing task I k (t) When the required software data is not cached in the edge server, the service delay includes the upload delay of user data, the computing task I k (t) The return delay of the required software data from the cloud to the edge server and the edge server processing computing tasks I k The processing delay of (t) can be expressed as:

[0091]

[0092] Where, To calculate task I in time slot t k (t) The required software data is not cached in the edge server to respond to the computing task I k (t) service delay, τ CDC For computing task I k (t) The return latency of the required software data from the cloud to the edge server.

[0093] At this time, the response calculation task I k The total energy consumption of (t) includes the transmission energy consumption of terminal k transmitting user data to the edge server, the computing energy consumption of task I k (t) The energy consumption of the return trip of the required software data from the cloud to the edge server and the energy consumption of the edge server processing the computing task I k The energy consumption of computing (t) can be expressed as:

[0094]

[0095] Where, To calculate task I in time slot t k (t) The required software data is not cached in the edge server to respond to the computing task I k Total energy consumption (t), e CDC For Task I k (t) The required backhaul energy consumption of software data from the cloud to the edge server.

[0096] In summary, the task request I of terminal k in time slot t can be k The service delay of (t) is comprehensively expressed as:

[0097]

[0098] The task request I of terminal k in time slot t can be k The total energy consumption of (t) is expressed as:

[0099]

[0100] In this invention, the total energy consumption within the scheduling period T is required not to exceed the set energy consumption cap. On this basis, it is hoped that the delay satisfaction rate within the scheduling period T is as high as possible, thereby simultaneously optimizing the long-term delay satisfaction rate and the total system energy consumption. The delay satisfaction rate within the scheduling period T is the average delay satisfaction rate of T time slots. The delay satisfaction rate within each time slot is the proportion of the service delay of all terminals in the corresponding time slot that does not exceed the maximum tolerable delay of the corresponding task.

[0101] Specifically, define ζ k (t) indicates request I k (t) Whether the service delay meets its maximum tolerance delay, when When k (t)=1, otherwise, ζ k (t)=0. Represents computing task I k (t) is the maximum tolerable time. In order to describe the average quality of service, the delay satisfaction rate in time slot t is defined as:

[0102]

[0103] In order to optimize the long-term delay satisfaction rate and the total energy consumption of the system at the same time, the long-term delay satisfaction rate is used as the optimization objective function, and the total energy consumption of the long-term system does not exceed the maximum energy consumption. as a constraint.

[0104] That is, the optimization objective can be expressed as:

[0105]

[0106] The energy consumption constraint can be expressed as:

[0107]

[0108] Combining the memory constraints, computing resource allocation constraints, and bandwidth resource allocation constraints described above, we can analyze and derive the original long-term scheduling model, as follows:

[0109]

[0110] It can be seen that the above long-term scheduling model is a long-term optimization problem. This optimization model corresponds to a multi-stage random mixed integer nonlinear programming problem. Since network parameters such as channel status and terminal request sequence change dynamically over time, there is a mutual coupling relationship between each time slot. There may be a certain trade-off relationship between delay satisfaction rate and system energy consumption. Improving one may lead to a decrease in the performance of the other, thereby affecting the overall network performance. Therefore, directly solving the above long-term optimization problem requires understanding the status of each time slot. When the number of time slots is too large, the model solution is more difficult.

[0111] Therefore, the present invention optimizes and transforms the above-mentioned original long-term scheduling model, applies Lyapunov optimization theory to split long-term constraints, and decouples the above-mentioned random optimization problem into single-stage deterministic sub-problems. Specifically, an energy-consuming virtual queue is constructed based on Lyapunov optimization theory, and the long-term energy consumption constraint can be converted into a queue stability constraint. In order to ensure the stability of the virtual queue and maximize the delay satisfaction rate, by minimizing the upper bound of the Lyapunov drift and penalty function, the multi-stage random problem can be converted into a deterministic sub-problem within each time slot, thereby decoupling the multi-stage random mixed integer nonlinear programming problem in the original model into a mixed integer nonlinear programming problem determined for each frame.

[0112] Specifically, we first construct an energy consumption virtual queue, and the dynamic evolution of the energy consumption virtual queue is expressed as:

[0113]

[0114] Where Q(t) is the length of the energy consumption virtual queue in time slot t, with an initial value of Q(0)=0, and E(t) is the total energy consumption in time slot t.

[0115] When the above energy consumption virtual queue is stable, it means that the long-term energy consumption constraint in the original model is met.

[0116] At this point, Lyapunov optimization theory can be used to derive the conditions that need to be met for the stability of the energy-consuming virtual queue.

[0117] Construct the Lyapunov function L that describes the stability of the system queue:

[0118]

[0119] The larger L is, the more congested the network is and the energy-consuming virtual queue is unstable.

[0120] Further construct the Lyapunov drift function:

[0121]

[0122] Find the conditional expectation of the above Lyapunov drift:

[0123]

[0124] Since the energy consumption E(t) has upper and lower bounds, the first term of the above conditional expectation is is bounded. At this time, in order to ensure the stability of the energy consumption queue, it is necessary to minimize the second term

[0125] Considering the optimization goal of maximizing the long-term delay satisfaction rate in the original model, a greedy method is used for optimization. A penalty function is constructed to minimize the energy consumption queue stability. That is, considering the long-term energy consumption constraints and the long-term energy consumption constraints, the optimization goal can be constructed as the upper limit of the drift plus penalty function:

[0126]

[0127] Where B is the constant term and V is the proportional coefficient, whose unit is the square of the energy consumption unit to ensure that the units of all factors are consistent.

[0128] In summary, the joint optimization problem of long-term delay satisfaction rate and system total energy consumption is transformed into a deterministic sub-problem within each time slot:

[0129]

[0130] Therefore, the original model can be transformed into the following scheduling model:

[0131]

[0132] Where, the cache strategy in each time slot t is c t ={c1(t),...,c N (t)}, the computing resource allocation strategy in each time slot t is f t ={f1(t),...,f K (t)} and the bandwidth resource allocation strategy in each time slot t is b t ={b1(t),...,b K (t)} is the parameter to be decided.

[0133] The deformed scheduling model optimizes each time slot of the scheduling period separately, thereby decoupling the multi-stage random mixed integer nonlinear programming problem in the original model into a mixed integer nonlinear programming problem determined for each frame, which facilitates the solution of the model.

[0134] S2. Convert all time slot sub-models into Markov decision processes and use deep reinforcement learning algorithms to find the optimal solution to obtain the memory, computing resources, and bandwidth resource allocation strategies for each time slot in the long-term scheduling cycle.

[0135] Specifically, the solution of all time slot sub-models can be constructed as a Markov decision process, and a deep reinforcement learning algorithm can be used to optimize the process to maximize the long-term delay satisfaction rate, thereby obtaining the memory, computing resources, and bandwidth resource allocation strategies for each time slot within the long-term scheduling cycle.

[0136] Using deep reinforcement learning algorithms requires defining the state space, action space, and reward function.

[0137] In one embodiment, a deep reinforcement learning algorithm based on a parameterized deep Q-network (PDQN) is specifically used, and its state space, action space, and reward function are defined as follows.

[0138] The state space is the input variables required to make decisions, expressed as

[0139] The action space includes resource allocation strategies, including caching strategies Computing resource and bandwidth resource allocation strategy The former is a discrete action and the latter is a continuous action.

[0140] Reward function R t is the negative of the sum of the Lyapunov drift and the penalty function.

[0141] Specifically, the reward function R t The expression is:

[0142]

[0143] like Figure 3 The following is the network structure of PDQN, where the policy network (Actor network) generates resource allocation decisions based on the environment, and the value network (Critic network) evaluates the corresponding Q value.

[0144] The Actor network will state S t Mapped to continuous action G t :

[0145]

[0146] Where, is random noise, φ μ Represents the current Actor network parameters, μ(S t ;φ μ ) indicates that the current network parameter is φ μ , status is S t Resource allocation decisions made by the Actor network.

[0147] The update process of the Actor network is to use the maximum Q value as the update target of the Actor network and update the network parameters by sampling the policy gradient. The policy gradient is:

[0148]

[0149] Where M represents the size of the training batch, φ Q Represents the current Critic network parameters, Represents the expected reward of the current Actor network.

[0150] Critic network input system state S t and continuous action G t , output the corresponding Q values ​​under different caching decisions, and then select the discrete action corresponding to the maximum Q value as the caching decision:

[0151] C t =argmaxQ(S t ,G t ;φ Q );

[0152] The critic network updates its parameters by minimizing the mean square error between the target Q value and the predicted Q value using gradient descent. The loss function is expressed as:

[0153]

[0154] Where γ represents the discount factor, S t ', G t ', They represent the system state of the next time slot, the computing resources and bandwidth resource allocation strategy generated in the next time slot, and the parameters of the target critic network respectively.

[0155] In order to solve the problem of overestimation of Q value, an asynchronous update strategy is adopted, that is, the Actor network is updated after updating the Critic network Δ times.

[0156] It should be noted that the deep reinforcement learning algorithm used for the solution is not limited to PDQN, but can also be other deep reinforcement learning algorithms such as the Deep Q-Network (DQN) algorithm, the enhanced Deep Q-Network (DQN-enhanced) algorithm, and the Deep Deterministic Policy Gradient (DDPG) algorithm. The PDQN algorithm is preferably used.

[0157] The present invention is described in detail below with reference to simulation:

[0158] The simulation parameters are set as follows: Consider a scenario consisting of three edge servers, each serving 50 terminals, and evaluate the average delay satisfaction rate and system energy consumption of all edge servers in 200 time slots. The size of the content library is 10, and for each task There is q n ={1,...,0.1}GB, e n ={10,...,1}MB, w n =0.1cycles / Byte, τ n =[2,...,4]s. The cache resources of the three edge servers are [10,20,30]GB, and the computing resources are [3,5,7]×10 7 cycle / s, bandwidth resources are [300,500,700]MHz. Transmit power is p k =0.5Watts, the noise power is N0=5×10 -13 Watts, the distance between the terminal and the edge server is uniformly distributed between [120, 255], and the antenna gain is A d =3, the carrier frequency is f c =915MHz, the path loss factor is d e =3. The delay and energy consumption of the backhaul link are τ CDC = 0.6s and e CDC =2J, energy efficiency coefficient is ε=10 -19 , the Zipf coefficient is α=1, and the energy consumption constraint is The proportional coefficient is V=1000.

[0159] Five benchmark scenarios are simulated below:

[0160] (1) Enhanced Deep Q-Network (DQN-enhanced): Utilizes the Deep Q-Network (DQN) algorithm to make caching decisions, and allocates bandwidth and computing resources proportionally based on user needs.

[0161] (2) Deep Q Network (DQN): The DQN algorithm is used to make cache decisions and evenly distribute bandwidth and computing resources to users.

[0162] (3) Deep Deterministic Policy Gradient (DDPG): Utilizes the Deep Deterministic Policy Gradient (DDPG) algorithm to make bandwidth and computing resource allocation decisions and cache software data based on popularity.

[0163] (4) Random: Caching decisions are randomly generated according to a normal distribution, and computing and bandwidth resources are evenly distributed among users.

[0164] (5) No-caching: No content is cached, and computing and bandwidth resources are evenly distributed among users.

[0165] The PDQN algorithm is used as an example to illustrate the convergence performance of the deep reinforcement learning algorithm. In PDQN, the Actor network contains two hidden layers with 400 and 300 neurons respectively, and the learning rate is 3×10 -5 The critic network consists of three hidden layers with 1000, 800 and 100 neurons respectively, and the learning rate is 3×10 -4 Other hyperparameters are as follows: soft update ratio is 0.01, noise size is 0.1, update frequency is Δ=2, discount factor is γ=0.9, and exploration probability is 0.9. Figure 4 The figure shows the convergence performance of the PDQN algorithm in one embodiment. As can be seen from the figure, as the number of training rounds increases, the average delay satisfaction rate and average reward value of all time slots in each round increase and converge quickly, demonstrating the good convergence performance of the PDQN algorithm.

[0166] like Figure 5 FIG. 1 is a performance comparison diagram of the time delay satisfaction rate under different schemes in one embodiment; Figure 6 The figure shows a performance comparison of the total energy consumption of the system under different schemes in one embodiment. Figure 5 and Figure 6As shown in the figure, compared with the random scheme (Random) and the no-caching scheme (No-caching), the performance achieved by establishing a scheduling model and optimizing with a deep reinforcement learning algorithm is better; and among the various deep reinforcement learning algorithms provided, the PDQN algorithm outperforms other deep reinforcement learning algorithms. Compared with the DQN-enhanced, DQN-based and DDPG-based schemes, the PDQN algorithm improves the average delay satisfaction rate by 5.2%, 16.9% and 27.7%, respectively, and reduces the average system energy consumption by 22.3%, 29.0% and 33.7%, respectively.

[0167] Example 2

[0168] The present invention also relates to an edge server, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in embodiment 1 when executing the computer program.

[0169] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Memory may be used to store computer programs and / or modules. The processor runs or executes computer programs and / or modules stored in the memory, and accesses data stored in the memory, thereby performing various functions of the electronic device.

[0170] Example 3

[0171] The present invention also relates to a resource joint scheduling system for edge-cloud integrated industrial Internet, which includes a terminal layer, an edge server layer and a cloud data center.

[0172] The terminal layer includes terminals that randomly access the network. Depending on different industrial Internet application scenarios, such as quality inspection and safety monitoring, the terminals will generate different computing-intensive tasks and upload them to the edge server for calculation.

[0173] The edge server layer includes an edge server cluster consisting of multiple edge servers, the edge servers being those described in Example 2. The edge servers dynamically make caching decisions and bandwidth and computing resource allocation decisions based on the channel status and user requests in the current time slot, and update memory and allocate resources in real time. Completing computing tasks on the edge servers requires user data and software data. User data is collected from terminals via wireless links, while software data needs to be downloaded from the cloud data center but can be pre-cached on the edge servers for reuse. When the software data required for a user-requested computing task is cached on the edge server, the data can be retrieved directly from memory, saving the latency and energy consumption required to retrieve it from the cloud data center.

[0174] Cloud data centers can provide the software and data required for each computing task, but this will bring additional backhaul latency and energy consumption.

[0175] Overall, this invention enables online caching decisions and bandwidth and computing resource allocation decisions based on the network environment and user needs, meeting the strict latency and energy requirements of the Industrial Internet of Things. Based on Lyapunov optimization theory, a virtual energy queue is constructed, transforming long-term energy consumption constraints into queue stability constraints. By minimizing the upper bound of the drift penalty function, the long-term delay satisfaction rate optimization problem is transformed into a deterministic subproblem within each time slot. On this basis, by constructing a Markov decision process and optimizing it using a deep reinforcement learning algorithm, caching decisions and bandwidth and computing resource allocation decisions can be determined.

[0176] The technical features of the above embodiments can be combined in any manner. To simplify the description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. It should be noted that the phrases "in one embodiment", "for example", "and another example", etc. of the present invention are intended to illustrate the present invention and are not intended to limit the present invention.

[0177] The above embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, and all such variations and improvements fall within the scope of protection of the present invention.

Claims

1. A resource joint scheduling method for edge-cloud integrated industrial Internet, characterized in that: include: Construct a scheduling model for the current edge server's long-term scheduling cycle, wherein the long-term scheduling cycle includes T time slots, and the long-term scheduling model includes T discrete time slot sub-models, each of which corresponds to a time slot, and each of which includes an optimization objective and constraints under the corresponding time slot. The optimization objective is to minimize the objective function under the corresponding time slot, and the objective function includes the factor Where E(t) is the total energy consumption caused by completing all tasks in time slot t, is the set total energy consumption limit, Q(t) is the energy consumption value of the energy consumption virtual queue in time slot t, and the queue in each time slot satisfies V is a set proportional coefficient, ζ(t) is the delay satisfaction rate of the completed task in time slot t; the constraints include memory constraints, computing resource allocation constraints, and bandwidth resource allocation constraints. The memory constraint indicates that the cache content in the corresponding time slot does not exceed the total memory of the edge server. The computing resource allocation constraint indicates that the computing resources allocated to all requests in the corresponding time slot do not exceed the total computing resources of the edge server. The bandwidth resource allocation constraint ensures that the bandwidth allocated to all requests in the corresponding time slot does not exceed the total bandwidth resources of the edge server. All time slot sub-models are converted into Markov decision processes, and a deep reinforcement learning algorithm is used for optimization to obtain the memory, computing resource and bandwidth resource allocation strategy for each time slot in the long-term scheduling cycle.

2. The resource joint scheduling method according to claim 1, characterized in that: The expression of the memory constraint is: c n (t)∈{0,1}; Where C is the total memory of the edge server itself, c n (t) indicates whether the software data required to execute the nth task is cached in the current edge server in the current time slot t. 0 means it is not stored in the current edge server, and 1 means it is stored in the current edge server. t ={c1(t),…,c N (t)} represents the cache strategy in time slot t, q n Indicates the software data size required to execute the nth type of task, N is the number of task types, It is a set of time slot indexes within the long-term scheduling period.

3. The resource joint scheduling method according to claim 1, wherein: The expression of the computing resource allocation constraint is: f k (t)∈[0,1]; Where, f k (t) represents the proportion of computing resources allocated to terminal k in the current time slot t, K is the number of terminals within the current edge server coverage, and f t ={f1(t),…,f K (t)} is the computing resource allocation strategy within time slot t, It is a set of time slot indexes within the long-term scheduling period.

4. The resource joint scheduling method according to claim 1, wherein: The expression of the bandwidth resource allocation constraint is: b k (t)∈[0,1]; Where b k (t) represents the bandwidth ratio allocated to terminal k in the current time slot t, K is the number of terminals within the coverage of the current edge server, and b t ={b1(t),…,b K (t)} is the bandwidth resource allocation strategy within time slot t, It is a set of time slot indexes within the long-term scheduling period.

5. The resource joint scheduling method according to claim 1, wherein: The expression of the optimization objective is:

6. The resource joint scheduling method according to claim 1, wherein: The expression of the delay satisfaction rate ζ(t) is: Where K is the number of terminals within the coverage of the current edge server, ζ k (t) represents task I k (t) Whether the service delay meets the maximum tolerable delay of the task, I k (t) represents the index of the computing task type generated by terminal k in time slot t, is the task type index set, N is the number of task types, T k (t) is task I k (t) service delay, Represents Task I k (t) maximum tolerable delay; Among them, the service delay T k (t) is expressed as: Where, Indicates execution of task I k (t) Whether the required software data is cached in the current edge server, 0 means it is not stored in the current edge server, and 1 means it is stored in the current edge server. Indicates execution of task I k (t) Task I when the required software data is already cached on the edge server k (t) service delay, To perform mission I k (t) Task I when the required software data is not cached in the edge server k (t) service latency; The expression of total energy consumption E(t) is: Where, E k (t) To complete Task I k (t) the total energy consumption caused, Indicates execution of task I k (t) Task I is completed when the required software data is cached on the edge server k (t) total energy consumption, To perform mission I k (t) Complete Task I when the required software data is not cached on the edge server k (t) total energy consumption.

7. The resource joint scheduling method according to claim 6, characterized in that: Including user data upload delay and edge server processing task I k (t) processing delay; Including the transmission energy consumption of terminal k transmitting user data to the edge server and the edge server processing task I k (t) calculation energy consumption; Including user data upload delay, Task I k (t) The return delay of the required software data from the cloud to the edge server and the edge server processing task I k (t) processing delay; Including the transmission energy consumption of terminal k transmitting user data to the edge server, task I k (t) The energy consumption of the return trip of the required software data from the cloud to the edge server and the energy consumption of the edge server processing the computing task I k (t) is the energy consumption of calculation.

8. The resource joint scheduling method according to any one of claims 1 to 7, characterized in that: The deep reinforcement learning algorithm is a parameterized deep Q-network algorithm.

9. An edge server comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A resource joint scheduling system for edge-cloud integrated industrial Internet, characterized in that: include: The terminal layer includes terminals that randomly access the network; an edge server layer, comprising an edge server cluster consisting of a plurality of edge servers, wherein the edge servers are the edge servers according to claim 9; Cloud data centers are used to provide the required software and data for the edge server layer.

Images