Cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning

Abstract

The present invention discloses a cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning, including: eAPs and cloud respectively construct a request scheduling decision model and a service orchestration decision model; the cloud combines the first resonance information sent by the eAP to obtain the state #imgabs0# using the attention mechanism to generate the second resonance information and transmit it to the eAP; the state #imgabs1# is input into the service orchestration decision model and optimized, combined with the lower guidance vector sent by the eAP to obtain the optimal service orchestration decision and transmit it to the edge server; the eAP combines the second resonance information sent by the cloud to obtain the state #imgabs2# using the attention mechanism to generate the first resonance information and transmit it to the cloud; the state #imgabs3# is input into the request scheduling decision model and optimized, combined with the upper guidance vector sent by the cloud to obtain the optimal request scheduling decision and transmit it to the edge server; the edge server processes the request according to the two decisions received. The present invention can effectively reduce scheduling costs and improve throughput.

Description

Technical Field

[0001] The present invention belongs to the field of big data technology, and specifically relates to a cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning. Background Art

[0002] With the explosive growth of data from network devices, centralized cloud computing currently faces many problems, including computing power, efficiency, latency, and privacy. The emergence of edge computing has fully unleashed the potential of edge devices and is widely regarded as a promising solution. As the amount of data processed by edge servers has increased dramatically, the new collaborative computing infrastructure between data centers of all sizes has unleashed the potential of edge cloud systems, which unify various computing resources and usher in an unprecedented era of prosperity. High returns have driven the rapid development of edge cloud systems, but have also made edge cloud systems more complex. Complex edge cloud systems have led to many unexpected problems in request distribution and service orchestration, such as complex hierarchical structures, limited collaboration, and information decentralization.

[0003] Edge cloud systems have a complex hierarchical structure, namely edge access proxies (eAPs), edge servers, and clouds, similar to the hierarchical structure of human society, where decisions and intelligence interact at different levels. However, existing edge intelligence ignores social hierarchical decision-making, resulting in inefficient scheduling problems. Secondly, there is a serious contradiction between the limited data and computing power of a node or an organization and advanced and complex large-scale learning models. Multiple parties in the edge cloud system have the need for data collaboration to form a complete model, but personal data privacy cannot be guaranteed through existing multi-device model training. In addition, with the development of edge cloud systems, the scope of complex and heterogeneous information dissemination within them is becoming wider and wider. However, existing request scheduling methods cannot distinguish valuable information from the collected data, reducing scheduling accuracy. Summary of the invention

[0004] In response to the above technical problems, the present invention proposes a cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning. Inspired by the efficient collaborative operation mechanism of human society, social learning is applied to complex edge cloud systems to achieve a balance between system long-term performance and system cost. To solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0005] A cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning includes the following steps:

[0006] S1, based on deep reinforcement learning algorithm and federated learning method, eAPs and cloud build request scheduling decision model and service orchestration decision model respectively;

[0007] S2, set the service scheduling period τ, and initialize the request scheduling period round number t=1;

[0008] S3, using the attention mechanism to obtain the state s′ of the orchestration service decision model U，t , and then combined with the first resonance information sent by eAP Again, the attention mechanism is used to U，t Update to get status Set the preset status on the cloud Input attention mechanism to get state Based on status With status Generate second resonance information Send to eAP;

[0009] S4, the state Input the service orchestration decision model, optimize the service orchestration decision model to achieve the balance between the time efficiency of the edge server and the cost of service deployment in the service orchestration stage, and combine the lower-level guidance vector g sent by the eAP L，t Get the optimal service orchestration decision and send it to the edge server;

[0010] S5, eAP uses the attention mechanism to obtain the state s′ of all request scheduling decision models L，t , and then combined with the second resonance information sent by the cloud Use the attention mechanism to the state s′ L，t Update to get status Set the preset status on the eAP Input attention mechanism to get state Based on status With status Generate first resonance information Send to the cloud;

[0011] S6, the state Input the request scheduling decision model, optimize the request scheduling decision model to achieve the balance between the eAP time efficiency and the cost generated by the request scheduling during the request scheduling phase, and combine the upper layer guidance vector g sent by the cloud U，t Obtain the optimal request scheduling decision and send the optimal request scheduling decision to the edge server;

[0012] s7, the edge server processes the request according to the received request scheduling decision and service orchestration decision. After the request processing is completed, execute t=t+1, judge t%τ==0, if yes, execute step s3, otherwise, execute step S5.

[0013] In step S1, the method for establishing the request scheduling decision model includes the following steps:

[0014] ①, construct the state, action and reward functions of the request scheduling decision model;

[0015] ② Each eAP trains the multi-agent deep reinforcement learning algorithm model based on the state, action and reward function constructed in step ① to obtain the basic model of request scheduling decision;

[0016] ③ Based on the federated learning method, the basic models of request scheduling decisions of all eAPs are averaged and weighted to obtain the request scheduling decision model.

[0017] The state of the request scheduling decision model includes the type of request, the latency requirement of the request, the available CPU computing resources of the edge server, the available memory, the available storage space and the total number of edge servers;

[0018] The action of the request scheduling decision model is to request the scheduling decision to adopt a b，t It means that its expression is a b，t = n, indicating that the request of eAP b at time frame t is assigned to edge server n;

[0019] The expression of the reward function of the request scheduling decision model is:

[0020]

[0021] In the formula, and represents the weight that controls the ratio of eAP throughput to cost, λ b represents the ratio of requests that violate the latency requirement within a time frame, ψ b represents the time utilization, ∈ represents the weight of the load balancing index between edge servers, and ν b Indicates the load balancing index of eAP b. represents the network cost of all requests in the first request queue transmitted to eAP b, represents the computational cost of edge server processing requests in the first request queue of eAP b, represents the request scheduling decision of eAP b at time frame t, Represents the collection of all cAPs, or proxy servers.

[0022] In step S1, the method for establishing the service orchestration decision model includes the following steps:

[0023] ①, construct the state, action and reward functions of the service orchestration decision model;

[0024] ② Each cloud trains the multi-agent deep reinforcement learning algorithm model based on the state, action, and reward function constructed in step ① to obtain the basic model of service orchestration decision-making;

[0025] ③ Based on the federated learning method, the basic service orchestration decision models of all clouds are averaged and weighted to obtain the service orchestration decision model.

[0026] The state of the service orchestration decision model includes available CPU computing resources, available memory, available storage space of the edge server, network latency between each eAP and the cloud, information of a second request queue for temporarily storing scheduled and unexecuted requests, and deployed services;

[0027] The action of the service orchestration decision model is also the service orchestration decision adopting a c，n，τ express, represents the time slot τ when the service on cloud c Deployed on edge server n;

[0028] The reward function of the service orchestration decision model is expressed as:

[0029]

[0030] In the formula, and represents the weight that controls the ratio of edge server throughput to cost, represents the service orchestration decision of the cloud at time slot τ, T represents the total delay of processing all processing requests in a time slot, represents the set of edge servers, |Q n | represents the second request queue Q that is not processed on edge server n n The queue length, K capex represents the cost of installing the network for operators, It represents the rental cost of leasing the memory w of edge server n during the service orchestration stage, which is also the service orchestration cost.

[0031] In step S3, the state The expression is:

[0032]

[0033] The state s′ U，t The expression is:

[0034]

[0035] In the formula, s U，t represents the state of all edge servers at time frame t, sUx，t represents the x-th type of state observed at time frame t, represents the attention weight of the x-th type of state, and X represents the total type of states of the orchestration service decision model.

[0036] In step S3, the second resonance information The expression is:

[0037]

[0038] In the formula, pU represents the state After standardization, Indicates status Normalized data.

[0039] In step s5, the state The expression is:

[0040]

[0041] The first resonance information The expression is:

[0042]

[0043] Where P U Indicates status After standardization, Indicates status Normalized data.

[0044] Beneficial effects of the present invention:

[0045] Socialized inter-layer guidance, that is, the request scheduling decision results of the lower layer will affect the business orchestration decision process of the upper layer. The upper layer only cares about the learning results from the lower layer, not the data, so as to help the upper layer make better decisions, reduce the scale of transmitted data and network bandwidth, and improve decision-making efficiency. Socialized intra-layer collaboration is proposed, that is, eAPs are gathered together in the form of an alliance, and social cooperation is developed within the alliance, i.e., within the layer, to train advanced complete large-scale models to achieve global optimal scheduling. Cross-layer social resonance is proposed, that is, the attention mechanism is used to filter out valuable information, thereby improving scheduling accuracy. The introduction of the resonance effect enables the attention mechanisms of the upper and lower layers to resonate simultaneously for the system goal, thereby improving system efficiency.

[0046] By introducing social guidance, social collaboration and resonant attention mechanisms, utilizing hierarchical associations, privacy data cooperation and centralizing valuable information, we can effectively reduce scheduling costs, improve time efficiency and increase system throughput. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 Schematic diagram of the system architecture of the present invention.

[0049] Figure 2 It is a schematic diagram of the process of the present invention.

[0050] Figure 3 Aggregate schematic diagram for the request scheduling decision model.

[0051] Figure 4 Schematic diagram of the process of establishing the request scheduling decision model and the service orchestration decision model.

[0052] Figure 5 This is a schematic diagram comparing the training efficiency of this application with other algorithms.

[0053] Figure 6 This is a schematic diagram of the performance comparison between this application and other algorithms.

[0054] Figure 7 The impact of request type and network bandwidth on the scheduling performance of this application.

[0055] Figure 8 The impact of resource competition and heterogeneous resource quantities on the scheduling performance of this application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] Embodiment 1: A cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning, comprising the following steps:

[0058] S1, based on deep reinforcement learning algorithm and federated learning method, eAPs and cloud build request scheduling decision model and service orchestration decision model respectively;

[0059] like Figure 1 and Figure 2As shown, the proxy server is used to receive requests sent by the user's network device. Each proxy server is provided with a first request queue, which is used to store the requests received by the eAP, i.e., the proxy server, as well as the arrival time and expiration time corresponding to the requests. The edge server processes the requests through service orchestration. Each edge server is embedded with a buffer, which is provided with a second request queue. The second request queue is used to temporarily store the scheduled and unexecuted requests. At the same time, the arrival time and expiration time corresponding to each request are also stored in the second request queue. The i-th request x sent by the proxy server b i，b When it is assigned to edge server n, edge server n will allocate a quantity f to it. n，i，b (cycles / second) of computing resources, and f n，i，b ≥0. In this application, Q n represents the second request queue currently stored in the buffer of edge server n, Q b represents the first request queue of eAP b, i.e., proxy server b, and and represents a collection of edge servers, represents a collection of proxy servers, and B represents a collection of proxy servers The total number of proxy servers in the, N represents the edge server set The total number of edge servers in the cloud, C represents the cloud collection The total number of clouds, Represents the set of services on all clouds. Represents a collection of services The total number of services in the middle. In the edge cloud system, due to the different perception ranges of intelligent agents at different levels, the upper cloud can only observe the length of the request queue on the edge server, but cannot observe how many requests in the system reach the lower eAP. The lower eAP does not perceive the costs incurred by the middle edge server and the upper cloud when coordinating services. Therefore, the intelligent agents at each layer only indirectly serve the overall goal of the entire system. The goals of different layers are different, but the goals of the upper and lower layers are interrelated. Better service orchestration at the upper layer is more conducive to request scheduling at the lower layer. This application introduces social learning. Through the guidance of the upper and lower layers, eAP can be prevented from making inappropriate decisions, such as sending requests to edge servers with insufficient resources. The edge server schedules the computing resources in the edge server through the cloud service to process the request.

[0060] like Figure 3 As shown, the method for establishing the request scheduling decision model includes the following steps:

[0061] ①, construct the state, action and reward functions of the request scheduling decision model;

[0062] The status includes the type of request, the latency requirement of the request, information of the first request queue, available CPU computing resources of the edge server, available memory, available storage space, and the total number of edge servers;

[0063] The action is to request that the scheduling decision adopt a b，t It means that its expression is a b，t =n, indicating that the request of eAPb in time frame t is assigned to edge server n.

[0064] Complex edge cloud systems involve various cost indicators that reflect the various costs incurred by operators, including the cost of installing the network K capex and the request processing cost K opex , K capex Also known as capital expenditure (CAPEX), it refers to the investment that operators need to bear. opex Also known as operating expenditure (OPEX), it mainly includes the cost of using the Internet, the cost of processing requests on edge servers (also known as computing costs), and the cost of service orchestration. Generally, capital expenditures are fixed, while operating expenditures are based on usage.

[0065] The request processing cost K opex The calculation formula is:

[0066]

[0067] In the formula, Indicates a transfer request x i，b The network cost, Indicates that the edge server processes the request x i，b The computational cost of It represents the rental cost of leasing the memory w of edge server n during the service orchestration stage, which is also the service orchestration cost.

[0068] The transmission request x i，b Network costs The calculation formula is:

[0069]

[0070] In the formula, represents the unit cost of using Internet transmission, in US dollars per bit, D i，b Indicates request x i，b The amount of data.

[0071] Edge server processes request x i，b The computational cost The calculation formula is:

[0072]

[0073] In the formula, represents the unit cost of using edge server computing resources, u represents the price impact factor set by the edge server service provider, and u>1, C i，b μ Indicates processing request x i，b The amount of CPU computing resources required.

[0074] The service orchestration stage request is stored in the leased edge server memory to facilitate the edge server to process the request. The service orchestration cost The calculation formula is:

[0075]

[0076] In the formula, represents the unit rental price of leasing edge server memory w, z n，w It represents the number of requests to rent the memory w of edge server n during the service orchestration phase.

[0077] In the request scheduling stage, the objective function, i.e. the reward function, is the balance between system time efficiency and the cost generated by request scheduling:

[0078]

[0079] In the formula, and Both represent the weights that control the ratio of eAP throughput to cost, λ b represents the ratio of requests that violate the latency requirement within a time frame, ψ b It represents the time utilization, reflecting the ratio of the total delay of requests completed within the delay requirement within a time frame to the total delay of all requests arriving at the proxy server. ∈ represents the weight of the load balancing index between the edge servers. ν b Indicates the load balancing index of eAP b. represents the network cost of transmitting all requests in the first request queue of eAP b within a time frame, represents the computational cost of processing all requests in the first request queue of eAPb by the edge server within a time frame, represents the request scheduling decision of eAP b at time frame t, and λ b ∈[0, 1], ψ b ∈[0, 1], ν b ∈[0.5, 1].

[0080] The ratio λ b The calculation formula is:

[0081]

[0082] In the formula, I(·) is the indicator function, T i，b Indicates processing request x i，b The total delay, r i，b Indicates request x i，b The delay requirement, γ(Q b ) represents the first request queue Q arriving at eAP b within a time frame b The number of requests.

[0083] Processing request x i，b The total delay T i，b The calculation formula is:

[0084]

[0085] In the formula, Indicates that the edge server processes the request x i，b The computation delay of Indicates request x i，b The transmission delay from eAP b to the edge server, Indicates request x i，b The queueing delay at the edge server before being processed.

[0086] The edge server processes the request x i，b The calculation delay The calculation formula is:

[0087]

[0088] In the formula, C i，b Indicates that the edge server processes the request x i，b The required computing resources are the number of CPU cycles.

[0089] Request x i，b Transmission delay from eAP b to edge server The calculation formula is:

[0090]

[0091] Where, ν b，n represents the communication throughput between eAP b and edge server n.

[0092] Since the computing resources of each edge server are limited, the queuing time caused by limited computing resources needs to be considered when processing requests in parallel.n |=0, and the remaining CPU, memory, and storage space resources of edge server n are greater than the corresponding resources required to process the request, then otherwise, Specifically, its value can be calculated by subtracting the time when the request arrives from the time when the request is scheduled.

[0093] The time utilization rate ψ b The calculation formula is:

[0094]

[0095] The load balancing index v of the eAP b b The calculation formula is:

[0096] ν b =1 / (1+exp{(-ξ))}; (11)

[0097] Where ξ represents the standard deviation of CPU usage and memory usage of all edge servers in the edge cloud system. ν is introduced to stabilize the edge cloud system and prevent many loads from being imposed on the edge servers. When ν is close to 0.5, the load is more balanced, so there is more CPU space and memory space for scheduling requests. These indicators improve long-term time efficiency while ensuring load balance on edge servers.

[0098] ② Each eAP trains the multi-agent deep reinforcement learning algorithm model based on the state, action and reward function constructed in step ① to obtain the basic model of request scheduling decision;

[0099] The network architecture of the multi-agent deep reinforcement learning algorithm model is the Advantage Actor Critic (A2C) algorithm.

[0100] ③ Based on the federated learning method, the basic models of request scheduling decisions of all eAPs are averaged and weighted to obtain the request scheduling decision model;

[0101]

[0102] In the formula, θ L represents the model parameters of the request scheduling decision model, represents the model parameters of the basic model of request scheduling decision of eAP b, p b represents the model weight of the basic model of request scheduling decision of eAP b, and ∑ b p b =1, p b >0.

[0103] The method for establishing the service orchestration decision model comprises the following steps:

[0104] ①, construct the state, action and reward functions of the service orchestration decision model;

[0105] The service orchestration decision model is used to select which service to place on which edge server. The status includes the edge server's available CPU computing resources, available memory, available storage space, network delay between each eAP and the cloud, second request queue information, deployed services, and service replicas. The role of the service replica is to quickly redeploy the service through the backup copy in time if an error occurs during the operation of the deployed service. Here, for each deployed service, a replica of the service will be deployed as a backup.

[0106] The action is also a service orchestration decision using a c，n，τ express, represents the time slot τ when the service on cloud c Deployed on edge server n, and

[0107] In the service orchestration stage, the objective function, or reward function, is to balance the internal system time efficiency and the cost generated by request scheduling:

[0108]

[0109] In the formula, and represents the weight that controls the ratio of edge server throughput to cost, represents the service orchestration decision of the cloud at time slot τ, T represents the total delay of processing all processing requests in a time slot, |Q n | represents the queue length of the unprocessed second request queue on edge server n.

[0110] The total delay T for processing all processing requests in a time slot is calculated as:

[0111]

[0112] Where, T i，b，τ represents the processing of request x during time slot τ i，b delay requirements.

[0113] ② Each cloud trains the multi-agent deep reinforcement learning algorithm model based on the state, action, and reward function constructed in step ① to obtain the basic model of service orchestration decision-making;

[0114] ③ Based on the federated learning method, the basic service orchestration decision models of all clouds are averaged and weighted to obtain the service orchestration decision model;

[0115]

[0116] In the formula, θ U represents the model parameters of the service orchestration decision model, represents the model parameters of the basic model of service orchestration decision of cloud c, p c represents the model weight of the basic model of service orchestration decision of cloud c, and ∑ c p c =1, and p c >0.

[0117] s2, set the service scheduling period τ, and initialize the request scheduling period round number t = 1;

[0118] In edge systems, frequent service orchestration will lead to more orchestration costs, causing some requests to time out during service switching, resulting in reduced throughput. At the same time, occasional service orchestration may cause the system to be unable to adapt to the ever-changing request scheduling requirements, resulting in reduced processing efficiency. This application sets a service orchestration cycle as a time slot, and a time slot is divided into several time frames. A request scheduling decision is executed once in each time frame, and a service orchestration decision is executed once in each service orchestration cycle.

[0119] S3, such as Figure 4 As shown, the attention mechanism is used to obtain the state s′ of the orchestration service decision model. U，t , and then combined with the first resonance information sent by eAP Again, the attention mechanism is used to U，t Update to get status Set the preset status on the cloud Input attention mechanism to get state Based on status With status Generate second resonance information Sent to eAP.

[0120] The updated state s′ of the service orchestration decision model U，t use It means that its expression is:

[0121]

[0122] The cloud uses the attention mechanism to obtain the state s′ of the service orchestration decision model U，t The expression is:

[0123]

[0124] In the formula, s U，t represents the state of all edge servers at time frame t, s Ux，t represents the x-th type of state observed at time frame t, represents the attention weight of the x-th type of state, X represents the total type of states of the orchestration service decision model, for example, the states of the orchestration service decision model include the available CPU computing resources of the edge server, available memory, available storage space, network delay between each eAP and the cloud, second request queue information, deployed services and service replicas, then X=7.

[0125] The second resonance information The expression is:

[0126]

[0127] Where P U Indicates status After standardization, Indicates status Normalized data.

[0128] The first resonance information is used as the input of the service orchestration decision model, and the second resonance information is used as the input of the request scheduling decision model. This enables the attention networks of the cloud and eAP to resonate at the same frequency, so that when one network changes, the other network also makes corresponding adjustments, thus generating resonance.

[0129] s4, the state Input the service orchestration decision model, optimize the service orchestration decision model to achieve the balance between the time efficiency of the edge server and the cost of service deployment in the service orchestration stage, and combine the lower-level guidance vector g sent by the eAP L，t Get the optimal service orchestration decision and send it to the edge server;

[0130] The optimal service orchestration decision is made through the lower-level guidance vector g L，t The lower layer guides the upper layer. Specifically, the cloud first generates a lower layer guidance basic vector based on the action output by the optimized service orchestration decision model, and transmits the lower layer guidance basic vector to the eAP. The eAP determines whether the action is allowed to be executed based on the edge server's constraints. The constraints include available memory constraints, available computing resource constraints, and available storage space constraints. If the edge server's constraints are met, the action is allowed to be executed. Otherwise, the action is not allowed to be executed. After the judgment is completed, the judgment result is returned to the cloud as a lower layer guidance vector. The cloud combines this judgment result and the action generated by the service orchestration decision model to generate the final optimal service orchestration decision. The lower layer guidance vector g L，t The expression is:

[0131] g L，t =V U *G U ; (19)

[0132] Where V U represents the vector generated by the action output by the service orchestration decision model, G U It represents the indicator vector of whether the action output by the service orchestration decision model is executed according to the constraints of the edge server. U = {1} indicates that the action output by the service orchestration decision model is allowed to be executed on the edge server. U = {0} indicates that the action output by the service orchestration decision model is not allowed to be executed on the edge server, and V U is a vector with values ​​of 0 or 1.

[0133] The goal of the entire system is to achieve a balance between the system's long-term throughput, system latency, and system cost. This application decomposes the overall system goal into a balance between the time efficiency of the request scheduling phase and the cost of the request scheduling phase. The formula for the overall system goal is:

[0134]

[0135] In the formula, and The weight representing the ratio of system throughput to cost, Represents the system throughput, reflecting the ratio of the number of requests completed within the delay requirement to the total number of requests reaching the proxy server ( λ represents the ratio of all eAPs), represents the time utilization rate, i.e., ψ, and represents the system time efficiency, Reflects the ratio of the actual request processing rate to the original request processing rate.

[0136] In order to avoid Φ→∞, the system throughput is used to measure the status of edge cloud system processing requests. The calculation formula of the system throughput is:

[0137]

[0138] In the formula, Φ represents the long-term throughput of the system, that is, the number of requests completed within the delay requirement, and γ τ (Q b ) represents the first request queue Q arriving at proxy server b within the τ time slot b The number of requests.

[0139] The calculation formula of the system long-term throughput Φ is:

[0140]

[0141] In the formula, γ τ (Qn) represents the second request queue Q arriving at edge server n within the τ time slot n The number of requests.

[0142] The time utilization rate The calculation formula is:

[0143]

[0144] In the formula, r b，τ represents the first request queue Q arriving at proxy server b within the τ time slot b The latency requirement of the request.

[0145] S5, eAP uses the attention mechanism to obtain the state s′ of all request scheduling decision models L，t , and then combined with the second resonance information sent by the cloud Use the attention mechanism to the state s′ L，t Update to get status Set the preset status on the eAP Input attention mechanism to get state Based on status With status Generate first resonance information Send to the cloud;

[0146] For state s′ L，t The updated status adopts It means that its expression is:

[0147]

[0148] The state s′ of the request scheduling decision model is obtained by using the attention mechanism L-t The expression is:

[0149]

[0150] In the formula, represents the attention weight of the y-th state of the request scheduling decision model, s Ly，t represents the y-th type of state observed at time frame t, s L，t represents the state of all eAPs at time frame t, and X represents the total type of the state of the request scheduling decision model.

[0151] The first resonance information The expression is:

[0152]

[0153] In the formula, p U Indicates status After standardization, Indicates status Normalized data.

[0154] S6, the state Input the request scheduling decision model, optimize the request scheduling decision model to achieve the balance between the eAP time efficiency and the cost generated by the request scheduling during the request scheduling phase, and combine the upper layer guidance vector g sent by the cloud U，t Obtain the optimal request scheduling decision and send the optimal request scheduling decision to the edge server;

[0155] The optimal request scheduling decision is made through the upper layer guidance vector g U，t The upper layer guides the lower layer. Specifically, the eAP first generates an upper layer guidance basic vector based on the action output by the optimized request scheduling decision model, and transmits the upper layer guidance basic vector to the cloud. The cloud determines whether the action is allowed to be executed based on the constraints of the edge server. If the constraints of the edge server are met, the action is allowed to be executed. Otherwise, the action is not allowed to be executed. After the judgment is completed, the judgment result is returned to the eAP as the upper layer guidance vector. The eAP combines this judgment result and the action generated by the request scheduling decision model to generate the final optimal request scheduling decision. The upper layer guidance vector g U，t The expression is:

[0156] g U，t =y L *G L ; (27)

[0157] Where V L represents the vector generated by the action output by the request scheduling decision model, G L Indicates the indicator vector of whether the action output by the request scheduling decision model is allowed to be executed according to the constraints of the edge server. L = {1} indicates that the action output by the request scheduling decision model is allowed to be executed on the edge server. L = {0} indicates that the action output by the request scheduling decision model is not allowed to be executed on the edge server, and V L is a vector with a value of 0 or 1

[0158] S7, the edge server processes the request according to the received request scheduling decision and service orchestration decision, executes t=t+1 after the request processing is completed, and determines t%r==0. If yes, executes step S3, otherwise, executes step S5.

[0159] like Figure 5 As shown, in terms of training efficiency, SocialEdge, i.e., the present application, has achieved the highest returns in both request distribution and service orchestration compared to other algorithms, because SocialEdge uses social collaboration to obtain additional information, such as data or model knowledge, social refinement, high-level guidance, and resonance attention mechanisms to improve the ability to obtain valuable information. Especially in the service orchestration stage, SocialEdge converges the fastest because it enables the model to quickly learn comprehensive and concentrated knowledge to help make scheduling decisions.

[0160] like Figure 6 As shown in the figure, in terms of algorithm comparison, the compared scheduling methods include: (i) KaiS (GCN is used for service orchestration, MADRL is used for request scheduling); (ii) DRL-s (DRL for orchestration); (iii) DRL-R (DRL for assignment); (iv) Greedy (for scheduling or orchestration), scheduling each request or service to the edge server with the most resources; (v) Random (for assignment or compilation), scheduling each request or service to a random edge server. Figure 5 This demonstrates the overall goal of SocialEdge: a trade-off between system performance and cost. Figure 6 It can be seen that by comparing different algorithms, SocialEdge has the most significant system goal and the second lowest system cost, and can use resources more effectively to meet accurate needs.

[0161] Bandwidth resources affect system stability and are now graded, with higher values ​​indicating lower average bandwidth and higher transmission delay. Figure 7 It can be clearly seen that as the bandwidth level increases, the system performance becomes lower and the cost increases. In addition, as the number of request types increases, the system goals and costs also increase. The time efficiency first decreases and then increases with the number of request types, indicating that when the number of request types is fixed, the service types should neither be too many nor too few, and should adapt to the number of request types.

[0162] Figure 8 The upper part shows the performance of SocialEdge when the edge server has heterogeneous network (i.e., CPU and memory) resources. While keeping the total amount of each resource unchanged, the resource variance between edge nodes is changed, and resource heterogeneity is divided into 5 levels according to the variance, with larger values ​​indicating higher heterogeneity. When the heterogeneity of CPU and memory resources is low, SocialEdge performs better under the existing request distribution. Although the performance of SocialEdge decreases with the increase of resource heterogeneity, it always maintains a high value in most cases, and the cost also varies with the heterogeneity of resources, reaching the minimum when it is moderate. Figure 8 The lower part shows that SocialEdge can adapt to the dynamic changes of the total amount of resources. The total amount of resources is not very important. The cost will increase first and then decrease as the resources decrease, which means that SocialEdge can also achieve the best performance with limited resources. SocialEdge benefits from social learning. It can sensitively perceive the load changes of various environmental resources and adjust policies to maintain efficient performance.

[0163] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning, characterized in that: The steps include: S1, multiple access edge server access agents (eAPs) build request scheduling decision models based on deep reinforcement learning algorithms and federated learning methods, and multiple clouds build service orchestration decision models based on deep reinforcement learning algorithms and federated learning methods; S2, set the service scheduling period τ, and initialize the request scheduling period round number t=1; S3, using the attention mechanism to obtain the state s of the orchestration service decision model ′ U,t , and then combined with the first resonance information sent by eAP Again, the attention mechanism is used to ′ U,t Update to get status Set the preset status on the cloud Input attention mechanism to get state Based on status With status Generate second resonance information Send to eAP; S4, the state Input the service orchestration decision model, optimize the service orchestration decision model to achieve the balance between the time efficiency of the edge server and the cost of service deployment in the service orchestration stage, and combine the lower-level guidance vector g sent by the eAP L,t Get the optimal service orchestration decision and send it to the edge server; S5, eAP uses the attention mechanism to obtain the state s of all request scheduling decision models L L ,t , and then combined with the second resonance information sent by the cloud Use the attention mechanism to L ′ ,t Update to get status Set the preset status on the eAP Input attention mechanism to get state Based on status With status Generate first resonance information Send to the cloud; S6, the state Input the request scheduling decision model, optimize the request scheduling decision model to achieve the balance between the eAP time efficiency and the cost generated by the request scheduling during the request scheduling phase, and combine the upper layer guidance vector g sent by the cloud U,t Obtain the optimal request scheduling decision and send the optimal request scheduling decision to the edge server; S7, the edge server processes the request according to the received request scheduling decision and service orchestration decision. After the request processing is completed, execute t=t+1, judge t%τ==0, if yes, execute step S3, otherwise, execute step S5.

2. According to claim 1, the cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning is characterized in that: In step S1, the method for establishing the request scheduling decision model includes the following steps: ①, construct the state, action and reward functions of the request scheduling decision model; ② Each eAP trains the multi-agent deep reinforcement learning algorithm model based on the state, action and reward function constructed in step ① to obtain the basic model of request scheduling decision; ③ Based on the federated learning method, the basic models of request scheduling decisions of all eAPs are averaged and weighted to obtain the request scheduling decision model.

3. According to claim 2, the cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning is characterized in that: The state of the request scheduling decision model includes the type of request, the latency requirement of the request, the available CPU computing resources of the edge server, the available memory, the available storage space and the total number of edge servers; The action of the request scheduling decision model is to request the scheduling decision to adopt a b,t It means that its expression is a b,t = n, indicating that the request of eAPb at time frame t is assigned to edge server n; The expression of the reward function of the request scheduling decision model is: In the formula, and represents the weight that controls the ratio of eAP throughput to cost, λ b represents the ratio of requests that violate the latency requirement within a time frame, ψ b represents the time utilization, ∈ represents the weight of the load balancing index between edge servers, and v b Indicates the load balancing index of eAPb. represents the network cost of all requests in the first request queue transmitted to eAPb, represents the computational cost of edge server processing requests in the first request queue of eAPb, It represents the request scheduling decision of eAPb in time frame t, and B represents the set of all eAPs, i.e., proxy servers.

4. The cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning according to claim 1 is characterized in that: In step S1, the method for establishing the service orchestration decision model includes the following steps: ①, construct the state, action and reward functions of the service orchestration decision model; ② Each cloud trains the multi-agent deep reinforcement learning algorithm model based on the state, action, and reward function constructed in step ① to obtain the basic model of service orchestration decision-making; ③ Based on the federated learning method, the basic service orchestration decision models of all clouds are averaged and weighted to obtain the service orchestration decision model.

5. The cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning according to claim 4 is characterized in that: The state of the service orchestration decision model includes available CPU computing resources, available memory, available storage space of the edge server, network latency between each eAP and the cloud, information of a second request queue for temporarily storing scheduled and unexecuted requests, and deployed services; The action of the service orchestration decision model is also the service orchestration decision adopting a c,n,τ express, represents the time slot τ when the service on cloud c Deployed on edge server n; The reward function of the service orchestration decision model is expressed as: In the formula, and represents the weight that controls the ratio of edge server throughput to cost, represents the service orchestration decision of the cloud at time slot τ, T represents the total delay of processing all processing requests in a time slot, represents the set of edge servers, |Q n | represents the second request queue Q that is not processed on edge server n n The queue length, K capex represents the cost of installing the network for the operator, It represents the rental cost of leasing the memory w of edge server n during the service orchestration stage, which is also the service orchestration cost.

6. The cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning according to claim 1 is characterized in that: In step S3, the state The expression is: The status ′ U,t The expression is: In the formula, s U,t represents the state of all edge servers at time frame t, s Ux,t represents the x-th type of state observed at time frame t, represents the attention weight of the x-th type of state, and X represents the total type of states of the orchestration service decision model.

7. The cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning according to claim 1 is characterized in that: In step S3, the second resonance information The expression is: Where P U Indicates status After standardization, Indicates status Normalized data.

8. The cloud-edge collaborative service orchestration and request scheduling optimization method based on social learning according to claim 1 is characterized in that: In step S5, the state The expression is: The first resonance information The expression is: Where P U Indicates status After standardization, Indicates status Normalized data.

Images