A computing resource scheduling method that adapts to dynamic environments in edge-end collaboration scenarios

Abstract

The present invention discloses a computing resource scheduling method that adapts to dynamic environments in edge-end collaboration scenarios. In dynamic edge-end collaboration scenarios, the method predicts the amount of business requests, comprehensively considers multi-dimensional resources to evaluate business requests, edge servers, and algorithm units, and matches business requests with edge servers. A deep reinforcement learning algorithm is used to solve the algorithm unit scheduling problem, making it adaptable to dynamically changing environments. At the same time, the method optimizes latency, edge layer contribution, and edge server resource utilization balance, optimizes latency, improves edge layer contribution, and makes edge server resource utilization more balanced. Simulation experiments show that the method proposed in the present invention can reduce latency, improve edge layer contribution, and improve edge server resource utilization balance to a certain extent.

Description

Technical Field

[0001] The present invention relates to the technical field of resource scheduling in edge-end collaboration scenarios, and in particular to a computing power resource scheduling method that adapts to dynamic environments in edge-end collaboration scenarios. Background Art

[0002] With the development of smart devices and the Internet of Things (IoT), a growing number of applications are rapidly emerging, such as virtual reality / augmented reality, facial recognition, and intelligent driving. These applications place high demands on computing power. In traditional cloud computing, large amounts of applications and data are transmitted to cloud data centers for processing. Cloud data centers have the powerful computing and storage capabilities to meet the needs of these applications. However, the distance between terminal devices and cloud centers is often long, which can result in long transmission delays. For some latency-sensitive services, cloud computing struggles to meet the low latency requirements.

[0003] Edge computing has experienced rapid development and widespread application in recent years. Unlike traditional cloud computing, edge computing deploys high-performance servers near users, bringing computing power to the edge of the network. This enhances computing capabilities at the edge of the network and allows compute-intensive and latency-sensitive services to be processed on edge servers.

[0004] Edge intelligence (EI), the integration of mobile edge computing (MEC) and artificial intelligence (AI) technologies, has recently emerged as a promising paradigm for supporting compute-intensive AI applications at the network edge. AI algorithms are widely used in applications such as virtual reality / augmented reality, facial recognition, and intelligent driving. For example, facial recognition requires algorithms such as CNNs. In edge intelligence, AI algorithms can be deployed on edge servers, where training and inference are performed, providing more efficient services for services like facial recognition. Compared to traditional cloud centers, edge servers are closer to users, resulting in shorter transmission latency, but edge server resources are limited.

[0005] To rationally utilize resources, CN114490018A (Prior Art 1) proposes a service scheduling algorithm based on resource feature matching. This introduces a resource feature matching mechanism into the service scheduling algorithm to address issues such as the inability to meet user MQoS requirements and underutilize resources in service scheduling scenarios. By introducing the concept of resource feature matching, execution efficiency is improved, energy consumption is reduced, and dynamic adjustment of optimization directions is supported.

[0006] CN113821317A (prior art 2) proposes a microservice scheduling method, device and equipment for edge-cloud collaboration, which decouples an application service program into multiple microservices, and based on an improved particle swarm optimization algorithm, takes service completion time and resource utilization of edge servers as optimization targets, and finally obtains a microservice scheduling strategy that achieves multi-objective optimization through continuous iteration; on the one hand, service scheduling enables service execution to be as close to the data source as possible, meeting the user's low-latency requirements; on the other hand, scheduling some microservices of non-latency-sensitive applications to cloud servers greatly alleviates the computing pressure of edge servers, avoids the problem of edge server overload, and affects service quality.

[0007] CN113391647A (prior art 3) proposes a multi-UAV edge computing service deployment and scheduling method and system, which relates to the field of UAV communication and service deployment technology. The method includes the following steps: service dependency and topology-aware microservice deployment; greedy microservice scheduling; load-aware microservice redeployment and failure-triggered service redeployment. The method has a fast service response speed: it can initialize microservice deployment and greedily schedule microservices based on the popularity of microservices, the dependencies between microservices, and the network topology, effectively reducing application response delay; good environmental adaptability: it can perceive the queue length and idle state of microservices running on edge computing nodes, thereby dynamically adjusting service function deployment when the service is congested or resources are idle, and better adapting to the changing execution environment; flexible failure adjustment: when a UAV edge node fails, the service can be redeployed in time to ensure service continuity.

[0008] In the existing scheduling methods, Technical Solution 1 introduces a resource feature matching mechanism into the service scheduling algorithm, but the resource features it considers are not comprehensive enough, and other resources may not be matched or reasonably utilized. Technical Solution 2 and Technical Solution 3 consider the optimization objectives of latency, resource utilization, etc. when performing service scheduling. In the complex environment of the edge collaboration scenario, their optimization objectives are not comprehensive enough. Technical Solution 2 uses a particle swarm optimization algorithm for microservice scheduling. Traditional heuristic algorithms generally give the optimal solution based on a certain problem, and are less efficient in solving problems in a dynamic environment. Therefore, the present invention has developed a set of algorithm unit scheduling methods that adapt to dynamic environments and make rational use of resources. Summary of the Invention

[0009] In response to the problem that the resource characteristics and optimization objectives considered in the existing technologies in dynamic edge collaboration scenarios are not comprehensive enough, the present invention proposes a computing resource scheduling method that adapts to the dynamic environment in the edge collaboration scenario. In the dynamic edge collaboration scenario, computing resources such as algorithm units are scheduled to adapt to the dynamically changing environment. At the same time, the latency is optimized, the contribution of the edge layer is improved, and the resource utilization of the edge server is more balanced.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] The present invention provides a computing power resource scheduling method adapted to a dynamic environment in an edge collaboration scenario, including the following steps:

[0012] S1. Construct resource evaluation models for service requests, edge servers, and algorithm units. Each resource evaluation model is modeled from four dimensions: computing, communication, storage, and memory;

[0013] S2. Based on historical service request volumes, make predictions to obtain future service request volumes, and use the prediction results as the basis for scheduling algorithm units;

[0014] S3. Evaluate service requests, edge servers, and algorithm units using resource metrics in the four dimensions of computing, communication, storage, and memory, and match service requests with edge servers according to the evaluation results;

[0015] S4. Considering the predicted service request volume, the matching result of service requests and edge servers, and the type of algorithm unit required by service requests, and taking into account latency, edge layer contribution, and the balance of resource utilization of edge servers, formulate a multi-objective optimization problem model;

[0016] S5. Based on the deep reinforcement learning algorithm, perform algorithm unit scheduling.

[0017] Further, in step S1, the service request is modeled as a quadruple request i , i =(tc <0 / / i , tb i , ts i , tm i ), 0≤i<M t , where the elements within the quadruple represent the computing requirement, communication requirement, storage requirement, and memory requirement of the service, and M t represents the number of service requests at the current time slot t;

[0018] The edge server is modeled as a quadruple E i =(C i , B i , S i , M i ), 0≤i<N, where the elements within the quadruple represent the computing power, communication capacity, storage capacity, and memory capacity of the server, and N represents the number of servers;

[0019] The algorithm unit is modeled as a quadruple r k =(c k , b k , s k , mk ), 0 ≤ k < W, where the elements within the quadruple respectively represent the computing resources, communication resources, storage resources, and memory resources required by the algorithm unit, and W represents the number of algorithm units.

[0020] Furthermore, in step S2, the method for predicting based on historical traffic request volume is as follows: Let the traffic request volume in the t-th time slot be X(t), the time series sliding window size be d, and the GRU deep learning algorithm is adopted. The traffic request volumes in the past d time slots are used to predict the traffic request volume in the next 1 time window.

[0021] Furthermore, in step S3, the resource metrics in the four dimensions of computing, communication, storage, and memory are used to evaluate the traffic requests and edge servers. First, the traffic requests and edge server data are normalized, and the process is as follows:

[0022] For the computing demand sequence of the service is normalized to obtain the normalization result Similarly, the communication demand, storage demand, and memory demand of the traffic request are normalized, and the computing, communication, storage, and memory resources owned by the edge server are normalized. The results are as follows:

[0023] The normalization result of the resource demand of request i is expressed as [[ID=l8]]

[0024] The normalization result of the resources owned by server i is expressed as

[0025] Then, different weights Q = (a, b, c, d) are assigned to the four metrics to obtain the comprehensive evaluation results TR i and ER i , as shown in the following formula:

[0026] TR i = GT i ·Q T

[0027] ER i = GE i ·Q T

[0028] where, TR i is the comprehensive evaluation result of traffic request i, ER i is the comprehensive evaluation result of edge server i, GE i is the normalization result of the resources owned by edge server i, GT i is the normalization result of the resource demand of traffic request i, and Q T is the weight of the four resource metrics.

[0029] Furthermore, in step S3, the service request and the edge server are graded using the Rank method, and matched using the Map method according to the grade;

[0030] Business requests and edge servers are ranked using the Rank method, as follows:

[0031] 1) Find the maximum and minimum values ​​in the comprehensive evaluation result sequence and calculate their difference;

[0032] 2) Define the number of levels to be divided rankcount;

[0033] 3) Divide the difference between the maximum and minimum values ​​in the comprehensive evaluation results by the number of levels to obtain the level interval gap, that is, the difference between the maximum and minimum values ​​in each level range is the gap; the level interval is divided into {[a0,a1),[a1,a2),...,[a rankcount-1 ,a rankcount ]}, where a n -a n-1 =gap;

[0034] 4) Use the evaluation results to correspond to the grade intervals to obtain the grading results;

[0035] The method of matching by level through Map method is as follows:

[0036] 1) According to the level of the business, select a server of the same level;

[0037] 2) If the level of a service is r, a server of level r is selected for it. If the number of servers of level r is greater than 1, the server with the least number of services on it is selected to match the service. If the level of a service is r but there is no server of level r, the server with the closest level is selected for it.

[0038] 3) The final matching result matrix A is obtained, which is expressed as follows:

[0039]

[0040] Among them, a i Indicates that business i is matched to server a i .

[0041] Furthermore, in step S4, the delay model construction process is as follows:

[0042] Business requests need to be transmitted to the server for processing, which can be divided into two situations:

[0043] Case 1: If the matched server j has a business request iRequired algorithm unit r k , that is, b t j,k =1, the service request is responded to directly on the server;

[0044] Case 2: If the matched server j has no business request i Required algorithm unit r k , that is, b t j,k =0, then transmit this business request to the central cloud;

[0045] In case 1, the delay from the terminal device to the server is Assume that the delay in case 1 is In case 2, if no response is received and the data needs to be transmitted to the central cloud, let the delay in case 2 be τ vc ; Define the delay of service i as It is expressed as follows:

[0046]

[0047] The average delay is defined as:

[0048]

[0049] Where M represents the number of service requests.

[0050] Furthermore, in step S4, the edge layer contribution model is constructed as follows:

[0051] use Indicates whether business request i can be responded to at the edge layer. Let the server matched by business request i be j, and the algorithm unit required by business request i be k. It is expressed as follows:

[0052]

[0053] Among them, if the matched server j has a business request i Required algorithm unit r k , that is, b t j,k =1, then the service request is responded to directly on the server; if the matched server j has no service request i Required algorithm unit r k , that is, b t j,k =0, then transmit this business request to the central cloud;

[0054] The contribution of the edge layer is expressed as H t, defined as the ratio of the number of services responded to at the edge layer in time slot t to the total number of services, and is expressed as follows:

[0055]

[0056] Among them, M t Indicates the number of service requests in the current time slot t.

[0057] Furthermore, in step S4, the edge server resource utilization balance model is constructed as follows:

[0058] Use rc j (y j ,z k ) represents the percentage of remaining resources of server j under a certain indicator, expressed as:

[0059]

[0060] Among them, y∈{C,B,S,M} represents the current resource amount of the server under a certain indicator, and z∈{c,b,s,m} represents the resources that the algorithm unit needs to occupy under a certain indicator;

[0061] Use RC t ={rc1,rc2,...,rc N} represents the current comprehensive resource amount of the server after the algorithm unit is deployed. The current resource amount of the server is expressed as:

[0062]

[0063] The resource utilization balance δ t Defined as 1 minus rc j The standard deviation of the sequence indicates the degree of balance of the current resources of each server. The degree of balance of edge server resource utilization is expressed by the following formula:

[0064]

[0065] in, Represents RC t The mean of the series.

[0066] Furthermore, the multi-objective optimization problem model of step S4 is constructed as follows:

[0067]

[0068] in, represents the average delay, H t represents the contribution of the edge layer, δ tIndicates the degree of balanced utilization of edge server resources. α, β, and γ represent the weight coefficients of latency, edge layer contribution, and balanced utilization of resources, respectively.

[0069] Furthermore, in step S5, the algorithm unit is deployed on the server through a scheduling method based on deep reinforcement learning, and the result is represented as a matrix B t :

[0070]

[0071] where b ij is a binary variable, if b ij If b is 1, it means that the algorithm unit j exists on the server i; if b ij If it is 0, it means that algorithm unit j does not exist on server i.

[0072] Compared with the prior art, the present invention has the following beneficial effects:

[0073] The present invention proposes a computing resource scheduling method that adapts to dynamic environments in edge collaboration scenarios. In dynamic edge collaboration scenarios, the method predicts the amount of business requests, comprehensively considers multi-dimensional resources to evaluate business requests, edge servers, and algorithm units, and matches business requests with edge servers. It uses a deep reinforcement learning algorithm to solve the algorithm unit scheduling problem, making it adaptable to dynamically changing environments. At the same time, it optimizes latency, edge layer contribution, and the balance of edge server resource utilization, optimizes latency, improves the contribution of the edge layer, and makes the resource utilization of edge servers more balanced. Simulation experiments have shown that the method proposed by the present invention can reduce latency, improve the contribution of the edge layer, and improve the balance of edge server resource utilization to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 A flow chart of a computing resource scheduling method that adapts to dynamic environments in edge collaboration scenarios provided in an embodiment of the present invention.

[0075] Figure 2 A schematic diagram of a method for matching service requests and edge servers provided in an embodiment of the present invention.

[0076] Figure 3 This is a delay comparison diagram provided by an embodiment of the present invention.

[0077] Figure 4 A comparison chart of edge layer contributions provided by an embodiment of the present invention.

[0078] Figure 5 A comparison chart of resource utilization balance provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0079] The resources of edge servers are limited. A reasonable algorithm unit scheduling method needs to be developed to make full use of the limited resources to serve business requests. The business requests in the environment are dynamically changing. When formulating the algorithm unit scheduling plan, the dynamic characteristics of the environment need to be considered, and at the same time, the service quality of users and the resource utilization of edge servers need to be taken into account. Therefore, this invention has developed a computing power resource scheduling method that adapts to the dynamic environment in the edge-end collaboration scenario.

[0080] To better understand this technical solution, the technical solutions in the embodiments of this invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this invention. Obviously, the described examples are only a part of the embodiments of this invention, rather than all of the embodiments. Based on the embodiments in this invention, all other embodiments obtained by those of ordinary skill in the art based on this application belong to the scope protected by this invention.

[0081] The computing power resource scheduling method that adapts to the dynamic environment in the edge-end collaboration scenario proposed by this invention, as Figure 1 shown, includes the following steps:

[0082] S1. Construct a resource evaluation model for business requests, edge servers, and algorithm units. Each resource evaluation model is modeled from four dimensions: computing, communication, storage, and memory.

[0083] Specifically, the business request is modeled as a quadruple request i =(tc i ,tb i ,ts i ,tm i ), 0≤i<M t , where the elements within the quadruple represent the computing requirements, communication requirements, storage requirements, and memory requirements of the business, and M t represents the number of business requests in the current time slot t.

[0084] The edge server is modeled as a quadruple E i =(C i ,B i ,S i ,M i ), 0≤i<N, where the elements within the quadruple represent the computing power, communication power, storage power, and memory power of the server, and N represents the number of servers.

[0085] The algorithm unit needs to occupy certain computing, communication, storage, and memory resources when running. The resources occupied by the algorithm unit of type r k are represented by the quadruple r k =(c k ,b k ,s k ,mk ), 0 ≤ k < W, where the elements in the quadruple represent the computing resources, communication resources, storage resources, and memory resources required by the algorithm unit, and W represents the number of algorithm units.

[0086] The volume of business requests has certain time characteristics. Therefore, first, a business prediction model is trained based on the historical data characteristics of the business for predicting future business request volumes. The GRU deep learning algorithm is used to actively predict the volume of business requests, and the prediction result is used as the basis for algorithm unit scheduling. Let the volume of business requests in the t-th time slot be X(t), and the time series sliding window size be d. The business request volumes in the past d time slots are used to predict the business request volume in the next 1 time window. GRU is used for prediction. In the training stage, the historical observation sample set is X = {X(t - d), (t - d + 1), …, X(t - 1)|X(t)}. {X(t - d), …, X(t - 1)} is used as the input of the GRU network model, and the prediction result X'(t) is output. By comparing X^'(t) and X(t), the model parameters are adjusted for model training.

[0087] S2. Predict based on historical business request volumes to obtain future business request volumes, and use the prediction results as the basis for algorithm unit scheduling.

[0088] Specifically, the method for predicting based on historical business request volumes is as follows: Let the volume of business requests in the t-th time slot be X(t), and the time series sliding window size be d. The GRU deep learning algorithm is used, and the business request volumes in the past d time slots are used to predict the business request volume in the next 1 time window. [[ID=I1]]

[0089] The present invention formulates a matching method for business and servers with resources as the main consideration factors. First, a comprehensive evaluation of business requirements and server resources is carried out to obtain a comprehensive evaluation result. Then, according to the evaluation result, the business and the server are graded. After grading, the business and the server are matched according to the grades. If this method is not adopted, it may lead to a situation where a business with high resource requirements is matched to a server with poor performance, resulting in the inability to meet business requirements. By adopting this method, the resources of the server can be utilized more reasonably, and at the same time, the matching result can provide an effective reference basis for subsequent algorithm unit scheduling work.

[0090] S3. Evaluate business requests, edge servers, and algorithm units using resource metrics in four dimensions: computing, communication, storage, and memory, and match business requests and edge servers according to the evaluation results.

[0091] Specifically, for the evaluation of business requests and edge servers, first, the data of business requests and edge servers are normalized, and the process is as follows:

[0092] Computational requirements sequence for all businesses Normalize and get the normalized result Similarly, we normalize the communication, storage, and memory requirements of the business requests, and normalize the computing, communication, storage, and memory resources owned by the edge server. The results are as follows:

[0093] The normalized resource requirement of request i is expressed as

[0094] The normalized result of the resources owned by server i is expressed as

[0095] Then assign different weights Q = (a, b, c, d) to the four indicators to obtain the comprehensive evaluation result TR i and ER i , as shown in the following formula:

[0096] TR i =GT i Q T

[0097] ER i =GE i Q T

[0098] Among them, TR i is the comprehensive evaluation result of business request i, ER i is the comprehensive evaluation result of edge server i, GE i is the normalized result of the resources owned by edge server i, GT i is the normalized result of resource requirements of business request i, Q T is the weight of the four resource indicators.

[0099] Then, the business request and the server are ranked using the Rank method and matched using the Map method, and the matching result is defined as matrix A.

[0100] Business requests and edge servers are ranked using the Rank method, as follows:

[0101] According to the comprehensive evaluation results TR i and ER i , classify services and servers:

[0102] Rank(TR i )=R TRi

[0103] Rank(ER i )=R ERi

[0104] For the rank method, it is defined as follows:

[0105] 1) Find the maximum and minimum values ​​in the comprehensive evaluation result sequence and calculate their difference;

[0106] 2) Define the number of levels to be divided, rankcount; for example, to divide into 5 levels, set rankcount = 5;

[0107] 3) Divide the difference between the maximum and minimum values ​​in the comprehensive evaluation results by the number of levels to obtain the level interval gap, that is, the difference between the maximum and minimum values ​​in each level range is the gap; the level interval is divided into {[a0,a1),[a1,a2),...,[a rankcount-1 ,a rankcount ]}, where a n -a n-1 =gap;

[0108] 4) Use the evaluation results to correspond to the grade intervals to obtain the grading results.

[0109] After classification, services and servers are matched according to their levels through the Map method, as follows:

[0110] 1) According to the level of the business, select a server of the same level;

[0111] 2) If the level of a service is r, a server of level r is selected for it. If the number of servers of level r is greater than 1, the server with the least number of services on it is selected to match the service. If the level of a service is r but there is no server of level r, the server with the closest level is selected for it.

[0112] 3) The final matching result matrix A is obtained, which is expressed as follows:

[0113]

[0114] Among them, a i Indicates that business i is matched to server a i .

[0115] Different businesses require different algorithm units to provide services for them, using the mapping relationship NS (request i )=r k Indicates that the algorithm unit required for business i is r k .

[0116] like Figure 2First, the business request and the server are graded by the comprehensive evaluation method and the grading method to obtain the grading results of the business request and the server, and then the business request and the server are matched by the matching method Map according to the grading results.

[0117] S4. Comprehensively predict the business request volume, the matching results between business requests and edge servers, and the type of algorithm unit required for business requests, taking into account latency, edge layer contribution, and the balance of edge server resource utilization, and formulate a multi-objective optimization problem model.

[0118] Specifically, the delay model construction process is as follows:

[0119] Business requests need to be transmitted to the server for processing, which can be divided into two situations:

[0120] Case 1: If the matched server j has a business request i Required algorithm unit r k , that is, b t j,k =1, the service request is responded to directly on the server;

[0121] Case 2: If the matched server j has no business request i Required algorithm unit r k , that is, b t j,k =0, then transmit this business request to the central cloud;

[0122] In case 1, the delay from the terminal device to the server is Assume that the delay in case 1 is In case 2, if no response is received and the data needs to be transmitted to the central cloud, let the delay in case 2 be τ vc ; Define the delay of service i as It is expressed as follows:

[0123]

[0124] The average delay is defined as:

[0125]

[0126] Where M represents the number of service requests.

[0127] The process of building the edge layer contribution model is as follows:

[0128] One of the optimization goals of this solution is to maximize the number of responses to business requests on the matched edge servers, thereby maximizing the role of the edge layer. Therefore, the edge layer contribution is set. The business and the server are matched using the above matching method. If the server has the algorithm unit required by the business request and can process the business request, the business can be responded to at the edge layer. Otherwise, the business cannot be responded to at the edge layer. The edge layer contribution is expressed as H t , defined as the ratio of the number of services responded to at the edge layer to the total number of services in the time slot t.

[0129] use Indicates whether business request i can be responded to at the edge layer. Let the server matched by business request i be j, and the algorithm unit required by business request i be k. It is expressed as follows:

[0130]

[0131] Among them, if the matched server j has a business request i Required algorithm unit r k , that is, b t j,k =1, then the service request is responded to directly on the server; if the matched server j has no service request i Required algorithm unit r k , that is, b t j,k =0, then transmit this business request to the central cloud;

[0132] The contribution of the edge layer is expressed as H t , defined as the ratio of the number of services responded to at the edge layer in time slot t to the total number of services, and is expressed as follows:

[0133]

[0134] Among them, M t Indicates the number of service requests in the current time slot t.

[0135] The edge server resource utilization balance model is constructed as follows:

[0136] The algorithm unit needs to occupy the resource space of the server. In order to keep the resource utilization of the server as balanced as possible, the present invention defines the resource utilization balance degree to represent the difference in resource utilization of each server.

[0137] Use rc j (y j ,z k ) represents the percentage of remaining resources of server j under a certain indicator, expressed as:

[0138]

[0139] Among them, y∈{C,B,S,M} represents the current resource amount of the server under a certain indicator, and z∈{c,b,s,m} represents the resources that the algorithm unit needs to occupy under a certain indicator; for example, rc j (C j ,c k ) represents the current remaining resource percentage of server j under the calculation indicator.

[0140] Use RC t ={rc1,rc2,...,rc N} represents the current comprehensive resource amount of the server after the algorithm unit is deployed. Since the four indicators have large differences in magnitude, the current resource amount of the server is expressed as:

[0141]

[0142] The constraint condition is that all four sub-terms in the following formula are not less than zero.

[0143] The resource utilization balance δ t Defined as 1 minus rc j The standard deviation of the sequence indicates the degree of balance of the current resources of each server. The degree of balance of edge server resource utilization is expressed by the following formula:

[0144]

[0145] in, Represents RC t The mean of the sequence, δ t The larger it is, the more balanced the resource utilization is.

[0146] From a business perspective, they need to receive responses from edge servers as quickly as possible, so latency is a key evaluation metric. The present invention aims for low latency. From the edge server perspective, the role of the edge layer should be maximized, ensuring that as many business requests as possible are responded to at the edge layer. At the same time, algorithm unit scheduling should ensure that resource utilization across servers is as balanced as possible. The present invention aims for a high degree of resource utilization balance. Therefore, the present invention formulates the overall optimization objective function G, as shown below.

[0147] The multi-objective optimization problem model (comprehensive optimization goal) is constructed as:

[0148]

[0149] in, represents the average delay, H trepresents the contribution of the edge layer, δ t Indicates the degree of balanced utilization of edge server resources. α, β, and γ represent the weight coefficients of latency, edge layer contribution, and balanced utilization of resources, respectively.

[0150] S5. Algorithm unit scheduling based on deep reinforcement learning algorithm.

[0151] In the deep reinforcement learning algorithm, the intelligent agent interacts and learns with the environment, makes scheduling decisions based on the real-time state of the environment, and can better adapt to the dynamically changing environment. Therefore, we use the deep reinforcement learning algorithm to solve the scheduling problem. In addition, the deployment scenario of the present invention includes multiple servers and multiple algorithm units. The state space and action space dimensions are high. The Q learning method in reinforcement learning is difficult to solve high-dimensional problems. Therefore, the DQN method that combines reinforcement learning and deep learning is adopted. This method interacts with the environment and realizes the dynamic scheduling of the algorithm unit based on the dynamically changing environmental information. In addition, in order to prevent the algorithm unit from occupying server resources for a long time, a life cycle is set for the algorithm unit. When the deployed algorithm unit reaches a certain life cycle T end After that, it will be automatically deleted and the corresponding resource space will be released. This method can dynamically schedule the algorithm unit and minimize the delay, improve the service response rate, and improve the balance of server resource utilization.

[0152] Specifically, the present invention deploys the algorithm unit on the server through a scheduling method based on deep reinforcement learning, and expresses the result as a matrix B t :

[0153]

[0154] where b ij is a binary variable, if b ij If b is 1, it means that the algorithm unit j exists on the server i; if b ij If it is 0, it means that algorithm unit j does not exist on server i.

[0155] The algorithm model training process is as follows:

[0156] First, define the state space, action space, and reward function in the algorithm unit scheduling environment. The Markov model is defined as follows:

[0157] State: The current state of the algorithm unit on the server state(t) = [s0,s1,...,s p ,...,s N*W-1 ],s p ∈{0,1}, if s pIf p / N = 1, then let p / N = i and p % N = j, where N is the number of servers and W is the number of algorithm units; that is, the integer part of the element serial number p divided by the number of servers is equal to i, and the remainder is equal to j, indicating that there is algorithm unit j on server i;

[0158] Action: The algorithm unit scheduling decision a(t) = [ac0, ac1,..., acW-1], 0 ≤ ac i < N, where ac i Indicates that algorithm unit i is deployed to server ac i ; If the algorithm unit already exists, it will not be redeployed; if the remaining resources of the server cannot meet the resource requirements of the algorithm unit, it will not be deployed;

[0159] Reward function: The reward function is set as the comprehensive optimization objective function;

[0160] R(t) = G t (16)

[0161] Then define the algorithm unit existence time matrix:

[0162]

[0163] where N is the number of servers, W is the number of algorithm units, and d i.j Indicates the time that algorithm unit j on server i has existed, which is used to determine whether the algorithm unit reaches the life cycle limit and whether it needs to be deleted.

[0164] The algorithm unit scheduling algorithm based on deep reinforcement learning is shown in Table 1.

[0165] Table 1 Algorithm unit scheduling algorithm based on deep reinforcement learning

[0166]

[0167]

[0168] Predict based on historical business request volume, randomly generate the resource volume and business requests of edge servers and the resource requirements of algorithm units, match business requests and edge servers, and train the scheduling algorithm based on deep reinforcement learning. Set the weights of delay, edge layer contribution degree, and edge server resource utilization balance degree in the comprehensive optimization objective function to be α = 1, β = 5, γ = 1 respectively. Use the trained algorithm unit scheduling method based on deep reinforcement learning to perform algorithm unit scheduling, and compare it with the random scheduling method and the greedy scheduling method that only optimizes delay. The comparison results under the three metrics are respectively as Figure 3 、 4 、5 shown. Figure 3The latency comparison results are shown, among which the greedy scheduling method that only optimizes latency has the best latency, the scheduling method based on deep reinforcement learning is second, and the random scheduling method is the worst. Figure 4 The comparison results of edge layer contribution are shown. Among them, the greedy scheduling method that only optimizes latency has the best edge layer contribution, followed by the scheduling method based on deep reinforcement learning, and the random scheduling method is the worst. Figure 5 The comparative results of the degree of resource utilization balance are shown, among which the scheduling method based on deep reinforcement learning obtains the best latency, the random scheduling method is second, and the greedy scheduling method that only optimizes latency is the worst. Because the greedy scheduling method that only optimizes latency only considers latency, and because the contribution of the edge layer is closely related to latency, the greedy scheduling method that only optimizes latency performs best under these two indicators. However, because it ignores other indicators, the degree of resource utilization balance is the worst. The scheduling method based on deep reinforcement learning comprehensively considers the three indicators, and compared with the random scheduling method, all of them are optimized to a certain extent. Simulation experiments show that the method proposed in the present invention can reduce latency, improve the contribution of the edge layer, and improve the degree of resource utilization balance of edge servers to a certain extent.

[0169] The foregoing description is merely a detailed description of the preferred embodiments and principles of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, any modifications, equivalent substitutions, improvements, etc. made based on the ideas provided by the present invention and within the spirit and principles of the present invention should be considered to be within the scope of protection of the present invention.

Claims

1. A computing resource scheduling method that adapts to dynamic environments in edge-end collaboration scenarios, characterized by: The steps include: S1. Build resource evaluation models for business requests, edge servers, and algorithm units. Each resource evaluation model is modeled from four dimensions: computing, communication, storage, and memory. S2. Predict future business request volumes based on historical business request volumes, and use the prediction results as the basis for scheduling algorithm units. S3. Evaluate business requests, edge servers, and algorithm units using resource indicators in four dimensions: computing, communication, storage, and memory. Match business requests with edge servers based on the evaluation results. S4. Formulate a multi-objective optimization problem model based on the predicted business request volume, the matching results between business requests and edge servers, and the type of algorithm unit required for the business requests, taking into account latency, edge layer contribution, and the balance of edge server resource utilization; S5. Algorithm unit scheduling based on deep reinforcement learning algorithm; The deep reinforcement learning algorithm model training process is as follows: Define the state space, action space, and reward function in the algorithm unit scheduling environment. The Markov model is defined as follows: State: The current state of the algorithm unit on the server state(t) = [s0,s1,...,s p ,...,s N*W-1 ],s p ∈{0,1}, if s p =1, then let p / N=i, p%N=j, where N is the number of servers and W is the number of algorithm units; that is, if the element number p divided by the number of servers is equal to i, and the remainder is equal to j, it means that algorithm unit j exists on server i; Action: Algorithm unit scheduling decision a(t) = [ac0, ac1,..., ac W-1 , 0 ≤ ac i < N, where ac i represents that algorithm unit i is deployed to server ac i ; if the algorithm unit already exists, it will not be redeployed; if the remaining resources of the server cannot meet the resource requirements of the algorithm unit, it will not be deployed; Reward function: The reward function is set as the comprehensive optimization objective function; R(t)=G t Then define the algorithm unit existence time matrix: Where N is the number of servers, W is the number of algorithm units, d i.j Indicates the time that algorithm unit j on server i has existed. It is used to determine whether the algorithm unit has reached its lifecycle limit and needs to be deleted.

2. The computing resource scheduling method adapted to dynamic environments in edge collaboration scenarios according to claim 1 is characterized in that: In step S1, the business request is modeled as a four-tuple request i =(tc i ,tb i ,ts i ,tm i ),0≤i <M t , where the elements in the quaternary group represent the computing requirements, communication requirements, storage requirements, and memory requirements of the business, respectively. t Indicates the number of service requests in the current time slot t; The edge server is modeled as a quadruple E i =(C i , B i , S i , M i ), 0 ≤ i < N, where the elements within the quadruple represent the computing power, communication power, storage capacity, and memory capacity of the server, and N represents the number of servers; The algorithm unit is modeled as a quadruple r k =(c k , b k , s k , m k ), 0 ≤ k < W, where the elements within the quadruple represent the computing resources, communication resources, storage resources, and memory resources required by the algorithm unit, and W represents the number of algorithm units.

3. The computing resource scheduling method adapted to dynamic environments in edge collaboration scenarios according to claim 1 is characterized in that: In step S2, the method for predicting based on historical business request volume is as follows: let the business request volume of the t-th time slot be X(t), the time series sliding window size be d, and the GRU deep learning algorithm be used to use the business request volume of the past d time slots to predict the business request volume of the next 1 time window.

4. The computing resource scheduling method adapted to dynamic environments in edge collaboration scenarios according to claim 1 is characterized in that: In step S3, the service request and edge server are evaluated using resource indicators in four dimensions: computing, communication, storage, and memory. First, the service request and edge server data are normalized. The process is as follows: Sequence of computing requirements for the business Normalize and get the normalized result Similarly, we normalize the communication, storage, and memory requirements of the business requests, and normalize the computing, communication, storage, and memory resources owned by the edge server. The results are as follows: The normalized resource requirement of request i is expressed as The normalized result of the resources owned by server i is expressed as Then assign different weights Q = (a, b, c, d) to the four indicators to obtain the comprehensive evaluation result TR i and ER i , as shown in the following formula: TR i =GT i ·Q T ER i =GE i Q T Among them, TR i is the comprehensive evaluation result of business request i, ER i is the comprehensive evaluation result of edge server i, GE i is the normalized result of the resources owned by edge server i, GT i is the normalized result of resource requirements of business request i, Q T is the weight factor of the four resource indicators.

5. The computing resource scheduling method adapted to dynamic environments in edge collaboration scenarios according to claim 1 is characterized in that: In step S3, the service request and the edge server are ranked using the Rank method, and matched using the Map method according to the ranking; Business requests and edge servers are ranked using the Rank method, as follows: 1) Find the maximum and minimum values ​​in the comprehensive evaluation result sequence and calculate their difference; 2) Define the number of levels to be divided rankcount; 3) Divide the difference between the maximum and minimum values ​​in the comprehensive evaluation results by the number of levels to obtain the gap between levels, that is, the difference between the maximum and minimum values ​​within each level is the gap; Level interval division {[a0,a1),[a1,a2),...,[a rankcount-1 ,a rankcount ]}, where a n -a n-1 =gap; 4) Use the evaluation results to correspond to the grade intervals to obtain the grading results; The method of matching by level through Map method is as follows: 1) According to the level of the business, select a server of the same level; 2) If the level of a service is r, a server of level r is selected for it. If the number of servers of level r is greater than 1, the server with the least number of services on it is selected to match the service. If the level of a service is r but there is no server of level r, the server with the closest level is selected for it. 3) The final matching result matrix A is obtained, which is expressed as follows: A=[a0,a1,...,a i ,...,a Mt-1 ],0≤a i <N Among them, a i Indicates that business i is matched to server a i .

6. The computing resource scheduling method adapted to dynamic environments in edge-end collaboration scenarios according to claim 1 is characterized in that: In step S4, the delay model construction process is as follows: Business requests need to be transmitted to the server for processing, which can be divided into two situations: Case 1: If the matched server j has a business request i Required algorithm unit r k , that is, b t j,k =1, the service request is responded to directly on the server; Case 2: If the matched server j has no business request i Required algorithm unit r k , that is, b t j,k =0, then transmit this business request to the central cloud; In case 1, the delay from the terminal device to the server is Assume that the delay in case 1 is In case 2, if no response is received, the data needs to be transmitted to the central cloud. Let the delay in case 2 be τ vc ; Define the delay of service i as It is expressed as follows: The average delay is defined as: Where M represents the number of service requests.

7. The computing resource scheduling method adapted to dynamic environments in edge collaboration scenarios according to claim 1 is characterized in that: In step S4, the edge layer contribution model is constructed as follows: use Indicates whether business request i can be responded to at the edge layer. Let the server matched by business request i be j, and the algorithm unit required by business request i be k. It is expressed as follows: Among them, if the matched server j has a business request i Required algorithm unit r k , that is, b t j,k =1, then the server will respond to the service request directly; if the matched server j has no service request i Required algorithm unit r k , that is, b t j,k =0, then transmit this business request to the central cloud; The contribution of the edge layer is expressed as H t , defined as the ratio of the number of services responded to at the edge layer in time slot t to the total number of services, and is expressed as follows: Among them, M t Indicates the number of service requests in the current time slot t.

8. The computing resource scheduling method adapted to dynamic environments in edge-end collaboration scenarios according to claim 1 is characterized in that: In step S4, the edge server resource utilization balance model is constructed as follows: Use rc j (y j ,z k ) represents the percentage of remaining resources of server j under a certain indicator, expressed as: Among them, y∈{C,B,S,M} represents the current resource amount of the server under a certain indicator, and z∈{c,b,s,m} represents the resources that the algorithm unit needs to occupy under a certain indicator; Use RC t ={rc1,rc2,...,rc N } represents the current comprehensive resource amount of the server after the algorithm unit is deployed. The current resource amount of the server is expressed as: The resource utilization balance δ t Defined as 1 minus rc j The standard deviation of the sequence indicates the degree of balance of the current resources of each server. The degree of balance of edge server resource utilization is expressed by the following formula: in, Represents RC t The mean of the series.

9. The computing resource scheduling method adapted to dynamic environments in edge collaboration scenarios according to claim 1 is characterized in that: The multi-objective optimization problem model of step S4 is constructed as follows: in, represents the average delay, H t represents the contribution of the edge layer, δ t Indicates the degree of balanced utilization of edge server resources. α, β, and γ represent the weight coefficients of latency, edge layer contribution, and balanced utilization of resources, respectively.

10. The computing resource scheduling method adapted to dynamic environments in edge-end collaboration scenarios according to claim 1 is characterized in that: In step S5, the algorithm unit is deployed on the server through a scheduling method based on deep reinforcement learning, and the result is represented as a matrix B t : where b ij is a binary variable, if b ij If b is 1, it means that the algorithm unit j exists on the server i; if b ij If it is 0, it means that algorithm unit j does not exist on server i.

Images