Resource allocation method and device, processing equipment, computer storage medium and program product

Abstract

The invention discloses a resource allocation method and device, processing equipment, a computer storage medium and a program product, and the method comprises the steps: receiving a task demand of a user, and enabling the task demand of the user to carry task data, computing resources and tolerant time delay of the user; acquiring current running state information of the edge computing network, and executing a corresponding scheduling decision on the user task demand based on deep reinforcement learning according to the running state information; issuing the user task demand to a corresponding edge computing node according to the scheduling decision, wherein the edge computing node is used for processing the user task demand; therefore, computing resources of the edge computing node are fully utilized, and the execution success rate of the user task is improved.

Description

Technical Field

[0001] The present application relates to the field of IT applications, and in particular to a resource allocation method and apparatus, processing equipment, computer storage medium and program product. Background Art

[0002] With the rapid development of mobile terminal applications, future wireless networks will need to have billions of mobile devices to support ultra-low latency communications and computing. However, the resource requirements of newly developed applications are growing, the computing power and battery capacity of user devices are still limited, and real-time information processing requires a lot of computing, resulting in high resource consumption.

[0003] In order to solve the problem of scarce user device resources, Mobile Edge Computing (MEC) technology migrates intensive computing tasks from user devices to the edge of the network and provides users with low-latency and flexible computing and communication services. Operators can take advantage of economies of scale when providing edge computing services, making edge computing services more affordable for the general public. However, how to allocate user task requests to different edge servers so that the workload between edge servers is well balanced, thereby shortening the response time delay of tasks and enhancing the user experience. An effective resource allocation method is urgently needed to solve this problem. Summary of the invention

[0004] To solve the above technical problems, embodiments of the present invention provide a resource allocation method and apparatus, a processing device, a computer storage medium, and a program product.

[0005] In a first aspect, a resource allocation method provided in an embodiment of the present application includes:

[0006] Receiving a task requirement of a user, wherein the task requirement of the user carries the task data, computing resources and tolerance delay of the user;

[0007] Obtain the current operating status information of the edge computing network, and execute corresponding scheduling decisions on the user task requirements based on the operating status information based on deep reinforcement learning;

[0008] The user task requirement is issued to the corresponding edge computing node according to the scheduling decision, and the edge computing node is used to process the user task requirement.

[0009] In a second aspect, the resource allocation device provided in an embodiment of the present application includes:

[0010] A receiving unit, configured to receive a task requirement of a user, wherein the task requirement of the user includes the task data, computing resources and tolerance delay of the user;

[0011] An acquisition unit, used to acquire current operating status information of the edge computing network, and execute corresponding scheduling decisions on the user task requirements based on deep reinforcement learning according to the operating status information;

[0012] A sending unit is used to send the user task requirement to the corresponding edge computing node according to the scheduling decision, and the edge computing node is used to process the user task requirement.

[0013] In a third aspect, the processing device provided in an embodiment of the present application includes: a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute any one of the above-mentioned resource allocation methods.

[0014] In a fourth aspect, a computer-readable storage medium provided in an embodiment of the present application is used to store a computer program, and the computer program enables a computer to execute any one of the above-mentioned resource allocation methods.

[0015] In a fifth aspect, the computer program product provided by the embodiments of the present application includes computer program instructions, which enable a computer to execute any one of the resource allocation methods described above.

[0016] In the technical solution of the embodiment of the present application, by receiving the user's task requirements, wherein the user's task requirements carry the user's task data, computing resources and tolerant delay; obtaining the current operating status information of the edge computing network, and executing corresponding scheduling decisions on the user's task requirements based on deep reinforcement learning according to the operating status information; issuing the user's task requirements to the corresponding edge computing nodes according to the scheduling decisions, and the edge computing nodes are used to process the user's task requirements; in this way, the user request is assigned to the appropriate edge computing node, making full use of the computing resources of the edge computing node, shortening the task response time, improving the execution success rate of the user task, and realizing the reasonable unloading of the user task and the overall optimization configuration of resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the framework of the resource allocation system of an embodiment of the present application;

[0018] Figure 2 This is a schematic diagram of the resource allocation method provided in the embodiment of the present application. Figure 1 ;

[0019] Figure 3 This is a schematic diagram of the resource allocation method provided in the embodiment of the present application. Figure 2 ;

[0020] Figure 4 This is a schematic diagram of the resource allocation method provided in the embodiment of the present application. Figure 3 ;

[0021] Figure 5 It is a flowchart of the model training method provided in the embodiment of the present application;

[0022] Figure 6 It is a schematic diagram of the framework of the DDPG model provided in the embodiment of the present application;

[0023] Figure 7 This is a schematic diagram of the resource allocation method provided in the embodiment of the present application. Figure 4 ;

[0024] Figure 8 It is a schematic diagram of the structure of the resource allocation device provided in an embodiment of the present application;

[0025] Fig. 9 It is a schematic structural diagram of a processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] It should be noted that the terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0028] It should be noted that the terms "first", "second", etc. in this document are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present disclosure, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0029] To facilitate understanding of the technical solutions of the embodiments of the present application, the relevant technologies of the embodiments of the present application are described below. The following related technologies can be arbitrarily combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0030] With the rapid development of mobile terminal applications, future wireless networks will need to have billions of mobile devices to support ultra-low latency communications and computing. However, the resource requirements of newly developed applications are growing, the computing power and battery capacity of user devices are still limited, and a large amount of computing is required for real-time information processing, resulting in high resource consumption. To solve the problem of scarce resources on user devices, the traditional approach is to utilize the abundant computing resources of remote cloud centers. User devices perform computationally intensive tasks by offloading them to the remote cloud to reduce their workload. However, since cloud centers are often far away from users, the long delay between the cloud and users may cause lags in applications with frequent user interactions, affecting the user experience.

[0031] To overcome the limitations of cloud computing, Mobile Edge Computing (MEC) technology migrates intensive computing tasks from user devices to the edge of the network and provides users with low-latency and flexible computing and communication services. This approach avoids offloading data from the user end to the remote cloud server, thereby reducing the round-trip transmission delay of data. Operators can take advantage of economies of scale when providing edge computing services, making edge computing services more affordable for the general public. For example, an operator released a preemptible (Spot) instance to sell the idle capacity of its data center. Their dynamic pricing model is based on the user's bid. If the user's bid price is higher than the current Spot instance price, its resource request is allocated. If the current instance price is higher than the user's bid price, the request will be terminated. A major problem faced by operators is how to allocate user task requests to different edge servers so that the workload between edge servers is well balanced, thereby shortening the task response time delay and enhancing the user experience. However, considering the limited computing resources of public edge servers, especially in areas with dense traffic, there is often a situation where a large number of tasks need to be processed in a short period of time, while the adjacent private edge servers are idle or have remaining computing resources that are not utilized. Therefore, how to cooperate with private edge servers to reasonably offload tasks and allocate resources for users in the network has become an urgent problem to be solved.

[0032] In the prior art, a distributed container scheduling method based on reinforcement learning under cloud-edge collaboration is proposed. First, the overall resource utilization and task delay benefits of the cloud-edge environment are defined, and a system model is established based on the container scheduling optimization goal. Secondly, the reinforcement learning state space and action space are constructed based on the system resource idle amount and container prevention strategy, and the benefit reward function of each step of reinforcement learning is defined. Finally, the state value function is calculated, and the optimal container scheduling strategy is obtained for container scheduling strategies with different benefit returns. The present invention effectively balances the cloud-edge system resource utilization and task delay by using reinforcement learning, and can better adapt to container scheduling tasks in a cloud-edge collaborative environment.

[0033] However, there are limitations and defects in the prior art. On the one hand, the cloud-edge collaborative architecture used in the prior art solution mainly considers the computing and storage resources of the edge server and the remote cloud center, so the objects of task offloading include the cloud and edge server. However, the process of diverting the task flow to the remote cloud center may face unpredictable network fluctuations and long delays, and a single edge server can only provide limited computing power and resources. Therefore, in real scenarios, through the collaborative computing of multiple adjacent edge servers, resource sharing and load balancing can be achieved to improve resource utilization; on the other hand, it can be seen from the prior art solution that the computing resources of public edge servers are limited, especially in areas with dense traffic, there are often a large number of tasks to be processed in a short period of time, and there are adjacent private edge servers in an idle state or there are residual computing resources that are not used. Therefore, how to combine private edge servers to reasonably unload tasks and allocate resources for users in the network has become an urgent problem to be solved. To this end, the following technical solution of this application is proposed.

[0034] To facilitate understanding of the technical solutions of the embodiments of the present application, the technical solutions of the present application are described in detail below through specific embodiments. The above related technologies can be combined arbitrarily with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application. The embodiments of the present application include at least part of the following contents.

[0035] The embodiments of the present application provide a resource allocation method, apparatus, device, computer-readable storage medium, and computer program product, which can allocate user requests to suitable edge computing nodes, make full use of the computing resources of edge computing nodes, shorten task response time, improve the execution success rate of user tasks, and achieve reasonable unloading of user tasks and overall optimization of resource configuration. The following describes an exemplary application of the processing device provided by the embodiments of the present application. The device provided by the embodiments of the present application can be implemented as a server. Below, an exemplary application when the device is implemented as a server will be described.

[0036] See also Figure 1 , Figure 1 Schematic diagram of the framework of the resource allocation system provided in the embodiment of the present application, showing the resource scheduling scenario of the joint private edge computing node provided in the present application. Figure 1As shown, the mobile edge computing network consists of the following parts: a group of user devices u, where the users in the system are represented by the set u = {1, 2, ..., U}, and the users are only associated with local public edge computing nodes. A group of edge computing node clusters K, where the edge computing node clusters are composed of public edge computing nodes M and private edge computing nodes N, represented by the sets M = {1, 2, ..., M} and N = {1, 2, ..., N} respectively. Each edge computing node covers a local area and does not overlap with each other. The edge computing nodes are transmitted through wired connections, and the private computing nodes are not directly associated with external users. A task scheduling center C, which can be an SDN controller, is used to schedule user tasks to other nearby public computing nodes for calculation or pay to schedule to nearby private edge computing nodes for calculation when the user cannot offload the task to the local public edge computing node. It should be noted that the edge computing node can also be called an edge server, the public edge computing node can also be called a public edge server, and the private edge computing node can also be called a private edge server.

[0037] In some embodiments, the server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The user device may be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, car terminal, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0038] Figure 2 This is a schematic diagram of the resource allocation method provided in the embodiment of the present application. Figure 1 ,like Figure 2 As shown, the resource allocation method is applied to Figure 1 The task scheduling center in the resource allocation system shown in the figure specifically includes the following steps:

[0039] Step 201: Receive a user's task requirement, wherein the user's task requirement carries the user's task data, computing resources, and tolerable delay.

[0040] In the embodiment of the present application, the task requirements of the user are received, and the system model is constructed according to the collaborative system architecture of the joint private edge server. The user devices in the system are set to be represented by the set u = {1, 2, ..., U}; the edge computing node cluster is composed of a public edge computing node M and a private edge computing node N, which are represented by the sets M = {1, 2, ..., M} and N = {1, 2, ..., N} respectively.

[0041] In some implementations, the state of the user equipment initially accessing the local public edge computing node in the initial environment is known and is represented by a U*M dimensional matrix, as shown in the following formula (1):

[0042]

[0043] Among them, x u,m (t)∈{0,1} is a binary variable, and the value 1 indicates that in the initial environment, user u offloads the task to the local public edge computing node m. At the same time, ∑ M x u,m (t) = 1, It is used to restrict users to only offload to one public edge computing node. Each task is represented as Among them, v u Indicates task data, that is, the data size of the user task; c u Represents computing resources, i.e., computing resources required for user tasks; Indicates the tolerance delay, that is, the maximum tolerance delay of the user task.

[0044] Step 202: Obtain the current operating status information of the edge computing network, and execute corresponding scheduling decisions based on the user task requirements based on deep reinforcement learning according to the operating status information.

[0045] In an embodiment of the present application, when the local public edge computing node resources of the user are tight and it is difficult to meet the task requirements of all local users, by obtaining the current operating status information of the edge computing network, the corresponding scheduling decision is executed based on the deep reinforcement learning according to the operating status information for the user task requirements, thereby offloading the task to a suitable adjacent edge computing node for execution. Specifically, the computing resource information of the first edge computing node and the second edge computing node is obtained, and the load prediction value of the third edge computing node is predicted at the same time. The operating status information of the edge computing node is passed into the deep reinforcement learning model, wherein the deep reinforcement learning model can be an improved deep deterministic policy gradient algorithm (Deepdeterministic policy gradient, DDPG). The DDPG model makes reasonable scheduling decisions for each user's task and allocates computing resources for user tasks scheduled to be executed on the edge computing node. Each time the DDPG model outputs a scheduling strategy, the environment returns an immediate reward value, and the DDPG model is updated using the reward value, so that the action output by the deep reinforcement learning model is more accurate, and finally the computing resources of the private edge computing nodes in the system are fully utilized.

[0046] In some embodiments, the edge computing network includes: a first edge computing node, a second edge computing node, and a third edge computing node, wherein the first edge computing node is a public edge server local to the user, the second edge computing node is a public edge server near the user, and the third edge computing node is a private edge server. Here, the first edge computing node and the second edge computing node can be collectively represented as public edge computing nodes or as public edge servers. The third edge computing node can be uniformly represented as a private edge computing node or a private edge server. Public edge computing nodes The running status information of the public edge computing node includes the task queue waiting to be processed in the public edge computing node and the computing power of the public server. The operating status information can be obtained by predicting the load of private edge computing nodes.

[0047] Step 203: Send the user task requirements to the corresponding edge computing nodes according to the scheduling decision, and the edge computing nodes are used to process the user task requirements.

[0048] Here, refer to Figure 3 , Figure 3 A schematic diagram of a resource allocation method provided in an embodiment of the present application Figure 2 ,like Figure 3As shown, it represents the process of offloading user tasks to appropriate adjacent edge computing nodes according to scheduling decisions, where the adjacent edge computing nodes can be public edge servers or private servers near the user. Specifically, the user device searches for available local public edge computing nodes for binary task offloading, and the local public edge computing nodes determine whether they meet all local user task requirements based on their own computing resources.

[0049] In some embodiments, the scheduling decision includes: when the current operating status information of the first edge computing node is insufficient to execute the user's task requirements, the user's task requirements are sent to the second edge computing node or the third edge computing node based on the load prediction value of the third edge computing node.

[0050] Specifically, refer to Figure 3 If the first edge computing node, i.e., the local public edge server, meets the task requirements of all local users, the user is charged according to the unified charging standard of the public edge server and resources are allocated. The user task is unloaded to the local public edge server, which performs the calculation and transmits the calculation result back to the user terminal, and the user terminal receives the return result. If the local public edge server does not meet the task requirements of all local users, it is determined to be unloaded to the second edge computing node or the third edge computing node according to the scheduling decision, that is, it is determined to be unloaded to a nearby public edge server or a private edge server. If the user task is unloaded to a nearby public edge server according to the scheduling decision, the user is charged according to the unified charging standard of the public edge server, and the nearby public edge server performs the calculation and transmits the calculation result back to the local public edge server, which transmits it back to the user terminal, and the user terminal receives the return result.

[0051] If the local public edge server does not meet the needs of all local user tasks, the user tasks are offloaded to the private edge server according to the scheduling decision. The private edge server performs the calculation and transmits the calculation results back to the local public edge server, which then transmits them back to the user terminal. The user terminal receives the returned results. When the task is executed, the local private server will charge the user according to the unified charging standard of the public edge server according to the time the task is executed on the private server, and at the same time charge the private server Pay additional rental fees n , the rental fee is positively correlated with the computing resources allocated to the user. The more computing resources the user occupies, the higher the fee charged per unit time. The rental fee can be specifically expressed as formula (2):

[0052]

[0053] in, i∈Nm represents the public edge server at time slot t Scheduling control decision for user u task, Indicates redirecting the tasks of user u associated with the local public edge server m to other edge servers for processing. m,n The charging standard of the private edge server can be specifically expressed as formula (3):

[0054]

[0055] Among them, μ is the pricing strategy of the public edge server, f n are the remaining computing resources and total computing resources of private edge server k respectively. Assume that the task of user u is offloaded to the local public server Processing on the server The computing resources allocated to user u are That is, the CPU frequency that can be used by the task offloaded by user u, then the user u task is on the local public server The processing time for the completed processing is It can be specifically expressed as formula (4):

[0056]

[0057] In some embodiments, the public edge server The total income can be expressed as R m , which can be specifically expressed as formula (5):

[0058]

[0059] The offloading decision and resource allocation problem in the scenario of public edge servers and private edge servers can be expressed as a problem of maximizing the benefits of the public server so that the user task can be completed within a limited time, as shown in the following formula (6):

[0060]

[0061] From the above, it can be seen that the resource allocation method provided in the embodiment of the present application receives the user's task requirements, wherein the user's task requirements carry the user's task data, computing resources and tolerated delay; obtains the current operating status information of the edge computing network, and executes corresponding scheduling decisions on the user's task requirements based on deep reinforcement learning according to the operating status information; sends the user's task requirements to the corresponding edge computing node according to the scheduling decision, and the edge computing node is used to process the user's task requirements; in this way, the user request is allocated to the appropriate edge computing node, which makes full use of the computing resources of the edge computing node, shortens the task response time, improves the execution success rate of the user task, and realizes the reasonable unloading of user tasks and the overall optimization configuration of resources.

[0062] In some implementations, obtaining current operating status information of the edge computing network includes:

[0063] Obtain current operating status information of the first edge computing node, the current operating status information of the first edge computing node includes the task queue to be processed in the first edge computing node and the computing resources of the first edge computing node; obtain current operating status information of the second edge computing node, the current operating status information of the second edge computing node includes the task queue to be processed in the second edge computing node and the computing resources of the second edge computing node; predict current operating status information of the third edge computing node, the current operating status information of the third edge computing node includes the load prediction value of the third edge computing node.

[0064] Here, the first edge computing node and the second edge computing node can be collectively represented as a public server Among them, que i Indicates the task queue waiting to be processed in the public server, f i Indicates the computing capacity of the public server.

[0065] In some embodiments, predicting the current operating status information of the third edge computing node, the current operating status information of the third edge computing node includes the load prediction value of the third edge computing node, including: obtaining a first time series of the third edge computing node, the first time series is an initial load time series of the third edge computing node; converting the first time series by a generalized power transform to obtain a load target value; determining a second time series based on the load target value by a maximum likelihood method, the second time series is a predicted load time series of the third edge computing node; calculating a first function by a first criterion, determining a first parameter based on the first function, and determining the load prediction value of the third edge computing node based on the second time series and the first parameter.

[0066] Here, the load of the third edge computing node, i.e., the private edge server, is predicted to obtain a load prediction value. The private server can realize functions such as controlling the sites visited by network users, recording the site information visited by users, and collecting user traffic. Therefore, the load value of the private server can be determined based on the network behavior of all users, and the load value changes to a certain extent and presents periodicity and time correlation. Exemplarily, the load of the private edge server can be predicted by using an improved auto-regressive moving average model (ARMA). Specifically, the original private edge server load time series is transformed by the generalized power transformation method, and the maximum likelihood method is used to determine the optimal transformation form of the private server load target value, which satisfies the modeling conditions of ARMA (p, q). For the converted private server load data, the ARMA model is ordered using the improved AIC criterion, and the converted private edge server load sequence is determined to be an ARMA (p, q) sequence. The load prediction value of the private edge server can be expressed as the following formula (7):

[0067]

[0068] Among them, X t represents the load forecast value at time t, c is a constant, represents the autoregressive coefficient of the ARMA model, X t-i is the actual load data before time t, θ i (i=1,2,…,q) are the parameters of the moving average model, ε t Represents the error term. In order to reduce the prediction error, the post-order one-step prediction is adopted. The one-step predicted load value of the private edge server at time t is recorded as load, which is specifically expressed as the following formula (8):

[0069]

[0070] In some embodiments, in the parameter estimation of the ARMA model, the values ​​of the order p and q will directly affect the accuracy and prediction time of the ARMA model prediction results. Too low an order will result in too large an error and cannot meet the prediction requirements of the load balancing system. Therefore, the AIC criterion is used to estimate the optimal values ​​of the order p and q. The AIC function is as follows (9):

[0071] AIC(k,j)=ln(σ 2 (k,j))+2(k+j) / N (9)

[0072] Among them, σ 2 is the residual variance, and N is the total number of samples. The orders p and q are determined by finding the minimum value of the function AIC(k,j).

[0073] In other implementations, for the converted private server load data, the BIC criterion may be used to determine the order of the ARMA model, and determine that the converted private edge server load sequence is an ARMA (p, q) sequence.

[0074] From the above, it can be seen that the resource allocation method provided in the embodiment of the present application converts the original third edge computing node load time series through the generalized power transformation method, and uses the maximum likelihood method to determine the optimal conversion form of the load target value to meet the modeling conditions; the model is ordered according to the converted load data, and a suitable order model is determined to predict the load of the private edge server; in this way, considering the privacy characteristics of the private edge server, it is proposed to use the collected historical load data to predict the future load of the private edge server by improving the time correlation of the ARMA model, so as to solve the technical problem that the real-time load of the private edge server cannot be obtained in the prior art.

[0075] In some embodiments, after the user task requirements are sent to the corresponding task edge computing nodes according to the scheduling decision, it also includes: updating the operating status information of the edge computing network. Here, by updating the operating status information of the first edge computing node, the second edge computing node, and the third edge computing node, the scheduling strategy output by the deep reinforcement learning model is more accurate, and finally the computing resources of the private edge server in the system are fully utilized. Under the premise of ensuring the delay constraint, the execution success rate of the user task and the overall benefit of the public edge server are improved, and the reasonable unloading of the user task and the overall optimization configuration of the resources are achieved.

[0076] Reference Figure 4 , Figure 4 This is a schematic diagram of the resource allocation method provided in the embodiment of the present application. Figure 3 ,like Figure 4 As shown, updating the running status information of the edge computing network specifically includes the following steps:

[0077] Step 401: Obtain the state space and action space of the edge computing network.

[0078] Here, at the decision time t, the state of the system can be expressed as S t ={S pub ,T,C,S pri},in, que i Indicates the task queue waiting to be processed in the public server, f i Indicates the computing power of the server. Indicates the information of the computing task; D i is the data size of the task, C iThe computing resources required to perform the task. is the maximum execution delay required by the task, r i Indicates the reward obtained after the computing task is executed. load i represents the predicted value of the private server load. The action space is composed of two parts: task offloading decision and computing resource allocation decision, which can be expressed as A t =(Y,F). Where Y represents the public edge server at time slot t For the scheduling control decision of user u’s task, F is the computing resource decision allocated to the edge server.

[0079] Step 402: Input the state space and action space into the DDPG algorithm model to obtain the Q value function and strategy function of the cumulative reward value obtained by the edge computing network after executing the action space in the state space.

[0080] Here, the reward function is defined based on the optimization goal of maximizing the public server benefits by completing the user task within a limited time. The reward function can be expressed as r t (s,a t ), used to evaluate the current state S t The action A selected by the model below t For example, when a task fails, the execution time of the user's task exceeds the maximum delay that the user can tolerate. The minimum reward is set to -10. When the task can be successfully executed, the overall revenue of the public server is used as the reward value of the current decision. The greater the revenue of the public server, the higher the reward value.

[0081] For example, the improved deep deterministic policy gradient algorithm (DDPG) is used to represent the policy as a policy function μ θ (s), maps state s to a deterministic action. After completing the deterministic strategy, the Bellman equation Q π (s,a) can be expressed as the following formula (10):

[0082] Q μ (s,a)=E s′～T(·|s,a) [R(s,a,s′)+γE a'～π(·|s′) [Q μ (s′,μ(s′)))]] (10)

[0083] Step 403: Update the Q value function and the policy function to update the operating status information of the edge computing network.

[0084] Here, after obtaining the Q value according to formula (10), the optimal strategy is solved through iterative update. The optimal strategy can be expressed as the following formula (11):

[0085]

[0086] In some embodiments, the DDPG algorithm model is a trained DDPG algorithm model.

[0087] Here, refer to Figure 5 , Figure 5 is a flow chart of a model training method provided in an embodiment of the present application, wherein the method further comprises:

[0088] Step 501: Obtain an action state for training and a target Q value function and a target policy function corresponding to the action state.

[0089] Step 502: Take the action state as the input of the DDPG algorithm model, and the target Q value function and the target policy function as the output of the DDPG algorithm to train the DDPG algorithm model, so as to obtain a primary DDPG algorithm model.

[0090] Step 503: Input the updated action state into the primary DDPG algorithm model to obtain the predicted Q value function and the predicted policy function.

[0091] Step 504: Obtain a loss function using the target Q value function, the target policy function, the predicted Q value function, and the predicted policy function.

[0092] Step 505: Use the loss function to iteratively update the primary DDPG algorithm model. When the number of iterative updates reaches a preset number, a trained DDPG algorithm model is obtained.

[0093] Here, refer to Figure 6 , Figure 6 A schematic diagram of the framework of the DDPG model provided in the embodiment of the present application. As shown in the figure, before training, an Actor network θ is first constructed. μ 、Critic network θ Q And the corresponding target network θ μ′ and θ Q′ The training or updating process of the DDPG algorithm model is mainly through updating the parameters of the Actor network and the Critic network, where the Actor network updates θ by maximizing the cumulative expected return. μ′ , the Critic network updates θ by minimizing the error between the evaluation value and the target value Q′For the target network artic, the DDPG algorithm model uses a soft update method, which can also be called exponential moving average (EMA). That is, a learning rate (or momentum) is introduced to make a weighted average of the old target network parameters and the new corresponding network parameters, and then assign them to the target network. Specifically, the action a is generated based on the online policy network π and random noise. t After exploring T times, the acquired experience data is stored in the experience replay pool Memory, and then N historical experience data are sampled from Memory to calculate the gradient of the Critic and Artic networks. Update the network parameter weights. The loss function of the critic network is defined as the following formula (12), and the loss function of the artic network is defined as the following formula (13). The target network parameters are updated by soft updating, and the training process is repeated until the model converges.

[0094]

[0095] For example, refer to Figure 6 First, we collect all the task sets of user devices from the environment, the computing resources of the public edge server, and the remaining resources predicted by the private edge server. Initialize state s0 and generate action a according to the current strategy and noise. t =μ(s t ,θ μ )+n t . Perform action a t And get a return R t and the state s of the next time slice t+1 , and store (s t ,a t ,R t ,s t+1 ) to the experience playback unit Memory. Sample N state data (s t ,a t ,R t ,s t+1 ), by calculating the gradient of critic and artic networks Update the Critic and Actor network weights. After multiple rounds of iterations, the DDPG algorithm model converges to the optimal solution and outputs the offloading decision and computing resource allocation plan for the user task set.

[0096] From the above, it can be seen that a resource allocation method provided by an embodiment of the present application adopts an improved DDPG algorithm to solve the problem of task scheduling and resource allocation in a joint private edge server scenario, and the predicted load result of the improved ARMA model is passed in as a state parameter. The model makes reasonable offloading decisions for each user's task and allocates computing resources for the tasks offloaded to the edge server for execution. Each time the model outputs a strategy, the environment returns an immediate reward value, which is used to update the neural network, making the actions output by the policy network more accurate, and finally making full use of the computing resources of the private edge servers in the system. Under the premise of ensuring the delay constraint, the execution success rate of user tasks and the overall benefits of public edge servers are improved, and the reasonable offloading of user tasks and the overall optimization configuration of resources are achieved.

[0097] Embodiment 1

[0098] Figure 7 This is a schematic diagram of a resource allocation method provided in an embodiment of the present application. Figure 4 ,like Figure 7 As shown, a task scheduling and resource allocation method for a joint private edge server based on an improved DDPG algorithm is provided, and the specific steps are as follows:

[0099] Step 701, start.

[0100] Step 702: Build a system model of a joint private edge server.

[0101] Here, the system model is constructed according to the collaborative system architecture of the joint private edge server. Assume that the edge computing network consists of the following parts: a group of user devices u, where the users in the system are represented by the set u = {1,2,…,U}, and the users are only associated with local public edge computing nodes. A group of edge computing node clusters K, where the edge computing node clusters are composed of public edge computing nodes M and private edge computing nodes N, represented by the sets M = {1,2,…,M}, N = {1,2,…,N} respectively, each edge computing node covers a local area and does not overlap with each other, and the edge computing nodes are transmitted through wired connections, and the private computing nodes are not directly associated with external users. A task scheduling center C, which can be an SDN controller, is used to schedule user tasks to other nearby public computing nodes for computing or pay to schedule to nearby private edge computing nodes for computing when the user cannot offload the task to the local public edge computing node.

[0102] Step 703: The ARMA model predicts the load of the private edge server.

[0103] Here, the improved ARMA model is used to predict the load of the private edge server. The private server can realize functions such as controlling the sites visited by network users, recording the site information visited by users, and collecting user traffic. Although the network behavior of a single user in the network is uncertain, the network behavior of all users causes the load value change of the private server to show a certain periodicity and time correlation. The original private edge server load time series is transformed by the generalized power transformation method, and the maximum likelihood method is used to determine the optimal transformation form of the server load target value to meet the ARMA (p, q) modeling conditions; for the converted server load data, the BIC criterion is used to determine the order of the ARMA model, and the transformed private edge server load sequence is determined to be an ARMA (p, q) sequence. The load prediction value at time t is as shown in the above formula (7), and the one-step prediction load value of the private edge server at time t is as shown in the above formula (8).

[0104] Step 704: Task modeling and setting optimization goals.

[0105] Here, when the task is dispatched to a nearby private edge server for execution, the private server will charge the user according to the unified charging standard of the public edge server based on the time the user task is executed at the private server, and at the same time pay an additional rental fee to the private server. The rental fee is positively correlated with the computing resources allocated to the user. The more computing resources the user occupies, the higher the fee charged per unit time. The charging standard of the private server is defined as the above formula (3), and the total revenue of the public server is expressed as the above formula (5). Then, the offloading decision and resource allocation problem in the joint private server scenario can be expressed as a problem of maximizing the benefits of the public server so that the user task can be completed within a limited time, that is, the above formula (6).

[0106] Step 705: Construct deep reinforcement learning state space and action space.

[0107] Here, the load prediction value obtained in step 703 is passed in as a state parameter to construct the state space and action space of deep reinforcement learning. At the decision time t, the state of the system can be expressed as S t ={S pub ,T,C,S pri},in, que i Indicates the task queue waiting to be processed in the public server, f i Indicates the computing power of the server. Indicates the information of the computing task; D i is the data size of the task, C i The computing resources required to perform the task. is the maximum execution delay required by the task, r i Indicates the reward obtained after the computing task is executed. load i represents the predicted value of the private server load. The action space is composed of two parts: task offloading decision and computing resource allocation decision, which can be expressed as A t =(Y,F). Where Y represents the public edge server at time slot t For the scheduling control decision of user u’s task, F is the computing resource decision allocated to the edge server.

[0108] Step 706: Set a reward function with reference to the optimization objective.

[0109] Here, the reward function is defined based on the optimization goal of maximizing the public server benefits by completing the user task within a limited time. The reward function can be expressed as r t (s,a t ), used to evaluate the current state S t The action A selected by the model below t For example, when a task fails, the execution time of the user's task exceeds the maximum delay that the user can tolerate. The minimum reward is set to -10. When the task can be successfully executed, the overall revenue of the public server is used as the reward value of the current decision. The greater the revenue of the public server, the higher the reward value.

[0110] Step 707: Calculate the state value function and iterate to obtain the optimal scheduling strategy.

[0111] Here, we use the improved Deep Deterministic Policy Gradient (DDPG) algorithm to express the policy as a policy function μ θ (s), maps state s to a deterministic action. After completing the deterministic strategy, the Bellman equation Q π (s,a) can be expressed as above formula (10). After obtaining the Q value according to formula (10), the optimal strategy is solved through iterative update. The optimal strategy can be expressed as above formula (11). After multiple rounds of iterations, the DDPG algorithm model converges to obtain the optimal solution and outputs the offloading decision and computing resource allocation plan of the user task set.

[0112] Step 708, end.

[0113] In summary, this application proposes a task scheduling and resource allocation method for a joint private edge server in an edge collaborative system, and includes the private edge server in the user task offloading target by paying. Considering the two different charging methods of users offloading to public edge servers and offloading to private edge servers. On the one hand, based on the privacy characteristics of private edge servers, this application cannot obtain the real-time load of private edge servers. By using the collected historical load data, the future load of private edge servers is predicted by improving the time correlation of the ARMA model. On the other hand, the improved DDPG algorithm is used to solve the problem of task scheduling and resource allocation in the joint private edge server scenario, and the predicted load result of the improved ARMA model is passed in as a state parameter. The algorithm makes reasonable offloading decisions for each user's task and allocates computing resources for the tasks offloaded to the edge server. Each time the algorithm outputs a strategy, the environment returns an immediate reward value, which is used to update the neural network, so that the action output by the policy network is more accurate, and finally the computing resources of the private edge servers in the system are fully utilized. Under the premise of ensuring the delay constraint, the execution success rate of user tasks and the overall benefit of public edge servers are improved, and the reasonable offloading of user tasks and the overall optimization configuration of resources are achieved.

[0114] Compared with the prior art, this application considers and utilizes the scenario of idle resources on private edge servers. Since public edge servers are built and deployed by major service providers, users can directly connect to the base station to offload tasks. A private edge server refers to an edge server that is not open to external users, such as a server in a company or campus network. The computing resources of a public edge server are limited, especially in areas with dense traffic. There are often a large number of tasks to be processed in a short period of time, while adjacent private edge servers are idle or have remaining computing resources that are not utilized. By incorporating private edge servers into the goal of collaborative computing through paid leasing, the computing resources of private edge servers are fully utilized, the task response time is shortened, and the overall revenue of operators is increased. On the other hand, unlike the prior art that does not consider the use of deep deterministic policy gradient algorithms to solve scheduling problems, this application reduces the correlation between unobservable errors and predictive variables by improving the ARMA model, thereby improving the accuracy of predicted load data. At the same time, the improved DDPG algorithm is more suitable for scenarios in continuous action spaces and has higher stability. The computing offloading model proposed in this application makes full use of the computing resources of the private edge servers in the system. Under the premise of ensuring latency constraints, it improves the execution success rate of user tasks and the overall benefits of public edge servers, and realizes the reasonable offloading of user tasks and the overall optimal allocation of resources.

[0115] Figure 8Schematic diagram of the structure of the resource allocation device provided in the embodiment of the present application, which is applied to a processing device such as Figure 8 As shown, the resource allocation device includes:

[0116] The receiving unit 801 is used to receive the task requirements of the user, wherein the task requirements of the user include the task data, computing resources and tolerance delay of the user.

[0117] The acquisition unit 802 is used to obtain the current operating status information of the edge computing network, and execute corresponding scheduling decisions on the user task requirements based on deep reinforcement learning according to the operating status information.

[0118] The sending unit 803 is used to send the user task requirement to the corresponding edge computing node according to the scheduling decision, and the edge computing node is used to process the user task requirement.

[0119] In some embodiments, the edge computing network includes:

[0120] A first edge computing node, a second edge computing node and a third edge computing node, wherein the first edge computing node is a public edge server local to the user, the second edge computing node is a public edge server near the user, and the third edge computing node is a private edge server.

[0121] In some embodiments, the acquisition unit 802 is also used to obtain the current operating status information of the first edge computing node, the current operating status information of the first edge computing node includes the task queue to be processed in the first edge computing node and the computing resources of the first edge computing node; obtain the current operating status information of the second edge computing node, the current operating status information of the second edge computing node includes the task queue to be processed in the second edge computing node and the computing resources of the second edge computing node.

[0122] In some implementations, the resource allocation device further includes a prediction unit 804, which is used to predict current operating status information of the third edge computing node, where the current operating status information of the third edge computing node includes a load prediction value of the third edge computing node.

[0123] In some embodiments, the prediction unit 804 is specifically used to obtain a first time series of the third edge computing node, where the first time series is the initial load time series of the third edge computing node; convert the first time series through a generalized power transform to obtain a load target value; determine a second time series based on the load target value through a maximum likelihood method, where the second time series is a predicted load time series of the third edge computing node; calculate a first function through a first criterion, determine a first parameter based on the first function, and determine a load prediction value of the third edge computing node based on the second time series and the first parameter.

[0124] In some implementations, the scheduling decision includes:

[0125] When the current operating status information of the first edge computing node is insufficient to execute the user's task requirements, the user's task requirements are sent to the second edge computing node or the third edge computing node based on the load prediction value of the third edge computing node.

[0126] In some implementations, the resource allocation device further includes an updating unit 805, configured to update the operating status information of the edge computing network.

[0127] In some embodiments, the update unit 805 is specifically used to obtain the state space and action space of the edge computing network; input the state space and action space into the DDPG algorithm model to obtain the Q value function and strategy function of the cumulative reward value obtained by the edge computing network after executing the action space in the state space; update the Q value function and the strategy function to update the operating status information of the edge computing network.

[0128] In some embodiments, the DDPG algorithm model is a trained DDPG algorithm model; the resource allocation device also includes a training unit 806, which is used to obtain the action state used for training and the target Q value function and the target policy function corresponding to the action state; the action state is used as the input of the DDPG algorithm model, and the target Q value function and the target policy function are used as the output of the DDPG algorithm to train the DDPG algorithm model to obtain a primary DDPG algorithm model; the updated action state is input into the primary DDPG algorithm model to obtain a predicted Q value function and a predicted policy function; the loss function is obtained using the target Q value function, the target policy function, the predicted Q value function and the predicted policy function; the primary DDPG algorithm model is iteratively updated using the loss function, and when the number of iterative updates reaches a preset number, the trained DDPG algorithm model is obtained.

[0129] Those skilled in the art should understand that Figure 8 The implementation functions of each unit in the resource allocation device shown can be understood by referring to the relevant description of the aforementioned method. Figure 8 The functions of each unit in the resource allocation device shown can be implemented by a program running on a processor, or by a specific logic circuit.

[0130] Fig. 9 900 is a schematic structural diagram of a processing device 900 provided in an embodiment of the present application. The processing device may be a server, Fig. 9 The processing device 900 shown includes a processor 910, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0131] Alternatively, if Fig. 9 As shown, the processing device 900 may further include a memory 920. The processor 910 may call and run a computer program from the memory 920 to implement the method in the embodiment of the present application.

[0132] The memory 920 may be a separate device independent of the processor 910 , or may be integrated into the processor 910 .

[0133] Alternatively, if Fig. 9 As shown, the processing device 900 may further include a transceiver 930, and the processor 910 may control the transceiver 930 to communicate with other devices, specifically, may send information or data to other devices, or receive information or data sent by other devices.

[0134] The transceiver 930 may include a transmitter and a receiver. The transceiver 930 may further include an antenna, and the number of antennas may be one or more.

[0135] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and performed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0136] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0137] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0138] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0139] Optionally, the computer-readable storage medium can be applied to the network device in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the network device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0140] Optionally, the computer-readable storage medium can be applied to the mobile terminal / terminal device in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0141] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0142] Optionally, the computer program product can be applied to the network device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the network device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0143] Optionally, the computer program product can be applied to the mobile terminal / terminal device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0144] The embodiment of the present application also provides a computer program.

[0145] Optionally, the computer program can be applied to the network device in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes implemented by the network device in the various methods in the embodiments of the present application. For the sake of brevity, they are not described here.

[0146] Optionally, the computer program can be applied to the mobile terminal / terminal device in the embodiments of the present application. When the computer program is run on a computer, the computer executes the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0147] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0148] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0149] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0150] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0151] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0152] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0153] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A resource allocation method, characterized in that: The method comprises: Receiving a task requirement of a user, wherein the task requirement of the user carries the task data, computing resources and tolerance delay of the user; Obtain the current operating status information of the edge computing network, and execute corresponding scheduling decisions on the user task requirements based on the operating status information based on deep reinforcement learning; The user task requirement is issued to the corresponding edge computing node according to the scheduling decision, and the edge computing node is used to process the user task requirement.

2. The method according to claim 1, characterized in that The edge computing network includes: A first edge computing node, a second edge computing node and a third edge computing node, wherein the first edge computing node is a public edge server local to the user, the second edge computing node is a public edge server near the user, and the third edge computing node is a private edge server.

3. The method according to claim 2, characterized in that The obtaining of the current operating status information of the edge computing network includes: Acquire current running status information of the first edge computing node, where the current running status information of the first edge computing node includes a task queue to be processed in the first edge computing node and computing resources of the first edge computing node; Acquire current running status information of the second edge computing node, where the current running status information of the second edge computing node includes a task queue to be processed in the second edge computing node and computing resources of the second edge computing node; Predict the current operating status information of the third edge computing node, where the current operating status information of the third edge computing node includes a load prediction value of the third edge computing node.

4. The method according to claim 3, characterized in that The predicting the current operating state information of the third edge computing node, wherein the current operating state information of the third edge computing node includes a load prediction value of the third edge computing node, includes: Acquire a first time series of the third edge computing node, where the first time series is an initial load time series of the third edge computing node; The first time series is transformed by generalized power transformation to obtain a load target value; Determine a second time series based on the load target value by a maximum likelihood method, where the second time series is a predicted load time series of the third edge computing node; A first function is calculated by a first criterion, a first parameter is determined based on the first function, and a load prediction value of the third edge computing node is determined based on the second time series and the first parameter.

5. The method according to any one of claims 1 to 4, characterized in that The scheduling decision includes: When the current operating status information of the first edge computing node is insufficient to execute the task requirements of the user, the task requirements of the user are sent to the second edge computing node or the third edge computing node based on the load prediction value of the third edge computing node.

6. The method according to any one of claims 1 to 4, characterized in that After sending the user task requirement to the corresponding task edge computing node according to the scheduling decision, the method further includes: Update the operating status information of the edge computing network.

7. The method according to claim 6, characterized in that The updating of the operating status information of the edge computing network includes: Obtaining the state space and action space of the edge computing network; Input the state space and action space into the DDPG algorithm model to obtain the Q value function and strategy function of the cumulative reward value obtained by the edge computing network after executing the action space in the state space; The Q value function and the policy function are updated to update the operating status information of the edge computing network.

8. The method according to claim 6, characterized in that The DDPG algorithm model is a trained DDPG algorithm model; the method further includes: Acquire an action state for training and a target Q value function and a target strategy function corresponding to the action state; The DDPG algorithm model is trained by taking the action state as the input of the DDPG algorithm model, and the target Q value function and the target policy function as the output of the DDPG algorithm to obtain a primary DDPG algorithm model; Input the updated action state into the primary DDPG algorithm model to obtain a predicted Q value function and a predicted policy function; Obtaining a loss function using the target Q value function, the target policy function, the predicted Q value function, and the predicted policy function; The primary DDPG algorithm model is iteratively updated using the loss function, and when the number of iterative updates reaches a preset number, a trained DDPG algorithm model is obtained.

9. A resource allocation device, characterized in that: The device comprises: A receiving unit, configured to receive a task requirement of a user, wherein the task requirement of the user includes the task data, computing resources and tolerance delay of the user; An acquisition unit, used to acquire current operating status information of the edge computing network, and execute corresponding scheduling decisions on the user task requirements based on deep reinforcement learning according to the operating status information; A sending unit is used to send the user task requirement to the corresponding edge computing node according to the scheduling decision, and the edge computing node is used to process the user task requirement.

10. A processing device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program causes a computer to execute the method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The method comprises computer program instructions which cause a computer to execute the method as claimed in any one of claims 1 to 8.

Images