Edge computing resource allocation method, device, electronic device and storage medium

Abstract

The present invention relates to an edge computing resource allocation method, device, electronic device and storage medium, and the method includes: obtaining the data transmission volume and position of several mobile terminals to be allocated resources, as well as the remaining load capacity and edge computing resources of all edge servers within the communication range of several mobile terminals; using the data transmission volume, position of several mobile terminals and the remaining load capacity and edge computing resources of all edge servers as a state set, using a preset state transition probability, using the selected action made according to the state transition probability as an action set, using the total revenue of several mobile terminals as a reward function, and using the influence of several mobile terminals on the reward as a discount factor to construct a Markov model; and performing deep reinforcement learning model training on the Markov model to determine the optimal resource allocation strategy. The present invention uses a deep reinforcement model to make multiple rounds of decisions to obtain the optimal resource allocation method and maximize the revenue of mobile terminals.

Description

Technical Field

[0001] The present invention relates to the field of edge computing resource allocation technology, and in particular to an edge computing resource allocation method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of computer technology, the number of mobile users is growing rapidly. Although the CPU processing power of mobile terminals is constantly improving, it is still difficult to complete a large number of tasks in a short period of time. Edge computing servers can provide efficient computing services for mobile terminals, which requires edge computing devices to efficiently allocate resources.

[0003] The existing methods for allocating computing resources have the following technical problems: 1. In actual scenarios, users are usually on the move, so their terminals may be covered by different edge servers at different times. Users have multiple options for allocation and will weigh price and processing time. 2. Under normal circumstances, edge servers set corresponding prices for computing resources, and users need to pay a certain fee based on their own computing needs before they can use the resources of the edge server. Therefore, the price of resources will also affect the allocation of mobile users. 3. Users' computing needs will continue to arise, so in each time period, users need to decide whether computing tasks should be executed locally on the mobile terminal or allocated to the edge server for processing.

[0004] Therefore, there is an urgent need to propose an edge computing resource allocation method, device, electronic device and storage medium to solve the technical problem of unreasonable edge computing resource allocation, thereby maximizing the long-term benefits of mobile terminals. Summary of the Invention

[0005] In view of this, it is necessary to provide an edge computing resource allocation method, device, electronic device and storage medium to solve the technical problem of unreasonable edge computing resource allocation in the prior art.

[0006] In one aspect, the present invention provides an edge computing resource allocation method, comprising:

[0007] Obtaining the data transmission volume and location of several mobile terminals to be allocated resources, as well as the remaining load capacity and edge computing resources of all edge servers within the communication range of the several mobile terminals;

[0008] Constructing a Markov model using the data transmission volume and location of the multiple mobile terminals, the remaining load capacity of all edge servers, and the edge computing resources as a state set, a preset state transition probability, selected actions made according to the state transition probability as an action set, the total benefits of the multiple mobile terminals as a reward function, and the impact of the multiple mobile terminals on the reward as a discount factor;

[0009] A deep reinforcement learning model is trained on the Markov model to determine the optimal resource allocation strategy.

[0010] In some possible implementations, determining the total revenue of the plurality of mobile terminals includes:

[0011] Determine the local consumption cost of the computing task in each of the mobile terminals, and determine the edge consumption cost of the computing task in each of the mobile terminals distributed to all edge servers;

[0012] determining a unit benefit of each of the mobile terminals according to the local consumption cost and the edge consumption cost;

[0013] The current uninstallation status of each of the plurality of mobile terminals is obtained, and the total revenue of the plurality of mobile terminals is determined according to the uninstallation status and the single revenue of the mobile terminal.

[0014] In some possible implementations, determining the edge consumption cost of allocating computing tasks in each mobile terminal to all edge servers includes:

[0015] Determine the edge consumption time of each computing task in the mobile terminal on all edge servers;

[0016] Obtain resource prices of edge servers, and determine edge processing costs of computing tasks in each mobile terminal on all edge servers based on each edge consumption time and the resource prices;

[0017] Determining a transmission cost of transmitting a computing task in each of the mobile terminals to all edge servers;

[0018] Each edge consumption cost is determined according to the transmission consumption cost and the edge processing cost.

[0019] Some possible implementations also include:

[0020] determining a total reward for the plurality of mobile terminals based on the reward function and the discount factor;

[0021] Determine the optimal allocation strategy based on the reward of the current state and the reward of performing the action in the next state.

[0022] In some possible implementations, deep reinforcement learning model training is performed on the Markov model to determine the optimal resource allocation strategy, including:

[0023] The first step is to initialize the DQN network;

[0024] Step 2: Determine the unloading action based on the greedy strategy;

[0025] Step 3: Get the reward and next state according to the uninstall action;

[0026] Step 4: Based on the current state, the current unloading action, the next state, and the reward, a state transition quadruple is formed, the state transition quadruple is stored in the experience pool, and previous experience is randomly drawn from the experience pool for learning to obtain the target value;

[0027] Step 5: Determine the loss function based on the target value;

[0028] Step 6: Update the network parameters according to the loss function;

[0029] Step 7: Determine whether the loss function converges. If so, stop training and obtain the optimal resource allocation strategy; if not, continue from step 2.

[0030] In some possible implementations, the allocation action is determined according to a greedy strategy, including:

[0031] Generate a random number according to the greedy strategy, wherein the random number ranges from 0 to 1;

[0032] If the random number is less than ε, the current state is input into the neural network to obtain the unloading action; if the random number is greater than ε, an unloading action is randomly selected from the action set;

[0033] Here, ε is a positive number less than 1.

[0034] In some possible implementations, updating network parameters according to the loss function includes:

[0035] Determining the offloading policy gradient by differentiation according to the loss function;

[0036] According to the offloading policy gradient, optimal neural network parameters are determined and updated.

[0037] On the other hand, the present invention also provides an edge computing resource allocation device, comprising:

[0038] an acquisition unit, configured to acquire the data transmission volume and location of a plurality of mobile terminals to be allocated resources, and the remaining load capacity and edge computing resources of all edge servers within the communication range of the plurality of mobile terminals;

[0039] a processing unit, configured to construct a Markov model using the data transmission volume and location of the plurality of mobile terminals, the remaining load capacity of all the edge servers, and the edge computing resources as a state set, a preset state transition probability, selected actions made according to the state transition probability as an action set, a total benefit of the plurality of mobile terminals as a reward function, and an influence of the plurality of mobile terminals on the reward as a discount factor;

[0040] The allocation unit is used to perform deep reinforcement learning model training on the Markov model to determine the optimal resource allocation strategy.

[0041] On the other hand, the present invention also provides an electronic device, including a memory and a processor, wherein:

[0042] The memory is used to store programs;

[0043] The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the edge computing resource allocation method described in any one of the above implementations.

[0044] On the other hand, the present invention also provides a computer-readable storage medium for storing computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the edge computing resource allocation method described in any of the above implementations.

[0045] Compared with the prior art, the beneficial effects of the present invention are: first, the data transmission volume and location of several mobile terminals to be allocated resources, as well as the remaining load capacity and edge computing resources of all edge servers within the communication range of several mobile terminals are obtained; then, the data transmission volume and location of several mobile terminals and the remaining load capacity and edge computing resources of all edge servers are used as the state set, the preset state transition probability, the selected action made according to the state transition probability is used as the action set, the total revenue of several mobile terminals is used as the reward function, and the influence of several mobile terminals on the reward is used as the discount factor to construct a Markov model; finally, the Markov model is trained by a deep reinforcement learning model to determine the optimal resource allocation strategy.

[0046] In summary, the present invention comprehensively considers the movement of mobile terminals and allocates resources under multiple rounds of decision-making, and determines the optimal edge computing resource allocation method based on the deep reinforcement learning model, thereby maximizing the long-term benefits of mobile terminals. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A schematic diagram of a flow chart of an embodiment of the edge computing resource allocation method provided by the present invention;

[0049] Figure 2 The present invention provides Figure 1 A flow chart of an embodiment of step S102;

[0050] Figure 3 The present invention provides Figure 2 Flow chart of step S201 in an embodiment;

[0051] Figure 4 The present invention provides Figure 1 A flow chart of another embodiment of step S102;

[0052] Figure 5 The present invention provides Figure 1 A flow chart of an embodiment of step S103;

[0053] Figure 6 This is an algorithm flow chart of an embodiment of the deep learning reinforcement model provided by the present invention;

[0054] Figure 7 The present invention provides Figure 5 A flow chart of an embodiment of step S506;

[0055] Figure 8 A schematic diagram of the structure of an embodiment of the edge computing resource allocation device provided by the present invention;

[0056] Figure 9 A schematic diagram of the structure of an embodiment of an edge computing resource allocation electronic device provided by the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0058] It should be understood that the schematic drawings are not drawn to scale. The flowcharts used in the present invention illustrate operations implemented according to some embodiments of the present invention. It should be understood that the operations in the flowcharts may be implemented out of sequence, and steps that do not have a logical contextual relationship may be reversed or performed simultaneously. In addition, those skilled in the art, guided by the present disclosure, may add one or more additional operations to the flowcharts or remove one or more operations from the flowcharts.

[0059] Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.

[0060] The embodiments of the present invention provide an edge computing resource allocation method, device, electronic device, and storage medium, which are described below.

[0061] The embodiment of the present invention provides an edge computing resource allocation method, combining Figure 1 Come and see, Figure 1 This is a flow chart of an embodiment of the edge computing resource allocation method provided by the present invention, including steps S101 to S104, wherein:

[0062] S101. Obtain the data transmission volume and location of several mobile terminals to be allocated resources, as well as the remaining load capacity and edge computing resources of all edge servers within the communication range of the several mobile terminals;

[0063] S102: Construct a Markov model using the data transmission volume and location of several mobile terminals, the remaining load capacity of all edge servers, and the edge computing resources as a state set, a preset state transition probability, selected actions based on the state transition probability as an action set, the total revenue of the several mobile terminals as a reward function, and the impact of the several mobile terminals on the reward as a discount factor;

[0064] S103. Conduct deep reinforcement learning model training on the Markov model to determine the optimal resource allocation strategy.

[0065] Compared with the prior art, the beneficial effects of the present invention are: first, the data transmission volume and location of several mobile terminals to be allocated resources, as well as the remaining load capacity and edge computing resources of all edge servers within the communication range of several mobile terminals are obtained; then, the data transmission volume and location of several mobile terminals and the remaining load capacity and edge computing resources of all edge servers are used as the state set, the preset state transition probability, the selected action made according to the state transition probability is used as the action set, the total revenue of several mobile terminals is used as the reward function, and the influence of several mobile terminals on the reward is used as the discount factor to construct a Markov model; finally, the Markov model is trained by a deep reinforcement learning model to determine the optimal resource allocation strategy, thereby maximizing the long-term revenue of the mobile terminals.

[0066] It's important to note that the long-term benefits of mobile terminals primarily depend on the rational allocation of edge computing resources. Therefore, by constructing a Markov model and performing multiple rounds of decision-making through deep reinforcement learning, we can make edge computing resource allocation more rational. Edge server resource allocation is generally related to the data transmission volume and location of the mobile terminal, as well as the remaining load capacity and computing resources of the edge server. These factors are used as the state set of the Markov model.

[0067] As a preferred embodiment, Figure 2 Come and see, Figure 2 The present invention provides Figure 1 Flowchart of step S102 in an embodiment, step S102 includes steps S201 to S203, wherein:

[0068] S201, determining the local consumption cost of a computing task in each mobile terminal, and determining the edge consumption cost of allocating the computing task in each mobile terminal to all edge servers;

[0069] S202: Determine the unit revenue of each mobile terminal based on the local consumption cost and the edge consumption cost;

[0070] S203: Obtain the current uninstallation status of each of the plurality of mobile terminals, and determine the total revenue of the plurality of mobile terminals according to the uninstallation status and the single revenue of the mobile terminal.

[0071] In the embodiment of the present invention, the total benefit of the mobile terminal is calculated based on the energy consumption costs of the local server and the edge server.

[0072] In a specific embodiment of the present invention, assuming that there are N edge servers and M mobile users, for general considerations, the present invention uses two-dimensional Euclidean coordinates to describe the locations of edge servers and mobile users, that is, the coordinates of the edge servers are represented as L n =(X n, Y n ), users are usually in a mobile state, and the trajectory of mobile user m is represented by l m (t)=(x m (t), y m (t)), binary OT m (t)=(d m (t), f m (t)), represents the computing demand of mobile user m in the tth time period, using Represents the uninstall decision of mobile users.

[0073] In the tth time period, the mobile terminal m will calculate the task OT m The benefit of (t) when it is assigned to edge server n for processing is:

[0074]

[0075] The total revenue of M mobile terminals is:

[0076]

[0077] Among them, U u (t) represents the total revenue of all mobile terminals, Indicates that mobile terminal m will calculate task OT m (t) The revenue when allocated to edge server n for processing, represents the local consumption cost of the computing task m in the mobile terminal, It represents the edge consumption cost of allocating computing tasks in mobile terminal m to edge server n.

[0078] As a preferred embodiment, Figure 3 Come and see, Figure 3 The present invention provides Figure 2 Flowchart of step S201 in an embodiment, step S201 includes steps S301 to S304, wherein:

[0079] S301, determining the edge consumption time of computing tasks in each mobile terminal on all edge servers;

[0080] S302: Obtain resource prices of edge servers, and determine the edge processing cost of computing tasks in each mobile terminal on all edge servers based on the time consumed and resource prices of each edge server;

[0081] S303: Determine the transmission cost of transmitting the computing task in each mobile terminal to all edge servers;

[0082] S304: Determine each edge consumption cost according to the transmission consumption cost and the edge processing cost.

[0083] In the embodiment of the present invention, the edge consumption cost is determined by the transmission consumption cost and the edge processing cost.

[0084] In a specific embodiment of the present invention, the mobile terminal m calculates the task OT m The time consumed by (t) allocated to edge server n is:

[0085]

[0086] in represents the time it takes for mobile terminal m to transmit the computing task to edge server n, represents the processing time of mobile terminal m at edge server n, d m (t) the data transmission volume of mobile terminal m, represents the transmission rate when the mobile terminal m transmits the data required for the task to the edge server n, f m (t) represents the computing demand of mobile terminal m in the tth time period, Indicates the computing resources allocated by the edge server to each task.

[0087] In time period t, mobile terminal m will calculate task OT m The transmission energy consumption (t) allocated to the edge server n is:

[0088]

[0089] Among them, P d is the data transmission power of the mobile terminal.

[0090] Computational Task OT m The cost of assigning (t) to edge server n for processing is:

[0091]

[0092] It should be noted that, in this embodiment, the distance from the mobile terminal m to the edge server n in the tth time period can be obtained by using the location of the edge server and the location of the mobile terminal:

[0093]

[0094] use represents the channel power gain between edge server n and mobile terminal m:

[0095]

[0096] Here, h0 refers to the received power when the reference distance d0 is 1m.

[0097] The transmission rate when the mobile terminal m transmits the data required for the task to the edge server n in the tth time period can be calculated as:

[0098]

[0099] Where B represents the transmission channel bandwidth between the mobile device and the edge server, σ 2 Represents the noise power between the mobile device and the edge server.

[0100] As a preferred embodiment, Figure 2 Come and see, Figure 2 The present invention provides Figure 1 Flow chart of another embodiment of step S102, step S102 includes steps S401 to S402, wherein:

[0101] S401, determining a total reward for a plurality of mobile terminals based on a reward function and a discount factor;

[0102] S402: Determine the optimal allocation strategy based on the reward of the current state and the reward of the action executed in the next state.

[0103] In an embodiment of the present invention, in order to interact with the mobile edge computing environment by selecting task offloading actions that maximize the long-term benefits of mobile terminals, a Marko model is constructed, an optimal action function is defined, and an optimal allocation strategy is determined.

[0104] In a specific embodiment of the present invention, the discount factor γ will affect the reward r of each subsequent time period. t Discounts are made, so by the tth time period, the total reward obtained by the mobile terminal is:

[0105] R t =r t +γr t+1 +γ 2 r t+2 +…+γ T-t r T

[0106] This example defines the optimal action-value function as:

[0107] Q * (s, a) = maxE[R t |s t =s,a t =a,π]

[0108] Its input includes the state s obtained by the mobile user observing the environment, the action a taken by the mobile user according to the strategy π, and the strategy π itself, where π is the mobile terminal task offloading strategy finally obtained in this example. The optimal value under all possible actions á is When , the optimal unloading strategy is to select action á to maximize the expected value When, that is:

[0109]

[0110] It should be noted that the Markov decision process is represented by a quintuple, namely (S, A, P, r, γ). In this example, the specific setting of the Markov decision process quintuple is as follows:

[0111] State set S: In the tth time period, the mobile user obtains the corresponding state by observing the edge computing environment, that is, s t ={rc t , d t , f t ,(x t ,y t )},in,

[0112] is a vector of length N, It represents the remaining load capacity of edge server n in the tth time period.

[0113] is a vector of length M, It means that in the tth time period, task OT m (t) The size of data that needs to be transferred when offloading to the edge server for processing.

[0114] is a vector of length M, It means that in the tth time period, the task OT is processed m (t) Required computing resources.

[0115] (x t ,y t )=((x1(t),y1(t)),(x2(t),y2(t)),…,(x M (t), y M (t))) is also a vector of length M, (x m (t), y m (t)) represents the coordinates of mobile user m in the tth time period.

[0116] Action set A: In the tth time period, the mobile user observes the state s t After that, the corresponding action will be selected and executed according to the strategy.

[0117] State transition probability: It means that in the current state s t Next, the mobile user performs action a t Then, transfer to the next state s t+1 probability.

[0118] Reward function: r t =r t (s t , a t ), in state s t Next, the mobile user takes action a t Afterwards, the environment will give corresponding feedback and return a reward r to the mobile user. t , and enter the next state s t+1 The purpose of this example is to maximize the long-term benefits of the consumer, so the reward value is directly set to the total benefit of the consumer, that is, r t =U u (t).

[0119] Discount factor: γ∈(0, 1], because the purpose of this example is to maximize the long-term benefits of mobile users, the discount factor is mainly used to represent the impact of subsequent mobile users' rewards on the current reward.

[0120] As a preferred embodiment, Figure 5 For example, Figure 5 The present invention provides Figure 1 Flowchart of step S103 in an embodiment, step S103 includes steps S501 to S507, wherein:

[0121] S501, initialize the DQN (Deep Q-Network) network;

[0122] S502, determining an unloading action according to a greedy strategy (∈-greedy);

[0123] S503. Obtain reward and next state according to the uninstall action;

[0124] S504: Based on the current state, the current unloading action, the next state, and the reward, a state transition quadruple is formed, the state transition quadruple is stored in the experience pool, and previous experience is randomly sampled from the experience pool for learning to obtain the target value;

[0125] S505. Determine a loss function based on the target value;

[0126] S506. Update network parameters according to the loss function;

[0127] S507 , determine whether the loss function converges. If so, stop training and obtain the optimal resource allocation strategy; if not, continue execution from S502 .

[0128] In the embodiment of the present invention, based on the DQN algorithm, training is repeated continuously to ultimately determine the optimal allocation strategy.

[0129] In a specific embodiment of the present invention, Figure 6 Come and see, Figure 6 This is an algorithm flow chart of an embodiment of the deep learning reinforcement model provided by the present invention. First, the experience pool D is initialized, the action value function Q with parameter θ is initialized, and the action value function Q with parameter θ is initialized. - = θ target action-value function

[0130] Then, the mobile user observes the environment and obtains the state s t And according to the greedy strategy, get the allocation action and execute the allocation action a t And observe the environment and get reward r t and the next state s t+1 , transfer the state tuple (s t , a t , r t , s t+1 ) is stored in the experience pool D. A set of samples (s k , a k , r k , s k+1 ) to train and get the target value:

[0131]

[0132] We can get a loss function that changes in each iteration i Right now:

[0133]

[0134] By repeating learning and training continuously, the loss function eventually converges and the final computing task allocation strategy is obtained.

[0135] As a preferred embodiment, the greedy strategy is as follows: generate a random number between 0 and 1, and if the random number is less than ε, then s t Input into the neural network to get the unloading action a t =argmaxQ(s,á;θ); If the random number is greater than ε, then randomly select an unloading action a from the action set A t Execute at After the action, the environment will give feedback and return the reward r t And enter the next state s t+1 , where ε is a positive number less than 1.

[0136] As a preferred embodiment, Figure 7 Come and see, Figure 7 The present invention provides Figure 5 Flowchart of step S506 in an embodiment, step S506 includes steps S701 to S702, wherein:

[0137] S701. Determine the offloading policy gradient by differentiation according to the loss function;

[0138] S702: Determine the optimal neural network parameters according to the offloading policy gradient, and update the neural network parameters.

[0139] In an embodiment of the present invention, new neural network parameters are obtained by performing gradient descent on the loss function, and training is continued based on the new network parameters to reduce the mean square error in the Bellman equation.

[0140] In a specific embodiment of the present invention, the gradient of the mobile terminal's uninstallation policy can be obtained by performing gradient descent on the loss function according to the neural network parameter θ:

[0141]

[0142] Update the neural network parameters θ and use a separate neural network to update the target value y during learning updates k , and then update the target network every ζ generation, copying the parameters of the Q network to the target network Right now And use it in the next ζ iterations Generate target value y k , set s t =s t+1 , which means entering the next state.

[0143] In order to better implement the edge computing resource allocation method in the embodiment of the present invention, based on the edge computing resource allocation method, the embodiment of the present invention also provides an edge computing resource allocation device, such as Figure 8 As shown, the edge computing resource allocation device 800 includes:

[0144] An acquisition unit 801 is configured to acquire data transmission volume and locations of a plurality of mobile terminals to be allocated resources, as well as remaining load capacity and edge computing resources of all edge servers within the communication range of the plurality of mobile terminals;

[0145] Processing unit 802 is configured to construct a Markov model using the data transmission volume and location of the plurality of mobile terminals, the remaining load capacity of all edge servers, and the edge computing resources as a state set, a preset state transition probability, selected actions made based on the state transition probability as an action set, the total benefits of the plurality of mobile terminals as a reward function, and the impact of the plurality of mobile terminals on the reward as a discount factor;

[0146] The allocation unit 803 is configured to perform deep reinforcement learning model training on the Markov model to determine an optimal resource allocation strategy.

[0147] The edge computing resource allocation device 800 provided in the above embodiment can implement the technical solution described in the above edge computing allocation method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above edge computing resource allocation method embodiment, which will not be repeated here.

[0148] like Figure 9 As shown, the present invention also provides an electronic device 900. The electronic device 900 includes a processor 901, a memory 902 and a display 903. Figure 9 Only some of the components of the electronic device 900 are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0149] In some embodiments, the processor 901 can be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run the program code stored in the memory 902 or process data, such as the edge computing resource allocation method in the present invention.

[0150] In some embodiments, processor 901 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, processor 901 may be local or remote. In some embodiments, processor 901 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an on-premises cloud, a multi-cloud, or any combination thereof.

[0151] In some embodiments, the memory 902 may be an internal storage unit of the electronic device 900, such as a hard disk or memory of the electronic device 900. In other embodiments, the memory 902 may also be an external storage device of the electronic device 900, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 900.

[0152] Furthermore, the memory 902 may include both an internal storage unit of the electronic device 900 and an external storage device. The memory 902 is used to store application software installed in the electronic device 900 and various data.

[0153] In some embodiments, the display 903 can be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 903 is used to display information on the electronic device 900 and to display a visual user interface. Components 901-903 of the electronic device 900 communicate with each other via a system bus.

[0154] In one embodiment, when the processor 901 executes the edge computing resource allocation program in the memory 902, the following steps may be implemented:

[0155] Obtaining the data transmission volume and location of several mobile terminals to be allocated resources, as well as the remaining load capacity and edge computing resources of all edge servers within the communication range of the several mobile terminals;

[0156] A Markov model is constructed using the data transmission volume and location of several mobile terminals, the remaining load capacity of all edge servers, and the edge computing resources as the state set, the preset state transition probability, the selected actions made according to the state transition probability as the action set, the total benefits of several mobile terminals as the reward function, and the impact of several mobile terminals on the reward as the discount factor.

[0157] A deep reinforcement learning model is trained on the Markov model to determine the optimal resource allocation strategy.

[0158] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 900 mentioned. The electronic device 900 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, or a laptop computer. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with iOS, Android, Microsoft or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 900 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0159] Accordingly, an embodiment of the present application also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by the processor, it can implement the steps or functions in the edge computing resource allocation method provided by the above-mentioned method embodiments.

[0160] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0161] The above is a detailed introduction to the edge computing resource allocation method, device, electronic device and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for allocating edge computing resources, characterized in that: include: Obtaining the data transmission volume and location of a number of mobile terminals to be allocated resources, as well as the remaining load capacity and edge computing resources of all edge servers within the communication range of the number of mobile terminals; The data transmission volume and location of the plurality of mobile terminals and the remaining load capacity and edge computing resources of all the edge servers are used as a state set, a preset state transition probability, a selected action made according to the preset state transition probability is used as an action set, the total income of the plurality of mobile terminals is used as a reward function, and the influence of the plurality of mobile terminals on the reward is used as a discount factor to construct a Markov model; Performing deep reinforcement learning model training on the Markov model to determine the optimal resource allocation strategy; Determining the total revenue of the plurality of mobile terminals includes: Determine the local consumption cost of the computing task in each of the mobile terminals, and determine the edge consumption cost of the computing task in each of the mobile terminals distributed to all edge servers; Determining a unit benefit of each of the mobile terminals according to the local consumption cost and the edge consumption cost; Acquire a current uninstallation state of each of the plurality of mobile terminals, and determine a total revenue of the plurality of mobile terminals according to the uninstallation state and a single revenue of the mobile terminal; Determining the edge consumption cost of allocating the computing tasks in each of the mobile terminals to all edge servers includes: Determine the edge consumption time of each computing task in the mobile terminal on all edge servers; Obtain resource prices of edge servers, and determine edge processing costs of computing tasks in each of the mobile terminals on all edge servers according to each edge consumption time and the resource prices; Determining the transmission consumption cost of transmitting the computing task in each of the mobile terminals to all edge servers; Determining each edge consumption cost according to the transmission consumption cost and the edge processing cost; Also includes: Determining a total reward for the plurality of mobile terminals according to the reward function and the discount factor; Determine the optimal allocation strategy based on the reward of the current state and the reward of executing the action in the next state.

2. The edge computing resource allocation method according to claim 1, characterized in that: The Markov model is trained by a deep reinforcement learning model to determine the optimal resource allocation strategy, including: The first step is to initialize the DQN network; Step 2: Determine the unloading action based on the greedy strategy; Step 3: Get the reward and next state according to the uninstall action; Step 4: According to the current state, the current unloading action, the next state and the reward, a state transition quadruple is formed, the state transition quadruple is stored in the experience pool, and the previous experience is randomly extracted from the experience pool for learning to obtain the target value; Step 5: Determine the loss function according to the target value; Step 6: Update network parameters according to the loss function; Step 7: Determine whether the loss function converges. If so, stop training and obtain the optimal resource allocation strategy; if not, continue from step 2.

3. The edge computing resource allocation method according to claim 2, characterized in that: According to the greedy strategy, the uninstallation action is determined, including: Generate a random number according to the greedy strategy, wherein the random number ranges from 0 to 1; If the random number is less than , input the current state into the neural network to obtain the unloading action; if the random number is greater than , randomly select an unloading action from the action set; in, is a positive number less than 1.

4. The edge computing resource allocation according to claim 2, characterized in that: According to the loss function, the network parameters are updated, including: According to the loss function, determining the offloading policy gradient by differentiation; According to the offloading policy gradient, optimal neural network parameters are determined, and the neural network parameters are updated.

5. An edge computing resource allocation device, characterized in that: include: An acquisition unit, used to acquire the data transmission volume and location of a number of mobile terminals to be allocated resources, and the remaining load capacity and edge computing resources of all edge servers within the communication range of the number of mobile terminals; A processing unit, configured to construct a Markov model using the data transmission volume and location of the plurality of mobile terminals and the remaining load capacity and edge computing resources of all the edge servers as a state set, a preset state transition probability, a selected action made according to the state transition probability as an action set, a total income of the plurality of mobile terminals as a reward function, and an influence of the plurality of mobile terminals on the reward as a discount factor; An allocation unit, used for performing deep reinforcement learning model training on the Markov model to determine an optimal resource allocation strategy; Determining the total revenue of the plurality of mobile terminals includes: Determine the local consumption cost of the computing task in each of the mobile terminals, and determine the edge consumption cost of the computing task in each of the mobile terminals distributed to all edge servers; Determining a unit benefit of each of the mobile terminals according to the local consumption cost and the edge consumption cost; Acquire a current uninstallation state of each of the plurality of mobile terminals, and determine a total revenue of the plurality of mobile terminals according to the uninstallation state and a single revenue of the mobile terminal; Determining the edge consumption cost of allocating the computing tasks in each of the mobile terminals to all edge servers includes: Determine the edge consumption time of each computing task in the mobile terminal on all edge servers; Obtain resource prices of edge servers, and determine edge processing costs of computing tasks in each of the mobile terminals on all edge servers according to each edge consumption time and the resource prices; Determining the transmission consumption cost of transmitting the computing task in each of the mobile terminals to all edge servers; Determining each edge consumption cost according to the transmission consumption cost and the edge processing cost; Also includes: Determining a total reward for the plurality of mobile terminals according to the reward function and the discount factor; Determine the optimal allocation strategy based on the reward of the current state and the reward of executing the action in the next state.

6. An electronic device, characterized in that: comprising a processor and a memory, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the edge computing resource allocation method described in any one of claims 1 to 4 above.

7. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the edge computing resource allocation method described in any one of claims 1 to 4 above.

Images