Method and system for mobile user task offloading and resource pricing in edge computing

Abstract

The application discloses a mobile user task unloading and resource pricing method and system in edge computing, constructs the problem of how the edge server sets the price of resources and how the mobile user unloads tasks to maximize their own long-term benefits into a Markov decision process, and then designs the edge server resource pricing strategy and the mobile user task unloading strategy under dynamic pricing based on the DDPG algorithm and the DQN algorithm. When the edge server is not in the full load state, it will receive the task request, and then the mobile user will send the data required for processing the task to the edge server. After the edge server completes the computing task, the computing result will be immediately returned to the mobile user. The application can take into account the benefits of both the mobile user and the edge server, and can maximize the long-term benefits of both as much as possible, improve the user experience, and also make the edge server obtain more profits.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of edge computing, and relates to a mobile user task offloading and resource pricing method and system in edge computing, in particular to a mobile user task offloading and resource pricing method and system in edge computing under dynamic pricing. BACKGROUND

[0002] Mobile edge computing sinks the function of a core network to the edge of a mobile access network to shorten the transmission time of a computing task and reduce the energy consumption of data transmission. Mobile edge computing can provide more efficient computing services for users, thereby improving the experience of the users. For computing offloading in mobile edge computing, offloading and pricing are both crucial. In an actual scenario, a user is usually in a mobile state, so the user can be covered by different edge servers at different time lengths. The user has multiple choices when offloading, and needs to decide whether to execute a computing task locally on a mobile device or offload the computing task to an edge server for processing. An edge server sets corresponding prices for computing resources, and the user needs to pay a certain fee to use the resources of the edge server according to the computing demand of the user, so the prices of the resources also affect the offloading of the mobile user. The pricing of resources by the edge server and the offloading of tasks by the user affect each other.

[0003] Typical resource pricing and task offloading usually follow the following process: an edge server sets corresponding prices for resources, and a user needs to pay a certain fee to use the resources of the edge server according to the computing demand of the user. If a mobile user chooses to offload a computing task to an edge server for processing, the mobile device initiates a request to the corresponding edge server. If the edge server is in a full load state, the edge server does not receive the task request, and the task is executed locally on the mobile device. If the edge server is not in a full load state, the edge server receives the task request, and the mobile device sends data required for processing the task to the edge server. The edge server processes the task immediately after receiving the task, and returns the computing result to the mobile device after the task is completed. However, the related research on task offloading and pricing in the mobile edge computing environment nowadays does not consider the movement of the user, that is, the user can be covered by different edge servers as the time length changes. In addition, most researches only consider single-round offloading and pricing. In fact, the computing demand of the mobile user can be continuously generated, so the user needs to make multiple rounds of offloading decisions to maximize the long-term benefit of the user. Corresponding to the multiple rounds of offloading of the mobile user, the edge server needs to make multiple rounds of pricing to maximize the long-term benefit of the edge server. SUMMARY

[0004] In order to solve the above technical problems, the application provides a method and system for mobile user task offloading and resource pricing in edge computing, which can maximize the long-term benefits of edge servers and mobile users.

[0005] The technical scheme adopted by the method of the application is as follows: a method for mobile user task offloading and resource pricing in edge computing, comprising the following steps: step 1: initializing the load of edge servers and the price of computing resources; step 2: a mobile user formulates an offloading strategy under the current load of edge servers and the price of computing resources; step 3: judging whether the mobile user chooses to execute locally or on a certain edge server according to the offloading strategy obtained in step 2; if choosing to execute locally, sequentially executing step 4; if choosing to execute on a certain edge server, executing step 5; step 4: the mobile user executes the computing task locally, and then executing step 9; step 5: judging whether the selected edge server in the offloading strategy obtained in step 2 is full; if full, executing step 4 again; if not full, sequentially executing step 6; step 6: the selected edge server executes the computing task; step 7: the selected edge server returns the computing result to the mobile user; step 8: the edge server dynamically adjusts the price of computing resources according to the current known information; step 9: judging whether the mobile user has computing demand; if yes, executing step 2 again; if not, sequentially executing step 10; step 10: the mobile user completes all computing tasks.

[0006] The technical scheme adopted by the system of the application is: a mobile user task offloading and resource pricing system in edge computing, comprising the following modules: a first module for collecting real-time orders submitted by passengers and related information of online car-hailing drivers; a second module for obtaining the value of a vehicle in different space-time states based on historical order data; specifically comprising the following sub-modules: a first module for initializing the load of an edge server and the price of computing resources; a second module for a mobile user to develop an offloading strategy under the current load of an edge server and the price of computing resources; a third module for determining whether the mobile user selects to execute locally or on a certain edge server according to the offloading strategy obtained in step 2; if selecting to execute locally, sequentially executing step 4; if selecting to execute on a certain edge server, executing step 5; a fourth module for a mobile user to execute a computing task locally, and then executing step 9; a fifth module for determining whether the selected edge server in the offloading strategy obtained in step 2 is full; if full, executing step 4 again; if not full, sequentially executing step 6; a sixth module for a selected edge server to execute a computing task; a seventh module for a selected edge server to return the computing result to a mobile user; an eighth module for an edge server to dynamically adjust the price of computing resources according to currently known information; a ninth module for determining whether a mobile user has computing requirements; if yes, executing step 2 again; if no, sequentially executing step 10; and a tenth module for a mobile user to complete all computing tasks.

[0007] The application models the problem of how an edge server develops a resource pricing strategy and how a mobile user develops a task offloading strategy to maximize its long-term revenue into a Markov decision process, respectively determines the state set, action set, state transition probability and reward of the edge server and the mobile user, and then designs the edge server resource pricing strategy and the mobile user task offloading strategy of the application based on the DDPG algorithm and the DQN algorithm.

[0008] In the past edge computing task offloading and resource pricing methods, or the task offloading strategy is designed to minimize time delay, energy consumption and cost, although it can maximize the long-term benefits of users, but ignores the long-term benefits of edge servers; or based on single round offloading and pricing to design user offloading strategy and edge server pricing strategy, while in fact the demand of mobile users may continue to generate, which needs multiple rounds of offloading and pricing to maximize the long-term benefits of users and edge servers; or without considering the case of user movement, that is, with the change of time length, the user may be covered by different edge servers. The method of mobile user and resource pricing in edge computing designed by the application considers the case of user movement and multiple rounds of offloading and pricing, which maximizes the long-term benefits of users, and also maximizes the long-term benefits of edge servers, thereby increasing the experience of users and making the edge servers obtain more profits. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 The edge computing mobile user task offloading and resource pricing method flowchart provided by the embodiment of the application. DETAILED DESCRIPTION

[0010] In order to facilitate those skilled in the art to understand and implement the application, the application will be further described in detail below in combination with the drawings and embodiments, and it should be understood that the embodiments described herein are only used to illustrate and explain the application, and are not used to limit the application.

[0011] The main purpose of the application is to provide an edge computing mobile user offloading strategy and resource pricing strategy, which can maximize the long-term benefits of mobile users and also maximize the long-term benefits of edge servers, increase the experience of users, and make the edge servers obtain more profits.

[0012] See Figure 1 The edge computing mobile user offloading and resource pricing method provided by the application has the characteristics that it includes the following steps:

[0013] Step 1: initialize the load of edge servers and the price of computing resources;

[0014] The specific implementation includes the following sub-steps:

[0015] Step 1.1: construct the pricing problem of edge servers as a Markov decision process, regard each edge server as an independent agent, and define the attributes of state S, action A and reward value r:

[0016] The edge server state set S: in the tth time length, the edge server obtains the corresponding state by observing the mobile edge computing environment, that is, s t ={rct ,lr t-1 ,c t-1 ,p t-1 ,num t-1},in:

[0017] It is a vector of length N. This represents the remaining load capacity of edge server n in the t-th time period;

[0018] It is a vector of length N. This represents the revenue obtained by edge server n in the (t-1)th time period;

[0019] It is a vector of length N. This represents the cost incurred by edge server n in processing computing tasks during the (t-1)th time period;

[0020] It is a vector of length N. This represents the price of computing resources for edge server n during the (t-1)th time period;

[0021] It is a vector of length N. This represents the number of computing tasks that are offloaded to edge server n during the (t-1)th time period.

[0022] Edge server action set A: In the t-th time period, the edge server observes state s t Then, the corresponding action will be selected and executed according to the strategy, that is... a t It is a vector of length N. This represents the pricing decision for the computing resources of edge server n during the t-th time period, i.e., the decision on how much to set the price of the computing resources of edge server n.

[0023] Edge server state transition probability: It represents the current state s t Below, the edge server performs action a. t Then, transition to the next state s. t+1 The probability of;

[0024] Edge server reward: r t =r t (s t ,a t ), in state s t Below, the edge server takes action a tAfter that, the environment will give the corresponding feedback, return a reward r to the edge server t And enter the next state s t+1 The purpose of this paper includes maximizing the long-term revenue of the edge server, so the reward value is directly set as the total revenue of the edge server, that is, r t = U s (t) ;

[0025] Edge server discount factor: γ∈0, 1, the discount factor is mainly used to represent the influence of the subsequent edge server reward on the current reward. The larger the discount factor, the more the edge server values the long-term reward, and the smaller the discount factor, the more the edge server values the short-term reward;

[0026] Step 1.2: The mobile user's computing task offloading problem is constructed as a Markov decision process, and the corresponding decision is to select to execute the computing task locally or offload to a certain edge server to execute the computing task;

[0027] Each mobile user is regarded as an independent agent, and the attributes of state S, action A and reward value r are defined:

[0028] Mobile user state set S: At the tth time length, the mobile user obtains the corresponding state by observing the mobile edge computing environment, that is, s t ′ = {rc t ,d t ,f t ,(x t ,y t ),p t}, wherein,

[0029] is a vector of length N, represents the remaining load capacity of the edge server n at the tth time length;

[0030] is a vector of length M, represents the data size required for transmission when the computing task OT m (t) is offloaded to the edge server for processing at the tth time length;

[0031] is a vector of length M, represents the computing resources required for processing the computing task OT m (t) at the tth time length;

[0032] (x t ,y t ) = ((x1(t),y1(t)),(x2(t),y2(t)),…,(xM (t),y M (t))) is also a vector of length M, (x m (t),y m (t) represents the coordinates of mobile user m at the t-th time interval;

[0033] is a vector of length N, represents the price of computing resource of edge server n at the t-th time interval;

[0034] Mobile user action set A: at the t-th time interval, mobile user will choose a corresponding action according to the policy after observing state s t ′, and execute it, i.e. a t ′ is a vector of length M, represents the offloading decision of mobile user m for computing task OT m (t) at the t-th time interval, i.e. whether task OT m (t) is executed locally on the mobile device or offloaded to one of the edge servers for processing;

[0035] Mobile user state transition probability: which represents the probability of moving from current state s t ′ to next state s t ′ after mobile user executes action a t+1 ′;

[0036] Mobile user reward: r t ′ = r t ′(s t ′, a t ′), after mobile user takes action a t ′ in state s t ′, the environment will give a feedback to the mobile user in the form of a reward r t ′ and move to next state s t+1 ′. The purpose of this paper includes maximizing the long-term revenue of the mobile user, so the reward value is directly set to the total revenue of the mobile user, i.e. r t ′ = U u (t);

[0037] Mobile user discount factor: γ′∈0,1, the discount factor is mainly used to represent the influence of the subsequent mobile user's reward on the current reward. The larger the discount factor, the more the mobile user values the long-term reward, and the smaller the discount factor, the more the mobile user values the short-term reward;

[0038] Step 1.3: Initialize the price of computing resource of edge server, set it as 0.5. In order to show the characteristics of user movement, the present application assumes that there are 4 edge servers, and the user can be covered by different edge servers as the time length changes.

[0039] Step 1.4: Initialize the computing resource and load capacity of edge server assigned to user, the computing resource is quantified by CPU frequency, set it as 3GHz; the load capacity represents the number of computing tasks that can be processed simultaneously on the edge server, set it as 6, the computing resource and load capacity of each edge server assigned to user are the same;

[0040] In this embodiment, the positions of edge servers and mobile users are described using two-dimensional Euclidean coordinates, that is, the coordinates of edge server m are represented as L n =(X n ,Y n ). At the same time, it is assumed that all edge servers are homogeneous, that is, all edge servers have the same coverage, computing task processing capacity and the same load upper limit. R is used to represent the coverage. F is used to represent the computing resource allocated by the edge server to each task, since the focus is on the task offloading strategy of mobile users, it is assumed that the computing resource allocated by the edge server to each task is the same, that is, the offloaded tasks all have the same F, which is quantified by CPU frequency. The load upper limit of edge server is represented as , that is, the number of computing tasks that can be processed simultaneously on the edge server.

[0041] Considering that users are usually in a mobile state, the trajectory of mobile user m can be represented as:

[0042] l m (t) = (x m (t), y m (t)), t ∈ [0, T] (1)

[0043] This embodiment divides the entire time length into several time length segments, that is, 1, 2, … T, (x m (t), y m (t)) represents the coordinates of mobile user m in the tth time length segment. Thus, the distance from mobile user m to edge server n in the tth time length segment can be calculated as:

[0044]

[0045] For the computing demand of mobile user m in the tth time length segment, it is defined as a two-tuple OT m (t) = (d m (t), f m(t)). Where OT m (t) represents the computation task of the tth time duration, d m (t) represents the computation task of the tth time duration, d m (t) represents the computation task of the tth time duration, d m (t) represents the computation task of the tth time duration, d m (t) represents the computation task of the tth time duration, d represents the offloading decision of mobile user m in the tth time duration, if represents the offloading decision of mobile user m in the tth time duration, if m (t) represents the computation task of the tth time duration, d represents the offloading decision of mobile user m in the tth time duration, if represents the offloading decision of mobile user m in the tth time duration, if represents the offloading decision of mobile user m in the tth time duration, if

[0046] Step 2: mobile user makes offloading strategy under the current edge server load and the price of computing resource;

[0047] The specific implementation includes the following sub-steps:

[0048] Step 2.1: initialize the experience pool of mobile user Let it be an empty set;

[0049] Step 2.2: establish Q network and target Q network, the input of the two networks is the state set S of mobile user and the action set A of mobile user, and the output is the task offloading strategy π' of mobile user;

[0050] The Q network corresponds to the action value function Q with parameters The target Q network corresponds to the target action value function with parameters Use neural network as a nonlinear function approximator to estimate the action value function to better select the strategy;

[0051] Step 2.3: observe the environment to get the state s t ', input s t ' to the Q network, then generate offloading action a t ' according to the ε-greedy strategy and execute, and then get the reward rt and the next state s t+1 ;

[0052] Step 2.4: The mobile user stores the state transition tuple (s t ', a t ', r t ', s t+1 ') into the experience pool , where s t ' represents the state of the mobile user at time t, a t ' represents the offloading action taken by the mobile user at time t, r t ' represents the reward obtained by the mobile user at time t, and s t+1 ' represents the state of the mobile user at time t+1.

[0053] The reward that the mobile user can obtain at the t-th time duration is:

[0054]

[0055] Step 2.5: Randomly take a certain number of samples from the experience pool for training, and constantly update the Q network, and update the target Q network at a certain frequency.

[0056] The action value function of the mobile user is:

[0057]

[0058] The loss function of the mobile user is:

[0059]

[0060] Step 3: Determine whether the mobile user selects to execute locally or on a certain edge server according to the offloading strategy obtained in step 2.

[0061] If the mobile user selects to execute locally, sequentially execute step 4.

[0062] If the mobile user selects to execute on a certain edge server, execute step 5.

[0063] Step 4: The mobile user executes the computing task locally, and then executes step 9.

[0064] Assuming that all mobile devices have the same task processing capability φ, which is quantified by CPU frequency. Therefore, the time duration required for executing the computing task OT m (t) locally at the t-th time duration can be calculated as:

[0065]

[0066] In general, the power consumption of a CPU can be modeled as a super-linear function of its own frequency, i.e.,

[0067] P l = κ u (φ) ζ ⑺

[0068] where κ u and ζ are model parameters that are pre-set according to the chip structure of the CPU of the mobile device.

[0069] Therefore, the energy consumed by the local execution of the computing task OT m (t) at the tth time duration is:

[0070]

[0071] Step 5: Determine whether the selected edge server in the offloading strategy obtained in Step 2 is full;

[0072] If yes, return to Step 4;

[0073] If no, proceed to Step 6;

[0074] Step 6: The selected edge server executes the computing task;

[0075] Suppose all mobile devices also have the same data transmission power P d . In order to ensure that the edge server can normally process the computing task, the mobile device needs to upload the data required by the task. At the tth time duration, the data transmission rate between the edge server n and the mobile user m is denoted as , and its calculation formula is as follows:

[0076]

[0077] where h0 represents the received power when the reference distance d0 is 1 meter.

[0078] Thus, the transmission rate of the mobile user m to the edge server n when transmitting the data required by the task at the tth time duration can be calculated as:

[0079]

[0080] In the above formula, B represents the bandwidth of the transmission channel between the mobile device and the edge server, and σ 2 represents the noise power between the mobile device and the edge server.

[0081] After obtaining the data transmission rate, the data transmission duration can be calculated as:

[0082]

[0083] Therefore, in the tth time period, the mobile user m will calculate the task OT m (t) consumes wireless transmission energy as:

[0084]

[0085] After the edge server receives the computing task, it will immediately process it. According to the resources required by the task, the time period spent by the edge server n in processing the task OT m (t) can be calculated as:

[0086]

[0087] The edge server will also consume a certain amount of energy when processing the computing task. Similar to the CPU power consumption formula of the mobile device, the power consumption formula of the edge server can be obtained as:

[0088]

[0089] where κ s and ζ' are model parameters set in advance according to the chip structure of the edge server CPU.

[0090] Therefore, in the tth time period, the energy consumed by the edge server n when processing the computing task OT m (t) on the edge server n is:

[0091]

[0092] After the edge server completes the task, it needs to return the obtained result to the mobile device. Since the result data after the computing task is processed is usually small, and the downlink rate from the edge server to the mobile device is very fast, the time period for returning the computing task result can be ignored when calculating.

[0093] Finally, in the tth time period, the total time period spent by the mobile user m for offloading the computing task OT m (t) to the edge server n for processing is:

[0094]

[0095] Step 7: The selected edge server returns the computing result to the mobile user;

[0096] Step 8: The edge server dynamically adjusts the price of the computing resource according to the currently known information;

[0097] The specific implementation includes the following sub-steps:

[0098] Step 8.1: Initialize the edge server experience pool Let it be an empty set;

[0099] Step 8.2: Establish the Critic network, Actor network, and corresponding target network. And μ′, the Critic network and the corresponding target network The input is the current state and action of the edge server, and the output is its value, which is used to determine whether the action is good or bad and to update the Actor network μ(s|θ). μ The inputs to the Actor network and the corresponding target network μ′ are the current state of the edge server, and the outputs are its pricing strategy.

[0100] The Critic network corresponds to parameters Action value function The Actor network corresponds to the parameter θ. μ The policy function μ(s|θ) μ ), corresponding to the target network The μ′ parameters are initially the same as those of the Critic and Actor networks;

[0101] Step 8.3: The edge server observes the mobile edge computing environment and obtains the status s. t Based on its own strategy, it selects an action from the action set and combines it with noise exploration to generate a pricing action. t And execute it, then receive the reward r. t and the next state s t+1 At this point, the price of edge server computing resources changes.

[0102] The reward that the edge server can obtain in the t-th time period is:

[0103]

[0104] Step 8.4: Indicate that the edge server will transfer the state transition tuple (s) t ,a t ,r t ,s t+1 Stored in the experience pool From this, a certain number of samples are randomly selected for training, where s t Indicates the state of the edge server at time t, a t r represents the pricing action taken by the edge server at time t. t s represents the revenue obtained by the edge server at time t. t+1 This indicates the state of the edge server at time t+1;

[0105] Step 8.5: The edge server will constantly update the Actor network and Critic network, while updating the target network;

[0106] The action value function of the edge server is:

[0107]

[0108] The loss function of the edge server is:

[0109]

[0110] Step 9: Determine whether the mobile user has computing requirements;

[0111] If so, return to Step 2;

[0112] If not, sequentially execute Step 10;

[0113] Step 10: The mobile user completes all computing tasks.

[0114] It should be understood that parts not elaborated in the specification are all prior art.

[0115] It should be understood that the above description of the preferred embodiments is more detailed and should not be considered as limiting the scope of patent protection of the present application. Ordinary skilled in the art can make substitutions or modifications without departing from the scope of the claims, which are all within the scope of protection of the present application. The scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for mobile user task offloading and resource pricing in edge computing, characterized in that, The method comprises the following steps: step 1, initializing the edge server load and the price of computing resources; step 2, the mobile user formulates an offloading strategy under the current edge server load and the price of computing resources; step 3, judging whether the mobile user chooses to execute locally or on a certain edge server according to the offloading strategy obtained in step 2; if choosing to execute locally, sequentially executing step 4; if choosing to execute on a certain edge server, executing step 5; step 4, the mobile user executes the computing task locally, and then executing step 9; step 5, judging whether the selected edge server in the offloading strategy obtained in step 2 is full; if full, executing step 4 again; if not full, sequentially executing step 6; step 6, the selected edge server executes the computing task; step 7, the selected edge server returns the computing result to the mobile user; step 8, the edge server dynamically adjusts the price of computing resources according to the current known information; step 9, judging whether the mobile user has computing requirements; if yes, executing step 2 again; if not, sequentially executing step 10; step 10, the mobile user completes all computing tasks; wherein the implementation of step 8 specifically comprises the following sub-steps: Step 8.1: Initialize the pool of edge server experiences , let it be the empty set; Step 8.2: Establish Critic network, Actor network and corresponding target network and , the input of Critic network and corresponding target network is the current state and action of edge server, and the output is its value, which is used to judge the action and update Actor network ; the input of Actor network and corresponding target network is the current state of edge server, and the output is its pricing strategy; The Critic network corresponds to parameters Action value function Actor networks have parameters policy function Corresponding target network and The initial parameters are the same as those of the Critic and Actor networks; Step 8.3: Edge server observes the mobile edge computing environment to get the state , selects an action from the action set according to its policy combined with exploration noise to generate the pricing action and executes, then gets the reward and the next state , at this time the price of edge server computing resources changes; Step 8.4: The edge server represents the state transition tuple into the experience pool and randomly takes a preset number of samples from it for training, wherein represents the state of the edge server at time t, represents the pricing action taken by the edge server at time t, represents the revenue obtained by the edge server at time t, represents the state of the edge server at time t+1; Step 8.5: the edge server constantly updates the Actor network and the Critic network, and simultaneously updates the target network.

2. The method for mobile user task offloading and resource pricing in edge computing according to claim 1, characterized in that The implementation of step 1 specifically comprises the following sub-steps: Step 1.1: Formulate the pricing problem of edge servers as a Markov Decision Process, where each edge server is treated as an independent agent, define the state S, action A and reward value R attributes: Edge server state set : In the tth time duration, the edge server obtains the corresponding state by observing the mobile edge computing environment, i.e. , wherein: is a vector of length N, represents the load capacity of the edge server at the i-th time duration, and the remaining load capacity of the edge server is a vector of length , represents the revenue obtained by the edge server in the th time duration, . It is a length of The vector, It means that in the first Each time period, edge server The cost of processing computational tasks; is a vector of length , represents the price of computing resources at the th time duration; and is the price of computing resources at the th time duration. is a vector of length , represents the number of computing tasks offloaded to the edge server in the th time duration Edge server action set : in the During a specific time period, the edge server observes the state. Then, the corresponding action will be selected and executed according to the strategy, that is... It is a length of The vector, It means that in the first Duration period, edge server Pricing decisions for computing resources, specifically deciding on edge servers. What should the price of computing resources be set at? Edge server state transition probabilities: representing the probability of transitioning to the next state from the current state after the edge server performs an action ; Edge server reward: In the state , the edge server takes an action After that, the environment will give feedback accordingly, return a reward to the edge server And enter the next state Set the reward value directly to the total income of the edge server, that is ; Edge server discount factor: The discount factor is mainly used to represent the influence of the subsequent edge server reward on the current reward. The larger the discount factor is, the more the edge server values the long-term reward, and the smaller the discount factor is, the more the edge server values the short-term reward. Step 1.2: constructing the computing task offloading problem of the mobile user as a Markov decision process, and the corresponding decision is to choose to execute the computing task locally or offload to a certain edge server to execute the computing task; Each mobile user is regarded as an independent agent, and the state S, action A and reward value R are defined as follows: Attributes are defined as follows: Set of mobile user states : At the first time duration, the mobile user obtains a corresponding state by observing the mobile edge computing environment, i.e. wherein, is a vector of length , represents the remaining load capacity of the edge server at the th time duration; and th time duration. is a vector of length , represents the data size that needs to be transmitted when the computing task is offloaded to the edge server for processing in the th time duration is a vector of length , represents the computing resources needed by the processing computing task in the th time duration It is also a length of The vector, It means that in the first During a specific time period, mobile users coordinates; is a vector of length , represents the price of the computing resources of the edge server at the th time duration Mobile user action set : In the th time duration, the mobile user will choose and perform the corresponding action according to the strategy after observing the state , that is is a vector with length , represents the mobile user 's action in the th time duration . The offloading decision of the computing task, that is, whether the task is executed locally on the mobile device or offloaded to one of the edge servers. Mobile user state transition probabilities: representing the probability of transitioning to a next state after the mobile user performs an action in the current state . Mobile user reward: In state , the mobile user takes an action After that, the environment gives a corresponding feedback, returning a reward to the mobile user And enters the next state The reward value is directly set to the total income of the mobile user, that is ; Mobile user discount factor: The discount factor is used to represent the influence of the future reward on the current reward. The larger the discount factor, the more the mobile user values long-term rewards, and the smaller the discount factor, the more the mobile user values short-term rewards. Step 1.3: initializing the price of computing resources of the edge server, setting it as 0.5, in order to show the characteristics of user movement, there are four edge servers, and the user is covered by different edge servers as the time length changes; Step 1.4: Initialize the computing resources allocated to the user and its load capacity of the edge server, the computing resources are quantified by CPU frequency, which is set to GHz; the load capacity represents the number of computing tasks that can be processed simultaneously on the edge server, which is set to 6, the computing resources and the load capacity allocated to the user of each edge server are the same.

3. The method for mobile user task offloading and resource pricing in edge computing according to claim 1, characterized in that The implementation of step 2 specifically comprises the following sub-steps: Step 2.1: Initialize the mobile user experience pool to the empty set; Step 2.2: Establish the Q-network and the target Q-network, the input of the two networks is the state set S of the mobile user and the action set A of the mobile user, and the output is the task offloading strategy of the mobile user ; Q network corresponds to a value function Q with parameters , the target Q network corresponds to a target value function with parameters , using a neural network as a nonlinear function approximator to estimate the value function to better select the policy; Step 2.3: Observe the environment to get the state , and input to the Q network, then according to - the greedy policy and generate the offloading action , and execute, then get the reward and the next state . ; Step 2.4: Mobile user shifts state transition tuple into the experience pool wherein represents the state of the mobile user at time t, represents the taken offloading action of the mobile user at time t, represents the gained reward of the mobile user at time t, represents the state of the mobile user at time t+1; Step 2.5: Randomly take out a preset number of samples from the experience pool for training, constantly update the Q network, and update the target Q network at a preset frequency. Step 2.5: Randomly take out a preset number of samples from the experience pool for training, constantly update the Q network, and update the target Q network at a preset frequency. 4.A system for mobile user task offloading and resource pricing in edge computing, characterized in that, The method comprises the following modules: a first module for initializing the edge server load and the price of computing resources; A second module for the mobile user to formulate an offloading strategy under the current edge server load and the price of computing resources; A third module for judging whether the mobile user chooses to execute locally or on a certain edge server according to the offloading strategy obtained in step 2; if choosing to execute locally, sequentially executing step 4; if choosing to execute on a certain edge server, executing step 5; a fourth module for the mobile user to execute the computing task locally, and then executing step 9; A fifth module for judging whether the selected edge server in the offloading strategy obtained in step 2 is full; if full, executing step 4 again; if not full, sequentially executing step 6; The sixth module is configured to perform a computing task by the selected edge server; the seventh module is configured to return the computing result to the mobile user by the selected edge server; and the eighth module is configured to dynamically adjust the price of the computing resource by the edge server according to currently known information. The ninth module is configured to determine whether the mobile user still has a computing demand; if yes, return to step 2; if no, sequentially execute step 10; and the tenth module is configured to complete all computing tasks by the mobile user; wherein the eighth module is specifically configured to execute the following sub-steps: Step 8.1 : Initialize the pool of edge server experiences , let it be the empty set; Step 8.2: Establish Critic network, Actor network and corresponding target network and , the input of Critic network and corresponding target network is the current state and action of edge server, and the output is its value, which is used to judge the action and update the Actor network ; the input of Actor network and corresponding target network is the current state of edge server, and the output is its pricing strategy; The Critic network corresponds to parameters Action value function Actor networks have parameters strategy function Corresponding target network and The initial parameters are the same as those of the Critic and Actor networks; Step 8.3: Edge server observes the mobile edge computing environment to get the state , selects an action from the action set according to its policy combined with exploration noise to generate the pricing action and executes, then gets the reward and the next state , at this time the price of edge server computing resources changes; Step 8.4: The edge server represents the state transition tuple into the experience pool and randomly takes a preset number of samples from it for training, wherein represents the state of the edge server at time t, represents the pricing action taken by the edge server at time t, represents the revenue obtained by the edge server at time t, represents the state of the edge server at time t+1; Step 8.5: The edge server constantly updates the Actor network and the Critic network, and simultaneously updates the target network.

Images