An adaptive mobile edge computing offloading and resource allocation method

Abstract

The application designs a self-adaptive mobile edge computing unloading and resource allocation method, mainly including the following steps: a user sends a task unloading request; a mobile edge computing system calculates the best unloading strategy of the task according to the data size of the task, the wireless channel information where the user is located and the available computing resources of each edge server in the mobile edge computing system and notifies the user; the user connects a base station according to the notified unloading strategy and sends a task; after the base station receives the task, the base station sends the task to an edge server connected to the base station, the edge server allocates resources to process the task; the edge server sends the processed result to the base station, and the base station returns the processed result to the user. The application adopts a deep reinforcement learning method, obtains the best task unloading scheme at the current time according to historical system information, and reasonably allocates resources to process the task under the limitation of task delay, so that the consumption of resources is reduced and the utilization rate of the edge server is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of edge computing, in particular to a self-adaptive mobile edge computing offloading and resource allocation method. BACKGROUND

[0002] In 5G and future communication network architecture, edge computing (MEC) technology as one of the key technologies, makes up for the lack of mobile device resources, and improves the service quality of application programs. However, in the MEC system, the available resources of the channel, base station and MEC server are changed with time, therefore, in order to make full use of the resources in the MEC system and improve the effective utilization of resources, it is necessary to study the offloading problem of multiple tasks and the joint allocation problem of multi-dimensional resources in the MEC system. SUMMARY

[0003] The present application aims at the problems in the above background technology, and proposes a self-adaptive mobile edge computing offloading and resource allocation method by jointly optimizing the multi-task offloading strategy and resource allocation strategy in the MEC system to cope with the dynamic changes of multi-dimensional resources in the MEC system, with the goal of minimizing the power consumption of the MEC server and the base station.

[0004] A self-adaptive mobile edge computing offloading and resource allocation method, the specific steps are as follows:

[0005] S1: build an edge computing system, and the user sends a task offloading request;

[0006] S2: after the mobile edge computing system receives the request, the best offloading strategy of the task is calculated according to the data size of the task, the wireless channel information and the available computing resources of each edge server, and the offloading strategy is notified to the user;

[0007] S3: the user connects the base station according to the offloading strategy notified by the mobile edge computing system and sends the task;

[0008] S4: after the base station receives the task, it is sent to the edge server connected thereto, and the edge server allocates resources to process the task;

[0009] S5: the edge server sends the result of processing the task to the base station, and the base station returns the result of processing the task to the user.

[0010] The step S1 specifically includes the following steps:

[0011] S11: Construct an edge computing system, which contains one master server, K mobile devices, and M base stations, wherein each base station is configured with one MEC server, and there are M MEC servers in total; in each time slot t, each mobile device sends only one task offloading request and can only select one base station to connect, because one base station is configured with one MEC server, therefore, after selecting the base station, the MEC server for task offloading is also selected; k represents the index of the mobile device / task, and k ∈ {1, 2,..., K}; m represents the index of the base station / MEC server, and m ∈ {1, 2,..., M}; the master server is used to collect information, including environmental information and available resources of the MEC server; the environmental information includes channel state information (CSI) of the mobile device connecting the base station. In time slot t, the environmental information known by the MEC system is actually the information at time t-1.

[0012] The step S2 adopts a deep reinforcement learning method to obtain a task offloading strategy, and specifically includes the following steps.

[0013] S21: A learning model is trained by using a deep reinforcement learning method.

[0014] S22: The best offloading strategy of the task is calculated by using the trained learning model.

[0015] The step S21 specifically includes the following steps.

[0016] S211: Construct a state space Define the state of the deep reinforcement learning model: Wherein L k (t) represents the data size of the task k at time slot t, represents the available computing resource of the MEC server m, h m,k (t) represents the CSI of the channel between the mobile device k and the base station m, {·} K represents the dimension of this value, {·} M represents the dimension of this value, {·} M×K represents the dimension of this value, M×K.

[0017] S212: Construct an action space Define the action of the deep reinforcement learning model: a(t) = {x k,m (t)} K×M , x k,m (t) ∈ {0, 1}, x k,m (t) = 0 represents that the task k does not select the MEC server m to process the task, and x k,m(t) = 1 means that the task k selects the MEC server m to process the task, {·} K×M The dimension of this value is K x M. The action a(t) represents which MEC server each mobile device selects to process its task at time slot t.

[0018]

[0019] S213: Define the strategy. According to the obtained environment state s(t), the action a(t) is selected by the strategy π(t): π(t): s(t)→a(t).

[0020] S214: Define the reward. According to the state s(t) and the selected action a(t), the reward r(t) = -∑ k∈K ∑ m∈M x m,k (t)(ρ m,k (t) + p m,k (t)) is obtained after the action a(t) is performed, where ρ m,k (t) represents the computing power of the MEC server m allocated to the task k, and p m,k (t) represents the transmission power of the base station m allocated to the task k.

[0021] S215: Build the neural network. A fully neural network (FNN) is used to build the neural network, and the state is used as the input of the neural network, and the Q(s, a) value corresponding to each action selected under the state is used as the output value of the neural network. The relu function is used as the activation function of each layer of the neural network: q i = relu(w i x q i-1 + b i ), where q i is the output of the i-th layer of the neural network, w i is the neuron weight of the i-th layer of the neural network, and b i is the bias of the i-th layer of the neural network.

[0022] S216: Train the neural network. The DQN algorithm of the deep reinforcement learning method is used to train the neural network. The experience replay mechanism is used, and the neural network stores the collected samples (s(t), a(t), r(t), s(t+1)) in the experience pool . Each time the neural network is trained, a set of samples is randomly selected from for training.

[0023] S217: After step S216 is completed, the trained learning model is obtained.

[0024] The step S213 specifically includes the following steps:

[0025] S2131: During training, an action is selected by using an epsilon-greedy algorithm. An action is randomly selected with a probability of epsilon, and an action with the maximum Q(s, a) output by the neural network is selected with a probability of 1-epsilon. The epsilon used in this paper is calculated by the following formula:

[0026] ε=v min +(v max -v min )e -η*episode (1)

[0027] where v min is the minimum exploration rate, v max is the maximum exploration rate, and η is the decay factor, and episode is the number of iterations in training.

[0028] The step S216 specifically includes the following steps:

[0029] S2161: Initialize all parameters θ = (w, b) of the neural network, initialize the learning rate α, the reward decay coefficient γ, the minimum exploration rate v min , the maximum exploration rate v max , the exploration rate decay factor η, the maximum number of iterations T, and initialize the experience pool

[0030] S2162: Let the iteration number t = 1, then the state where L k (t) represents the data size of task k, represents the available computing power of MEC server m, and h m,k (t) represents the CSI of the channel between mobile device k and base station m. Take s(t) as the input of the neural network to obtain the Q value output corresponding to all actions in this state.

[0031] S2163: Calculate the greed rate ε according to formula (1), and select the action a(t) according to the epsilon-greedy algorithm, and obtain the state

[0032] S2164: Let Use s c (t) and a(t) to obtain the reward r(t), and store the sample (s(t), a(t), r(t), s(t+1)) into the experience pool

[0033] S2165: Extract N samples (s i ,a i ,r i ,s i′ ), and calculate the target value

[0034] S2166: Minimize the mean square error and use to update the network parameters.

[0035] S2167: t = t + 1, repeat step S2162 until t = T. After the iteration, a trained deep reinforcement learning model can be obtained. Taking the state as the input of the model, the maximum value of the output of the model corresponds to the optimal action, i.e. the best offloading decision.

[0036] The step S22 specifically comprises:

[0037] Using the trained learning model obtained in step S21, taking the data size of the task, the wireless channel information and the available computing resources of each edge server as the input of the learning model, and selecting the action corresponding to the maximum output value in the output of the learning model as the offloading strategy.

[0038] The step S3 specifically comprises the following steps:

[0039] S31: Calculate the transmission rate of task k offloaded to base station m:

[0040]

[0041] where B k,m is the bandwidth between task k and base station m, σ 2 is the channel noise power, and P k is the transmission power of mobile device k.

[0042] S32: According to formula (2), calculate the time of task k offloaded to base station m:

[0043]

[0044] The value of x k,m (t) in formula (3) is calculated by step S22.

[0045] The step S4 specifically comprises the following steps:

[0046] S41: Calculate the processing time of task k on MEC server m:

[0047]

[0048] where ρ m,k (t) is the computing power allocated by MEC server m to task k, and ω kFor the load of task k (unit: cycles / bit), C m For the switched capacitors of the MEC server, V m C m The voltage at both ends.

[0049] Step S5 specifically includes the following steps:

[0050] S51: Calculate the transmission rate at which base station m returns the result of task k to mobile device k after processing.

[0051]

[0052] Where, p m,k The transmission power allocated to base station m for task k.

[0053] S52: According to formula (5), calculate the transmission time for base station m to return the result of task k to mobile device k:

[0054]

[0055] Among them, L ′ k The size of the data after task k is processed.

[0056] S53: To minimize the resource consumption of MEC servers and base stations, a joint optimization problem is established for the computation offloading and resource allocation of multiple tasks in an edge computing system.

[0057]

[0058]

[0059] Among them, D max The maximum latency of the task. This represents the available computing power of MEC server m. Let m be the available transmission power of base station m.

[0060] S54: Obtain the optimal action, x, from the deep reinforcement learning model in step S2. m,k The value of (t) is then used to determine the values ​​of all x. m,k Substituting the value of (t) into formula (7), formula (7) can be simplified to:

[0061]

[0062]

[0063] Solving formula (8) with the convex optimization toolkit can obtain the optimal resource allocation scheme of the edge computing system. In step S2, the reward r(t) is defined as the optimization target value of formula (8), which is used to train the deep reinforcement learning model to obtain the optimal offloading strategy.

[0064] Compared with the prior art, the present application has the following advantages:

[0065] (1) The present method considers that the MEC system knows the historical channel information at the time of sending the offloading task by the user, rather than the instantaneous channel information at the offloading time, and obtains the optimal task offloading strategy at the current time by using the historical channel information through the deep reinforcement learning method.

[0066] (2) The present method considers the time-varying nature of the channel and the MEC system resources and the total delay of task processing, and proposes a hybrid optimization problem of task offloading and resource allocation, to realize dynamic task offloading and MEC system resource allocation. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 A flowchart of the adaptive mobile edge computing offloading and resource allocation method of the present application.

[0068] Figure 2 The time relationship between the MEC system and the environment system regarding the channel information of the present application.

[0069] Figure 3 A DQN structure diagram in the deep reinforcement learning algorithm in the embodiment of the present application.

[0070] Figure 4 An algorithm flowchart in the embodiment of the present application. DETAILED DESCRIPTION

[0071] The present application will be described in detail below in combination with the drawings and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present application, and gives a detailed implementation manner and specific operation process, but the protection scope of the present application is not limited to the following embodiments.

[0072] As shown in Figure 1 An adaptive mobile edge computing offloading and resource allocation method, which comprises the following steps:

[0073] S1: Construct an edge computing system, and a user sends a task offloading request.

[0074] S11: Construct an edge computing system, which contains one master server, K mobile devices, and M base stations, each of which is configured with an MEC server, a total of M MEC servers; in each time slot t, each mobile device sends only one task offloading request and can only select one base station to connect, because one base station is configured with an MEC server, so the selected base station is selected to offload the task; k represents the index of the mobile device / task, and k ∈ {1, 2,..., K}; m represents the index of the base station / MEC server, and m ∈ {1, 2,..., M}; the master server is used to collect information, including environmental information and available resources of the MEC server. The environmental information includes channel state information (CSI) of the mobile device connecting the base station. As shown in Figure 2 , at time slot t, the environmental information known by the MEC system is actually the information at time t-1.

[0075] S2: After the mobile edge computing system receives the request, the best offloading strategy of the task is calculated according to the data size of the task, the wireless channel information, and the available computing resources of each edge server, and the offloading strategy is notified to the user.

[0076] The method process is as follows: Figure 3 .

[0077] S21: A learning model is trained by using a deep reinforcement learning method.

[0078] S22: The best offloading strategy of the task is calculated by using the trained learning model.

[0079] The steps contained in S21 are as follows:

[0080] S211: Construct a state space Define the state of the deep reinforcement learning model: where L k (t) represents the data size of task k at time slot t, represents the available computing power of MEC server k, h m,k (t) represents the CSI of the channel between mobile device k and base station k, {·} K represents the dimension of this value is K, {·} M represents the dimension of this value is M, {·} M×K represents the dimension of this value is MxK.

[0081] S212: Construct an action space Define the action of the deep reinforcement learning model: a(t) = {xk,m (t)} K×M x k,m (t)∈{0,1},x k,m (t) = 0 indicates that task k did not select MEC server m to process the task, x k,m (t) = 1 indicates that task k selects MEC server m to process the task, {·} K×M The dimension of this value is K×M. Action a(t) indicates which MEC server each mobile device selects to handle its task in time slot t.

[0082] S213: Define the policy. Based on the obtained environmental state s(t), select the action a(t) through the policy π(t): π(t): s(t) → a(t).

[0083] S214: Define the reward. Based on the state s(t) and the selected action a(t), after performing action a(t), the reward r(t) obtained from the environmental feedback is: s(t) × a(t).

[0084] S2141: During training, an ε-greedy algorithm is used to select actions. Actions are randomly selected with a probability of ε, and the action that maximizes the neural network output Q(s,a) is selected with a probability of 1-ε. The ε used in this paper is calculated using the following formula:

[0085] ε=v min +(v max -v min )e -η*episode (1)

[0086] Among them, v min To minimize the exploration rate, v max η is the maximum exploration rate, η is the decay factor, and episode is the number of iterations during training.

[0087] S215: Constructing a Neural Network. A neural network is constructed using an Open-ended Neural Network (FNN). The state is used as the input to the neural network, and the value of each action Q(s,a) in that state is selected as the output value of the neural network. The ReLU function is used as the activation function for each layer of the neural network: q i =relu(w i ×q i-1 +b i ), where q i w is the output of the i-th layer in the neural network. i Let b be the weight of the neuron in the i-th layer of the neural network. i Let be the bias of the i-th layer of the neural network.

[0088] S216: Training the Neural Network. The Deep Reinforcement Learning (DQN) algorithm is used to train the neural network. An experience replay mechanism is employed, where the neural network stores the collected samples (s(t), a(t), r(t), s(t+1)) in an experience pool. In this process, during each training session, the neural network... A set of samples is randomly selected for training. The algorithm flow of this method is as follows: Figure 4 As shown, it includes the following steps:

[0089] S2161: Initialize all parameters of the neural network θ = (w, b), initialize the learning rate α, the reward decay coefficient γ, and the minimum exploration rate v. min Maximum exploration rate v max Exploration rate decay factor η, maximum number of iterations T, initial experience pool

[0090] S2162: Let the iteration number t = 1, then the state Among them, L k (t) represents the amount of data in task k. h represents the available computing power of MEC server m. m,k s(t) represents the CSI of the channel between mobile device k and base station k. Using s(t) as the input to the neural network, we obtain the Q-value outputs corresponding to all actions in this state.

[0091] S2163: Calculate the greedy rate ε according to formula (1), select action a(t) according to the ε-greedy algorithm, and obtain the state at the next moment.

[0092] S2164: Order Use s c Samples (t), a(t), r(t), and s(t+1) are rewarded with r(t), and the sample (s(t), a(t), r(t), s(t+1)) is stored in the experience pool.

[0093] S2165: Draw N samples from the experience pool (s i ,a i ,r i ,s i ′ ), and calculate the target value y. i =r i +

[0094]

[0095] S2166: Minimize mean square error and use Update network parameters.

[0096] S2167: t = t + 1, repeat step S2162 until t = T. After the iteration, a trained deep reinforcement learning model can be obtained. Taking the state as the input of the model, the action corresponding to the maximum value of the model's output is the optimal action, i.e., the best unloading decision.

[0097] The steps included in S22 are as follows:

[0098] S221: The trained deep reinforcement learning model is obtained from step S21. The data size of the task, the wireless channel information, and the available computing resources of each edge server are used as inputs to the learning model. The action with the largest output value is selected as the offloading strategy from the output of the learning model.

[0099] S3: The user connects to the base station and sends a task according to the offloading policy notified by the edge computing system.

[0100] S31: Calculate the transmission rate of task k offloaded to base station m:

[0101]

[0102] Among them, B k,m Let σ be the bandwidth between task k and base station m. 2 P is the channel noise power. k Let k be the transmission power of the mobile device.

[0103] S32: Calculate the time for task k to be unloaded to base station m according to formula (2):

[0104]

[0105] In formula (3) x k,m The value of (t) is calculated in step S22.

[0106] S4: The base station receives the task and sends it to the edge server connected to it. The edge server allocates resources to process the task.

[0107] S41: Calculate the processing time of task k on MEC server m:

[0108]

[0109] Where, ρ m,k (t) represents the computational power allocated to task k by MEC server m, ω k For the load of task k (unit: cycles / bit), C m For the switched capacitors of the MEC server, V m C m The voltage at both ends.

[0110] S5: The edge server sends the results of the task processing to the base station, which then returns them to the user.

[0111] S51: Calculate the transmission rate at which base station m returns the result of task k to mobile device k after processing.

[0112]

[0113] Where, p m,k The transmission power allocated to base station m for task k.

[0114] S52: According to formula (5), calculate the transmission time for base station m to return the result of task k to mobile device k:

[0115]

[0116] Among them, L ′ k The size of the data after task k is processed.

[0117] S53: To minimize the resource consumption of MEC servers and base stations, a joint optimization problem is established for the computation offloading and resource allocation of multiple tasks in an edge computing system.

[0118]

[0119]

[0120] Among them, D max The maximum latency of the task. This represents the available computing power of MEC server m. Let m be the available transmission power of base station m.

[0121] S54: Obtain the optimal action, x, from the deep reinforcement learning model in step S2. m,k The value of (t) is then used to determine the values ​​of all x. m,k Substituting the value of (t) into formula (7), formula (7) can be simplified to:

[0122]

[0123]

[0124] By solving equation (8) using the convex optimization toolkit, the optimal resource allocation scheme for the edge computing system can be obtained. In step S2, the reward r(t) is defined as the optimization objective value of equation (8) and used to train a deep reinforcement learning model to obtain the optimal unloading strategy.

Claims

1. An adaptive mobile edge computing offloading and resource allocation method, characterized in that, Includes the following steps: S1: Build an edge computing system, and the mobile device sends a task unloading request; S2: After receiving the request, the mobile edge computing system calculates the best offloading strategy for the task based on the data volume of the task, the wireless channel information, and the available computing resources of each edge server, and then notifies the mobile device of the offloading strategy. S3: The mobile device connects to the base station and sends a task according to the offloading policy notified by the mobile edge computing system; S4: After receiving the task sent by the mobile device, the base station sends it to the edge server connected to it, and the edge server allocates resources to process the task. S5: The edge server sends the result of the task processing to the base station, and the base station returns the result of the task processing to the mobile device; Step S1 includes: S11: Build an edge computing system, which includes a main server. Mobile devices There are [number] base stations, each equipped with one Mobile Edge Computing (MEC) server, totaling [number]. Each MEC server; in each time slot t, each mobile device sends only one task offload request and can only select one base station to connect to, because one base station is configured with one MEC server. Therefore, the selected base station determines which MEC server is selected for task offload. Indicates the index of the mobile device / task, and ; This represents the index of the base station / MEC server, and The main server is used to collect information, including environmental information and the available resources of the MEC server. The environmental information includes the channel state information (CSI) of the mobile device connecting to the base station. In time slot t, the environmental information known to the MEC system is the information at time t-1. Step S2 employs a deep reinforcement learning-based method to obtain a task unloading strategy, including: S21: Train a learning model using a deep reinforcement learning-based method; S22: Calculate the optimal unloading strategy for the task using the trained learning model; Step S21 includes: S211: Constructing the state space Define the state of the deep reinforcement learning model: ,in This indicates that in time slot t, the task The size of the data, Indicates MEC server Available computing resources Indicates mobile device With base station CSI of the inter-channel The dimension representing this value is , The dimension representing this value is , The dimension representing this value is ; S212: Constructing the Action Space Define the actions of the deep reinforcement learning model: , , Indicates task MEC server not selected To handle the task. Indicates task Select MEC server To handle the task. The dimension representing this value is ;action This indicates which MEC server each mobile device selects to process its task in time slot t. ; S213: Define the strategy; based on the obtained environmental state. Through strategy Select Action : ; S214: Define reward; based on state and the chosen action Execute actions Afterwards, receive rewards based on environmental feedback. ,in Indicates MEC server Assigned to task Calculated power, base station Assigned to task The transmission power; S215: Construct a neural network; use a fully connected neural network (FNN) to construct the neural network, taking the state as the input of the neural network, and selecting the Q(s,a) value corresponding to each action in this state as the output value of the neural network; use the ReLU function as the activation function for each layer of the neural network: ,in, This represents the output of the i-th layer in the neural network. Let be the weights of the neurons in the i-th layer of the neural network. This represents the bias of the i-th layer of the neural network; S216: Training the neural network; using the deep reinforcement learning method DQN algorithm to train the neural network; employing an experience playback mechanism, the neural network processes the collected samples... Stored in the experience pool In this process, during each training session, the neural network... A set of samples is randomly selected for training. S217: After step S216 is completed, the trained learning model is obtained; Step S22: Using the trained learning model obtained in step S21, the data size of the task, the wireless channel information, and the available computing resources of each edge server are used as inputs to the learning model, and the action corresponding to the maximum output value is selected as the unloading strategy from the output of the learning model.

2. The adaptive mobile edge computing offloading and resource allocation method according to claim 1, characterized in that, Step S213 includes the following steps: S2131: During training, use - Greedy algorithm selects actions; The probability of randomly selecting an action is 1- The probability-based selection neural network maximizes the action whose output Q(s,a); the method employed... Calculated using the following formula: (1) in, To minimize the exploration rate, To maximize the exploration rate, As the attenuation factor, This represents the number of iterations during training.

3. The adaptive mobile edge computing offloading and resource allocation method according to claim 2, characterized in that, Step S216 includes the following steps: S2161: Initialize all parameters of the neural network Initialize learning rate Reward decay coefficient Minimum exploration rate Maximum exploration rate Exploration rate decay factor Maximum number of iterations Initialize the experience pool ; S2162: Let the number of iterations be... Then the state ,in, Indicates task The size of the data, Indicates MEC server Available computed power, Indicates base station With mobile devices CSI of the channel between; As input to the neural network, the Q-values ​​corresponding to all actions in that state are output. S2163: Calculate the greedy rate according to formula (1) and according to Greedy algorithm selects actions And obtain the state at the next moment. ; S2164: Order ,use and Receive reward and the sample Store in experience pool ; S2165: Draw N samples from the experience pool. And calculate the target value. ; S2166: Minimize mean square error and use Update network parameters; S2167: Repeat step S2162 until... After the iteration is complete, a trained deep reinforcement learning model can be obtained. The state is used as the input of the model, and the action corresponding to the maximum value of the model's output is the optimal action, i.e. the best unloading decision.

4. The adaptive mobile edge computing offloading and resource allocation method according to claim 3, characterized in that: In step S3, the task Unload to base station The transmission rate is: (2) in, For mobile devices With base station The bandwidth of the channel between them Channel noise power, For mobile devices The transmission power; according to formula (2), the task Unload to base station The time is: (3) In formula (3) The value is calculated in step S22.

5. The adaptive mobile edge computing offloading and resource allocation method according to claim 4, characterized in that: In step S4, the task On the MEC server The processing time is: (4) in, MEC server Assigned to task Calculated power, For the task Load, unit: cycles / bit For the switched capacitors of the MEC server, for The voltage at both ends.

6. The adaptive mobile edge computing offloading and resource allocation method according to claim 5, characterized in that: In step S5, the base station The task The processed results are returned to the mobile device. The transmission rate is: (5) in, For base stations For the task The allocated transmission power; then, according to formula (5), the base station The task The processed results are returned to the mobile device. The transmission time is: (6) in, For the task The size of the processed data.

7. The adaptive mobile edge computing offloading and resource allocation method according to claim 6, characterized in that: In step S4, with the goal of minimizing the resource consumption of MEC servers and base stations, a joint optimization problem of computation offloading and resource allocation for multiple tasks in the edge computing system is established: (7) in, The maximum latency of the task. Indicates MEC server Available computed power, For base stations Available transmission power.

8. The adaptive mobile edge computing offloading and resource allocation method according to claim 7, characterized in that: In step S22, the state of time slot t is used as the input to the deep reinforcement learning model, and the optimal action is obtained from the deep reinforcement learning model, i.e. The value, then, all Substituting the value into formula (7), formula (7) simplifies to: (8) By solving formula (8) using the convex optimization toolkit, the optimal resource allocation scheme for the edge computing system can be obtained, reducing resource waste and improving the utilization rate of the edge computing system; In step S2164, the reward will be... Defined as the optimization objective value of formula (8), it is used to train the deep reinforcement learning model.

Images