Multi-user dependent task unloading method based on deep Q learning and CH algorithm

Abstract

The invention discloses a multi-user dependent task unloading method based on deep Q learning and a CH algorithm, and relates to the technical field of task unloading. Comprising the following steps: determining a single BS / SC and multiple servers as a network model, determining a communication condition and a calculation mode, then establishing a dependency task model by using a DAG, and completing MEC system modeling; according to the method, a target function is established, a DTO-DQN-CH hybrid optimization algorithm is provided, task unloading optimization applying deep Q learning is described, a load balancing mechanism based on a CH algorithm is introduced, and finally a complete process of DTO-DQN-CH is obtained. Three simulation tasks are set, MD-TSDDQN and DTO-GA-CH serve as a control group, and the influence of the load balance effect between MECs, the UD number and the MEC number on task unloading optimization is evaluated. According to the invention, the purpose of minimizing energy consumption and time delay can be achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of task offloading, and more particularly to a multi-user dependent task offloading method based on deep Q learning and CH algorithm. Background Art

[0002] Dependency task offloading is the core enabling technology of edge computing and the basis for achieving low-latency services. It makes up for the shortcomings of centralized processing in cloud computing and is deeply integrated with multi-access edge computing (MEC).

[0003] Although relying on task offloading can effectively reduce task processing latency, it also places higher demands on offloading strategies. Therefore, more attention should be paid to issues such as offloading strategy optimization and load balancing of MEC servers.

[0004] Therefore, it is an urgent problem for those skilled in the art to propose a multi-user dependent task offloading method based on deep Q learning and CH algorithm to solve the difficulties existing in the existing technology. Summary of the Invention

[0005] In view of this, the present invention provides a multi-user dependent task offloading method based on deep Q learning and CH algorithm to solve the technical problems existing in the prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solutions: The multi-user dependent task offloading method based on deep Q learning and CH algorithm includes the following steps: S1. Determine the single BS / SC, multi-server network model and determine the communication conditions and calculation methods. Then use DAG to establish a dependent task model to complete the MEC system modeling. S2. Establish the objective function and propose the DTO-DQN-CH hybrid optimization algorithm. Describe the task offloading optimization using deep Q learning and introduce the load balancing mechanism based on the CH algorithm. Finally, derive the complete DTO-DQN-CH process. S3. Set three simulation tasks, take MD-TSDDQN and DTO-GA-CH as control groups, and evaluate the load balancing effect between MECs, the number of UDs, and the impact of the number of MECs on task offloading optimization.

[0007] Optionally, in S1, a single BS / SC and multiple servers network model is determined, and the communication conditions and calculation methods are determined. Then, a dependent task model is established using DAG. The specific contents of completing the MEC system modeling are as follows: The first and last UD subtasks are executed locally, and the communication delay between the BS / SC and the server and the transmission delay of the task calculation results are not included in the calculation. Under the condition that the UD task is passed to the MEC for processing, the transmission rate expression of any UD is: ; in, , They represent the channel bandwidth, the transmit power from UD to BS / SC, the channel gain of the wireless propagation channel from UD to BS / SC, the distance from UD to BS / SC, the path loss index, and the noise power spectral density respectively; According to the offloading strategy, the subtask will be executed locally or on the MEC server, which will generate two types of delay and energy consumption. The corresponding computing model is established, namely local computing and edge computing, using UD i To distinguish different UDs, where i=1, 2, 3..., UD i No. j The subtask is executed locally Indicates that it is executed on the MEC server using express; when When , the task processing delay and the energy consumption required by UD to execute the task are generated. The calculation formula for the processing delay of the corresponding subtask is: ,in, For the j The number of CPU cycles required for each subtask, UD i The computing power of the energy consumption is calculated as follows: ; when When , the task transmission delay and the processing delay of the MEC server executing the task are generated. The transmission delay calculation formula is: ,in, is the data volume of the corresponding subtask, UD i The transmission rate; the processing delay calculation formula is: , UD is the computing resource allocated to the corresponding subtask. i No. j The transmission energy consumption calculation formula of each subtask is: ,in, UD i Transmit power to BS / SC, is the transmission delay; Then use DAG to build a dependent task model, To express the jThe predecessor subtask of a subtask, then j The expression for the transmission completion time of each subtask is: , is the local preparation time of the corresponding subtask, Sure, is the processing delay of the corresponding subtask; in, For the corresponding local running subtask k The completion time of the predecessor subtask, The subtask executed on the MEC server j The first of the predecessor subtasks k The completion time of the predecessor subtask, j Subtask transmission completion time: , executed on the MEC server j The execution completion time of each subtask is: , To run the j Preparation time for each subtask, To handle delay, is the transmission delay; Finally, run the j The preparation time for each subtask is: , complete the MEC system modeling.

[0008] Optionally, the specific content of establishing the objective function in S2 is: Under the premise of not violating the maximum tolerable delay of UDs, an objective function is established: ; satisfy ,in , is the weight between delay and energy consumption, is the total task processing delay, is the total energy consumption, is the total delay of task processing, UD i The sum of the energy consumption of local execution subtasks, is the total transmission energy consumption, UD i The sum of the local processing delays of all subtasks, UD i The sum of the local processing energy consumption of all subtasks.

[0009] Optionally, the specific content of the DTO-DQN-CH hybrid optimization algorithm proposed in S2 is: First, we give an overview of the DTO-DQN-CH hybrid algorithm. We divide the MEC architecture with multiple UDs, a single BS / SC, and multiple MEC servers into two layers: the edge layer and the user layer. At the edge layer, the BS / SC operation consists of three parts: the first part calculates the completion time of dependent subtasks; the second part optimizes task offloading based on deep Q learning; and the third part implements load balancing task allocation according to the CH algorithm.

[0010] Optionally, the specific content of the description of task offloading optimization using deep Q learning in S2 is: First, in each training iteration, the task processing delay and the generated energy consumption are taken as the current state , task processing delay is the completion time of the last subtask in the DAG structure, and the processing delay of all UDs is , the energy consumption of each task is the sum of the local execution energy consumption and transmission energy consumption of all subtasks in the current user task. The energy consumption of all UDs is ; Second, actions are selected based on the Q-values ​​learned by Q-learning. The action space is composed of the offloading strategies of each UD. Since the first and last subtasks in the dependent task model must be executed locally, the actions of UDs are expressed as: ,in, X is the uninstallation strategy corresponding to UD; Third, when the established objective function is maximized, the objective function is set as the reward function: ; When the state s(t-1) is greater than the state s(t), the reward value is the target value, otherwise it is -1; Fourth, we use the experience replay mechanism to store the transition process of the agent's interaction with the environment in a replay buffer. Each time, we randomly extract a small batch of samples from the buffer to train Q learning. By introducing target Q learning, we can make Q learning more efficient.

[0011] Optionally, the specific content of the load balancing mechanism based on the CH algorithm in S2 is: The hash space, , mapped to the ring, and then mapped the MEC server and tasks to the hash ring. Each MEC server node corresponds to multiple virtual nodes, which are distributed in different positions on the hash ring.

[0012] Optionally, the complete process of obtaining DTO-DQN-CH in S2 is as follows: First, it receives information about dependent tasks from UDs and initializes the dependent task model built using DAG. When the BS / SC receives a task offloading request from the UDs, the DTO-DQN-CH starts trial-and-error training of the DQN and finds the optimal offloading strategy that reduces processing latency and energy consumption. After obtaining the optimal offloading strategy, the offloading strategy information is sent from the BS / SC to the UDs. The UDs offload some subtasks to the MEC server according to the offloading strategy. The MEC server executes the offloaded subtasks and transmits the execution results back to the UDs.

[0013] Optionally, MD-TSDDQN uses DQN to learn the optimal offloading of dependent tasks, and DTO-GA-CH uses genetic algorithm GA to find the optimal offloading of dependent tasks.

[0014] It can be seen from the above technical solution that compared with the prior art, the present invention discloses a multi-user dependent task offloading method based on deep Q learning and CH algorithm, which has the following beneficial effects: For a single BS / SC, multi-server architecture, and under the premise of satisfying dependent task constraints, deep Q learning and the CH algorithm are combined to optimize the offloading strategy of dependent tasks and the load of MEC servers, thereby minimizing energy consumption and latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0016] Figure 1 Flowchart of the multi-user dependent task offloading method based on deep Q learning and CH algorithm provided by the present invention; Figure 2 This is a system model diagram of a single BS / SC and multiple MEC servers provided by the present invention; Figure 3 A framework diagram of related task offloading and dynamic load balancing in edge computing provided by the present invention; Figure 4 The DQN-based task offloading optimization diagram provided by the present invention; Figure 5 Select the MEC server node diagram for the task provided by the present invention; Figure 6 The DTO-DQN-CH flow chart provided by the present invention; Figure 7 A comparison chart of load ratios provided by the present invention; Figure 8 A comparison chart of average task processing delays with increasing numbers of UDs provided by the present invention; Figure 9 A comparison chart of total energy consumption with increasing number of UDs provided by the present invention; Figure 10 A comparison chart of the average task processing delay with increasing number of MECs provided by the present invention; Figure 11 A comparison chart of the total energy consumption provided by the present invention as the number of MECs increases. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0018] See also Figure 1 and Figure 2 As shown, the present invention discloses a multi-user dependent task offloading method based on deep Q learning and CH algorithm, comprising the following steps: S1. Determine the single BS / SC, multi-server network model and determine the communication conditions and calculation methods. Then use DAG to establish a dependent task model to complete the MEC system modeling. S2. Establish the objective function and propose the DTO-DQN-CH hybrid optimization algorithm. Describe the task offloading optimization using deep Q learning and introduce the load balancing mechanism based on the CH algorithm. Finally, derive the complete DTO-DQN-CH process. S3. Set three simulation tasks, take MD-TSDDQN and DTO-GA-CH as control groups, and evaluate the load balancing effect between MECs, the number of UDs, and the impact of the number of MECs on task offloading optimization.

[0019] Furthermore, in S1, a single BS / SC and multiple servers are determined as the network model, and the communication conditions and calculation methods are determined. Then, a dependent task model is established using DAG. The specific contents of completing the MEC system modeling are as follows: The first and last UD subtasks are executed locally, and the communication delay between the BS / SC and the server and the transmission delay of the task calculation results are not included in the calculation. Under the condition that the UD task is passed to the MEC for processing, the transmission rate expression of any UD is: ; in, , They represent the channel bandwidth, the transmit power from UD to BS / SC, the channel gain of the wireless propagation channel from UD to BS / SC, the distance from UD to BS / SC, the path loss index, and the noise power spectral density respectively; According to the offloading strategy, the subtask will be executed locally or on the MEC server, which will generate two types of delay and energy consumption. The corresponding computing model is established, namely local computing and edge computing, using UD i To distinguish different UDs, where i=1, 2, 3..., UD i No. j The subtask is executed locally Indicates that it is executed on the MEC server using express; when When , the task processing delay and the energy consumption required by UD to execute the task are generated. The calculation formula for the processing delay of the corresponding subtask is: ,in, For the j The number of CPU cycles required for each subtask, UD i The computing power of the energy consumption is calculated as follows: ; when When , the task transmission delay and the processing delay of the MEC server executing the task are generated. The transmission delay calculation formula is: ,in, is the data volume of the corresponding subtask, UD i The transmission rate; the processing delay calculation formula is: , UD is the computing resource allocated to the corresponding subtask. i No. j The transmission energy consumption calculation formula of each subtask is: ,in, UD i Transmit power to BS / SC, is the transmission delay; Then use DAG to build a dependent task model, To express the j The predecessor subtask of a subtask, then j The expression for the transmission completion time of each subtask is: , is the local preparation time of the corresponding subtask, Sure, is the processing delay of the corresponding subtask; in, For the corresponding local running subtask k The completion time of the predecessor subtask, The subtask executed on the MEC server j The first of the predecessor subtasks k The completion time of the predecessor subtask, j Subtask transmission completion time: , executed on the MEC server j The execution completion time of each subtask is: , To run the j Preparation time for each subtask, To handle delay, is the transmission delay; Finally, run the j The preparation time for each subtask is: , complete the MEC system modeling.

[0020] Furthermore, the specific content of establishing the objective function in S2 is: Under the premise of not violating the maximum tolerable delay of UDs, an objective function is established: ; satisfy ,in , is the weight between delay and energy consumption, is the total task processing delay, is the total energy consumption, is the total delay of task processing, UD i The sum of the energy consumption of local execution subtasks, is the total transmission energy consumption, UD i The sum of the local processing delays of all subtasks, UD i The sum of the local processing energy consumption of all subtasks.

[0021] Furthermore, the specific content of the DTO-DQN-CH hybrid optimization algorithm proposed in S2 is: First, we give an overview of the DTO-DQN-CH hybrid algorithm. We divide the MEC architecture with multiple UDs, a single BS / SC, and multiple MEC servers into two layers: the edge layer and the user layer. At the edge layer, the BS / SC operation consists of three parts: the first part calculates the completion time of dependent subtasks; the second part optimizes task offloading based on deep Q learning; and the third part implements load balancing task allocation according to the CH algorithm.

[0022] See also Figure 3 As shown, Figure 3 The upper half is the edge layer, and the lower half is the user layer.

[0023] For further information, see Figure 4 As shown, The specific content of S2 describing the task offloading optimization using deep Q learning is as follows: First, in each training iteration, the task processing delay and the generated energy consumption are taken as the current state , task processing delay is the completion time of the last subtask in the DAG structure, and the processing delay of all UDs is , the energy consumption of each task is the sum of the local execution energy consumption and transmission energy consumption of all subtasks in the current user task. The energy consumption of all UDs is ; Second, actions are selected based on the Q-values ​​learned by Q-learning. The action space is composed of the offloading strategies of each UD. Since the first and last subtasks in the dependent task model must be executed locally, the actions of UDs are expressed as: ,in, X is the uninstallation strategy corresponding to UD; Specifically, in action selection, an improved combination of greedy strategy and Boltzmann search strategy is adopted. - Greedy strategy, ,in, The Q value of each action calculated by Q learning, is the probability of each action.

[0024] Third, when the established objective function is maximized, the objective function is set as the reward function: ; When the state s(t-1) is greater than the state s(t), the reward value is the target value, otherwise it is -1; Specifically, when the established objective function is maximized, the selected behavior obtains more rewards.

[0025] Fourth, we use the experience replay mechanism to store the transition process of the agent's interaction with the environment in a replay buffer. Each time, we randomly extract a small batch of samples from the buffer to train Q learning. By introducing target Q learning, we can make Q learning more efficient.

[0026] Specifically, the target Q learning generates the label y of transition k as: , represents the Q value of each action in the action set A, is the weight of target Q learning.

[0027] The DQN agent trains the Q network by randomly selecting a small set of samples (s(t), a(t), r(t), s(t+1)) from the experience replay buffer and minimizing the loss function To train the target Q value of the Q network, the DQN agent trains the Q network by randomly selecting a small set of samples (s(t), a(t), r(t), s(t+1)) in the experience replay buffer and minimizing the loss function To train the target Q value of the Q network, the loss function is: ,in, For a small batch of samples.

[0028] Furthermore, the specific content of the load balancing mechanism based on the CH algorithm in S2 is: The hash space, , mapped to the ring, and then mapped the MEC server and tasks to the hash ring. Each MEC server node corresponds to multiple virtual nodes, which are distributed in different positions on the hash ring.

[0029] Specifically, the MEC server and virtual node are numbered "MEC_m#0" and "MEC_m#Virtual_h," where "0" represents the MEC server number and "h" represents the virtual node number. The hash values ​​of the MEC server and virtual node are calculated using the FjV hash function. The task node is numbered "UD_m#task_j," where m is the UD number and j is the subtask number.

[0030] See also Figure 5 As shown in the figure, an example is given to illustrate how the task node selects the MEC server. First, the MEC physical nodes and virtual nodes are mapped to the hash ring, and the task node is also mapped to the hash ring. Then, the nearest MEC node is determined clockwise based on the location of the task node. Finally, the virtual node is mapped to the physical node, for example .

[0031] The hash value of the MEC server is , the hash value of the MEC server virtual point is , the hash value of the task node is .

[0032] The hash function FjV can be used to map a random number to a fixed number. The location of the hash ring of the MEC server, virtual node, and task node can be determined by Calculation. Among them, jp can represent the node location of MEC server, virtual server and subtask respectively; hash represents The hash value of The hash value and The hash value of .

[0033] Specifically, the advantage of the CH algorithm lies in its load balancing capability and server scalability.

[0034] For further information, see Figure 6 As shown, the complete process of obtaining DTO-DQN-CH in S2 is as follows: First, it receives information about dependent tasks from UDs and initializes the dependent task model built using DAG. When the BS / SC receives a task offloading request from the UDs, the DTO-DQN-CH starts trial-and-error training of the DQN and finds the optimal offloading strategy that reduces processing latency and energy consumption. After obtaining the optimal offloading strategy, the offloading strategy information is sent from the BS / SC to the UDs. The UDs offload some subtasks to the MEC server according to the offloading strategy. The MEC server executes the offloaded subtasks and transmits the execution results back to the UDs.

[0035] Furthermore, MD-TSDDQN adopts DQN to learn the optimal offloading of dependent tasks, and DTO-GA-CH adopts genetic algorithm GA to find the optimal offloading of dependent tasks.

[0036] Specifically, MD-TSDDQN adopts DQN to learn the optimal offloading of dependent tasks, compared to DTO-DQN-C which ignores the load balancing among multiple MECs.

[0037] Specifically, set the following parameter values: UD i The transmission power is 0.2W, the noise power spectrum density is 174dBm / Hz, and the UD i The distance from the BS / SC is [100,200]m, the path loss exponent is 4, the channel bandwidth is 10MHz, and the UD i The computing power of the network is [0.5, 1.0] GHz, and the computing power of MEC is 8 GHz.

[0038] The following tasks are set: ① To evaluate the load balancing effect between MECs, set up 3 MEC servers and 5 UDs. The load ratio represents the ratio of the offloaded subtasks on each MEC server to the total number of offloaded subtasks on all MEC servers, such as Figure 7 As shown in Figure 2, it can be seen that DTO-DQN-CH distributes the offloaded subtasks to the three MEC servers in the most balanced way.

[0039] ② To evaluate the impact of the number of UDs on task offloading optimization, the number of UDs is increased from 4 to 7, and the number of MEC servers is 4. The average task processing delay and total energy consumption on UDs are compared, see Figure 8 、 Figure 9 ③ To evaluate the impact of the number of MECs on task offloading optimization, the number of MEC servers is increased from 3 to 6, and the number of UDs is 5. The average task processing delay and total energy consumption of the three on UDs are compared. Figure 10 、 Figure 11 As shown in the figure, DTO-DQN-CH all show good performance.

[0040] In summary, the proposal of DTO-DQN-CH is necessary for the optimization of multi-server and multi-user dependent task offloading and is effective in terms of performance.

[0041] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0042] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-user dependent task offloading method based on deep Q learning and CH algorithm, characterized by: The following steps are involved: S1. Determine the single BS / SC, multi-server network model and determine the communication conditions and calculation methods. Then use DAG to establish a dependent task model to complete the MEC system modeling. S2. Establish the objective function and propose the DTO-DQN-CH hybrid optimization algorithm. Describe the task offloading optimization using deep Q learning and introduce the load balancing mechanism based on the CH algorithm. Finally, derive the complete DTO-DQN-CH process. S3. Set three simulation tasks, take MD-TSDDQN and DTO-GA-CH as control groups, and evaluate the load balancing effect between MECs, the number of UDs, and the impact of the number of MECs on task offloading optimization.

2. The multi-user dependent task offloading method based on deep Q learning and CH algorithm according to claim 1 is characterized in that: In S1, a single BS / SC and multiple servers network model is used, and the communication conditions and calculation methods are determined. Then, a dependent task model is established using DAG. The specific contents of completing the MEC system modeling are as follows: The first and last UD subtasks are executed locally, and the communication delay between the BS / SC and the server and the transmission delay of the task calculation results are not included in the calculation. Under the condition that the UD task is passed to the MEC for processing, the transmission rate expression of any UD is: ； in, , They represent the channel bandwidth, the transmit power from UD to BS / SC, the channel gain of the wireless propagation channel from UD to BS / SC, the distance from UD to BS / SC, the path loss index, and the noise power spectral density respectively; According to the offloading strategy, the subtask will be executed locally or on the MEC server, which will generate two types of delay and energy consumption. The corresponding computing model is established, namely local computing and edge computing, using UD i To distinguish different UDs, where i=1, 2, 3..., UD i No. j The subtask is executed locally Indicates that it is executed on the MEC server using express; when When , the task processing delay and the energy consumption required by UD to execute the task are generated. The calculation formula for the processing delay of the corresponding subtask is: ,in, For the j The number of CPU cycles required for each subtask, UD i The computing power of the energy consumption is calculated as follows: ； when When , the task transmission delay and the processing delay of the MEC server executing the task are generated. The transmission delay calculation formula is: ,in, is the data volume of the corresponding subtask, UD i The transmission rate; the processing delay calculation formula is: , UD is the computing resource allocated to the corresponding subtask. i No. j The transmission energy consumption calculation formula of each subtask is: ,in, UD i Transmit power to BS / SC, is the transmission delay; Then use DAG to build a dependent task model, To express the j The predecessor subtask of a subtask, then j The expression for the transmission completion time of each subtask is: , is the local preparation time of the corresponding subtask, Sure, is the processing delay of the corresponding subtask; in, For the corresponding local running subtask k The completion time of the predecessor subtask, The subtask executed on the MEC server j The first of the predecessor subtasks k The completion time of the predecessor subtask, j Subtask transmission completion time: , executed on the MEC server j The execution completion time of each subtask is: , To run the j Preparation time for each subtask, To handle delay, is the transmission delay; Finally, run the j The preparation time for each subtask is: , complete the MEC system modeling.

3. The multi-user dependent task offloading method based on deep Q learning and CH algorithm according to claim 1 is characterized in that: The specific content of establishing the objective function in S2 is: Under the premise of not violating the maximum tolerable delay of UDs, an objective function is established: ； satisfy ,in , is the weight between delay and energy consumption, is the total task processing delay, is the total energy consumption, is the total delay of task processing, UD i The sum of the energy consumption of local execution subtasks, is the total transmission energy consumption, UD i The sum of the local processing delays of all subtasks, UD i The sum of the local processing energy consumption of all subtasks.

4. The multi-user dependent task offloading method based on deep Q learning and CH algorithm according to claim 1 is characterized in that: The specific content of the DTO-DQN-CH hybrid optimization algorithm proposed in S2 is: First, we give an overview of the DTO-DQN-CH hybrid algorithm. We divide the MEC architecture with multiple UDs, a single BS / SC, and multiple MEC servers into two layers: the edge layer and the user layer. At the edge layer, the BS / SC operation consists of three parts: the first part calculates the completion time of dependent subtasks; the second part optimizes task offloading based on deep Q learning; and the third part implements load balancing task allocation according to the CH algorithm.

5. The multi-user dependent task offloading method based on deep Q learning and CH algorithm according to claim 1 is characterized in that: The specific content of S2 describing the task offloading optimization using deep Q learning is as follows: First, in each training iteration, the task processing delay and the generated energy consumption are taken as the current state , task processing delay is the completion time of the last subtask in the DAG structure, and the processing delay of all UDs is , the energy consumption of each task is the sum of the local execution energy consumption and transmission energy consumption of all subtasks in the current user task. The energy consumption of all UDs is ; Second, actions are selected based on the Q-values ​​learned by Q-learning. The action space is composed of the offloading strategies of each UD. Since the first and last subtasks in the dependent task model must be executed locally, the actions of UDs are expressed as: ,in, X is the uninstallation strategy corresponding to UD; Third, when the established objective function is maximized, the objective function is set as the reward function: ; When the state s(t-1) is greater than the state s(t), the reward value is the target value, otherwise it is -1; Fourth, we use the experience replay mechanism to store the transition process of the agent's interaction with the environment in a replay buffer. Each time, we randomly extract a small batch of samples from the buffer to train Q learning. By introducing target Q learning, we can make Q learning more efficient.

6. The multi-user dependent task offloading method based on deep Q learning and CH algorithm according to claim 1 is characterized in that: The specific content of the load balancing mechanism based on the CH algorithm in S2 is: The hash space, , mapped to the ring, and then mapped the MEC server and tasks to the hash ring. Each MEC server node corresponds to multiple virtual nodes, which are distributed in different positions on the hash ring.

7. The multi-user dependent task offloading method based on deep Q learning and CH algorithm according to claim 1 is characterized in that: The specific content of the complete process of obtaining DTO-DQN-CH in S2 is: First, it receives information about dependent tasks from UDs and initializes the dependent task model built using DAG. When the BS / SC receives a task offloading request from the UDs, the DTO-DQN-CH starts trial-and-error training of the DQN and finds the optimal offloading strategy that reduces processing latency and energy consumption. After obtaining the optimal offloading strategy, the offloading strategy information is sent from the BS / SC to the UDs. The UDs offload some subtasks to the MEC server according to the offloading strategy. The MEC server executes the offloaded subtasks and transmits the execution results back to the UDs.

8. The multi-user dependent task offloading method based on deep Q learning and CH algorithm according to claim 1 is characterized in that: MD-TSDDQN uses DQN to learn the optimal offloading of dependent tasks, and DTO-GA-CH uses genetic algorithm GA to find the optimal offloading of dependent tasks.

Images