A Multi-User Dependency Task Unloading Method Based on Deep Q-Learning and CH Algorithm

Abstract

This invention discloses a multi-user dependent task offloading method based on deep Q-learning and the CH algorithm, belonging to the field of task offloading technology. It includes: determining a single BS / SC and multiple servers as the network model and defining communication conditions and computation methods; then using a DAG to establish a dependent task model to complete MEC system modeling; establishing an objective function; proposing a DTO-DQN-CH hybrid optimization algorithm; describing the task offloading optimization using deep Q-learning; introducing a load balancing mechanism based on the CH algorithm; and finally obtaining the complete DTO-DQN-CH process; setting three simulation tasks, using MD-TSDDQN and DTO-GA-CH as control groups, to evaluate the load balancing effect between MECs, the number of UDs, and the impact of the number of MECs on task offloading optimization. This invention can achieve the goal of minimizing energy consumption and latency.

Description

Technical Field

[0001] This invention relates to the field of task offloading technology, and more specifically to a multi-user dependent task offloading method based on deep Q-learning and CH algorithm. Background Technology

[0002] Dependency offloading is a core enabling technology for edge computing, the foundation for achieving low-latency services, and makes up for the shortcomings of centralized processing in cloud computing. It is also deeply integrated with multi-access edge computing (MEC).

[0003] While relying on task offloading can effectively reduce task processing latency, it also places higher demands on the offloading strategy. Therefore, greater emphasis should be placed on optimizing the offloading strategy and addressing load balancing issues on the MEC server.

[0004] Therefore, proposing a multi-user dependency task offloading method based on deep Q-learning and CH algorithm to solve the difficulties of existing technologies is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a multi-user dependency task unloading method based on deep Q-learning and CH algorithm to solve the technical problems existing in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A multi-user dependency task offloading method based on deep Q-learning and CH algorithm includes the following steps:

[0008] S1. Determine the network model as a single BS / SC and multiple servers, and determine the communication conditions and computing methods. Then, use DAG to establish a dependent task model to complete the MEC system modeling.

[0009] S2. Establish the objective function, propose the DTO-DQN-CH hybrid optimization algorithm, describe the task offloading optimization using deep Q-learning, introduce the load balancing mechanism based on the CH algorithm, and finally obtain the complete process of DTO-DQN-CH.

[0010] S3. Set up three simulation tasks, using MD-TSDDQN and DTO-GA-CH as control groups, to evaluate the load balancing effect between MECs, the number of UDs, and the impact of the number of MECs on task offloading optimization.

[0011] Optionally, in S1, a single BS / SC and multiple servers are selected as the network model, and the communication conditions and computation methods are determined. Then, a dependent task model is established using DAG. The specific content of completing the MEC system modeling is as follows:

[0012] If the first and last subtasks of UD are executed locally, and the latency of communication between BS / SC and the server, as well as the latency of transmitting the returned task calculation results, are not included in the calculation, then the transmission rate expression for any UD is as follows, provided that the UD task is transmitted to MEC for processing:

[0013] ;

[0014] in, , These represent the channel bandwidth, the transmit power from UD to BS / SC, the channel gain of the wireless propagation channel from UD to BS / SC, the distance from UD to BS / SC, the path loss exponent, and the noise power spectral density, respectively.

[0015] Based on the offloading strategy, subtasks may execute locally or on the MEC server, resulting in two types of latency and energy consumption. Corresponding computational models are established: local computing and edge computing, using UD... i To distinguish different UDs, where i = 1, 2, 3..., UD i The j Each subtask is executed locally. This indicates that execution on the MEC server will use... express;

[0016] when At that time, the processing latency of the generated task and the energy consumption required for UD to execute the task are calculated. The processing latency of the corresponding subtask is calculated as follows: ,in, For the first j The number of CPU cycles required for each subtask For UD i The computing power and energy consumption are calculated as follows:

[0017] ;

[0018] when At that time, task transmission delay and MEC server task execution processing delay are generated. The transmission delay calculation formula is: ,in, This represents the data volume for the corresponding subtask. For UD i The transmission rate; the processing delay calculation formula is: , UD allocates computing resources to the corresponding subtasks i The j The formula for calculating the transmission energy consumption of each sub-task is as follows: ,in, For UD i Transmit power to BS / SC For transmission delay;

[0019] Then use DAG to build a dependency task model, and use To indicate the first j The predecessor subtask of each subtask, then j The expression for the completion time of each subtask transmission is: , The local preparation time for the corresponding subtask is determined by... Sure, This refers to the processing delay of the corresponding subtask;

[0020] in, For the corresponding locally running subtask's first k Completion time of each precursor sub-task For the subtask executed on the MEC server j The first of the preceding subtasks k The completion time of the first precursor sub-task, the first j Subtask transmission completion time: The first execution on the MEC server j The completion time for each subtask is:

[0021] , To run the first on the server j Preparation time for each sub-task To handle latency, For transmission delay;

[0022] Finally, run the first one on the server. j The preparation time for each sub-task is:

[0023] Complete the MEC system modeling.

[0024] Optionally, the specific content of establishing the objective function in S2 is as follows:

[0025] Without violating the maximum tolerable delay of UDs, establish an objective function:

[0026] ;

[0027] satisfy ,in , As the weight between latency and energy consumption, For the overall task processing delay, Total energy consumption, The total latency for task processing. For UD iThe sum of energy consumption for executing subtasks locally. For total transmission energy consumption, For UD i The sum of the local processing latency of all subtasks. For UD i The sum of local processing energy consumption of all subtasks.

[0028] Optionally, the specific details of the DTO-DQN-CH hybrid optimization algorithm proposed in S2 are as follows:

[0029] First, the DTO-DQN-CH hybrid algorithm is described in general. The MEC architecture with multiple UDs, a single BS / SC, and multiple MEC servers is divided into two layers: the edge layer and the user layer. In the edge layer, the operation of the BS / SC consists of three parts: the first part is to calculate the completion time of dependent subtasks; the second part is to optimize task offloading based on deep Q-learning; and the third part is to implement load balancing task allocation according to the CH algorithm.

[0030] Optionally, the specific details of the task offloading optimization for applying deep Q-learning in S2 are as follows:

[0031] First, in each training iteration, the task processing latency and energy consumption are taken as the current state. Task processing latency This is the completion time of the last subtask in the DAG structure, and the processing latency of all UDs is... Energy consumption of each task The sum of the local execution energy consumption and transmission energy consumption of all subtasks in the current user task, and the energy consumption of all UDs is: ;

[0032] Second, actions are selected based on the Q-value learned through Q-learning; the action space is composed of the unloading strategy of each UD. Since the first and last subtasks in the dependency task model must be executed locally, the actions of UDs are represented as follows: ,in, X The corresponding uninstallation strategy for UD;

[0033] Third, when the established objective function is maximized, the objective function is set as the reward function: ;

[0034] The reward is the target value when the state s(t-1) is greater than the state s(t); otherwise, it is -1.

[0035] Fourth, an experience replay mechanism is used to store the transition process of agent-environment interaction in a replay buffer. Each time, a small batch of samples is randomly selected from the buffer to train Q-learning, introducing target Q-learning to make Q-learning more effective.

[0036] Optionally, the specific details of the load balancing mechanism based on the CH algorithm in S2 are as follows:

[0037] hash space, The data is mapped onto a ring, and then the MEC servers and tasks are mapped onto a hash ring. Each MEC server node corresponds to multiple virtual nodes, which are distributed at different positions on the hash ring.

[0038] Optionally, the complete process for obtaining DTO-DQN-CH in S2 is as follows:

[0039] First, it receives dependency task information from UDs and initializes the dependency task model built with DAG. When BS / SC receives a task offloading request from UDs, DTO-DQN-CH starts trial and error training of DQN and finds the optimal offloading strategy to reduce processing latency and energy consumption.

[0040] After obtaining the optimal unloading strategy, the BS / SC sends the unloading strategy information to the UDs. The UDs unload some subtasks to the MEC server according to the unloading strategy. The MEC server executes the unloaded subtasks and sends the execution results back to the UDs.

[0041] Optionally, MD-TSDDQN uses DQN to learn the optimal unloading of dependent tasks, while DTO-GA-CH uses the genetic algorithm GA to find the optimal unloading of dependent tasks.

[0042] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a multi-user dependency task offloading method based on deep Q-learning and CH algorithm, the beneficial effects of which are:

[0043] For a single BS / SC multi-server architecture, and under the premise of satisfying dependent task constraints, deep Q-learning and CH algorithm are combined to optimize the offloading strategy of dependent tasks and the load of MEC server, thereby achieving the goal of minimizing energy consumption and latency. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0045] Figure 1 A flowchart of the multi-user dependency task unloading method based on deep Q-learning and CH algorithm provided by the present invention;

[0046] Figure 2 This is a system model diagram of a single BS / SC and multiple MEC servers provided by the present invention;

[0047] Figure 3 This invention provides a framework diagram for task offloading and dynamic load balancing in edge computing.

[0048] Figure 4 This invention provides a task offloading optimization diagram based on DQN.

[0049] Figure 5 The present invention provides a task selection MEC server node diagram;

[0050] Figure 6 The DTO-DQN-CH flowchart provided for this invention;

[0051] Figure 7 The load ratio comparison chart provided for this invention;

[0052] Figure 8 A comparison chart of average task processing latency for the increased number of UDs provided by the present invention;

[0053] Figure 9 A comparison chart of the total energy consumption of the increased number of UDs provided by this invention;

[0054] Figure 10 A comparison chart showing the average task processing latency increased by the number of MECs provided by this invention;

[0055] Figure 11 This is a comparison chart showing the total energy consumption as the number of MECs increases, as provided by the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] See Figure 1 and Figure 2 As shown, this invention discloses a multi-user dependency task offloading method based on deep Q-learning and the CH algorithm, including the following steps:

[0058] S1. Determine the network model as a single BS / SC and multiple servers, and determine the communication conditions and computing methods. Then, use DAG to establish a dependent task model to complete the MEC system modeling.

[0059] S2. Establish the objective function, propose the DTO-DQN-CH hybrid optimization algorithm, describe the task offloading optimization using deep Q-learning, introduce the load balancing mechanism based on the CH algorithm, and finally obtain the complete process of DTO-DQN-CH.

[0060] S3. Set up three simulation tasks, using MD-TSDDQN and DTO-GA-CH as control groups, to evaluate the load balancing effect between MECs, the number of UDs, and the impact of the number of MECs on task offloading optimization.

[0061] Furthermore, in S1, a single BS / SC and multiple servers are selected as the network model, and the communication conditions and computation methods are determined. Then, a dependent task model is established using DAG. The specific content of the MEC system modeling is as follows:

[0062] If the first and last subtasks of UD are executed locally, and the latency of communication between BS / SC and the server, as well as the latency of transmitting the returned task calculation results, are not included in the calculation, then the transmission rate expression for any UD is as follows, provided that the UD task is transmitted to MEC for processing:

[0063] ;

[0064] in, , These represent the channel bandwidth, the transmit power from UD to BS / SC, the channel gain of the wireless propagation channel from UD to BS / SC, the distance from UD to BS / SC, the path loss exponent, and the noise power spectral density, respectively.

[0065] Based on the offloading strategy, subtasks may execute locally or on the MEC server, resulting in two types of latency and energy consumption. Corresponding computational models are established: local computing and edge computing, using UD... i To distinguish different UDs, where i = 1, 2, 3..., UD i The j Each subtask is executed locally. This indicates that execution on the MEC server will use... express;

[0066] when At that time, the processing latency of the generated task and the energy consumption required for UD to execute the task are calculated. The processing latency of the corresponding subtask is calculated as follows: ,in, For the first jThe number of CPU cycles required for each subtask For UD i The computing power and energy consumption are calculated as follows:

[0067] ;

[0068] when At that time, task transmission delay and MEC server task execution processing delay are generated. The transmission delay calculation formula is: ,in, This represents the data volume for the corresponding subtask. For UD i The transmission rate; the processing delay calculation formula is: , UD allocates computing resources to the corresponding subtasks i The j The formula for calculating the transmission energy consumption of each sub-task is as follows: ,in, For UD i Transmit power to BS / SC For transmission delay;

[0069] Then use DAG to build a dependency task model, and use To indicate the first j The predecessor subtask of each subtask, then j The expression for the completion time of each subtask transmission is: , The local preparation time for the corresponding subtask is determined by... Sure, This refers to the processing delay of the corresponding subtask;

[0070] in, For the corresponding locally running subtask's first k The completion time of each precursor sub-task. For the subtask executed on the MEC server j The first of the preceding subtasks k The completion time of the first precursor sub-task, the first j Subtask transmission completion time: The first execution on the MEC server j The completion time for each subtask is:

[0071] , To run the first on the server j Preparation time for each sub-task To handle latency, For transmission delay;

[0072] Finally, run the first one on the server. j The preparation time for each sub-task is:

[0073] Complete the MEC system modeling.

[0074] Furthermore, the specific details of establishing the objective function in S2 are as follows:

[0075] Without violating the maximum tolerable delay of UDs, establish an objective function:

[0076] ;

[0077] satisfy ,in , As the weight between latency and energy consumption, For the overall task processing delay, Total energy consumption, The total latency for task processing. For UD i The sum of energy consumption for executing subtasks locally. For total transmission energy consumption, For UD i The sum of the local processing latency of all subtasks. For UD i The sum of local processing energy consumption of all subtasks.

[0078] Furthermore, the specific details of the DTO-DQN-CH hybrid optimization algorithm proposed in S2 are as follows:

[0079] First, the DTO-DQN-CH hybrid algorithm is described in general. The MEC architecture with multiple UDs, a single BS / SC, and multiple MEC servers is divided into two layers: the edge layer and the user layer. In the edge layer, the operation of the BS / SC consists of three parts: the first part is to calculate the completion time of dependent subtasks; the second part is to optimize task offloading based on deep Q-learning; and the third part is to implement load balancing task allocation according to the CH algorithm.

[0080] See Figure 3 As shown, Figure 3 The upper part is the edge layer, and the lower part is the user layer.

[0081] Further, see Figure 4 As shown,

[0082] The specific details of the task offloading optimization for applying deep Q-learning in S2 are as follows:

[0083] First, in each training iteration, the task processing latency and energy consumption are taken as the current state. Task processing latency This is the completion time of the last subtask in the DAG structure, and the processing latency of all UDs is... Energy consumption of each task The sum of the local execution energy consumption and transmission energy consumption of all subtasks in the current user task, and the energy consumption of all UDs is: ;

[0084] Second, actions are selected based on the Q-value learned through Q-learning; the action space is composed of the unloading strategy of each UD. Since the first and last subtasks in the dependency task model must be executed locally, the actions of UDs are represented as follows: ,in, X The corresponding uninstallation strategy for UD;

[0085] Specifically, an improved approach combining a greedy strategy and a Boltzmann search strategy is adopted in the action selection process. - Greedy strategy ,in, The Q-value for each action is calculated for Q-learning. The probability of each action.

[0086] Third, when the established objective function is maximized, the objective function is set as the reward function: ;

[0087] The reward is the target value when the state s(t-1) is greater than the state s(t); otherwise, it is -1.

[0088] Specifically, the chosen behavior yields greater rewards when the established objective function is maximized.

[0089] Fourth, an experience replay mechanism is used to store the transition process of agent-environment interaction in a replay buffer. Each time, a small batch of samples is randomly selected from the buffer to train Q-learning, introducing target Q-learning to make Q-learning more effective.

[0090] Specifically, the label y of the transition k learned by the target Q is expressed as:

[0091] , The Q-value represents the value of each action in action set A. These are the weights for learning the target Q.

[0092] The DQN agent trains the Q-network by randomly selecting a small set of samples (s(t), a(t), r(t), s(t+1)) from the experience replay buffer, and minimizes the loss function. To train the Q-network, the DQN agent trains the Q-network by randomly selecting a small set of samples (s(t), a(t), r(t), s(t+1)) from the experience replay buffer and minimizing the loss function. The target Q-value and loss function for training the Q-network are: ,in, This is a small batch of samples.

[0093] Furthermore, the specific details of the load balancing mechanism based on the CH algorithm in S2 are as follows:

[0094] hash space, The data is mapped onto a ring, and then the MEC servers and tasks are mapped onto a hash ring. Each MEC server node corresponds to multiple virtual nodes, which are distributed at different positions on the hash ring.

[0095] Specifically, the MEC server and virtual node are numbered "MEC_m#0" and "MEC_m#Virtual_h", where "0" represents the MEC server number and "h" represents the virtual node number. The hash values ​​of the MEC server and virtual node are calculated using the FjV hash function. The task node number is "UD_m#task_j", where m is the UD number and j is the subtask number.

[0096] See Figure 5 The following example illustrates how a task node selects a MEC server. First, MEC physical and virtual nodes are mapped to a hash ring, and the task node is also mapped to the hash ring. Then, the nearest MEC node is determined clockwise based on the task node's position. Finally, the found virtual node is mapped to a physical node, for example... .

[0097] The hash value of the MEC server is The hash value of the MEC server virtual point is The hash value of the task node is .

[0098] A hash function FjV can be used to map random numbers to a fixed number. The location of the hash ring of MEC servers, virtual nodes, and task nodes can be determined by... Calculation. Here, jp can represent the node location of the MEC server, virtual server, and subtask, respectively; hash represents... hash value, hash value and The hash value.

[0099] Specifically, the advantages of the CH algorithm lie in its load balancing capabilities and server scalability.

[0100] Further, see Figure 6 As shown, the complete process of obtaining DTO-DQN-CH in S2 is as follows:

[0101] First, it receives dependency task information from UDs and initializes the dependency task model built with DAG. When BS / SC receives a task offloading request from UDs, DTO-DQN-CH starts trial and error training of DQN and finds the optimal offloading strategy to reduce processing latency and energy consumption.

[0102] After obtaining the optimal unloading strategy, the BS / SC sends the unloading strategy information to the UDs. The UDs unload some subtasks to the MEC server according to the unloading strategy. The MEC server executes the unloaded subtasks and sends the execution results back to the UDs.

[0103] Furthermore, MD-TSDDQN uses DQN to learn the optimal unloading of dependent tasks, while DTO-GA-CH uses the genetic algorithm GA to find the optimal unloading of dependent tasks.

[0104] Specifically, MD-TSDDQN uses DQN to learn the optimal offloading of dependent tasks, while DTO-DQN-C ignores load balancing among multiple MECs.

[0105] Specifically, set the following parameter values: UD i The transmission power is 0.2W, the noise power spectral density is 174dBm / Hz, and the UD... i The distance to BS / SC is [100, 200] m, the path loss exponent is 4, the channel bandwidth is 10 MHz, and UD i The computing power of the [0.5, 1.0] GHz is [0.5, 1.0] GHz, while the computing power of the MEC is 8 GHz.

[0106] Set up the following tasks: ① To evaluate the load balancing effect among MECs, configure 3 MEC servers and 5 UDs. The load ratio represents the proportion of offloaded subtasks on each MEC server relative to the total number of offloaded subtasks across all MEC servers, such as... Figure 7 As shown, DTO-DQN-CH distributes the offload subtasks across the three MEC servers in the most balanced manner.

[0107] ② To evaluate the impact of the number of UDs on task offloading optimization, the number of UDs was increased sequentially from 4 to 7, with a MEC server count of 4. Average task processing latency and total energy consumption on the UDs were compared; see [link to relevant documentation]. Figure 8 , Figure 9As shown. ③ To evaluate the impact of the number of MECs on task offloading optimization, the number of MEC servers was increased from 3 to 6, and the number of UDs was 5. See [link to comparison of average task processing latency and total energy consumption on UDs]. Figure 10 , Figure 11 As shown, both DTO-DQN-CH exhibit good performance.

[0108] In summary, the proposal of DTO-DQN-CH is necessary and effective in terms of performance for optimizing the unloading of multi-server, multi-user dependent tasks.

[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0110] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-user dependency task offloading method based on deep Q-learning and CH algorithm, characterized in that, Includes the following steps: S1. Determine the network model as a single BS / SC and multiple servers, and determine the communication conditions and computing methods. Then, use DAG to establish a dependent task model to complete the MEC system modeling. S2. Establish the objective function, propose the DTO-DQN-CH hybrid optimization algorithm, describe the task offloading optimization using deep Q learning, introduce the load balancing mechanism based on the CH algorithm, and finally obtain the complete process of DTO-DQN-CH. S3. Set up three simulation tasks, using MD-TSDDQN and DTO-GA-CH as control groups, to evaluate the load balancing effect among MECs, the number of UDs, and the impact of the number of MECs on task offloading optimization. The specific details of the DTO-DQN-CH hybrid optimization algorithm proposed in S2 are as follows: First, the DTO-DQN-CH hybrid algorithm is described in general. The MEC architecture with multiple UDs, a single BS / SC, and multiple MEC servers is divided into two layers: the edge layer and the user layer. In the edge layer, the operation of the BS / SC consists of three parts: the first part is to calculate the completion time of dependent subtasks; the second part is to optimize task offloading based on deep Q-learning; and the third part is to implement load balancing task allocation according to the CH algorithm. The specific details of the load balancing mechanism based on the CH algorithm in S2 are as follows: hash space, The data is mapped onto a ring, and then the MEC servers and tasks are mapped onto a hash ring. Each MEC server node corresponds to multiple virtual nodes, which are distributed at different positions on the hash ring.

2. The multi-user dependency task offloading method based on deep Q-learning and CH algorithm according to claim 1, characterized in that, In S1, a single BS / SC and multiple servers are determined as the network model, and the communication conditions and computing methods are defined. Then, a dependent task model is established using DAG. The specific content of the MEC system modeling is as follows: If the first and last subtasks of UD are executed locally, and the latency of communication between BS / SC and the server, as well as the latency of transmitting the returned task calculation results, are not included in the calculation, then the transmission rate expression for any UD, assuming the UD task is transmitted to MEC for processing, is as follows: ； in, , These represent the channel bandwidth, the transmit power from UD to BS / SC, the channel gain of the wireless propagation channel from UD to BS / SC, the distance from UD to BS / SC, the path loss exponent, and the noise power spectral density, respectively. Based on the offloading strategy, subtasks may execute locally or on the MEC server, resulting in two types of latency and energy consumption. Corresponding computational models are established: local computing and edge computing, using UD... i To distinguish different UDs, where i = 1, 2, 3..., UD i The j Each subtask is executed locally. This indicates that execution on the MEC server uses... express; when At that time, the processing latency of the generated task and the energy consumption required for UD to execute the task are calculated. The processing latency of the corresponding subtask is calculated as follows: ,in, For the first j The number of CPU cycles required for each subtask For UD i The computing power and energy consumption are calculated as follows: ； when At that time, task transmission delay and MEC server task execution processing delay are generated. The transmission delay calculation formula is: ,in, This represents the data volume for the corresponding subtask. For UD i The transmission rate; the processing delay calculation formula is: , UD allocates computing resources to the corresponding subtasks i The j The formula for calculating the transmission energy consumption of each sub-task is as follows: ,in, For UD i Transmit power to BS / SC For transmission delay; Then use DAG to build a dependency task model, and use To indicate the first j The predecessor subtask of each subtask, then j The expression for the completion time of each subtask transmission is: , The local preparation time for the corresponding subtask is determined by... Sure, This refers to the processing delay of the corresponding subtask; in, For the corresponding locally running subtask's first k The completion time of each precursor sub-task. For the subtask executed on the MEC server j The first of the preceding subtasks k The completion time of the first precursor sub-task, the first j Subtask transmission completion time: The first execution on the MEC server j The completion time for each subtask is: , To run the first on the server j Preparation time for each sub-task To handle latency, For transmission delay; Finally, run the first one on the server. j The preparation time for each sub-task is: Complete the MEC system modeling.

3. The multi-user dependency task offloading method based on deep Q-learning and CH algorithm according to claim 1, characterized in that, The specific details of establishing the objective function in S2 are as follows: Without violating the maximum tolerable delay of UDs, establish an objective function: ； satisfy ,in , As the weight between latency and energy consumption, For the overall task processing delay, Total energy consumption, The total latency for task processing. For UD i The sum of energy consumption for executing subtasks locally. For total transmission energy consumption, For UD i The sum of the local processing latency of all subtasks. For UD i The sum of the local processing energy consumption of all subtasks.

4. The multi-user dependency task offloading method based on deep Q-learning and CH algorithm according to claim 1, characterized in that, The specific details of the task offloading optimization for applying deep Q-learning in S2 are as follows: First, in each training iteration, the task processing latency and energy consumption are taken as the current state. Task processing latency This is the completion time of the last subtask in the DAG structure, and the processing latency of all UDs is... Energy consumption of each task The sum of the local execution energy consumption and transmission energy consumption of all subtasks in the current user task, and the energy consumption of all UDs is: ; Second, select actions based on the Q-value learned through Q-learning; The action space is composed of the unloading strategy of each UD. Since the first and last subtasks in the dependency task model must be executed locally, the actions of UDs are represented as follows: ,in, X The corresponding uninstallation strategy for UD; Third, when the established objective function is maximized, the objective function is set as the reward function: ; The reward is the target value when the state s(t-1) is greater than the state s(t); otherwise, it is -1. Fourth, an experience replay mechanism is used to store the transition process of agent-environment interaction in a replay buffer. Each time, a small batch of samples is randomly selected from the buffer to train Q-learning, introducing target Q-learning to make Q-learning more effective.

5. The multi-user dependency task offloading method based on deep Q-learning and CH algorithm according to claim 1, characterized in that, The complete process for obtaining DTO-DQN-CH in S2 is as follows: First, it receives dependency task information from UDs and initializes the dependency task model built with DAG. When BS / SC receives a task offloading request from UDs, DTO-DQN-CH starts trial and error training of DQN and finds the optimal offloading strategy to reduce processing latency and energy consumption. After obtaining the optimal unloading strategy, the BS / SC sends the unloading strategy information to the UDs. The UDs unload some subtasks to the MEC server according to the unloading strategy. The MEC server executes the unloaded subtasks and sends the execution results back to the UDs.

6. The multi-user dependency task offloading method based on deep Q-learning and CH algorithm according to claim 1, characterized in that, MD-TSDDQN uses DQN to learn the optimal unloading of dependent tasks, while DTO-GA-CH uses the genetic algorithm GA to find the optimal unloading of dependent tasks.

Images