An active migration type offloading method and system for AGV dependent computing tasks

Abstract

The application provides an active migration type unloading method and system for AGV dependent computing tasks, comprising: constructing an AGV task unloading model based on a VEC system; dividing the received task into subtasks by the AGV to obtain a task sequential unloading process; dividing the state space and action space of the AGV task unloading model based on the task sequential unloading process; calculating the time delay to obtain the overall energy consumption of the task unloading; establishing an active migration type unloading scheme to update the state of the unloading task and the obtained result in the current time slot and the next time slot; defining the overall energy consumption of the task unloading as an optimization problem and converting the optimization problem into a finite time domain time-varying Markov decision problem; solving to obtain the minimum overall energy consumption required for completing the task unloading, and completing the active migration type unloading for the AGV dependent computing task. The application can realize efficient and energy-saving service migration and computing unloading in the vehicle-mounted edge computing network.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of vehicle edge computing, and particularly relates to an active migration type unloading method and system for AGV dependent computing tasks. BACKGROUND

[0002] In the field of high-end equipment cable manufacturing, logistics automation and intelligentization are the key to improving production efficiency and reducing costs. In response to the call of the state to promote the digital transformation of manufacturing, it is necessary to carry out digital transformation and upgrading of old factories. This transformation is not only the key to improving production efficiency and reducing costs, but also the only way to realize the modernization of the field of high-end equipment cable manufacturing. In order to ensure real-time production process control in the complex manufacturing logistics scene of high-end equipment cable, it is necessary to provide ultra-high reliability and millimeter-level latency for the automatic guided vehicle (AGV) executing tasks. High complexity real-time computing often requires explosive computing resources and strict time delay, but the computing power of AGV is limited and cannot meet these requirements. Therefore, vehicle edge computing (VEC) is proposed to solve the above problems by deploying computing services near AGV to greatly reduce the delay.

[0003] Most of the current research makes two assumptions. First, all base stations are required to be deployed with backhaul. However, when carrying out digital transformation and upgrading based on traditional factories, there are challenges in installing fiber backhaul links. Especially in old factories, the transformation of infrastructure is often limited by physical space and existing structure, making the deployment of fiber backhaul links complex and costly. Second, in the actual production process of industrial workshops, the generated work tasks are often not independent subtasks, and the dependency between tasks exists in most actual scene businesses, where the output of some subtasks is the input of other subtasks. However, traditional service migration methods generally only focus on whether to migrate and ignore the dependency between tasks. In addition, due to the short wave path of 5G signals, especially in the millimeter wave frequency band (24GHz-86GHz), it is easily blocked by physical obstacles, which can cause signal quality to decline or even signal interruption. Considering the mobility of AGV, intermittent wireless connection can cause the failure of computing offloading and migration, therefore, there is an urgent need for an active migration type unloading method and system for AGV dependent computing tasks, which enables AGV to independently decide when to offload tasks and obtain intermediate results based on statistical information of vehicle motion. SUMMARY

[0004] Considering the influence of intermittent computing and communication, the present application provides an active migration type unloading method and system for AGV dependent computing tasks to realize efficient and energy-saving service migration and computing offloading in vehicle edge computing networks, which is used to improve the energy efficiency of edge computing networks and promote the development of edge computing.

[0005] To achieve the above object, the application provides the following scheme:

[0006] An active migration type unloading method for AGV dependent computing tasks, comprising the following steps:

[0007] Based on a VEC system, an AGV task unloading model is constructed; wherein the VEC system comprises a plurality of base stations equipped with VEC servers;

[0008] The AGV divides the received task into subtasks, obtains a task sequential unloading process according to the dependency relationship between the subtasks, and establishes a subtask unloading index based on the task sequential unloading process;

[0009] The position of the AGV is mapped to the corresponding base station, and the state space and action space of the AGV task unloading model are divided based on the task sequential unloading process and the subtask unloading index;

[0010] Based on the state space and action space of the AGV task unloading model, the migration delay, unloading delay, computing delay and result acquisition delay are calculated, and the total energy consumption of task unloading is obtained;

[0011] An active migration type unloading scheme is established, and the state update of unloading tasks and result acquisition in the current time slot and the next time slot is carried out based on the migration delay, unloading delay, computing delay and result acquisition delay;

[0012] The total energy consumption of task unloading is defined as an optimization problem, and the optimization problem is converted into a finite time-varying Markov decision problem of joint unloading and migration optimization;

[0013] The finite time-varying Markov decision problem is solved by a deep double Q network algorithm based on a Markov decision process combined with the state update, the minimum total energy consumption required for task unloading is obtained, and the active migration type unloading for AGV dependent computing tasks is completed.

[0014] Preferably, the dependency relationship between the subtasks indicates that the output of a previous subtask is the input of a subsequent subtask;

[0015] The mth unloaded subtask is represented as:

[0016] Wherein, L m represents the size of the subtask d m , α m represents the computing intensity of CPU cycles required for processing the task, Q w (m) represents the updated intermediate result after the completion of the subtask d m .

[0017] Qw (m) is represented as:

[0018] where w i is the output size of each subtask d i , succ(d i ) is the set of all successor tasks in the task topology graph that depend on the output of d i .

[0019] Preferably, the state space is defined as S = Φ x M x K,

[0020] where Φ = {1, 2,..., N} x {1, 2,..., N} is the set of base stations that the AGV is connected to in the last two time slots, M = {1, 2,..., M} is the set of tasks to be offloaded, and K = {0, 1, 2,..., M} is the set of the latest acquired task results, with 0 indicating that no task result has been acquired before the current time slot;

[0021] The action space is defined as A = {0, 1, 2}, representing the set of possible actions, with the offloading and result acquisition at each time slot t defined as action a t , a t = 0 indicating that the AGV is in standby state, a t = 1 indicating that the AGV offloads tasks without acquiring results in the current time slot, and a t = 2 indicating that the AGV offloads tasks and acquires intermediate results in the current time slot.

[0022] Preferably, the state is represented by a composite state s t = (n t , m t , k t ) e S when updating the state of the offloading task and the result acquisition of the current time slot and the next time slot, where the substate n t = (n t , n t-1 ) represents the indices of the base stations to which the AGV is connected at time slots t and t-1, m t represents the index of the task to be offloaded, and k t represents the index of the latest acquired task result at time slot t.

[0023] Preferably, the active migration offloading scheme established includes:

[0024] Determining whether the AGV has a base station switching at the current time slot t, and if so, performing migration of the intermediate result Q w (m) at the new base station and updating the index m t = k t+ 1, and make a decision on whether to obtain the intermediate result or not;

[0025] The AGV decides whether to unload the task in the current time slot t, if yes, the task is unloaded and calculated, and a timeout judgment is made based on a preset first time limit, if it is decided not to unload the task or the timeout, the task is recalculated in the next time slot, and k t+1 = m t , until k t+1 = M, the calculation of M tasks is completed; if it is decided to unload the task and there is no timeout, k t+1 = k t is updated, m t+1 = m t + 1, and a decision is made on whether to obtain the intermediate result or not;

[0026] If it is decided to obtain the intermediate result, a timeout judgment is made based on a preset second time limit during the obtaining process, if there is no timeout, k t+1 = m t is updated, until k t+1 = M, the calculation of M tasks is completed; if it is decided not to obtain the intermediate result or the timeout, k t+1 = k t is kept, until k t+1 = M, the calculation of M tasks is completed.

[0027] Preferably, the finite-horizon time-varying Markov decision problem is to minimize the expected cumulative communication energy consumption during T time slots.

[0028] Preferably, the method for obtaining the minimum total energy consumption required for task offloading completion is:

[0029] Solving step 1: initialize the parameters of the deep double Q network of the Markov decision process, record the initial state s∈S and the initial time slot t = 1; wherein the initialization includes setting the initial weights of the target network and the main network of the deep double Q network to be the same, and initializing the experience replay buffer;

[0030] Solving step 2: starting from the current state s t , an action a t is selected by using an ε-greedy strategy, after execution, a new state s t+1 and a reward R are observed, an experience tuple (s t , a t , R, s t+1 ) is stored in the experience replay buffer, and the current state is updated to s t+1 ;

[0031] Solving step 3: sampling from the experience replay buffer, and using the target network to calculate the target Q value of the sampled experience;

[0032] Solving step 4: training the main network based on the sampling experience and the target Q value, and updating the target network;

[0033] Solving step 5: repeating solving steps 2-4 until the expected cumulative communication energy consumption during T time slots is minimum, and the minimum total energy consumption required for task offloading is obtained.

[0034] The application also provides an active migration offloading system for AGV-dependent computing tasks, which is used to implement the active migration offloading method, and comprises:

[0035] A task offloading model construction module is configured to construct an AGV task offloading model based on a VEC system, wherein the VEC system comprises a plurality of base stations equipped with VEC servers.

[0036] An index establishment module is configured to divide the received task into subtasks by the AGV, obtain a task sequential offloading process according to the dependency relationship between the subtasks, and establish a subtask offloading index based on the task sequential offloading process.

[0037] A model space division module is configured to map the position of the AGV to a corresponding base station, divide the state space and action space of the AGV task offloading model based on the task sequential offloading process and the subtask offloading index.

[0038] An energy consumption calculation module is configured to calculate the migration delay, offloading delay, computing delay and result acquisition delay based on the state space and action space of the AGV task offloading model, and obtain the total energy consumption of task offloading.

[0039] A state updating module is configured to establish an active migration offloading scheme, and perform state updating of offloading tasks and result acquisition in the current time slot and the next time slot based on the migration delay, offloading delay, computing delay and result acquisition delay.

[0040] A problem transformation module is configured to define the total energy consumption of task offloading as an optimization problem, and transform the optimization problem into a finite-time Markov decision problem of joint offloading and migration optimization.

[0041] A problem solving module is configured to solve the finite-time Markov decision problem by combining the state updating based on the deep double Q network algorithm of the Markov decision process, obtain the minimum total energy consumption required for task offloading, and complete the active migration offloading for AGV-dependent computing tasks.

[0042] Preferably, in the index establishment module, the dependency relationship between the subtasks indicates that the output of a previous subtask is the input of a subsequent subtask.

[0043] The mth offloaded subtask is represented as:

[0044] wherein, L m represents the size of the subtask d m , alpha m represents the computational intensity of CPU cycles required to process the task, Q w (m) represents the intermediate result updated after the completion of the subtask d m .

[0045] Q w (m) is represented as:

[0046] wherein, w i represents the output size of each subtask d i , succ(d i ) represents the set of all successor tasks in the task topology graph that depend on the output of d i .

[0047] Compared with the prior art, the beneficial effects of the present application are: the present application can realize efficient offloading and migration under the consideration of the influence of intermittent calculation and communication. The optimization problem is converted into a time-varying Markov decision problem in a limited time domain, and the time of offloading the task and obtaining the result is determined with the optimization goal of minimizing the total energy consumption of completing all tasks. The deep double Q network (DDQN) algorithm based on MDP is adopted to solve the energy minimization problem, and the real-time offloading and obtaining strategy is obtained, so as to realize efficient and energy-saving service migration and calculation offloading. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the embodiments are briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to these drawings without creative labor for those skilled in the art.

[0049] Figure 1 The flow chart of the active migration offloading method for AGV-dependent computing tasks in the first embodiment of the present application;

[0050] Figure 2 The AGV task offloading model in the first embodiment of the present application;

[0051] Figure 3 The active migration offloading scheme flow chart of the active migration offloading method for AGV-dependent computing tasks in the first embodiment of the present application;

[0052] Figure 4 The application scenario schematic diagram in the second embodiment of the present application. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Example 1

[0056] like Figure 1 As shown in Figure 1, an active migration offloading method for AGV-dependent computing tasks can be used to migrate intermediate results using AGV connections for various application scenarios such as smart factories. It includes the following steps:

[0057] S1: Based on the VEC system, an AGV task offloading model is constructed; wherein the VEC system includes several base stations equipped with VEC servers; in this embodiment, Figure 2 As shown in the figure, the VEC system where the mobile AGV is located contains N base stations. Each base station is equipped with a VEC server, which can provide edge computing. There is no overlapping coverage area between the base stations. The AGV can only be connected to one base station during movement.

[0058] S2: AGV divides the received task into subtasks, obtains the task sequence unloading process according to the dependency relationship between subtasks, and establishes the subtask unloading index [1, 2, ..., M] based on the task sequence unloading process.

[0059] A further embodiment is that the dependency relationship between subtasks means that the output of a previous subtask is the input of a subsequent subtask;

[0060] The mth offloaded subtask is represented as:

[0061] Among them, L m Represents subtask d m The size of α m Indicates the computational intensity of the CPU cycles required to process the task, Q w (m) represents subtask d m Intermediate results updated upon completion;

[0062] Q w (m) is expressed as:

[0063] where w i denotes the output size of each subtask d i , and succ(d i ) denotes the set of all successor tasks in the task topology graph that depend on the output of d i .

[0064] S3: Map the position of AGV to the corresponding base station, based on the task order unloading process and the subtask unloading index, divide the state space and action space of the AGV task unloading model.

[0065] Further embodiments are that the state space is defined as S = Φ × M × K,

[0066] where Φ = {1, 2,..., N} × {1, 2,..., N} is the set of base stations that AGV is connected to in the last two time slots, M = {1, 2,..., M} is the set of tasks to be unloaded, and K = {0, 1, 2,..., M} is the set of recently obtained task results, 0 indicates that no task result is obtained before the current time slot; the state space refers to the set of all states, each state in the state space can be represented by a composite state. The state is the state updated in step S5.

[0067] The action space is defined as A = {0, 1, 2}, which represents the set of possible actions, the unloading and result obtaining at each time slot t is defined as action a t , a t = 0 indicates that the AGV is in standby state, a t = 1 indicates that the AGV unloads the task in the current time slot without obtaining the result, and a t = 2 indicates that the AGV unloads the task and obtains the intermediate result in the current time slot.

[0068] S4: Based on the state space and action space of the AGV task unloading model, calculate the migration delay, unloading delay, calculation delay, and result obtaining delay to obtain the total energy consumption of task unloading;

[0069] The unloading delay calculation process is as follows:

[0070] The unloading delay caused by unloading task m to the edge server is:

[0071]

[0072] where, denotes the uplink transmission rate of AGV to base station n at time slot t, and the calculation formula is:

[0073]

[0074] where, Bu denotes the uplink bandwidth, P u denotes the transmit power of the vehicle, N0denotes the noise power, H n (t) denotes the channel gain power between the AGV and the base station n, and is calculated as:

[0075] H n (t) = ψ t g t Ad n (t) -γ

[0076] where ψ t denotes the minimum scale fast fading power component, which is assumed to be exponentially distributed with unit mean, g t denotes the lognormal shadowing with standard deviation, A denotes the path loss constant, d n (t) denotes the distance between the AGV and the base station n at time slot t, and γ denotes the attenuation exponent. d n (t) is calculated as:

[0077]

[0078] where r t denotes the position of the AGV at time slot t, z n denotes the position of the connected base station n, h v and h B denote the antenna heights of the AGV and the base station n, respectively.

[0079] The delay calculation process is as follows:

[0080] The computation delay of the task successfully offloaded to the base station is:

[0081]

[0082] where f c (n) denotes the CPU cycle frequency available to the VEC server for task processing;

[0083] The delay calculation process for obtaining the result is as follows:

[0084] The delay caused by obtaining the intermediate result of the computation completed is:

[0085]

[0086] where B denotes the downlink transmission rate of the base station n to the AGV at time slot t, and is calculated as:

[0087]

[0088] where Bd denotes the downlink bandwidth, P B denotes the transmit power of the base station.

[0089] The calculation process of the migration delay is as follows:

[0090] If there is a migration of the result, the migration delay caused by the uplink of the intermediate result is denoted as:

[0091]

[0092] The calculation process of the total energy consumption of task offloading is as follows:

[0093] The instantaneous energy communication cost calculation formula of the action at state s t is as follows:

[0094] E t (s,a)=P u ·min(tm t +tu t ,T0)+P r ·min(td t ,max(T0-tm t -tu t -tc t ,0))

[0095] Step S5: Establish an active migration offloading scheme, and update the states of the offloading tasks and the acquisition of results in the current time slot and the next time slot based on the migration delay, the offloading delay, the calculation delay, and the delay of acquiring the results; specifically, in order to avoid invalid offloading caused by a timeout fault, according to the active migration offloading scheme, a decision is made for each time slot to determine the time of offloading tasks and the acquisition of results. As shown in the following table. Figure 3

[0096] When updating the states of the offloading tasks and the acquisition of results in the current time slot and the next time slot, the state is represented by a composite state s t =(n t ,m t ,k t )∈S, wherein the substate n t =(n t ,n t-1 ) represents the index of the base station connected to the AGV at time slots t and t-1, m t represents the index of the task to be offloaded, and k t represents the index of the latest acquired task result at time slot t.

[0097] Further embodiments are that the established active migration offloading scheme includes:

[0098] ​determines whether the AGV has a base station switching at the current time slot t, if yes, i.e. n t-1 ≠ n t , the migration of the intermediate result Q w (m) is performed at the new base station, and the task index m t to be offloaded is updated as k t +1, and the offloading decision is made; if no, the offloading decision is directly made;

[0099] The AGV determines whether to offload the task at the current time slot t, if yes, the task is offloaded and calculated, and a timeout judgment is made based on a preset first time limit, if it is decided not to offload the task or timeout, the task is recalculated at the next time slot, and m t+1 is updated as m t , until k t+1 =M, the calculation of the M tasks is completed, wherein the initial value of k t+1 is 0; if it is decided to offload the task and there is no timeout, k t+1 is updated as k t , and m t+1 is updated as m t +1, and a decision is made whether to obtain the intermediate result;

[0100] If it is decided to obtain the intermediate result, a timeout judgment is made based on a preset second time limit during the obtaining process, if there is no timeout, k t+1 is updated as m t , until k t+1 =M, the calculation of the M tasks is completed; if it is decided not to obtain the intermediate result or timeout, k t+1 is kept as k t , until k t+1 =M, the calculation of the M tasks is completed.

[0101] S6: define the total energy consumption of task offloading as an optimization problem, and convert the optimization problem into a finite time-varying Markov decision problem of joint offloading and migration optimization;

[0102] Further embodiments are that the finite time-varying Markov decision problem is to minimize the expected cumulative communication energy consumption during T time slots, the problem jointly optimizes offloading and migration, and the goal is to minimize the total energy consumption for completing all tasks.

[0103] The defined problem of minimizing the expected cumulative communication energy consumption during T time slots can be expressed as:

[0104]

[0105] wherein π t is a time-dependent deterministic policy, and E(·) represents the average value of all possible cumulative energy consumptions in a random process.

[0106] Furthermore, since no more energy will be consumed once the final task is obtained, the final state will not be transferred accordingly. Therefore, we use E t ′(s,π t (s)) to replace E t (s,π t (s)) is used as the energy consumption expression in the defined optimization problem, E t ′(s,π t (s)) can be expressed as:

[0107]

[0108] Among them, S end Represents a collection of end states.

[0109] S7: By combining a deep double-Q network algorithm based on the Markov decision process with state updates, we solve the finite-time time-varying Markov decision problem, obtain the minimum overall energy consumption required to complete task offloading, and complete active migration offloading for AGV-dependent computing tasks.

[0110] A further implementation method is to obtain the minimum overall energy consumption required to complete task offloading as follows:

[0111] Solution step 1: Initialize the parameters of the deep double-Q network of the Markov decision process, record the initial state s∈S and the initial time slot t=1; the initialization includes setting the initial weights of the target network and the main network of the deep double-Q network to be the same, and initializing the experience replay buffer;

[0112] Solution step 2: From the current state s t Start, use ε-greedy strategy to select action a t , observe the new state s after execution t+1 With reward R, the experience tuple (s t ,a t ,R,s t+1 ) is stored in the experience replay buffer and the current state is updated to s t+1 ;

[0113] Solution step 3: Sample from the experience replay buffer and use the target network to calculate the target Q value of the sampled experience to stabilize the learning process.

[0114] Solution Step 4: Train the main network based on the sampled experience and the target Q value, and update the target network. Specifically, the main network is trained using the sampled experience and the target Q value, and the strategy is optimized by minimizing the difference between the predicted and target Q values. The weights of the main network are periodically copied to the target network to maintain the stability of the target network.

[0115] Solving step 5: loop solving steps 2-4 until the expected cumulative communication energy consumption during T time slots is minimum, and the minimum total energy consumption required for task offloading is obtained.

[0116] Embodiment two

[0117] Reference Figure 4 , step S1: this example takes an AGV car in a straight line motion at a constant speed in an industrial park as an example, there are N base stations on both sides of the AGV car movement road, the road is L meters long, there are T m time slots, the AGV speed is v, there are M sub-tasks, the size of each sub-task is L m , the required computing intensity is a m , and the size of the sub-task result output is w m .

[0118] Specifically, the following steps are implemented:

[0119] Step S2: the AGV receives the task, divides the task into several sub-tasks, and determines the sequential offloading process according to the dependency relationship between the tasks.

[0120] Step S3: map the position of the AGV to the corresponding base station to reduce the state dimension, and define the state space of the system as s t =(n t ,m t ,k t )∈S, and the action space as a t ∈A={0,1,2}.

[0121] Step S4: calculate the task offloading delay tu t (s,a), the task computing delay tc t (s,a), the intermediate result acquisition delay td t (s,a) and the result migration delay tm t (s,a) and the total energy consumption E t (s,a).

[0122] Step S5: according to the active migration offloading scheme, determine the task index m t+1 of the next time slot offloading and the index k t+1 of the task for obtaining the result.

[0123] Specifically, step S5 includes:

[0124] Step S51: make a migration judgment, observe whether the AGV has a base station switching in the current time slot, and judge whether n t+1 is equal to n t .

[0125] Step S52: making an offloading decision, the system decides whether to offload the task at time slot t, updating m t+1 ;

[0126] Step S53: after successful offloading, deciding whether to obtain intermediate results, updating k t+1 .

[0127] Step S6: defining an optimization problem and converting the optimization problem into a time-varying Markov decision problem jointly optimizing offloading and migration, the goal being to minimize the energy consumption required to complete the task;

[0128] Step S7: using a deep double Q network (DDQN) algorithm based on MDP to solve the energy minimization problem;

[0129] Specifically, step S7 includes:

[0130] Step S71: initializing the DDQN algorithm parameters, recording the initial state s e S and the initial time slot t = 1;

[0131] Step S72: starting from the current state s t , using an e-greedy strategy to select an action a t , observing the new state s t+1 and the reward R after execution, then storing the experience tuple (s t , a t , R, s t+1 ) in the experience replay buffer, and updating the current state to s t+1 ;

[0132] Step S73: sampling from the experience replay buffer, calculating the target Q value;

[0133] Step S74: training the main network, periodically updating the target network;

[0134] Step S75: repeating steps S62 to S64 until the expected cumulative communication energy consumption during T time slots is minimized.

[0135] Embodiment three

[0136] The application also provides an active migration offloading system for AGV-dependent computing tasks, for implementing the active migration offloading method, comprising:

[0137] A task offloading model construction module is configured to construct an AGV task offloading model based on a VEC system, wherein the VEC system comprises a plurality of base stations equipped with VEC servers;

[0138] The index establishing module is configured to divide the received task into subtasks, obtain a task sequential offloading process according to a dependency relationship between the subtasks, and establish a subtask offloading index based on the task sequential offloading process.

[0139] The model space dividing module is configured to map a position of the AGV to a corresponding base station, divide a state space and an action space of an AGV task offloading model based on the task sequential offloading process and the subtask offloading index.

[0140] The energy consumption calculating module is configured to calculate a migration delay, an offloading delay, a calculation delay and a result obtaining delay based on the state space and the action space of the AGV task offloading model, and obtain a total energy consumption of the task offloading.

[0141] The state updating module is configured to establish a proactive migration offloading scheme, and perform state updating of offloading tasks and result obtaining in a current time slot and a next time slot based on the migration delay, the offloading delay, the calculation delay and the result obtaining delay.

[0142] The problem converting module is configured to define the total energy consumption of the task offloading as an optimization problem, and convert the optimization problem into a finite time-varying Markov decision problem of joint offloading and migration optimization.

[0143] The problem solving module is configured to solve the finite time-varying Markov decision problem by a deep double Q network algorithm based on a Markov decision process in combination with the state updating, obtain a minimum total energy consumption required for completing the task offloading, and complete the proactive migration offloading for the AGV dependent computing task.

[0144] In a further implementation, in the index establishing module, the dependency relationship between the subtasks indicates that an output of a previous subtask is an input of a next subtask.

[0145] The mth offloaded subtask is represented as:

[0146] wherein, L m represents a size of the subtask d m , a m represents a calculation intensity of a CPU cycle required for processing the task, and Q w (m) represents an intermediate result updated after the subtask d m is completed.

[0147] Q w (m) is represented as:

[0148] wherein, w i represents an output size of each subtask d i , and succ(d i) represents a set containing all successor tasks in the task topology graph that depend on the output of d i

[0149] The above-described embodiments are merely intended to describe the preferred modes of the present application, and are not intended to limit the scope of the present application. Various modifications and improvements of the present application made by those skilled in the art without departing from the design spirit of the present application shall fall within the scope of the present application defined by the claims.​

Claims

1. An active migration offloading method for AGV-dependent computing tasks, characterized in that: The following steps are involved: An AGV task offloading model is constructed based on the VEC system; wherein the VEC system includes several base stations equipped with VEC servers; AGV divides the received task into subtasks, obtains the task sequence unloading process according to the dependency relationship between subtasks, and establishes the subtask unloading index based on the task sequence unloading process; Mapping the position of the AGV to the corresponding base station, and dividing the state space and action space of the AGV task offloading model based on the task sequence offloading process and the subtask offloading index; Based on the state space and action space of the AGV task offloading model, the migration delay, offloading delay, calculation delay and result acquisition delay are calculated to obtain the overall energy consumption of task offloading; Establish an active migration offloading solution to update the status of offloading tasks and results in the current and next time slots based on migration latency, offloading latency, computation latency, and result acquisition latency. The overall energy consumption of task offloading is defined as an optimization problem, and the optimization problem is transformed into a finite-horizon time-varying Markov decision problem of joint offloading and migration optimization; By combining the deep double-Q network algorithm based on the Markov decision process with the state update, the finite-time domain time-varying Markov decision problem is solved, the minimum overall energy consumption required to complete task offloading is obtained, and active migration offloading for AGV-dependent computing tasks is completed.

2. The active migration offloading method for AGV-dependent computing tasks according to claim 1 is characterized in that: The dependency relationship between the subtasks means that the output of the previous subtask is the input of the next subtask; The mth offloaded subtask is represented as: Among them, L m Represents subtask d m The size of α m Indicates the computational intensity of the CPU cycles required to process the task, Q w (m) represents subtask d m Intermediate results updated upon completion; Q w (m) is expressed as: Among them, w i Represents each subtask d i The output size of succ(d i ) indicates that the task topology graph contains the i The set of all successor tasks of the output.

3. The active migration offloading method for AGV-dependent computing tasks according to claim 2 is characterized in that: The state space is defined as S = Φ × M × K, Where Φ = {1, 2, ..., N} × {1, 2, ..., N} is the set of base stations connected to the AGV in the last two time slots, M = {1, 2, ..., M} is the set of tasks to be offloaded, and K = {0, 1, 2, ..., M} is the set of recently obtained task results, where 0 indicates that no task result has been obtained before the current time slot. The action space is defined as A = {0, 1, 2}, which represents the set of possible actions. The unloading and acquisition results at each time slot t are defined as action a t , a t =0 means AGV is in standby state, a t =1 means that AGV unloads the task in the current time slot without obtaining the result, a t =2 means that the AGV both unloads tasks and obtains intermediate results in the current time slot.

4. The active migration offloading method for AGV-dependent computing tasks according to claim 2, characterized in that: When the state of unloading tasks and obtaining results in the current time slot and the next time slot is updated, the state is composed of the composite state s t =(n t ,m t ,k t )∈S represents, where substate n t =(n t ,n t-1 ) represents the index of the base station to which the AGV is connected at time slots t and t-1, m t Indicates the task index to be offloaded, k t Indicates the index of the most recently obtained task result at time slot t.

5. The active migration offloading method for AGV-dependent computing tasks according to claim 4 is characterized in that: The established active migration offloading solution includes: Determine whether the AGV has a base station switch in the current time slot t. If so, execute the intermediate result Q at the new base station w Migrate (m) and update the task index m to be uninstalled t =k t +1, make uninstall decision; if not, make uninstall decision directly; The AGV decides whether to unload the task in the current time slot t. If so, it unloads and calculates the task, and makes a timeout judgment based on the preset first time limit. If it decides not to unload the task or the timeout is exceeded, it recalculates the task in the next time slot and updates m. t+1 =m t , until k t+1 = M, complete the calculation of M tasks; if it is decided to offload the task and there is no timeout, update k t+1 =k t , m t+1 =m t +1, make a decision on whether to obtain the intermediate results; If it is decided to obtain the intermediate result, a timeout judgment is performed based on the preset second time limit during the acquisition process. If there is no timeout, k is updated. t+1 =m t , until k t+1 = M, complete the calculation of M tasks; if it is decided not to obtain the intermediate results or timeout, keep k t+1 =k t , until k t+1 =M, complete the calculation of M tasks.

6. The active migration offloading method for AGV-dependent computing tasks according to claim 1, characterized in that: The finite-horizon time-varying Markov decision problem is to minimize the expected cumulative communication energy consumption during T time slots.

7. The active migration offloading method for AGV-dependent computing tasks according to claim 5, characterized in that: The method to obtain the minimum overall energy consumption required to complete task offloading is: Solution step 1: Initialize the parameters of the deep double-Q network of the Markov decision process, record the initial state s∈S and the initial time slot t=1; the initialization includes setting the initial weights of the target network and the main network of the deep double-Q network to be the same, and initializing the experience replay buffer; Solution step 2: From the current state s t Start, use ε-greedy strategy to select action a t , observe the new state s after execution t+1 With reward R, the experience tuple (s t ,a t ,R,s t+1 ) is stored in the experience replay buffer and the current state is updated to s t+1 ; Solution step 3: Sample from the experience replay buffer and use the target network to calculate the target Q value of the sampled experience; Solution step 4: training the main network based on the sampling experience and the target Q value, and updating the target network; Solution step 5: Loop through steps 2-4 until the expected cumulative communication energy consumption during T time slots is minimized, and obtain the minimum overall energy consumption required to complete task offloading.

8. An active migration offloading system for AGV-dependent computing tasks, used to implement the active migration offloading method according to any one of claims 1 to 7, characterized in that: include: A task offloading model construction module is used to construct an AGV task offloading model based on a VEC system; wherein the VEC system includes a plurality of base stations equipped with VEC servers; The index building module is used for AGV to divide the received tasks into subtasks, obtain the task sequence unloading process according to the dependency relationship between the subtasks, and build the subtask unloading index based on the task sequence unloading process; A model space partitioning module is used to map the position of the AGV to the corresponding base station, and to partition the state space and action space of the AGV task offloading model based on the task sequence offloading process and the subtask offloading index; An energy consumption calculation module is used to calculate the migration delay, offloading delay, calculation delay and result acquisition delay based on the state space and action space of the AGV task offloading model to obtain the overall energy consumption of task offloading; The status update module is used to establish an active migration offloading solution and update the status of offloading tasks and results in the current and next time slots based on the migration delay, offloading delay, calculation delay, and result acquisition delay. a problem conversion module, configured to define the overall energy consumption of the task offloading as an optimization problem, and convert the optimization problem into a finite-horizon time-varying Markov decision problem for joint offloading and migration optimization; The problem-solving module is used to solve the finite-time domain time-varying Markov decision problem by combining the state update with the deep double-Q network algorithm based on the Markov decision process, obtain the minimum overall energy consumption required to complete the task offloading, and complete the active migration offloading for AGV-dependent computing tasks.

9. The active migration offloading system for AGV-dependent computing tasks according to claim 8, It is characterized by: In the index building module, the dependency relationship between the subtasks means that the output of the previous subtask is the input of the next subtask; The mth offloaded subtask is represented as: Among them, L m Represents subtask d m The size of α m Indicates the computational intensity of the CPU cycles required to process the task, Q w (m) represents subtask d m Intermediate results updated upon completion; Q w (m) is expressed as: Among them, w i Represents each subtask d i The output size of succ(d i ) indicates that the task topology graph contains the i The set of all successor tasks of the output.

Images