Active migration type unloading method and system for AGV dependent computing task

Abstract

The invention provides an active migration type unloading method and system for an AGV dependent computing task, and the method comprises the steps: building an AGV task unloading model based on a VEC system; the AGV divides the received tasks into sub-tasks and obtains a task sequential unloading process; dividing a state space and an action space of the AGV task unloading model based on a task sequence unloading process; calculating time delay to obtain total energy consumption of task unloading; establishing an active migration type unloading scheme, and performing unloading tasks of the current time slot and the next time slot and state updating of an obtained result; defining total energy consumption of task unloading as an optimization problem, and converting the optimization problem into a finite time domain time-varying Markov decision problem; and solving to obtain the minimum total energy consumption required by task unloading completion, and completing active migration type unloading oriented to the AGV dependent computing task. According to the invention, efficient and energy-saving service migration and computing unloading in the vehicle-mounted edge computing network can be realized.

Description

Technical Field

[0001] The present invention belongs to the field of vehicle edge computing technology, and specifically relates to an active migration unloading method and system for AGV-dependent computing tasks. Background Art

[0002] In the field of high-end equipment cable manufacturing, logistics automation and intelligence are the key to improving production efficiency and reducing costs. In response to the country's call to promote the digital transformation of the manufacturing industry, it is now necessary to digitally transform and upgrade old factories. This transformation is not only the key to improving production efficiency and reducing costs, but also the only way to achieve modernization in the field of high-end equipment cable manufacturing. In order to ensure real-time production process control in the complex manufacturing logistics scenarios of high-end equipment cables, it is necessary to provide ultra-high reliability and millimeter-level latency for the automatic guided vehicles (AGVs) performing tasks. High-complexity real-time computing often requires explosive computing resources and strict time delays, but the computing power of AGVs is limited and insufficient to meet these requirements. Therefore, vehicle edge computing (VEC) is proposed to solve the above problems, and latency is greatly reduced by deploying computing services near AGVs.

[0003] Most studies at this stage make two assumptions. First, all base stations are required to be deployed with backhaul. However, when digital transformation and upgrading are carried out on the basis of traditional factories, the fiber backhaul link is not easy to install. Especially in old factories, the transformation of infrastructure is often limited by physical space and existing structures, making the deployment of fiber backhaul links complicated and costly. Secondly, in the actual production process of industrial workshops, the generated tasks are often not independent subtasks. The dependencies between tasks exist in most actual scenario businesses, and the outputs of some subtasks are the inputs of other subtasks. However, traditional service migration methods generally only focus on whether to migrate and ignore the dependencies between tasks. In addition, due to the short range of 5G signals, especially the millimeter wave band (24GHz-86GHz), it is easily blocked by physical obstacles, which will lead to signal quality degradation or even signal interruption. At the same time, considering the mobility of AGV, intermittent wireless connections may lead to the failure of computing offloading and migration. Therefore, there is an urgent need for an active migration offloading method and system for AGV-dependent computing tasks, so that AGV can independently decide when to offload tasks online and obtain intermediate results based on the statistical information of vehicle movement. Summary of the invention

[0004] Taking into account the impact of intermittent computing and communication, the present invention provides an active migration unloading method and system for AGV-dependent computing tasks, so as to realize efficient and energy-saving service migration and computing offloading in the vehicle-mounted edge computing network, which is used to improve the energy efficiency of the edge computing network and promote the development of edge computing.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] An active migration offloading method for AGV-dependent computing tasks includes the following steps:

[0007] Based on the VEC system, an AGV task offloading model is constructed; wherein the VEC system includes several base stations equipped with VEC servers;

[0008] AGV divides the received task into subtasks, obtains the task sequence unloading process according to the dependency relationship between subtasks, and establishes the subtask unloading index based on the task sequence unloading process;

[0009] Mapping the position of the AGV to the corresponding base station, and dividing the state space and action space of the AGV task offloading model based on the task sequence offloading process and the subtask offloading index;

[0010] Based on the state space and action space of the AGV task offloading model, the migration delay, offloading delay, calculation delay and result acquisition delay are calculated to obtain the overall energy consumption of task offloading;

[0011] Establish an active migration offloading scheme, and update the status of offloading tasks and results in the current time slot and the next time slot based on the migration delay, offloading delay, calculation delay, and result acquisition delay;

[0012] The overall energy consumption of the task offloading is defined as an optimization problem, and the optimization problem is transformed into a finite-horizon time-varying Markov decision problem of joint offloading and migration optimization;

[0013] By combining the deep double Q network algorithm based on the Markov decision process with the state update, the finite time domain time-varying Markov decision problem is solved, the minimum overall energy consumption required to complete the task offloading is obtained, and the active migration offloading for AGV-dependent computing tasks is completed.

[0014] Preferably, the dependency relationship between the subtasks means that the output of a previous subtask is the input of a subsequent subtask;

[0015] The mth offloaded subtask is represented as:

[0016] Among them, L m Represents subtask d m The size of α m Indicates the computational intensity of the CPU cycles required to process the task, Q w (m) represents subtask d m Intermediate results updated upon completion;

[0017] Qw (m) is expressed as:

[0018] Among them, w i Represents each subtask d i The output size of succ(d i ) indicates that the task topology graph contains the i The set of all successor tasks of the output of .

[0019] Preferably, the state space is defined as S=Φ×M×K,

[0020] Where Φ = {1, 2, ..., N} × {1, 2, ..., N} is the set of base stations connected by the AGV in the last two time slots, M = {1, 2, ..., M} is the set of tasks to be offloaded, K = {0, 1, 2, ..., M} is the set of recently acquired task results, and 0 means that no task result has been acquired before the current time slot;

[0021] The action space is defined as A = {0, 1, 2}, which represents a set of possible actions. The unloading and acquisition results at each time slot t are defined as action a t , a t =0 means AGV is in standby state, a t =1 means that AGV unloads the task in the current time slot without obtaining the result, a t =2 means that the AGV both unloads tasks and obtains intermediate results in the current time slot;

[0022] Preferably, when the status of unloading tasks and obtaining results in the current time slot and the next time slot is updated, the state is composed of the composite state s t =(n t ,m t ,k t )∈S represents, where substate n t =(n t ,n t-1 ) represents the index of the base station to which the AGV is connected at time slots t and t-1, m t Indicates the task index to be uninstalled, k t Indicates the index of the most recently obtained task result at time slot t.

[0023] Preferably, the established active migration unloading solution includes:

[0024] Determine whether the AGV has a base station switch in the current time slot t. If so, execute the intermediate result Q at the new base station w (m) and update the task index m to be uninstalled t =k t+1, make an uninstall decision; if not, make an uninstall decision directly;

[0025] The AGV decides whether to unload the task in the current time slot t. If so, it unloads and calculates the task, and makes a timeout judgment based on the preset first time limit. If it decides not to unload the task or the timeout occurs, it recalculates the task in the next time slot and updates m. t+1 =m t , until k t+1 =M, complete the calculation of M tasks; if it is decided to offload the task and there is no timeout, update k t+1 =k t , m t+1 =m t +1, make a decision on whether to obtain the intermediate results;

[0026] If it is decided to obtain the intermediate result, a timeout judgment is performed based on the preset second time limit during the acquisition process. If there is no timeout, k is updated. t+1 =m t , until k t+1 = M, complete the calculation of M tasks; if it is decided not to obtain the intermediate results or timeout, keep k t+1 =k t , until k t+1 =M, complete the calculation of M tasks.

[0027] Preferably, the finite-time domain time-varying Markov decision problem is to minimize the expected cumulative communication energy consumption during T time slots.

[0028] Preferably, the method for obtaining the minimum overall energy consumption required to complete task offloading is:

[0029] Solution step 1: Initialize the parameters of the deep double Q network of the Markov decision process, record the initial state s∈S and the initial time slot t=1; the initialization includes setting the initial weights of the target network and the main network of the deep double Q network to be the same, and initializing the experience playback buffer;

[0030] Solution step 2: From the current state s t Start, use the ε-greedy strategy to select action a t , observe the new state s after execution t+1 With reward R, the experience tuple (s t ,a t ,R,s t+1 ) is stored in the experience replay buffer and the current state is updated to s t+1 ;

[0031] Solution step 3: Sample from the experience replay buffer and use the target network to calculate the target Q value of the sampled experience;

[0032] Solution step 4: training the main network based on the sampling experience and the target Q value, and updating the target network;

[0033] Solution step 5: Loop through steps 2-4 until the expected cumulative communication energy consumption during T time slots is minimized, and obtain the minimum overall energy consumption required to complete the task offloading.

[0034] The present invention also provides an active migration unloading system for AGV-dependent computing tasks, which is used to implement the active migration unloading method, including:

[0035] A task offloading model building module is used to build an AGV task offloading model based on a VEC system; wherein the VEC system includes a number of base stations equipped with VEC servers;

[0036] An index building module is used for AGV to divide the received tasks into subtasks, obtain the task sequence unloading process according to the dependency relationship between the subtasks, and build the subtask unloading index based on the task sequence unloading process;

[0037] A model space division module, used to map the position of the AGV to the corresponding base station, and divide the state space and action space of the AGV task offloading model based on the task sequence offloading process and the subtask offloading index;

[0038] An energy consumption calculation module is used to calculate the migration delay, unloading delay, calculation delay and result acquisition delay based on the state space and action space of the AGV task unloading model to obtain the overall energy consumption of task unloading;

[0039] A status update module is used to establish an active migration offloading scheme, and update the status of the offloading task and the result of the current time slot and the next time slot based on the migration delay, offloading delay, calculation delay and the delay of obtaining the result;

[0040] A problem conversion module, used for defining the overall energy consumption of the task offloading as an optimization problem, and converting the optimization problem into a finite-horizon time-varying Markov decision problem of joint offloading and migration optimization;

[0041] The problem-solving module is used to solve the finite-time domain time-varying Markov decision problem through a deep double-Q network algorithm based on the Markov decision process combined with the state update, obtain the minimum overall energy consumption required to complete the task offloading, and complete the active migration offloading for AGV-dependent computing tasks.

[0042] Preferably, in the index building module, the dependency relationship between the subtasks indicates that the output of a previous subtask is the input of a subsequent subtask;

[0043] The mth offloaded subtask is represented as:

[0044] Among them, L m Represents subtask d m The size of α m Indicates the computational intensity of the CPU cycles required to process the task, Q w (m) represents subtask d m Intermediate results updated upon completion;

[0045] Q w (m) is expressed as:

[0046] Among them, w i Represents each subtask d i The output size of succ(d i ) indicates that the task topology graph contains the i The set of all successor tasks of the output of .

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention can achieve efficient offloading and migration while taking into account the influence of intermittent computing and communication. The optimization problem is transformed into a time-varying Markov decision problem in a finite time domain, and the time for offloading tasks and obtaining results is determined with the optimization goal of minimizing the total energy consumption of completing all tasks. The MDP-based deep double Q network (DDQN) algorithm is used to solve the energy minimization problem, and a real-time offloading and acquisition strategy is obtained, thereby achieving efficient and energy-saving service migration and computing offloading. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 It is a flow chart of an active migration offloading method for AGV-dependent computing tasks in Embodiment 1 of the present invention;

[0050] Figure 2 The AGV task offloading model in the first embodiment of the present invention;

[0051] Figure 3 It is a flow chart of an active migration unloading solution of an active migration unloading method for AGV-dependent computing tasks in Embodiment 1 of the present invention;

[0052] Figure 4 This is a schematic diagram of an application scenario in the second embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Embodiment 1

[0056] like Figure 1 As shown in FIG, an active migration offloading method for AGV-dependent computing tasks can be used to migrate intermediate results using AGV connections for various application scenarios such as smart factories. The method includes the following steps:

[0057] S1: Based on the VEC system, an AGV task offloading model is constructed; wherein the VEC system includes several base stations equipped with VEC servers; in this embodiment, Figure 2 As shown, the VEC system where the mobile AGV is located includes N base stations. Each base station is equipped with a VEC server, which can provide edge computing. There is no overlapping coverage area between the base stations. The AGV can only be connected to one base station during movement.

[0058] S2: AGV divides the received task into subtasks, obtains the task sequence unloading process according to the dependency relationship between the subtasks, and establishes the subtask unloading index [1,2,...,M] based on the task sequence unloading process.

[0059] A further implementation method is that the dependency relationship between subtasks indicates that the output of a previous subtask is the input of a subsequent subtask;

[0060] The mth offloaded subtask is represented as:

[0061] Among them, L m Represents subtask d m The size of α m Indicates the computational intensity of the CPU cycles required to process the task, Q w (m) represents subtask d m Intermediate results updated upon completion;

[0062] Q w (m) is expressed as:

[0063] Among them, w i Represents each subtask d i The output size of succ(d i ) indicates that the task topology graph contains the i The set of all successor tasks of the output of .

[0064] S3: Map the location of the AGV to the corresponding base station, and divide the state space and action space of the AGV task offloading model based on the task sequence offloading process and the subtask offloading index.

[0065] A further embodiment is that the state space is defined as S=Φ×M×K,

[0066] Wherein, Φ = {1, 2, ..., N} × {1, 2, ..., N} is the set of base stations connected by the AGV in the last two time slots, M = {1, 2, ..., M} is the set of tasks to be unloaded, K = {0, 1, 2, ..., M} is the set of recently acquired task results, 0 means that no task result has been acquired before the current time slot; the state space refers to the set of all states, and each state in the state space can be represented by a composite state. The state is the state updated in step S5.

[0067] The action space is defined as A = {0, 1, 2}, which represents the set of possible actions. The unloading and acquisition results at each time slot t are defined as action a t , a t =0 means AGV is in standby state, a t =1 means that AGV unloads the task in the current time slot without obtaining the result, a t =2 means that the AGV both unloads tasks and obtains intermediate results in the current time slot.

[0068] S4: Based on the state space and action space of the AGV task offloading model, the migration delay, offloading delay, calculation delay, and result acquisition delay are calculated to obtain the overall energy consumption of task offloading;

[0069] The unloading delay calculation process is as follows:

[0070] The offloading delay caused by offloading task m to the edge server is:

[0071]

[0072] in, represents the uplink transmission rate from AGV to base station n at time slot t, and the calculation formula is:

[0073]

[0074] Among them, Bu represents the uplink bandwidth, P u represents the transmission power of the vehicle, N0 represents the noise power, H n (t) represents the channel gain power between AGV and base station n, and its calculation formula is:

[0075] H n (t) = ψ t g t Ad n (t) -γ

[0076] Among them, ψ t represents the minimum scale fast fading power component, which is assumed to be exponentially distributed with unit mean, g t represents the lognormal shading with standard deviation, A represents the path loss constant, d n (t) represents the distance between the AGV and the base station n at time slot t, and γ represents the attenuation index. n The calculation formula of (t) is:

[0077]

[0078] Among them, r t represents the position of the AGV at time slot t, z n represents the location of the connected base station n, h v and h B They represent the antenna heights of AGV and base station n respectively.

[0079] The calculation process of delay is as follows:

[0080] The computational delay of the task successfully offloaded to the base station is:

[0081]

[0082] Among them, f c (n) represents the CPU cycle frequency available for task processing by the VEC server;

[0083] The delay calculation process for obtaining the result is as follows:

[0084] The delay caused by obtaining the intermediate results of the calculation is:

[0085]

[0086] in, represents the downlink transmission rate from base station n to AGV in time slot t, and the calculation formula is:

[0087]

[0088] Among them, Bd represents the downlink bandwidth, P B Indicates the transmit power of the base station.

[0089] The migration delay is calculated as follows:

[0090] If there is migration of results, the migration delay caused by the uplink of intermediate results is expressed as:

[0091]

[0092] The overall energy consumption calculation process of task offloading is as follows:

[0093] In status t The instantaneous energy communication cost calculation formula of the action at is:

[0094] E t (s,a)=P u ·min(tm t +tu t ,T0)+P r ·min(td t ,max(T0-tm t -tu t -tc t ,0))

[0095] Step S5: Establish an active migration offloading scheme, and update the status of offloading tasks and results in the current time slot and the next time slot based on the migration delay, offloading delay, calculation delay, and delay in obtaining results; specifically, in order to avoid invalid offloading caused by timeout failure, make a decision for each time slot according to the active migration offloading scheme to determine the time for offloading tasks and obtaining results. Figure 3 shown.

[0096] When the state of unloading tasks and obtaining results in the current time slot and the next time slot is updated, the state is composed of the composite state s t =(n t ,m t ,k t )∈S represents, where substate n t =(n t ,n t-1 ) represents the index of the base station to which the AGV is connected at time slots t and t-1, m t Indicates the task index to be uninstalled, k t Indicates the index of the most recently obtained task result at time slot t.

[0097] A further implementation method is that the established active migration unloading solution includes:

[0098] Determine whether the AGV has a base station switch in the current time slot t. If so, that is, n t-1 ≠n t , then the intermediate result Q is executed at the new base station w (m) and update the task index m to be uninstalled t =k t +1, make an uninstall decision; if not, make an uninstall decision directly;

[0099] The AGV decides whether to unload the task in the current time slot t. If so, it unloads and calculates the task, and makes a timeout judgment based on the preset first time limit. If it decides not to unload the task or the timeout occurs, it recalculates the task in the next time slot and updates m. t+1 =m t , until k t+1 =M, complete the calculation of M tasks, where k t+1 The initial value of is 0; if it is decided to offload the task and there is no timeout, update k t+1 =k t , m t+1 =m t +1, make a decision on whether to obtain the intermediate results;

[0100] If it is decided to obtain the intermediate result, a timeout judgment is performed based on the preset second time limit during the acquisition process. If there is no timeout, k is updated. t+1 =m t , until k t+1 = M, complete the calculation of M tasks; if it is decided not to obtain the intermediate results or timeout, keep k t+1 =k t , until k t+1 =M, complete the calculation of M tasks.

[0101] S6: The overall energy consumption of task offloading is defined as an optimization problem, and the optimization problem is transformed into a finite-horizon time-varying Markov decision problem for joint offloading and migration optimization;

[0102] A further implementation method is that the finite-horizon time-varying Markov decision problem is to minimize the expected cumulative communication energy consumption during T time slots. The problem jointly optimizes offloading and migration, with the goal of minimizing the total energy consumption of completing all tasks.

[0103] The problem of minimizing the expected cumulative communication energy consumption during T time slots can be expressed as:

[0104]

[0105] Among them, π t is a time-dependent deterministic strategy, and E(·) represents the average value of all possible cumulative energy consumption in the random process.

[0106] Furthermore, since no more energy will be consumed once the final task is obtained, the final state will not be transferred accordingly. Therefore, we use E t ′(s,π t (s)) to replace E t (s,π t (s)) is used as the energy consumption expression in the defined optimization problem, E t ′(s,π t (s)) can be expressed as:

[0107]

[0108] Among them, S end Represents a collection of end states.

[0109] S7: Through the deep double Q network algorithm based on the Markov decision process combined with state update, the finite time domain time-varying Markov decision problem is solved to obtain the minimum overall energy consumption required to complete the task offloading, and complete the active migration offloading for AGV-dependent computing tasks.

[0110] A further implementation method is that the method for obtaining the minimum overall energy consumption required to complete task offloading is:

[0111] Solution step 1: Initialize the parameters of the deep double Q network of the Markov decision process, record the initial state s∈S and the initial time slot t=1; the initialization includes setting the initial weights of the target network and the main network of the deep double Q network to be the same, and initializing the experience playback buffer;

[0112] Solution step 2: From the current state s t Start, use the ε-greedy strategy to select action a t , observe the new state s after execution t+1 With reward R, the experience tuple (s t ,a t ,R,s t+1 ) is stored in the experience replay buffer and the current state is updated to s t+1 ;

[0113] Solution step 3: Sample from the experience replay buffer and use the target network to calculate the target Q value of the sampled experience to stabilize the learning process.

[0114] Solution step 4: Train the main network based on the sampled experience and the target Q value, and update the target network; specifically, train the main network using the sampled experience and its target Q value, and optimize the strategy by minimizing the difference between the predicted and target Q values. Periodically copy the weights of the main network to the target network to maintain the stability of the target network.

[0115] Solution step 5: Loop through steps 2-4 until the expected cumulative communication energy consumption during T time slots is minimized, and obtain the minimum overall energy consumption required to complete the task offloading.

[0116] Embodiment 2

[0117] Reference Figure 4 , Step S1: This example takes an AGV car that moves at a uniform speed in a straight line in an industrial park as an example. There are N base stations on both sides of the AGV car's moving road. The road is L meters long and has a total of T m time slots, the AGV speed is v, there are M subtasks in total, and the size of each subtask is L m , the required computational intensity is α m , the size of the subtask result output is w m .

[0118] Specifically, it is achieved through the following steps:

[0119] Step S2: The AGV receives the task, divides the task into several subtasks, and determines the sequential unloading process according to the dependencies between the tasks.

[0120] Step S3: Map the position of the AGV to the corresponding base station to reduce the state dimension, and define the state space of the system as s t =(n t ,m t ,k t )∈S, the action space is a t ∈A={0,1,2}.

[0121] Step S4: Calculate the offloading task delay tu t (s,a), task calculation delay tc t (s,a), intermediate result acquisition delay td t (s,a) and the resulting migration delay tm t (s, a) and overall energy consumption E t (s,a).

[0122] Step S5: According to the active migration offloading scheme, determine the task index m to be offloaded in the next time slot t+1 , the index k of the task that gets the result t+1 ;

[0123] Specifically, step S5 includes:

[0124] Step S51: Perform migration judgment, observe whether the AGV switches to a base station in the current time slot, and judge whether n t+1 Is it equal to n t ;

[0125] Step S52: Make an unloading decision. The system decides whether to unload the task in time slot t and updates m t+1 ;

[0126] Step S53: After successful unloading, decide whether to obtain the intermediate results and update k t+1 .

[0127] Step S6: define the optimization problem and transform it into a time-varying Markov decision problem of joint offloading and migration optimization, the goal of which is to minimize the energy consumption required to complete the task;

[0128] Step S7: using a deep double Q network (DDQN) algorithm based on MDP to solve the energy minimization problem;

[0129] Specifically, step S7 includes:

[0130] Step S71: Initialize DDQN algorithm parameters, record the initial state s∈S and the initial time slot t=1;

[0131] Step S72: From the current state s t Start, use the ε-greedy strategy to select action a t , observe the new state s after execution t+1 and reward R, then the experience tuple (s t ,a t ,R,s t+1 ) is stored in the experience replay buffer and the current state is updated to s t+1 ;

[0132] Step S73: sampling from the experience playback buffer and calculating the target Q value;

[0133] Step S74: training the main network and regularly updating the target network;

[0134] Step S75: Repeat steps S62 to S64 until the expected cumulative communication energy consumption during T time slots is minimized.

[0135] Embodiment 3

[0136] The present invention also provides an active migration unloading system for AGV-dependent computing tasks, which is used to implement an active migration unloading method, including:

[0137] A task offloading model building module is used to build an AGV task offloading model based on a VEC system; wherein the VEC system includes a number of base stations equipped with VEC servers;

[0138] An index building module is used for AGV to divide the received tasks into subtasks, obtain the task sequence unloading process according to the dependency relationship between the subtasks, and build the subtask unloading index based on the task sequence unloading process;

[0139] The model space partitioning module is used to map the location of the AGV to the corresponding base station, and to partition the state space and action space of the AGV task offloading model based on the task sequence offloading process and the subtask offloading index;

[0140] The energy consumption calculation module is used to calculate the migration delay, unloading delay, calculation delay and result acquisition delay based on the state space and action space of the AGV task offloading model to obtain the overall energy consumption of task offloading;

[0141] A status update module is used to establish an active migration offloading scheme, and update the status of the offloading task and the result of the current time slot and the next time slot based on the migration delay, offloading delay, calculation delay and the delay of obtaining the result;

[0142] A problem conversion module is used to define the overall energy consumption of task offloading as an optimization problem and convert the optimization problem into a finite-horizon time-varying Markov decision problem for joint offloading and migration optimization;

[0143] The problem-solving module is used to solve the finite-time time-varying Markov decision problem through a deep double-Q network algorithm based on the Markov decision process combined with state update, obtain the minimum overall energy consumption required to complete the task offloading, and complete the active migration offloading for AGV-dependent computing tasks.

[0144] A further implementation method is that, in the index building module, the dependency relationship between subtasks indicates that the output of a previous subtask is the input of a subsequent subtask;

[0145] The mth offloaded subtask is represented as:

[0146] Among them, L m Represents subtask d m The size of α m Indicates the computational intensity of the CPU cycles required to process the task, Q w (m) represents subtask d m Intermediate results updated upon completion;

[0147] Q w (m) is expressed as:

[0148] Among them, w i Represents each subtask d i The output size of succ(d i) indicates that the task topology graph contains the i The set of all successor tasks of the output of .

[0149] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. An active migration offloading method for AGV-dependent computing tasks, characterized in that: The following steps are involved: Based on the VEC system, an AGV task offloading model is constructed; wherein the VEC system includes several base stations equipped with VEC servers; AGV divides the received task into subtasks, obtains the task sequence unloading process according to the dependency relationship between subtasks, and establishes the subtask unloading index based on the task sequence unloading process; Mapping the position of the AGV to the corresponding base station, and dividing the state space and action space of the AGV task offloading model based on the task sequence offloading process and the subtask offloading index; Based on the state space and action space of the AGV task offloading model, the migration delay, offloading delay, calculation delay and result acquisition delay are calculated to obtain the overall energy consumption of task offloading; Establish an active migration offloading scheme, and update the status of offloading tasks and results in the current time slot and the next time slot based on the migration delay, offloading delay, calculation delay, and result acquisition delay; The overall energy consumption of the task offloading is defined as an optimization problem, and the optimization problem is transformed into a finite-horizon time-varying Markov decision problem of joint offloading and migration optimization; By combining the deep double Q network algorithm based on the Markov decision process with the state update, the finite time domain time-varying Markov decision problem is solved, the minimum overall energy consumption required to complete the task offloading is obtained, and the active migration offloading for AGV-dependent computing tasks is completed.

2. The active migration offloading method for AGV-dependent computing tasks according to claim 1 is characterized in that: The dependency relationship between the subtasks means that the output of the previous subtask is the input of the next subtask; The mth offloaded subtask is represented as: Among them, L m Represents subtask d m The size of α m Indicates the computational intensity of the CPU cycles required to process the task, Q w (m) represents subtask d m Intermediate results updated upon completion; Q w (m) is expressed as: Among them, w i Represents each subtask d i The output size of succ(d i ) indicates that the task topology graph contains the i The set of all successor tasks of the output of .

3. The active migration offloading method for AGV-dependent computing tasks according to claim 2 is characterized in that: The state space is defined as S = Φ × M × K, Where Φ = {1, 2, ..., N} × {1, 2, ..., N} is the set of base stations connected by the AGV in the last two time slots, M = {1, 2, ..., M} is the set of tasks to be offloaded, K = {0, 1, 2, ..., M} is the set of recently acquired task results, and 0 means that no task result has been acquired before the current time slot; The action space is defined as A = {0, 1, 2}, which represents a set of possible actions. The unloading and acquisition results at each time slot t are defined as action a t , a t =0 means AGV is in standby state, a t =1 means that AGV unloads the task in the current time slot without obtaining the result, a t =2 means that the AGV both unloads tasks and obtains intermediate results in the current time slot.

4. The active migration offloading method for AGV-dependent computing tasks according to claim 2, characterized in that: When the status of unloading tasks and obtaining results in the current time slot and the next time slot is updated, the state is composed of the composite state s t =(n t ,m t ,k t )∈S represents, where substate n t =(n t ,n t-1 ) represents the index of the base station to which the AGV is connected at time slots t and t-1, m t Indicates the task index to be uninstalled, k t Indicates the index of the most recently obtained task result at time slot t.

5. The active migration offloading method for AGV-dependent computing tasks according to claim 4 is characterized in that: The active migration offloading solution established includes: Determine whether the AGV has a base station switch in the current time slot t. If so, execute the intermediate result Q at the new base station w (m) and update the task index m to be uninstalled t =k t +1, make an uninstall decision; if not, make an uninstall decision directly; The AGV decides whether to unload the task in the current time slot t. If so, it unloads and calculates the task, and makes a timeout judgment based on the preset first time limit. If it decides not to unload the task or the timeout occurs, it recalculates the task in the next time slot and updates m. t+1 =m t , until k t+1 =M, complete the calculation of M tasks; if it is decided to offload the task and there is no timeout, update k t+1 =k t , m t+1 =m t +1, make a decision on whether to obtain the intermediate results; If it is decided to obtain the intermediate result, a timeout judgment is performed based on the preset second time limit during the acquisition process. If there is no timeout, k is updated. t+1 =m t , until k t+1 = M, complete the calculation of M tasks; if it is decided not to obtain the intermediate results or timeout, keep k t+1 =k t , until k t+1 =M, complete the calculation of M tasks.

6. The active migration offloading method for AGV-dependent computing tasks according to claim 1, characterized in that: The finite-time time-varying Markov decision problem is to minimize the expected cumulative communication energy consumption during T time slots.

7. The active migration offloading method for AGV-dependent computing tasks according to claim 5, characterized in that: The method to obtain the minimum overall energy consumption required to complete task offloading is: Solution step 1: Initialize the parameters of the deep double Q network of the Markov decision process, record the initial state s∈S and the initial time slot t=1; the initialization includes setting the initial weights of the target network and the main network of the deep double Q network to be the same, and initializing the experience replay buffer; Solution step 2: From the current state s t Start, use the ε-greedy strategy to select action a t , observe the new state s after execution t+1 With reward R, the experience tuple (s t ,a t ,R,s t+1 ) is stored in the experience replay buffer and the current state is updated to s t+1 ; Solution step 3: Sample from the experience replay buffer and use the target network to calculate the target Q value of the sampled experience; Solution step 4: training the main network based on the sampling experience and the target Q value, and updating the target network; Solution step 5: Loop through steps 2-4 until the expected cumulative communication energy consumption during T time slots is minimized, and obtain the minimum overall energy consumption required to complete the task offloading.

8. An active migration unloading system for AGV-dependent computing tasks, used to implement the active migration unloading method according to any one of claims 1 to 7, characterized in that: include: A task offloading model building module is used to build an AGV task offloading model based on a VEC system; wherein the VEC system includes a number of base stations equipped with VEC servers; An index building module is used for AGV to divide the received tasks into subtasks, obtain the task sequence unloading process according to the dependency relationship between the subtasks, and build the subtask unloading index based on the task sequence unloading process; A model space division module, used to map the position of the AGV to the corresponding base station, and divide the state space and action space of the AGV task offloading model based on the task sequence offloading process and the subtask offloading index; An energy consumption calculation module is used to calculate the migration delay, unloading delay, calculation delay and result acquisition delay based on the state space and action space of the AGV task unloading model to obtain the overall energy consumption of task unloading; A status update module is used to establish an active migration offloading scheme, and update the status of the offloading task and the result of the current time slot and the next time slot based on the migration delay, offloading delay, calculation delay and the delay of obtaining the result; A problem conversion module, used for defining the overall energy consumption of the task offloading as an optimization problem, and converting the optimization problem into a finite-horizon time-varying Markov decision problem of joint offloading and migration optimization; The problem-solving module is used to solve the finite-time domain time-varying Markov decision problem through a deep double-Q network algorithm based on the Markov decision process combined with the state update, obtain the minimum overall energy consumption required to complete the task offloading, and complete the active migration offloading for AGV-dependent computing tasks.

9. The active migration offloading system for AGV-dependent computing tasks according to claim 8, It is characterized in that In the index building module, the dependency relationship between the subtasks indicates that the output of the previous subtask is the input of the next subtask; The mth offloaded subtask is represented as: Among them, L m Represents subtask d m The size of α m Indicates the computational intensity of the CPU cycles required to process the task, Q w (m) represents subtask d m Intermediate results updated upon completion; Q w (m) is expressed as: Among them, w i Represents each subtask d i The output size of succ(d i ) indicates that the task topology graph contains the i The set of all successor tasks of the output of .

Images