Intelligent three-dimensional library task unloading method and system based on edge computing

Abstract

The invention discloses an intelligent three-dimensional library task unloading method and system based on edge computing, and relates to the technical field of mobile edge computing. The method comprises the steps of obtaining storage system operation data and computing resources, and determining a to-be-unloaded task; a task unloading model is constructed based on the to-be-unloaded task and the computing resources, a service cache model is constructed according to a cache decision of an edge server for a service program, a communication model is constructed according to the transmission capability of a communication process, and a computing model is constructed according to the computing power of the edge server and the dynamic allocation of the computing resources; and optimizing the task unloading model by using a deep reinforcement learning algorithm to obtain an optimal unloading decision. According to the method, service caching, task unloading and cooperation and computing resource allocation are jointly optimized, the task average processing time delay of the intelligent three-dimensional library system is jointly optimized in a long-time dimension, and meanwhile the real-time performance, reliability and resource utilization efficiency of the intelligent three-dimensional library system are further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of mobile edge computing, and in particular, to an intelligent automated storage and retrieval system (AS / RS) task offloading method and system based on edge computing. Background Art

[0002] The statements in this section merely provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] The Industrial Internet of Things (IIoT) is the application and extension of Internet of Things (IoT) information technology in the manufacturing industry. It is the combined product of computer, communication, and microelectronics technologies. It applies intelligent terminals with sensing, control, computing, and other capabilities, as well as advanced technologies such as intelligent computing, to the industrial production process, greatly improving production efficiency and accelerating the modern intelligentization of traditional industries. At the same time, with the rapid development of the modern logistics and warehousing industries, the intelligent automated storage and retrieval system (hereinafter referred to as "intelligent AS / RS"), as an advanced warehousing system integrating automation technology, Internet of Things technology, and artificial intelligence technology, has gradually become an important means to improve logistics efficiency and reduce labor costs. In the intelligent AS / RS, the inbound, outbound, storage, and management of materials are all completed by automated equipment and intelligent systems, thus realizing efficient and accurate warehousing operations. However, with the continuous growth of the order volume and the increase in the complexity of the warehousing system, the intelligent AS / RS faces a series of challenges such as insufficient computing resources and data transmission delays. Therefore, in order to process the large amount of data generated by sensors, devices, and systems in the intelligent AS / RS and execute complex computing tasks, computing offloading has become an important strategy. Computing offloading is to offload computing tasks from terminal devices or sensors to edge computing nodes or cloud servers to achieve distributed computing and optimal utilization of resources. However, the latency problem and service program problem existing in computing offloading have become bottlenecks restricting system performance. In the intelligent AS / RS system, the real-time requirement of tasks is usually very high. Therefore, formulating a reasonable service caching strategy and reducing computing offloading latency are crucial for ensuring timely response and improving system efficiency.

[0004] Traditional intelligent automated storage and retrieval systems rely on cloud computing architectures, and all task processing is executed remotely in the cloud. This centralized computing method may have a significant impact on task real-time performance due to the need to upload data to the cloud for processing and then return it, especially in high-concurrency scenarios. At the same time, frequent data transmission increases the network bandwidth burden, which may affect system performance when the task data volume is large or the network environment is poor. Moreover, the reliability is insufficient, and cloud or network failures may cause tasks to be unable to be completed or delayed, thus affecting the stability of the entire system. Edge computing technology can significantly reduce data transmission latency, relieve cloud pressure, and improve system reliability by deploying computing resources to edge nodes close to the data source. However, the application of edge computing in intelligent automated storage and retrieval systems still faces challenges. The tasks in intelligent automated storage and retrieval systems are diverse and dynamically changing, and different tasks have quite different requirements for computing resources and real-time performance. How to make a reasonable decision on which tasks should be offloaded to edge servers is a key issue. Moreover, the resources of edge servers are limited, and the number of a certain service that can be deployed in the edge computing system is also limited. Under such conditions, mobile terminals cannot unconditionally offload computing tasks to any edge server. Therefore, before making a task offloading decision at the mobile terminal, the edge computing system should first decide which services of each application need to be cached on each edge server. Then, how to select and dynamically update the cached services to meet task requirements is a technical difficulty. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the purpose of the present invention is to provide an intelligent automated storage and retrieval task offloading method and system based on edge computing. By jointly optimizing service caching, task offloading and collaboration, and computing resource allocation, the average task processing delay of the intelligent automated storage and retrieval system is jointly optimized in a long time dimension, and at the same time, the real-time performance, reliability, and resource utilization efficiency of the intelligent automated storage and retrieval system are further improved.

[0006] To achieve the above purpose, the present invention is implemented through the following technical solutions:

[0007] The first aspect of the present invention provides an intelligent automated storage and retrieval task offloading method based on edge computing, including the following steps:

[0008] Obtain the operation data and computing resources of the warehousing system, and determine the tasks to be offloaded;

[0009] Construct a task offloading model based on the tasks to be offloaded and computing resources. Among them, the task offloading model includes a service caching model, a communication model, and a computing model. Construct a service caching model according to the caching decision of the edge server for service programs, construct a communication model according to the transmission capacity of the communication process, and construct a computing model according to the dynamic allocation of the computing power and computing resources of the edge server;

[0010] Optimize the task offloading model using the deep reinforcement learning algorithm to obtain the optimal offloading decision.

[0011] Furthermore, the warehousing system includes three levels: the terminal device layer, the edge server layer, and the cloud server layer. The terminal device layer consists of terminal devices. The edge layer includes edge servers that receive the offloaded data from the terminal layer in real time and perform calculations. The cloud server layer receives data from the edge servers through the cloud and performs further calculations.

[0012] Furthermore, the specific steps for constructing the service cache model according to the cache decision of the edge server for the service program are as follows:

[0013] Construct the server cache framework;

[0014] Assume that when the computing task is executed locally, there is no need to download data or install the corresponding service program. When the computing task is offloaded from the local to the edge server, the edge server needs to pre-cache the service program specific to the task in advance;

[0015] Set the storage capacity constraint according to the cache decision of the edge server for the service program.

[0016] Furthermore, the specific steps for constructing the communication model according to the transmission capacity of the communication process are as follows:

[0017] Calculate the signal-to-noise ratio of the part of the task transmitted by the terminal device to the edge server through the channel;

[0018] Summarize the signal-to-noise ratios of all channels to obtain the cumulative signal-to-noise ratio;

[0019] According to Shannon's theorem, use the cumulative signal-to-noise ratio to calculate the data transmission rate of the edge server.

[0020] Even further, the part of the task transmitted by the terminal device to the edge server through the channel includes three cases: complete offloading, no offloading, and partial offloading.

[0021] Furthermore, the specific steps for constructing the computing model according to the dynamic allocation of the computing power and computing resources of the edge server are as follows:

[0022] Calculate all the time delays of the terminal device for task processing on the edge server, and summarize all the time delays to obtain the total time delay;

[0023] Calculate the average execution delay according to the total time delay, and dynamically allocate computing resources according to the average execution delay.

[0024] Even further, when calculating the average execution delay, use the maximum value of the computing resources of the edge server as the constraint.

[0025] Further, the specific steps for optimizing the task offloading model using the deep reinforcement learning algorithm are as follows:

[0026] Set the state space, action space, and reward function;

[0027] Construct a Markov decision model based on the set parameters;

[0028] Set the objective function based on the Markov decision model, and use the DDQN algorithm to solve the objective function to obtain the optimal offloading decision.

[0029] Furthermore, the objective function aims at minimizing the average delay of the warehousing system.

[0030] The second aspect of the present invention provides an intelligent automated storage and retrieval system task offloading system based on edge computing, including:

[0031] A data acquisition module, configured to acquire the operation data and computing resources of the warehousing system, and determine the tasks to be offloaded;

[0032] A model construction module, configured to construct a task offloading model based on the tasks to be offloaded and computing resources. Among them, the task offloading model includes a service caching model, a communication model, and a computing model. The service caching model is constructed according to the caching decision of the edge server for service programs, the communication model is constructed according to the transmission capacity during the communication process, and the computing model is constructed according to the dynamic allocation of the computing power and computing resources of the edge server;

[0033] A model optimization module, configured to optimize the task offloading model using the deep reinforcement learning algorithm to obtain the optimal offloading decision.

[0034] The above one or more technical solutions have the following beneficial effects:

[0035] The present invention discloses an intelligent automated storage and retrieval system task offloading method and system. Aiming at the defects that the current task offloading or service caching problem optimization methods are single and mostly use traditional algorithms or heuristic methods, the present invention introduces the DDQN algorithm for the joint optimization of the average delay of task offloading and service caching, further improving the real-time performance, reliability, and resource utilization efficiency of the intelligent automated storage and retrieval system, and providing a more intelligent and efficient solution for the intelligent warehousing system.

[0036] Based on the challenges faced by the current intelligent automated storage and retrieval system, such as insufficient computing resources and data transmission delays, especially in the case of a sharp increase in the order volume, the latency problem caused by the traditional cloud computing architecture affects the real-time performance of tasks. By introducing the deep reinforcement learning algorithm DDQN, the task offloading and service caching strategies are optimized in the intelligent automated storage and retrieval system, reducing the computing offloading latency and improving the resource utilization efficiency. This method offloads computing tasks to edge servers and reasonably caches service programs on edge servers according to the dynamically changing task requirements, thus significantly enhancing the real-time performance, reliability, and efficiency of the system. This technical solution is particularly applicable to modern intelligent warehousing systems and has high application value.

[0037] Advantages of additional aspects of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which form a part of this specification, are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0039] Figure 1 Schematic diagram of the structure of the warehousing system in Embodiment 1 of the present invention;

[0040] Figure 2 Flowchart of the intelligent automated storage and retrieval task offloading method based on edge computing in Embodiment 1 of the present invention;

[0041] Figure 3 Structure diagram of the DDQN algorithm in Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0043] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof;

[0044] Embodiment 1:

[0045] Embodiment 1 of the present invention provides an intelligent automated storage and retrieval task offloading method based on edge computing, including the following steps:

[0046] Step 1: Obtain the operation data and computing resources of the warehousing system, and determine the tasks to be offloaded.

[0047] In a specific implementation, as Figure 1 shown, the warehousing system is an EWM-AS / RS system. The EWM-AS / RS system is an advanced automated warehousing system based on cloud-edge collaboration, including three levels: the terminal device layer, the edge server layer, and the cloud server layer. The terminal device layer consists of terminal devices, including stackers, AGVs, RGVs, robotic arms, conveyor belts, and robots, etc. The edge layer includes edge servers. The edge layer receives the offloaded data from the terminal layer in real time and then performs calculations in the edge servers. The cloud server layer receives data from the edge servers through the cloud and performs further calculations.

[0048] In this embodiment, the warehousing system deploys edge computing nodes according to a predetermined area layout. The edge nodes are connected to terminal devices (such as stackers and AGV cars) through a high-bandwidth and low-latency network. Each edge node is configured with a storage module with service caching capabilities and a high-performance computing module to support the dynamic offloading and processing of multiple tasks. Define the main parameters in the system operation. K is the total number of types of service programs, and the size of a single service program is 20GB. Q is the total number of edge servers. In this embodiment, K = 5 and Q = 5. The storage capacity of each edge server is 40GB. D = 10 terminal devices carrying tasks. The total computing resources of each server are 10GHz. At the beginning of each time slot t, each terminal device generates a computing task l d (t), the task size is uniformly distributed in [10, 50]MB. The local computing power of each device is 2GHz. The channel bandwidth B = 20MHz, and the channel noise power v 2 = 10 -9 W, the transmit power y d = 0.5W. At each time slot t, for device d, server q, and channel j, a channel gain that conforms to a uniform distribution is randomly generated

[0049] Step 2: Build a task offloading model based on the tasks to be offloaded and computing resources.

[0050] Based on the EWM-AS / RS system constructed in Step 1, build a mathematical model for task offloading, service caching, and computing resource allocation. Specifically, the task offloading model includes a service caching model, a communication model, and a computing model. Build a service caching model according to the caching decision of the edge server for service programs, build a communication model according to the transmission capacity during the communication process, and build a computing model according to the dynamic allocation of the computing power and computing resources of the edge server.

[0051] Step 2.1: Construct a service cache model according to the cache decision of the edge server for the service program.

[0052] Step 2.1.1: Construct a server cache framework.

[0053] In a specific implementation, in the construction of an intelligent stereoscopic warehouse system, different computing tasks generated, such as: AGV cart route planning, storage location planning for shelving, etc., will require different service programs to calculate the tasks. By dynamically caching the service programs required for high-frequency tasks, the system can avoid cloud transmission overhead, quickly respond to real-time tasks, and at the same time make full use of the limited resources of the edge server. Therefore, it is very necessary to construct a service cache framework.

[0054] Considering the service cache framework, the system provides K types of service programs, where K = {1, 2,..., K} represents the set of service programs; H q,k (t) represents the cache decision of the edge server q for the k-th type of service program at time slot t, and H q,k (t) ∈ {0, 1}, where H q,k (t) = 1 means that the edge server q has cached the k-th type of service program at time slot t; the storage capacity constraint of the edge server q is as follows:

[0055] C1:

[0056] C2: H q,k (t) ∈ {0, 1}.

[0057] Among them, δ k represents the data size of the k-th type of service program, with the unit of MB; θ q represents the total storage capacity size of the edge server q, with the unit of GB.

[0058] Step 2.1.2: For simplicity, assume that when the computing task is executed locally, there is no need to download data or install the corresponding service program, and when the computing task is offloaded from the local to the edge server, the edge server needs to pre-cache the service program specific to this task.

[0059] Step 2.1.3: Set the storage capacity constraint according to the cache decision of the edge server for the service program.

[0060] According to the above assumption, therefore, the service cache constraint of the edge server q at time slot t is as follows:

[0061] C3:

[0062] C4:

[0063] Among them, It is indicated that the computing task generated by the terminal device d in time slot t requires the support of the k-th type of service program.

[0064] It should be particularly noted that the service cache model is not only a constraint condition of the objective function in the subsequent steps, but also the core input and decision object of the DDQN deep reinforcement learning optimization. It provides the real-time cache status for the algorithm to perceive the environment, and dynamically adjusts the cache policy through the DDQN algorithm. Its construction effect will directly affect the task offloading efficiency, and the system needs to dynamically adjust the cache content to match the task requirements.

[0065] Step 2.2: Construct a communication model according to the transmission capacity of the communication process.

[0066] Step 2.2.1: Calculate the signal-to-noise ratio of the part of the task transmitted by the terminal device to the edge server through the channel. Among them, the part of the task transmitted by the terminal device to the edge server through the channel includes three cases: full offloading, no offloading, and partial offloading.

[0067] In a specific implementation manner, a terminal device d such as a stacker can offload the computing task to the edge server through the V2R communication link. Define the variable indicating the ratio of the part of the task of the terminal device d such as a stacker unloaded to the edge server q through channel j in time slot t. Its value is between 0 and 1, where 1 represents full offloading, 0 represents no offloading, and a value between 0 and 1 represents partial offloading. B is the bandwidth of the channel. Define indicating the channel gain of the terminal device d transmitting data to the edge server q through channel j at time slot t, y d is the transmission power of device d, and v 2 is the noise power.

[0068] Then, the signal-to-noise ratio of device d transmitting the part of the task to the edge server q on channel j in time slot t is as follows:

[0069]

[0070] Step 2.2.2: Aggregate the signal-to-noise ratios of all channels to obtain the cumulative signal-to-noise ratio λ d,q (t).

[0071] Due to the use of multi-channel transmission, the cumulative signal-to-noise ratio of device d offloading to server q is the aggregation of the signal-to-noise ratios of each channel j as follows:

[0072]

[0073] Step 2.2.3: According to Shannon's theorem, use the cumulative signal-to-noise ratio to calculate the data transmission rate of the edge server.

[0074] According to Shannon's theorem, the channel data transmission rate is as follows:

[0075] C = Blog2(1 + S / N).

[0076] Among them, C represents the limit value of the data transmission rate; B represents the channel bandwidth; S represents the signal power, and N represents the noise power. The transmission rate V d,q (t) of the terminal d to the edge server q in the time slot t is as follows:

[0077]

[0078] Step 2.3: Construct a computing model according to the dynamic allocation of edge server computing power and computing resources.

[0079] Step 2.3.1: Calculate all the delays of the terminal device for task processing in the edge server, and summarize all the delays to obtain the total delay.

[0080] In a specific implementation, the local computing power of the device is defined as fn, the task volume is l, and the offloading rate is offrate, where the value of offrate is between 0 and 1, 1 represents full offloading, 0 represents no offloading, and between 0 and 1 represents partial offloading. The weight coefficient of time is ω. In the dynamic network environment of MEC, each edge server needs to process tasks of different sizes. At the same time, the computing resources of the edge server are limited. In addition, the computing resources of the edge server are not static. f d,q (t) represents the computing resources allocated by the edge server q to the terminal d in the time slot t, and the maximum value of the computing resources of the edge server q is fm. The constraint conditions are as follows:

[0081] C5:

[0082] For simplicity, it is assumed that the constrained computing tasks can be directly executed locally without installing the corresponding service program; in the time slot t, the computing task l d (t) has a local processing delay cost T local (t) as follows:

[0083]

[0084] C6: 0 ≤ offrate ≤ 1.

[0085] When data needs to be transmitted to the edge server for processing, there will be a data transmission delay; in the time slot t, the transmission delay T d (t) of the computing task l tran (t) generated by the terminal device d to the edge server is as follows:

[0086] T tran \(l(t) = l\) d \(l(t) \cdot offrate \cdot \omega / C\)

[0087] C represents the data transmission rate, which is obtained from the communication model constructed in Step 2.2:

[0088] \(C = V\) d,q \(= B \log_2(1 + \lambda\) d,q )

[0089] When the data is transmitted to the edge server for computing, at time slot \(t\), the computing task \(l\) generated by the terminal device \(d\) d \(l(t)\) is offloaded to the edge server \(q\) for computing according to the offrate offloading ratio, and the computing delay \(T\) mec \(l(t)\) is as follows:

[0090] \(T\) mec \(l(t) = l\) d \(l(t) \cdot offrate \cdot \omega / f\) d,q \(l(t)\)

[0091] The total processing delay \(T\) of the computing task generated by the vehicle terminal \(d\) in time slot \(t\) total \(l(t)\) is as follows:

[0092] \(T\) total \(l(t) = T\) local \(l(t) + T\) tran \(l(t) + T\) mec \(l(t)\)

[0093] Step 2.3.2: Calculate the average execution delay according to the total delay, and dynamically allocate computing resources according to the average execution delay. Among them, the maximum computing resource of the edge server is used as a constraint when calculating the average execution delay.

[0094] Through the above analysis, \(T\) avg \(l(T, Q)\) represents the average execution delay of all terminals in the system during the entire time slot, \(T = \{1, 2, \ldots, T\}\) represents a set of several time slots, and \(D = \{1, 2, \ldots, M\}\) represents a set of terminal devices such as stackers, as follows:

[0095]

[0096] Step 3: Optimize the task offloading model using the deep reinforcement learning algorithm to obtain the optimal offloading decision.

[0097] Step 3.1: Set the state space, action space, and reward function.

[0098] State space construction: The state space defines the overall state of the system at time slot t, including information such as caches, tasks, and resources. Define to represent the service cache status of each edge server at time slot t, and initialize to represent the idle computing resources of each edge server at time slot t, with an initial value of 10 GHz, k d (t) is the type of service program required for the task of terminal device d at time slot t, with an initial value of 0, indicating that no service program is required, L d (t) identifies the task size generated by terminal device d at time slot t, with an initial value of 0 GB; θ q (t) represents the remaining storage service program capacity size of edge server q at time slot t. The initial value is 40 GB, and the state space S of the system t is as follows:

[0099] S t ={L d (t), μ edge (t), q edge (t), k d (t), θ q (t)}.

[0100] Action space construction: Define the task offloading of each terminal device and the service cache decision of the edge server. At time slot t, when terminal device d receives a new task L d (t), it needs to decide how to process it. x d,q (t) represents the proportion of the task that terminal device d offloads to edge server q at time slot t, with a value range of [0, 1]. 0 means the task is completely processed locally, 1 means the task is completely offloaded to the edge server for processing, and a value between 0 and 1 means partial offloading of the task. The initial value is x d,q (t)=0; y q,k (t) represents whether edge server q caches the service program k at time slot t, with a value of {0, 1}, and the initial value is y q,k (t)=0. Therefore, the action space a t can be described as the offloading decision and is as follows:

[0101] a t ={x d,q (t), y q,k (t)}.

[0102] Reward function construction: The reward function R(t) evaluates the impact of the current decision A(t) on the system performance. The goal is to minimize the total average delay of the system, which is expressed as:

[0103] r t =r{S t , at} = -T avg (T, D).

[0104] Step 3.2: Construct a Markov decision model according to the set parameters.

[0105] Determine the Markov decision model as a triple which are the states during task offloading which represents the state space, represents the action values generated by all terminal devices, and represents the reward value obtained after the terminal device makes a decision. The triple is specifically implemented as s d (t), a d (t) and r d (t), where s d (t) represents the state information of the device, a d (t) represents the action of the device, and r d (t) represents the reward value after making a decision, obtained according to the setting in Step 3.1.

[0106] Step 3.3: Set the objective function based on the Markov decision model, and use the Double Deep Q-Network (DDQN) algorithm to solve the objective function to obtain the optimal offloading decision. Among them, the objective function aims at the minimum average delay of the warehousing system.

[0107] In a specific implementation, as Figure 2 shown, given the task size and the service program corresponding to the task, construct the state space, action space, and reward function of the deep reinforcement learning DDQN algorithm, execute the training process until the maximum number of iterations or convergence, aiming at the minimum average delay of the system, and use DDQN to solve the optimal offloading decision. The optimal offloading decision includes the optimal service caching decision, offloading decision, and computing resource allocation.

[0108] First, initialize the task size of each terminal device d and the service program corresponding to the task, as shown in Table 1.

[0109] Table 1 Initialization of task size and service program parameter table

[0110] Terminal device d <![CDATA[Task size l d (t)(MB)]]> Required service program K 1 20 3 2 35 5 3 40 2 4 15 1 5 25 4 6 30 3 7 45 5 8 50 2 9 12 1 10 28 4 .

[0111] Build the DDQN deep reinforcement learning algorithm as Figure 3 shown, define γ = 0.9 to represent the decay discount factor, initialize the experience replay pool D, and store the historical state transitions (s t , a t, r t , s t+1 ), randomly set the parameters of the evaluation network Q e and the target network Q t . The parameters of both are the same, i.e., w e and w t . Obtain the current state S from the state space t , and select an action a in the action space t ; Based on the state s t and the action a t , determine the next state S t+1 ; Based on the action a t , the state S t+1 , and the reward function to obtain the reward value r t ; Based on the state S t , the action a t , the state S t+1 , and the reward value r t to obtain a set of experience tuples (s t , a t , r t , s t+1 ); Put the experience tuple (s t , a t , r t , s t+1 ) into the experience replay buffer D.

[0112] The DDQN model outputs an action according to the ε-greedy policy. If random exploration is selected (i.e., an event with a probability of ε occurs), the DDQN model outputs a random action; otherwise, the DDQN model outputs the action corresponding to the maximum value of Q(s t , a t ; w e ).

[0113] ε The ε-greedy policy is as follows:

[0114]

[0115] At the beginning of training, dynamically adjust the parameter of ε to explore other possible optimal behaviors as much as possible, avoiding the problem that the neural network falls into the local optimal solution, thus improving the overall optimization effect and generalization ability. Put the experience tuple (s t , a t , r t , s t+1 ) obtained according to the exploration strategy into the experience replay buffer D for updating the experience replay buffer D.

[0116] Take numbers from the experience replay pool D to calculate the value of the evaluation network Q e . The formula is as follows:

[0117] Q = r t + γQ(s t+1 , argmax a Q(s t+1 , a; w e ); w t )。

[0118] Use the gradient descent method to solve the loss function and update the parameters w of the evaluation network Q e , and at the same time update the parameters w of the target network Q every ψ times t , and the loss function is as follows:

[0119] L = [Q - Q(s, a; w t )] 2 。

[0120] By continuously iterating, solve the minimum average delay T avg (T, D) under different service cache decisions, offloading and cooperation decisions, and computing resource allocations, and obtain the finally trained task offloading model. This model couples the service cache decision and task offloading, and solves the problem of the separation of caching and offloading in traditional methods by simultaneously defining the offloading ratio and service cache decision in the action space. At the same time, compared with the traditional DDQN for handling discrete action spaces, this application needs to handle continuous actions, expanding the action space support ability of the traditional DDQN. And for the problem that the traditional DDQN performs well in a static environment but may converge slowly in the face of the dynamics of an intelligent storage library (such as network fluctuations and sudden changes in task requirements), this application incorporates dynamic parameters (such as the remaining resources of the server) into the state space and a dynamic ε-greedy strategy that uses a high exploration rate in the early stage and gradually decreases to balance exploration and exploitation, adapting to the dynamic task offloading requirements.

[0121] Embodiment 2:

[0122] Embodiment 2 of the present invention provides an intelligent storage library task offloading system based on edge computing, including:

[0123] A data acquisition module, configured to acquire the operation data of the warehousing system and computing resources, and determine the tasks to be offloaded;

[0124] A model construction module, configured to construct a task offloading model based on the tasks to be offloaded and computing resources, where the task offloading model includes a service cache model, a communication model, and a computing model. The service cache model is constructed according to the cache decision of the edge server for the service program, the communication model is constructed according to the transmission capacity during the communication process, and the computing model is constructed according to the computing power of the edge server and the dynamic allocation of computing resources;

[0125] A model optimization module, configured to optimize a task offloading model using a deep reinforcement learning algorithm to obtain an optimal offloading decision.

[0126] Each step involved in the second embodiment above corresponds to the first method embodiment, and the specific implementation manner can refer to the relevant description part of the first embodiment.

[0127] Those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to implement. The present invention is not limited to any specific combination of hardware and software.

[0128] Although the specific implementation manner of the present invention has been described above in conjunction with the accompanying drawings, it is not a limitation on the protection scope of the present invention. Those skilled in the art should understand that based on the technical solution of the present invention, various modifications or deformations that can be made without creative efforts by those skilled in the art are still within the protection scope of the present invention.

Claims

1. An intelligent automated storage and retrieval system task offloading method based on edge computing, characterized in that, It includes the following steps: Obtain the operation data and computing resources of the warehousing system, and determine the tasks to be offloaded; Construct a task offloading model based on the tasks to be offloaded and computing resources. Among them, the task offloading model includes a service cache model, a communication model, and a computing model. Construct the service cache model according to the cache decision of the edge server for the service program, construct the communication model according to the transmission capacity of the communication process, and construct the computing model according to the dynamic allocation of the computing power and computing resources of the edge server; Optimize the task offloading model using the deep reinforcement learning algorithm to obtain the optimal offloading decision.

2. The intelligent automated storage and retrieval system task offloading method based on edge computing according to claim 1, wherein The warehousing system includes three levels: the terminal device layer, the edge server layer, and the cloud server layer. The terminal device layer is composed of terminal devices. The edge layer includes edge servers. The edge layer receives the offloaded data from the terminal layer in real time and then performs calculations in the edge servers. The cloud server layer receives data from the edge servers through the cloud and performs further calculations.

3. The intelligent automated storage and retrieval system task offloading method based on edge computing according to claim 1, wherein The specific steps for constructing the service cache model according to the cache decision of the edge server for the service program are as follows: Construct a server cache framework; Assume that when the computing task is executed locally, there is no need to download data or install the corresponding service program. When the computing task is offloaded from the local to the edge server, the edge server needs to pre-cache the service program specific to the task; Set the storage capacity constraint according to the cache decision of the edge server for the service program.

4. The intelligent automated storage and retrieval system task offloading method based on edge computing according to claim 2, wherein The specific steps for constructing the communication model according to the transmission capacity of the communication process are as follows: Calculate the signal-to-noise ratio of the part of the task transmitted by the terminal device to the edge server through the channel; Aggregate the signal-to-noise ratios of all channels to obtain the cumulative signal-to-noise ratio; According to Shannon's theorem, use the cumulative signal-to-noise ratio to calculate the data transmission rate of the edge server.

5. The intelligent automated storage and retrieval system task offloading method based on edge computing according to claim 1, wherein The part of the task transmitted by the terminal device to the edge server through the channel includes three cases: complete offloading, non-offloading, and partial offloading.

6. The intelligent automated storage and retrieval system task offloading method based on edge computing according to claim 1, wherein The specific steps for constructing the computing model according to the dynamic allocation of the computing power and computing resources of the edge server are as follows: Calculate all the time delays of the terminal device for task processing in the edge server, and aggregate all the time delays to obtain the total time delay; Calculate the average execution delay according to the total time delay, and dynamically allocate computing resources according to the average execution delay.

7. The intelligent vertical warehouse task offloading method based on edge computing according to claim 6, characterized in that When calculating the average execution delay, use the maximum value of the computing resources of the edge server as the constraint.

8. The intelligent automated storage and retrieval system task offloading method based on edge computing according to claim 1, wherein The specific steps for optimizing the task offloading model using the deep reinforcement learning algorithm are as follows: Set the state space, action space, and reward function; Construct a Markov decision model according to the set parameters; Set the objective function based on the Markov decision model, and use the DDQN algorithm to solve the objective function to obtain the optimal offloading decision.

9. The intelligent automated storage and retrieval system task offloading method based on edge computing according to claim 1, wherein The objective function aims at the minimum average time delay of the warehousing system.

10. An intelligent automated storage and retrieval system based on edge computing, characterized in that, It includes: A data acquisition module configured to obtain the operation data and computing resources of the warehousing system and determine the tasks to be offloaded; A model construction module configured to construct a task offloading model based on the tasks to be offloaded and computing resources. Among them, the task offloading model includes a service cache model, a communication model, and a computing model. Construct the service cache model according to the cache decision of the edge server for the service program, construct the communication model according to the transmission capacity of the communication process, and construct the computing model according to the dynamic allocation of the computing power and computing resources of the edge server; A model optimization module, configured to optimize a task offloading model using a deep reinforcement learning algorithm to obtain an optimal offloading decision.

Images