Multi-unmanned aerial vehicle task offloading and cache optimization method and system based on edge computing

Abstract

The application discloses a kind of based on edge computing's multi-unmanned plane task unloading and cache optimization method and system, method includes: establishing the system model by T-UAV layer, B-UAV layer and ground user layer;With the minimum total cost of system model as target under satisfying preset constraint condition, obtain optimization problem and modeling as multi-agent Markov decision process, and the optimal task unloading and cache joint strategy are solved.In the present application, in order to reduce the computing load of edge server, improve system response capability, it is divided into unmanned plane of storage computing result and auxiliary edge computing unmanned plane;When unmanned plane meets repetitive task, selectively store task result by cache mechanism, similar computing task can share the computing result submitted by previous task, propose the joint decision of unloading and cache effectively reduce the time delay and energy consumption of task unloading and processing, further improve the efficiency and accuracy of multi-unmanned plane task unloading and cache.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mobile communication, and in particular to a multi-unmanned aerial vehicle task offloading and cache optimization method and system based on edge computing. BACKGROUND

[0002] Mobile Edge Computing (MEC) represents a new computing paradigm aimed at bringing computing and storage resources closer to mobile users. MEC technology belongs to the field of distributed computing, with its calculations distributed across multiple mobile devices. The main goal of this model is to improve the efficiency and performance of mobile network services by deploying resources at the network edge. One key advantage is the reduction of communication latency, allowing delay-sensitive applications such as augmented reality and virtual reality to run more smoothly by processing data directly near the user. In addition, mobile edge computing can improve bandwidth utilization by directly processing and storing data over short distances, reducing the pressure on the core network and improving overall network performance. Location-aware services are also an important feature, providing strong support for navigation and location-based services by obtaining user geographic location information. At the same time, mobile edge computing supports large-scale device connectivity, providing a convenient channel for Internet of Things devices to communicate directly with cloud services. In terms of security and privacy, by processing data at the network edge, the need for data transmission in the network is reduced, improving data security and privacy. In addition, mobile edge computing is flexible and scalable, allowing for the deployment and management of computing resources at the edge according to different network needs.

[0003] Unmanned aerial vehicle-assisted edge computing is an innovative technology that combines unmanned aerial vehicles with edge computing, enabling highly mobile and flexible supply of computing and communication resources. The key advantage of this technology is its ability to provide instant computing capabilities, particularly suitable for scenarios that require high-performance computing at specific locations, such as emergency rescue. By integrating advanced computing devices on unmanned aerial vehicle platforms, users can quickly deploy computing tasks without relying on fixed infrastructure. Chinese patent CN116634466A discloses a method for offloading and resource allocation of edge computing tasks based on unmanned aerial vehicle cooperation multi-access, which utilizes unmanned aerial vehicles to collaborate in mobile edge computing services for computing and cache resources, minimizing service latency and ensuring service fairness among user devices. However, this method is based on the different types of unmanned aerial vehicle services and ignores the offloading and caching problems faced by repetitive tasks in actual applications. SUMMARY

[0004] The application provides a multi-unmanned aerial vehicle (UAV) task offloading and cache optimization method and system based on edge computing, and aims to solve the technical problem that the existing multi-access edge computing method for UAV cooperation ignores the accuracy and energy consumption of offloading and caching in the face of task repetition in actual application.

[0005] The first aspect of the application provides a multi-UAV task offloading and cache optimization method based on edge computing, which comprises the following steps:

[0006] A system model is established, which comprises a T-UAV layer, a B-UAV layer and a ground user layer; the T-UAV layer provides a cache UAV for selectively storing the processing results of tasks based on a task cache mechanism; the B-UAV layer provides multiple UAVs for providing mobile edge computing services for the ground user layer;

[0007] The environmental state data of the system model is obtained based on the task request of the ground user layer, and an optimization problem is obtained based on the environmental state data and aiming at minimizing the total cost composed of the total time delay, the total flight energy consumption and the total energy consumption under the condition of meeting the preset constraint condition;

[0008] The optimization problem is modeled as a multi-agent Markov decision process, and a MATD3 algorithm is used to solve it to obtain an optimal task offloading and cache joint strategy.

[0009] Specifically, in the T-UAV layer, the T-UAV is suspended at a fixed height, and its three-dimensional coordinates are represented as In the B-UAV layer, I B-UAVs are contained, and the B-UAV set is represented as , The three-dimensional coordinates of the i-th B-UAV are represented as The flight angle of the i-th B-UAV is represented as In the ground user layer, N users are contained, and the user set is represented as The three-dimensional coordinates of the n-th ground user are represented as The speed of the n-th ground user follows a uniform distribution .

[0010] Specifically, the task cache mechanism comprises the following steps:

[0011] The popularity, freshness and size factor of the current task are determined according to the task request, and the function value of the current task is calculated;

[0012] The processing result size of the current task and the available cache space size of the cache UAV are obtained, and the available cache space of the cache UAV is updated based on the processing result size of the current task and the function value and the available cache space size of the cache UAV.

[0013] Specifically, the step of obtaining the processing result size of the current task and the available cache space size of the cache UAV, and updating the available cache space of the cache UAV based on the processing result size of the current task and the function value and the available cache space size of the cache UAV, comprises:

[0014] obtaining the processing result size of the current task and the available cache space size of the cache UAV, and determining whether the available cache space size of the cache UAV is greater than or equal to the processing result size of the current task;

[0015] if yes, determining whether the current task is cached; if yes, updating the function value of the current task, and marking the minimum value of the function values of all tasks in the cache space as ; if no, adding the current task into the cache space, taking the next task as the current task, and jumping to the step of obtaining the processing result of the task and determining whether the available cache space size of the cache UAV is greater than or equal to the processing result size of the current task;

[0016] if no, determining whether the minimum value of the function values of all tasks in the cache space is greater than or equal to the processing result size of the current task; if yes, not updating the available cache space size of the cache UAV; if no, deleting the task result corresponding to the minimum value from the cache space of the cache UAV to update the available cache space size of the cache UAV.

[0017] Specifically, the minimized total cost of the system model is represented as follows:

[0018]

[0019] In the formula: Pi,t represents the transmission power of the UAV i at the time t, Xk,t represents the joint decision of offloading and caching matching of the task k at the time slot t; wi represents the weight value associated with the energy consumption of the system model, Xn represents the binary variable of the connection relationship between the user n and the B-UAV i, E represents the total energy consumption of the system model; wi represents the weight value associated with the flight energy consumption, Ei represents the flight energy consumption of the B-UAV i; wi represents the weight value associated with the delay of the system model, T represents the total delay of the system model;

[0020] The preset constraint condition specifically comprises:

[0021]

[0022] C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV, C3 and C4 represent the horizontal flight boundary constraint of the UAV, C5 represents the flight height constraint of the UAV, C5 represents the flight height constraint of the UAV, C5 represents the flight height constraint of the UAV, C6 represents the constraint of the transmission power of the UAV, C7 represents the minimum collision distance of the two UAVs; C8 represents the constraint of the joint decision of the task unloading and cache matching; C9 represents the time delay constraint of the system model processing task, C9 represents the time delay constraint of the system model processing task,

[0023] Specifically, the total time delay of the system model is represented as:

[0024]

[0025] C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV, C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV, C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV, C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV,

[0026] The total energy consumption of the system model is represented as:

[0027]

[0028] C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV, C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV, C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV, C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV,

[0029] Specifically, the step of modeling the optimization problem as a multi-agent Markov decision process and combining the MATD3 algorithm to solve and obtain the optimal task unloading and cache joint strategy comprises:

[0030] The optimization problem is modeled as a multi-agent Markov decision process, the B-UAV is taken as an agent, position information of all UAVs in the B-UAV layer is taken as a state of the agent, a flight trajectory, a transmission power and a task processing decision of the UAV are taken as an action of the agent, a minimum total cost under a preset constraint condition is set as a reward, and violation of a UAV collision constraint, flight of the UAV out of a boundary and non-service of the user by the UAV are taken as punishments, each agent adopts the MATD3 algorithm for solving, an optimal task offloading and caching joint strategy is obtained, for the user in a service range, a B-UAV assisted calculation can be selected, the T-UAV is selected for offloading, and a result sent by the T-UAV is selected to be sent to the user to process a task, for a repetitive task, a result of calculation is cached in the UAV, repetitive tasks can share the result stored in the cache, time delay and energy consumption of task offloading and processing are effectively reduced, and efficiency and accuracy of task offloading and caching of the multi-UAV task offloading and caching are further improved.

[0031] The second aspect of the present application provides a multi-UAV task offloading and caching optimization system based on edge computing, the system comprising:

[0032] A system model establishing unit is configured to establish a system model comprising a T-UAV layer, a B-UAV layer and a ground user layer; the T-UAV layer provides a caching UAV for selectively storing a processing result of a task; the B-UAV layer provides a plurality of UAVs for providing mobile edge computing services for the ground user layer;

[0033] An optimization problem generating unit is configured to acquire environment state data of the system model based on a task request of the ground user layer, and obtain an optimization problem according to the environment state data under a preset constraint condition, with a minimum total cost of total time delay, total flight energy consumption and total energy consumption as a target of the system model;

[0034] An optimal strategy generating unit is configured to model the optimization problem as a multi-agent Markov decision process, and solve the optimization problem by using a MATD3 algorithm to obtain an optimal task offloading and caching joint strategy.

[0035] The third aspect of the present application provides a computer device comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps of the multi-UAV task offloading and caching optimization method according to any one of the above when executing the computer program.

[0036] The fourth aspect of the present application provides a computer readable storage medium having a computer program stored thereon, and the computer program implements the steps of the multi-UAV task offloading and caching optimization method according to any one of the above when executed by a processor.

[0037] From the above technical solution can be seen, the present application has the following advantages:

[0038] The application provides a multi-unmanned aerial vehicle task offloading and cache optimization method and system based on edge computing, and the method comprises the following steps: a system model is established, which comprises a T-UAV layer, a B-UAV layer and a ground user layer; the T-UAV layer provides a cache unmanned aerial vehicle, which is used for selectively storing the processing result of a task based on a task cache mechanism; the B-UAV layer provides a plurality of unmanned aerial vehicles, which are used for providing mobile edge computing services for the ground user layer; environment state data of the system model is acquired based on a task request of the ground user layer, and an optimization problem is obtained by taking the minimum total cost composed of total time delay, total flight energy consumption and total energy consumption of the system model as an objective under the condition that a preset constraint condition is met; the optimization problem is modeled as a multi-agent Markov decision process, and a MATD3 algorithm is used to solve the optimization problem, so that an optimal task offloading and cache joint strategy is obtained.

[0039] In the application, in order to reduce the computing load of an edge server and improve the system response capability, UAVs (unmanned aerial vehicles) are divided into T-UAVs for storing computing results and B-UAVs for assisting edge computing; meanwhile, considering that the UAVs will encounter repetitive tasks when processing computing tasks, the computing results are cached in the UAVs, the generality of task requests is utilized, the task results are selectively stored through a cache mechanism, similar computing tasks can share the computing result submitted by a previous task, and a joint decision of offloading and caching is proposed, so that, for users within the service range, B-UAVs can be selected to assist computing, or the tasks can be offloaded to T-UAVs for computing and the cached results of the T-UAVs can be sent to the users to process the tasks, thereby effectively reducing the time delay and energy consumption of task offloading and processing, further improving the efficiency and accuracy of multi-unmanned aerial vehicle task offloading and caching, and solving the technical problems of the existing unmanned aerial vehicle cooperative multi-access edge computing method, i.e., the accuracy and energy consumption of offloading and caching are neglected in actual application. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only show some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0041] Figure 1 A step flowchart of a multi-unmanned aerial vehicle task offloading and cache optimization method based on edge computing provided by the present application is shown in the figure.

[0042] Figure 2A system architecture diagram of a system model provided by an embodiment of the present application is shown in FIG. 1.

[0043] Figure 3 A cache update flowchart of a task cache mechanism provided by an embodiment of the present application is shown in FIG. 6.

[0044] Figure 4 A structure block diagram of a multi-UAV task offloading and cache optimization system based on edge computing provided by an embodiment of the present application is shown in FIG. 7. DETAILED DESCRIPTION

[0045] An embodiment of the present application provides a multi-UAV task offloading and cache optimization method and system based on edge computing, which is used to solve the technical problem that the existing UAV cooperative multi-access edge computing method ignores the accuracy of offloading and cache and energy consumption faced by the repetitiveness of tasks in actual application.

[0046] In order to make the purposes, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the embodiments described below are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0047] Referring to FIG. 1, Figures 1-3 The first aspect of the present application provides a multi-UAV task offloading and cache optimization method based on edge computing, which comprises the following steps:

[0048] In step 101, a system model composed of a T-UAV layer, a B-UAV layer and a ground user layer is established. The T-UAV layer provides a cache UAV for selectively storing the processing results of tasks. The B-UAV layer provides multiple UAVs for providing mobile edge computing services for the ground user layer.

[0049] Referring to FIG. 1, Figure 2 In the system model, there is only one UAV in the top layer T-UAV, and there are multiple UAVs in the middle layer B-UAV. In order to reduce the computing load of the edge server and improve the system response capability, the UAV (unmanned aerial vehicle) is divided into a T-UAV for storing the computing results and a B-UAV for assisting edge computing. For some areas where it is difficult to arrange base stations or the base stations are far away but the human flow is large, such as some remote scenic spots, the flexibility of the UAV can better provide computing services for these ground users.

[0050] The top layer unmanned aerial vehicle can accept the calculation task result of the task and save it, and the second layer B-UAV can fly randomly within a certain range to provide services for users within the coverage range. In particular, the T-UAV is at the topmost position at a fixed height, and the calculation result of the task stored in the T-UAV as a task memory can be stored in the T-UAV, and the storage capacity of the T-UAV is limited. When the lower layer B-UAV is insufficient in computing resources, the T-UAV can also serve as a temporary offloading node to assist the B-UAV with insufficient computing resources to process the task.

[0051] In the system model of multi-unmanned aerial vehicle assisted edge computing established by the application, the T-UAV layer includes one T-UAV, the B-UAV layer includes I B-UAVs, and the ground user layer includes N users, in particular:

[0052] In the T-UAV layer, the T-UAV is suspended at a fixed height, and its three-dimensional coordinates are represented as ;

[0053] In the B-UAV layer, I B-UAVs are included, and the B-UAV set is represented as , represents the three-dimensional coordinates of the i-th B-UAV, represents the flight angle of the i-th B-UAV, and the maximum flight speed is , and the flight trajectory of the i-th B-UAV is represented as ;

[0054] In the ground user layer, N users are included, and the user set is represented as , the three-dimensional coordinates of the n-th ground user are represented as , and the speed of the n-th ground user follows a uniform distribution ; for the task request of the user, the task set can be represented as , each task includes three parts represented by a set , represents, represents the size of the task data volume, is the number of cpu cycles required for processing the task, is the result size after processing the task.

[0055] In step 102, the environment state data of the system model is obtained based on the task request of the ground user layer, and the optimization problem is obtained based on the environment state data under the condition that the preset constraint condition is met, and the minimum total cost of the system model composed of total delay, total flight energy consumption and total energy consumption is taken as the target.

[0056] It should be noted that when a task request is received from the ground user layer, the environmental state data of the system model needs to be obtained to determine the task processing status; the environmental state data may include the UAV's location information, transmission power, and the decision on task processing for associated users.

[0057] Furthermore, considering the repetitive nature of tasks and the universality of task processing results, this invention uses the T-UAV as a cache drone. Due to limited cache space, task results are selectively stored according to the proposed caching mechanism, leveraging the universality of task requests. Similar computational tasks can share the computation results submitted by the previous task. When a user has a computational need, if the requested task is repetitive, the result can be directly sent from the cache drone to the user, reducing computational latency and energy consumption. The B-UAV serves ground mobile users. For users within the service range, they can choose to use the B-UAV for auxiliary computation, offloading the computation to the T-UAV and having the T-UAV send cached results to the user to process the task.

[0058] Therefore, in task caching, this invention chooses to cache frequently requested tasks in the T-UAV, which can effectively reduce system latency and energy consumption. Specifically, the task caching mechanism includes steps S1 and S2:

[0059] Step S1: Determine the popularity, freshness, and size factor of the current task based on the task request, and calculate the function value of the current task. In this invention, each task K is assigned a function value. , Used to determine the priority of task caching.

[0060] Among them, the function value Represented as:

[0061]

[0062] in, This indicates that the popularity of a task is related to the number of times the task is requested. This indicates that the freshness of a task is related to its creation time. This is a value related to the size of the task data; , and These represent different weighting parameters. In this invention, it can be calculated... The value is used to update the T-UAV cache space.

[0063] Regarding popularity:

[0064] Task request probability is an important concept, representing the likelihood that a user will request a task. A higher request probability indicates that the task is more popular. request probability can be defined as:

[0065]

[0066] where, denotes the number of requests for task k, a denotes the number of times task k is requested.

[0067] Task access time refers to the time from when the task is requested to when the task result is retrieved from the cache space and sent to the user. The shorter the access time, the faster the task retrieval. Since the location of the B-UAV is constantly changing, we use the average access time to represent the access time of task . can be represented as:

[0068]

[0069] where, denotes the time when the a-th request for task is completed (i.e., the time when the task result is retrieved and returned to the user), denotes the time when the a-th request for task is initiated, and the length of the access time is directly related to the user's experience.

[0070] In task requests, the average time interval between task requests is also important. If the time interval is short, it means that the user requests the task frequently, and the task average request interval time can be defined as:

[0071]

[0072] where, denotes the time of the most recent request for task k, denotes the time of the first request for task k, and thus the popularity of task k can be written as:

[0073]

[0074] 2) For Freshness:

[0075] The freshness of a task is the difference between the cache time of the task and the current time. Tasks have a time limit, and considering the freshness of this feature can ensure that the tasks in the cache space are in a relatively new state.

[0076] ​

[0077] wherein, denotes the current time, denotes the generation time of the task. The smaller the difference is, the fresher the task is.

[0078] 3) For the size factor:

[0079] The cached tasks are not only affected by popularity and freshness, but also by the data size associated with the task. By introducing the size factor, the tasks can be cached more effectively according to the storage resource requirements.

[0080] For example, if a task has a high popularity value, but due to its data size, it also needs a large amount of processing resources, then its size factor will be lower than a task with a similar popularity value, but with a smaller data size.

[0081]

[0082] wherein, denotes the size of the input data, is the largest task data size in the request set.

[0083] Step S2: Obtain the processing result size of the current task and the available cache space size of the cache drone, and update the available cache space of the cache drone based on the processing result size of the current task, the function value and the available cache space size of the cache drone.

[0084] Let ξ be the size of the total cache space, and be the available size of the cache space, be the size of the task result. Please refer to Figure 3 , the following are the caching strategies for different situations, which can assume that the available cache space .

[0085] When :

[0086] Step 1. If the task is not cached, calculate the of the task, and add the processing result of this task to the cache space, update the size of the cache space , then compare the of all tasks in the cache space, find the minimum value and mark it as .

[0087] Step 2. If the task is already in the cache space, update the value of the task, and find the in the task again.

[0088] When :

[0089] Step 3. Calculate the .

[0090] Step 4. If , the cache space is not updated.

[0091] Step 5. If , the corresponding task result is evicted from the cache space, and the

[0092] Update the available cache space size .

[0093] Step 6. If the updated cache space size is not enough to place , jump back to the comparison step of and .

[0094] In the present application, the task cache mechanism of updating the cache space according to the function value of the task, considering the size, application frequency and freshness of the task, can maximize the cache hit rate due to the limited cache space.

[0095] The target of the system model is to minimize the cost of completing the task under the constraints of meeting the maximum delay of the UAV flight and transmission power and task processing. Specifically, the description of each module model, the specification of the optimization problem and the optimization method in the system model is as follows:

[0096] I. Communication model:

[0097] 1. Communication model between user and B-UAV layer

[0098] The distance formula between the user and the B-UAV is as follows:

[0099]

[0100] In the formula: denotes the distance between user n and B-UAV i at time t.

[0101]

[0102] wherein, is the power gain with a reference distance of 1m; is the channel gain of user n and B-UAV i at time t, which follows the spatial free space path loss model.

[0103] ​​Task transmission rate from users to B-UAVs is denoted as:

[0104]

[0105] where is the total bandwidth, is the number of users connected with UAV i, is the transmit power of user n, is the power of Gaussian white noise.

[0106] 2. B-UAV and T-UAV layer communication model

[0107] B-UAV and T-UAV distance formula is denoted as:

[0108]

[0109] where, denotes the distance between UAV i and UAV s.

[0110]

[0111] where, is the power gain with reference distance of 1m; denotes the communication gain between B-UAV i and T-UAV s, which follows the spatial free space path loss model.

[0112] User transmission rate to T-UAVs is denoted as:

[0113]

[0114] where, denotes the communication bandwidth between B-UAV i and T-UAV s, is the transmit power of UAV i at time t, is the power of Gaussian white noise.

[0115] II. Task offloading and caching model

[0116] To facilitate subsequent resource allocation, the present application uses denotes the joint decision of task offloading and caching matching, denotes that the user gives the task to the associated UAV for processing, denotes that the task is offloaded to the T-UAV for processing, denotes the selection of task caching.

[0117] where, at each time period, the UAV needs to check the users' locations and divide them into I clusters , denotes the set of users served by UAV i. The user association can be obtained by the heuristic algorithm k-means, which computes the sum of squared errors SSE according to the users' locations:

[0118]

[0119] where, is the computation of the cluster's centroid, is the total number of users in the set The goal of the K-Means method is to minimize SSE so that users within the same cluster are as close as possible. To achieve this, first, I centroids are randomly selected and each user is assigned to the closest cluster. Second, the centroids of each cluster are recomputed and the least squares clustering state is updated; this process is iterated until the centroids of all clusters no longer change, and finally the clustering result that minimizes SSE is obtained. We denote a binary variable to indicate the connection between user n and UAV i, to indicate that user n is connected to B-UAV, otherwise not connected, and a user can only be connected to at most one UAV at the same time.

[0120] 1. User offloads task to B-UAV computation:

[0121] The transmission delay of user n transmitting computation task k to B-UAV i at time t is

[0122]

[0123] The computation delay of task k is

[0124] where,

[0125] denotes the computation resource allocated to user n by UAV i. Then, the time for the task result to be transmitted back is

[0126]

[0127] The total computation delay of task k for user n by B-UAV i is

[0128]

[0129]

[0130] Energy consumption of user offloading task to B-UAV computation​​​​ may be expressed as:

[0131]

[0132] where k represents the effective switching capacitor coefficient, is a constant value for the transmission power of the user, is a variable value for the transmission power of the UAV i.

[0133] 2. The user offloads the task to the T-UAV through the B-UAV for calculation:

[0134] The time delay of the B-UAV transmitting the task to the T-UAV may be expressed as:

[0135]

[0136] In the formula: is the transmission rate between the UAV i and the UAV s.

[0137] The calculation time delay of the T-UAV may be expressed as:

[0138]

[0139] wherein, represents the calculation resource of the T-UAV.

[0140] The total time delay of the task result returning is:

[0141]

[0142]

[0143] wherein, represents the time delay of the task result returning from the T-UAV to the B-UAV.

[0144] The total time delay of the T-UAV calculation is:

[0145]

[0146] Therefore, the energy consumption generated by the T-UAV calculation may be expressed as:

[0147]

[0148] 3. Buffering:

[0149] Unlike the task offloading computation, the latency of caching requires two parts: 1) the latency of reading the required task result from the cache space. 2) the latency of sending the task result to the user.

[0150]

[0151] where the first part is the reading cache time, the second part is the latency of sending the result back to the user through the B-UAV.

[0152] However, when the user chooses the caching way to process the task, the requested content may not be cached in the T-UAV, at this time the user chooses to send the task to the T-UAV for computation, at this time the concept of cache hit rate is introduced, and the time required for caching is expressed in the form of expectation , that is:

[0153]

[0154] The formula of the cache hit rate can be expressed as:

[0155]

[0156] The energy consumption required for caching is mainly the energy consumed for sending the result to the user , which can be expressed as:

[0157]

[0158] In the same way as the computation latency, the energy consumption of caching can be expressed by the following formula:

[0159]

[0160] III. Latency and energy consumption model

[0161] 1. Flight energy consumption of B-UAV:

[0162] In the present application, the B-UAV flies in the same attitude, and the B-UAV flies at a uniform speed in each time slot, so its flight power consumption includes two parts: the energy consumption of horizontal flight and the energy consumption of overcoming air resistance.

[0163] The power of its horizontal flight can be expressed as:

[0164]

[0165] In the formula, M represents the total mass of the UAV, and A is the area of the propeller blade of the UAV.

[0166] The energy consumed for overcoming air resistance Can be modeled as:

[0167]

[0168] Where, The air resistance coefficient related to the spiral blade, A is the area of the UAV spiral blade, and η represents the air density. Therefore, the flight energy consumption of the UAV Can be written as:

[0169]

[0170] 2、Total delay model:

[0171] Total delay model Can be written as:

[0172]

[0173] 3、Total energy consumption model:

[0174] Total energy consumption model Can be written as:

[0175]

[0176] Four, optimization problem

[0177] In the present application, according to the environmental state data, the system model composed of the total delay, the total flight energy consumption and the total energy consumption is minimized, and the total cost is minimized under the satisfaction of the preset constraint condition, so that the minimum total cost of the system model is represented as follows:

[0178]

[0179] In the formula: The weight value associated with the energy consumption of the system model is represented as, The weight value associated with the flight energy consumption is represented as, The weight value associated with the system model delay is represented as.

[0180] The preset constraint condition specifically includes:

[0181]

[0182] In the formula: C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV, The maximum speed of the UAV is represented as, The maximum value of the UAV in the x-axis is represented as, C5 represents the maximum value of the UAV on the y-axis; C5 represents the flight height constraint of the UAV, C6 represents the maximum value of the UAV on the z-axis; C6 represents the constraint of the transmission power of the UAV, C7 represents the maximum transmission power of the UAV; C7 represents the collision distance constraint of the two UAVs, C8 represents the minimum collision distance of the two UAVs; C8 represents the constraint of the joint decision of the task unloading and cache matching; C9 represents the time delay constraint of the system model processing task, C9 represents the maximum time delay value; C10 represents the connection constraint of the user and the UAV.

[0183] Step 103, modeling the optimization problem as a multi-agent Markov decision process, and solving it by using the MATD3 algorithm to obtain the optimal task unloading and cache joint strategy.

[0184] In the UAV-assisted edge computing system, the UAV needs to determine its own position, transmission power and task processing decision for the associated user to obtain the minimum total system cost. Considering that the action of the UAV (the movement of the UAV, the transmission power and the task processing decision) may affect the environment state, the total cost of the system is determined by the current state of the system and the joint action of all UAVs. In addition, the previous state and the previous action jointly convert the system environment into a new state.

[0185] In this case, the optimization problem can be converted into a multi-agent Markov decision process I is the set of agents, S is the set of all agents, is the action of agent i, P represents the state transition probability, is the reward of agent i, denotes the discount factor. Among them, taking B-UAV as the agent, taking the position information of all UAVs in the B-UAV layer as the state, taking the flight trajectory of the UAV, the transmission power and the task processing decision as the action of the agent, setting the minimum total cost under the condition of satisfying the preset constraint as the reward and taking the violation of the UAV collision constraint, the UAV flying out of the boundary and the user not being served by the UAV as the punishment.

[0186] The specific description of the multi-agent Markov decision process is as follows:

[0187] 1) Agent I: Each B-UAV is an agent, which jointly learns to obtain its position, transmission power and joint decision for covering users to obtain the minimum system cost.

[0188] 2) State space : For the optimization problem, the state space of each time slot is composed of the positions of all UAVs, that is

[0189] 3) Action space The action of each agent includes three parts: the flight trajectory of the UAV, the launch power of the UAV and the joint decision, , wherein represents the speed of the agent i in the X, Y and Z directions, respectively represent the launch power and the joint decision. Among them, is a discrete action, which can be continuous by the form of probability.

[0190] 4) Reward function In order to solve the above optimization problem, I agents should cooperatively minimize the total cost of the system under certain constraint conditions. If the constraint is not met, the corresponding penalty is added in the reward function. For the reward function , wherein is the penalty for violating the UAV collision constraint, is the penalty for the UAV flying out of the boundary, represents the penalty for the existence of users not served by the UAV.

[0191] , wherein represents the cost of each UAV i, .

[0192] In order to solve the above multi-agent MDP problem, considering the high-dimensional continuous action space of the task offloading optimization problem, the MATD3 algorithm is proposed, and each UAV uses the MATD3 algorithm, which includes a participant network and two critic networks. Through the two critic networks, each UAV can handle the Q value overestimation problem in a critic framework. In addition, in order to improve the learning stability, a target actor network and a target critic network are adopted. After each iteration is completed, the cache space of the T-UAV is updated according to the optimal strategy of the iteration and the above cache update mechanism.

[0193] Referring to Figure 4 , the second aspect of the present application also provides a multi-UAV task offloading and cache optimization system based on edge computing, which comprises:

[0194] The system model establishing unit 201 is used to establish a system model composed of a T-UAV layer, a B-UAV layer and a ground user layer; the T-UAV layer provides a cache UAV for selectively storing the processing results of tasks; the B-UAV layer provides a plurality of UAVs for providing mobile edge computing services for the ground user layer;

[0195] The optimization problem generation unit 202 is configured to obtain environment state data of a system model based on a task request of a ground user layer, and obtain an optimization problem based on the environment state data and a minimum total cost composed of a total time delay, a total flight energy consumption and a total energy consumption as an objective under a preset constraint condition.

[0196] The optimal strategy generation unit 203 is configured to model the optimization problem as a multi-agent Markov decision process, and solve the optimization problem by using a MATD3 algorithm to obtain an optimal task offloading and caching joint strategy.

[0197] The third aspect of the present application provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps of the multi-unmanned aerial vehicle task offloading and caching optimization method according to any one of the above when executing the computer program.

[0198] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the multi-unmanned aerial vehicle task offloading and caching optimization method according to any one of the above when executed by a processor.

[0199] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0200] In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other manners. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can be in another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0201] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0202] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0203] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the entire or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0204] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for multi-UAV task offloading and cache optimization based on edge computing, characterized in that, The method comprises: A system model is established by a T-UAV layer, a B-UAV layer and a ground user layer; the T-UAV layer provides a cache unmanned aerial vehicle (UAV) for selectively storing a processing result of a task based on a task cache mechanism; the B-UAV layer provides a plurality of UAVs for providing a mobile edge computing service for the ground user layer; Based on a task request of the ground user layer, environment state data of the system model is obtained, and under a preset constraint condition, an optimization problem is obtained according to the environment state data, with a minimized total cost of the system model composed of a total time delay, a total flight energy consumption and a total energy consumption as a target; The optimization problem is modeled as a multi-agent Markov decision process, and a MATD3 algorithm is used to solve the optimization problem, to obtain an optimal task offloading and cache joint strategy.

2. The multi-UAV task offloading and caching optimization method of claim 1, wherein, In the T-UAV layer, T-UAVs are suspended at a fixed height, whose three-dimensional coordinates are denoted as ; in the B-UAV layer, there are I B-UAVs, and the set of B-UAVs is denoted as , denotes the three-dimensional coordinates of the i-th B-UAV, denotes the flight angle of the i-th B-UAV, and the flight trajectory of the i-th B-UAV is denoted as ; in the ground user layer, there are N users, and the set of users is denoted as , the three-dimensional coordinates of the n-th ground user are denoted as , and the speed of the n-th ground user follows a uniform distribution . 3.The method of claim 1, wherein, The task cache mechanism comprises: The popularity, freshness and size factor of a current task are determined according to a task request, and a function value of the current task is calculated; The processing result size of the current task and the available cache space size of the cache UAV are obtained, and the available cache space of the cache UAV is updated based on the processing result size of the current task and the function value and the available cache space size of the cache UAV.

4. The multi-UAV task offloading and caching optimization method of claim 3, wherein, The step of obtaining the processing result size of the current task and the available cache space size of the cache UAV, and updating the available cache space of the cache UAV based on the processing result size of the current task and the function value and the available cache space size of the cache UAV, comprises: The processing result size of the current task and the available cache space size of the cache UAV are obtained, and it is determined whether the available cache space size of the cache UAV is greater than or equal to the processing result size of the current task; If yes, it is judged whether the current task is cached; if yes, the function value of the current task is updated, and the minimum value among the function values of all tasks in the cache space is marked as ; if no, the current task is added to the cache space, the next task is taken as the current task, and the process of obtaining the processing result of the task is jumped to, and it is judged whether the available cache space size of the cache drone is greater than or equal to the processing result size of the current task. If not, it is determined whether the minimum value in the function values of all tasks in the cache space is greater than or equal to the processing result size of the current task; if yes, the available cache space size of the cache UAV is not updated; if not, the task result corresponding to the minimum value is deleted from the cache space of the cache UAV to update the available cache space size of the cache UAV.

5. The multi-UAV task offloading and caching optimization method of claim 2, wherein, The minimized total cost of the system model is represented as follows: In the formula: represents the transmission power of the UAV i at time t, represents the joint decision of the task k at time slot t to unload and match the cache; represents the weight value associated with the energy consumption of the system model, represents the binary variable of the connection relationship between the user n and the B-UAV i, represents the total energy consumption of the system model; represents the weight value associated with the flight energy consumption, represents the flight energy consumption of the B-UAV i; represents the weight value associated with the delay of the system model, represents the total delay of the system model; The preset constraint condition specifically comprises: In the formula: C1 represents the constraint of the flight angle of the UAV; C2 represents the speed constraint of the UAV, represents the maximum value of the speed of the UAV; C3 and C4 represent the horizontal flight boundary constraint of the UAV, represents the maximum value of the UAV on the x-axis, represents the maximum value of the UAV on the y-axis; C5 represents the flight height constraint of the UAV, represents the maximum value of the UAV on the z-axis; C6 represents the constraint of the transmission power of the UAV, represents the maximum transmission power of the UAV; c7 represents the constraint of the collision distance of two UAVs, represents the minimum collision distance of two UAVs; C8 represents the constraint of the joint decision of the unloading and caching matching of the task; C9 represents the time delay constraint of the system model processing task, represents the maximum time delay value; C10 represents the connection constraint of the user and the UAV.

6. The multi-UAV task offloading and caching optimization method of claim 5, wherein, total latency of the system model is represented as: In the formula: denotes the total computation delay of B-UAVi for task k of user n at time t, denotes the total computation delay of B-UAVi for task k of user n at time t, denotes the expected time for user n to send task k to T-UAVs for computation through B-UAVi; Total energy consumption of the system model is represented as: wherein: represents the energy consumption of user n offloading task k to B-UAVi computing task k, represents the energy consumption of T-UAVi computing user n's task k, represents the energy consumption of T-UAVs caching user n's task result.

7. The multi-UAV task offloading and caching optimization method of claim 5, wherein, The step of modeling the optimization problem as a multi-agent Markov decision process and solving the optimization problem by combining the MATD3 algorithm to obtain the optimal task offloading and cache joint strategy, comprises: The optimization problem is modeled as a multi-agent Markov decision process, the B-UAV is taken as an agent, the position information of all UAVs in the B-UAV layer is taken as a state, the flight trajectory, the launch power and the task processing decision of the UAV are taken as the action of the agent, the minimized total cost under the preset constraint condition is set as a reward and the violation of the UAV collision constraint, the UAV flying out of the boundary and the user not being served by the UAV are taken as punishments, each agent is solved by the MATD3 algorithm to obtain the optimal task offloading and cache joint strategy.

8. An edge computing based multi-unmanned aerial vehicle task offloading and cache optimization system, characterized in that, The system comprises: The system model establishment unit is configured to establish a system model composed of a T-UAV layer, a B-UAV layer, and a ground user layer; the T-UAV layer provides a cache UAV for selectively storing processing results of tasks; and the B-UAV layer provides a plurality of UAVs for providing mobile edge computing services for the ground user layer. The optimization problem generation unit is configured to obtain environment state data of the system model based on a task request of the ground user layer, and obtain an optimization problem based on the environment state data and a preset constraint condition, wherein the optimization problem has a minimum total cost composed of a total time delay, a total flight energy consumption, and a total energy consumption as an objective. The optimal strategy generation unit is configured to model the optimization problem as a multi-agent Markov decision process, and solve the optimization problem by using a MATD3 algorithm to obtain an optimal task offloading and cache joint strategy. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the multi-UAV task offloading and cache optimization method according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the multi-UAV task offloading and cache optimization method according to any one of claims 1-7.

Images