Multi-unmanned aerial vehicle task unloading and caching optimization method and system based on edge computing

Abstract

The invention discloses a multi-UAV (unmanned aerial vehicle) task unloading and caching optimization method and system based on edge calculation. The method comprises the following steps: establishing a system model composed of a T-UAV layer, a B-UAV layer and a ground user layer; and obtaining an optimization problem by taking the minimum total cost of the system model as a target under the condition that a preset constraint condition is met, modeling as a multi-agent Markov decision process, and solving to obtain an optimal task unloading and caching joint strategy. In the invention, in order to reduce the calculation load of an edge server and improve the system response capability, an unmanned aerial vehicle for storing a calculation result and an unmanned aerial vehicle for assisting edge calculation are divided; when the unmanned aerial vehicle encounters repetitive tasks, task results are selectively stored through a caching mechanism, similar calculation tasks can share calculation results submitted by a previous task, and a joint decision of unloading and caching is proposed to effectively reduce time delay and energy consumption of task unloading and processing. And the efficiency and accuracy of task unloading and caching of the multiple unmanned aerial vehicles are further improved.

Description

Technical Field

[0001] The present invention relates to the field of mobile communication technology, and in particular to a multi-UAV task offloading and cache optimization method and system based on edge computing. Background Art

[0002] Mobile Edge Computing (MEC) represents a new computing paradigm that aims to bring computing and storage resources closer to mobile users. MEC technology belongs to the distributed computing method, in which computing is distributed on multiple mobile devices. The main goal of this model is to improve the efficiency and performance of mobile network services by deploying resources at the edge of the network. A key advantage is to reduce communication latency. By processing data directly near the user, latency-sensitive applications such as augmented reality and virtual reality can run more smoothly. In addition, mobile edge computing can also improve bandwidth utilization. By directly processing and storing data over a shorter distance, it reduces the pressure on the core network and improves overall network performance. Location-aware services are also one of its important features. By obtaining the user's geographic location information, it provides strong support for services such as navigation and location-based services. At the same time, mobile edge computing supports large-scale device connections and provides a convenient channel for IoT devices to communicate directly with cloud services. In terms of security and privacy, by processing data at the edge of the network, the need for data transmission in the network is reduced, and the security and privacy of data are improved. In addition, mobile edge computing is also flexible and scalable, and computing resources can be deployed and managed at the edge according to different network requirements.

[0003] Drone-assisted edge computing is an innovative technology that integrates drones with edge computing to achieve highly mobile and flexible supply of computing and communication resources. The key advantage of this technology is that it can provide computing power with instant response, which is particularly suitable for scenarios that require high-performance computing at specific locations, such as emergency rescue. By integrating advanced computing equipment on the drone platform, users can quickly deploy computing tasks without relying on fixed infrastructure. The Chinese patent with application number CN116634466A discloses a method for offloading and resource allocation of multi-access edge computing tasks based on drone collaboration, which uses drones to collaborate on mobile edge computing services for computing and caching resources, minimize service delays, and ensure service fairness between user devices. However, this method is based on the different types of drone services, and ignores the offloading and caching problems faced by the repetitiveness of tasks in actual applications. Summary of the invention

[0004] The present invention provides a multi-UAV task offloading and cache optimization method and system based on edge computing, which is used to solve the technical problems of accuracy and energy consumption of offloading and caching faced by existing UAV collaborative multi-access edge computing methods that ignore the repeatability of tasks in practical applications.

[0005] The first aspect of the present application provides a multi-UAV task offloading and cache optimization method based on edge computing, the method comprising:

[0006] A system model consisting of a T-UAV layer, a B-UAV layer and a ground user layer is established; the T-UAV layer provides a cache UAV for selectively storing task processing results based on a task cache mechanism; the B-UAV layer provides multiple UAVs for providing mobile edge computing services for the ground user layer;

[0007] Acquire environmental status data of the system model based on the task request of the ground user layer, and obtain an optimization problem based on the environmental status data with the goal of minimizing the total cost of the system model consisting of total delay, total flight energy consumption and total energy consumption under preset constraints;

[0008] The optimization problem is modeled as a multi-agent Markov decision process and solved using the MATD3 algorithm to obtain the optimal task offloading and caching joint strategy.

[0009] Specifically, in the T-UAV layer, the T-UAV is suspended at a fixed height, and its three-dimensional coordinates are expressed as ; In the B-UAV layer, there are I B-UAVs, where the B-UAV set is represented as , represents the 3D coordinates of the i-th B-UAV, represents the flight angle of the ith B-UAV, and the flight trajectory of the ith B-UAV is expressed as ; In the ground user layer, there are N users, where the user set is represented as , the three-dimensional coordinates of the nth ground user are expressed as , the speed of the nth ground user follows a uniform distribution .

[0010] Specifically, the task caching mechanism includes:

[0011] Determine the popularity, freshness and size factor of the current task according to the task request, and calculate the function value of the current task;

[0012] The size of the processing result of the current task and the size of the available cache space of the cache drone are obtained, and the available cache space of the cache drone is updated based on the size of the processing result of the current task and the function value and the size of the available cache space of the cache drone.

[0013] Specifically, the step of obtaining the size of the processing result of the current task and the size of the available cache space of the cache drone, and updating the available cache space of the cache drone based on the size of the processing result of the current task and the function value and the size of the available cache space of the cache drone includes:

[0014] Obtain the size of the processing result of the current task and the size of the available cache space of the cache drone, and determine whether the size of the available cache space of the cache drone is greater than or equal to the size of the processing result of the current task;

[0015] If yes, determine whether the current task has been cached; if yes, update the function value of the current task and mark the minimum value of the function values ​​of all tasks in the cache space as ; If not, add the current task to the cache space, take the next task as the current task and jump to the step of obtaining the processing result of the task, and determine whether the size of the available cache space of the cache drone is greater than or equal to the size of the processing result of the current task;

[0016] If not, determine whether the minimum value of the function values ​​of all tasks in the cache space is greater than or equal to the size of the processing result of the current task; if so, do not update the available cache space size of the cache drone; if not, delete the task result corresponding to the minimum value from the cache space of the cache drone to update the available cache space size of the cache drone.

[0017] Specifically, the minimized total cost of the system model is expressed as follows:

[0018]

[0019] Where: represents the transmission power of UAV i at time t, represents the joint decision of offloading and cache matching of task k in time slot t; represents the weight value associated with the energy consumption of the system model, A binary variable representing the connection relationship between user n and B-UAVi, Represents the total energy consumption of the system model; Represents the weight value associated with flight energy consumption, represents the flight energy consumption of B-UAVi; represents the weight value associated with the system model delay, Represents the total delay of the system model;

[0020] The preset constraints include:

[0021]

[0022] Where: C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV, Indicates the maximum speed of the drone; C3 and C4 indicate the horizontal flight boundary constraints of the drone. Indicates the maximum value of the drone on the x-axis, Indicates the maximum value of the drone on the y-axis; C5 indicates the flight altitude constraint of the drone. Indicates the maximum value of the drone on the z-axis; C6 indicates the constraint of the drone’s transmission power. represents the maximum transmission power of the UAV; c7 represents the constraint of the collision distance between two UAVs. represents the minimum collision distance between two UAVs; C8 represents the joint decision constraint of task offloading and cache matching; C9 represents the delay constraint of the system model processing task. Indicates the maximum delay value; C10 indicates the connection constraint between the user and the drone.

[0023] Specifically, the total delay of the system model is It is expressed as:

[0024]

[0025] Where: represents the total computational delay of B-UAVi for task k of user n at time t, represents the total computational delay of B-UAVi offloading task k of user n to T-UAVs at time t, represents the expected time for user n to send task k to T-UAVs for calculation via B-UAVi;

[0026] The total energy consumption of the system model It is expressed as:

[0027]

[0028] Where: represents the energy consumption of task k when user n offloads the task to B-UAVi. represents the energy consumption of task k calculated by user n by T-UAVi, represents the energy consumption of T-UAVs to cache the task results of user n.

[0029] Specifically, the optimization problem is modeled as a multi-agent Markov decision process, and solved in combination with the MATD3 algorithm to obtain the optimal task offloading and caching joint strategy, including:

[0030] The optimization problem is modeled as a multi-agent Markov decision process, with B-UAV as the agent, the position information of all drones in the B-UAV layer as the state, the flight trajectory, transmission power and task processing decision of the drone as the action of the agent, the minimization of the total cost under the preset constraints is set as the reward, and the violation of the drone collision constraint, the drone flying out of the boundary and the user not being served by the drone are set as the penalty. Each agent uses the MATD3 algorithm to solve and obtain the optimal task offloading and caching joint strategy. For users within the service range, B-UAV auxiliary calculation can be selected, which is offloaded to T-UAV calculation and T-UAV sends the cached results to the user to process the task. For repetitive tasks, the calculation results are cached in the drone, and repeated tasks can share the results stored in the cache, effectively reducing the delay and energy consumption of task offloading and processing, and further improving the efficiency and accuracy of multi-drone task offloading and caching.

[0031] The second aspect of the present application provides a multi-UAV task offloading and cache optimization system based on edge computing, the system comprising:

[0032] A system model building unit, used to build a system model consisting of a T-UAV layer, a B-UAV layer and a ground user layer; the T-UAV layer provides a cache drone for selectively storing task processing results; the B-UAV layer provides multiple drones for providing mobile edge computing services for the ground user layer;

[0033] an optimization problem generating unit, configured to obtain environmental state data of a system model based on a task request of a ground user layer, and obtain an optimization problem according to the environmental state data with a goal of minimizing a total cost of the system model consisting of a total delay, a total flight energy consumption, and a total energy consumption under preset constraints;

[0034] The optimal strategy generation unit is used to model the optimization problem as a multi-agent Markov decision process and solve it using the MATD3 algorithm to obtain the optimal task offloading and caching joint strategy.

[0035] A third aspect of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any of the multi-UAV task offloading and cache optimization methods described above are implemented.

[0036] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the multi-UAV task offloading and cache optimization methods described above.

[0037] It can be seen from the above technical solutions that the present invention has the following advantages:

[0038] The present invention provides a multi-UAV task offloading and cache optimization method and system based on edge computing, wherein the method comprises: establishing a system model composed of a T-UAV layer, a B-UAV layer and a ground user layer; the T-UAV layer provides a cache UAV for selectively storing task processing results based on a task cache mechanism; the B-UAV layer provides multiple UAVs for providing mobile edge computing services for the ground user layer; obtaining environmental state data of the system model based on task requests of the ground user layer, and obtaining an optimization problem based on the environmental state data with the goal of minimizing the total cost of the system model consisting of total delay, total flight energy consumption and total energy consumption under preset constraints; modeling the optimization problem as a multi-agent Markov decision process, and solving it using the MATD3 algorithm to obtain the optimal task offloading and cache joint strategy.

[0039] In the present invention, in order to reduce the computing load of the edge server and improve the system responsiveness, UAVs (unmanned aerial vehicles) are divided into T-UAVs for storing computing results and B-UAVs for assisting edge computing; at the same time, considering that unmanned aerial vehicles may encounter repetitive tasks when processing computing tasks, the computing results are cached in the unmanned aerial vehicles. The present invention utilizes the universality of task requests, selectively stores task results through a cache mechanism, and allows similar computing tasks to share the computing results submitted by the previous task. A joint decision of offloading and caching is also proposed. For users within the service range, B-UAV can be selected to assist in computing, or offload computing to T-UAV and T-UAV can send cached results to users to process tasks while minimizing costs, effectively reducing the delay and energy consumption of task offloading and processing, and further improving the efficiency and accuracy of multi-UAV task offloading and caching, thereby solving the technical problems of accuracy and energy consumption of offloading and caching faced by the existing unmanned aerial vehicle collaborative multi-access edge computing methods that ignore the repetitiveness of tasks in actual applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 A flowchart of a multi-UAV task offloading and cache optimization method based on edge computing provided by an embodiment of the present invention;

[0042] Figure 2A system architecture diagram of a system model provided by an embodiment of the present invention;

[0043] Figure 3 A cache update flow chart of a task cache mechanism provided by an embodiment of the present invention;

[0044] Figure 4 A structural block diagram of a multi-UAV task offloading and cache optimization system based on edge computing provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The embodiment of the present invention provides a multi-UAV task offloading and cache optimization method and system based on edge computing, which is used to solve the technical problems of accuracy and energy consumption of offloading and caching faced by the existing UAV collaborative multi-access edge computing method which ignores the repeatability of tasks in practical applications.

[0046] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] See also Figure 1-Figure 3 The first aspect of the present invention provides a multi-UAV task offloading and cache optimization method based on edge computing, the method comprising:

[0048] Step 101, establish a system model consisting of a T-UAV layer, a B-UAV layer and a ground user layer; the T-UAV layer provides a cache drone for selectively storing the processing results of the task; the B-UAV layer provides multiple drones for providing mobile edge computing services for the ground user layer.

[0049] See also Figure 2 In the system model, the top-level T-UAV has only one drone, while the middle-level B-UAV has multiple drones. In order to reduce the computing load of the edge server and improve the system responsiveness, the present invention divides UAVs into T-UAVs for storing computing results and B-UAVs for assisting edge computing. For some areas where it is difficult to deploy base stations or the base stations are far away but the flow of people is large, such as some remote scenic spots, the flexibility of drones can better provide computing services for these ground users.

[0050] Among them, the top-level UAV will receive the calculation results of the task and save them, and the second-level B-UAV can fly arbitrarily within a certain range to provide services to users within the coverage area. In particular, the T-UAV is at the top at a fixed altitude. The task results calculated by the T-UAV as a task storage can be stored in the T-UAV. The storage capacity of the T-UAV is limited. When the computing resources of the lower-level B-UAV are insufficient, the T-UAV can also serve as a temporary unloading node to assist the B-UAV with insufficient computing resources to process tasks.

[0051] In the system model of multi-UAV assisted edge computing established by the present invention, the T-UAV layer includes one T-UAV, the B-UAV layer includes I B-UAVs, and the ground user layer includes N users. Specifically:

[0052] In the T-UAV layer, the T-UAV is suspended at a fixed height, and its three-dimensional coordinates are expressed as ;

[0053] In the B-UAV layer, there are I B-UAVs, where the B-UAV set is represented as , represents the 3D coordinates of the i-th B-UAV, represents the flight angle of the i-th B-UAV, and the maximum flight speed is , the flight trajectory of the i-th B-UAV is expressed as ;

[0054] In the ground user layer, there are N users, where the user set is represented as , the three-dimensional coordinates of the nth ground user are expressed as , the speed of the nth ground user follows a uniform distribution ; For user task requests, the task collection can be used Indicates that each task consists of three parts using a collection express express, Indicates the size of the task data. The number of CPU cycles required to process the task. It is the result size after task processing.

[0055] Step 102, based on the task request of the ground user layer, the environmental status data of the system model is obtained, and the optimization problem is obtained based on the environmental status data with the goal of minimizing the total cost of the system model consisting of total delay, total flight energy consumption and total energy consumption while satisfying preset constraints.

[0056] It should be noted that when a task request is received from the ground user layer, it is necessary to obtain the environmental status data of the system model to determine the processing status of the task; wherein the environmental status data may include the location information of the UAV, the transmission power, and the decision on the task processing for the associated user.

[0057] In addition, the present invention takes into account the repetitiveness of tasks and the universality of task processing results, and uses T-UAV as a cache UAV. Due to limited cache space, the task results are selectively stored according to the proposed cache mechanism, and the universality of task requests is utilized. Similar computing tasks can share the computing results submitted by the previous task. When a user has computing needs, if the request is a repeated task, the result can be directly sent from the cache UAV to the user end, reducing computing latency and energy consumption. B-UAV serves ground mobile users. For users within the service range, B-UAV-assisted computing can be selected, offloading computing to T-UAV, and T-UAV sends the cached results to the user to process the task.

[0058] Therefore, when caching tasks, the present invention chooses to cache some tasks frequently requested by users in the T-UAV, which can effectively reduce the system latency and energy consumption. Specifically, the task caching mechanism includes steps S1 and S2:

[0059] Step S1: Determine the popularity, freshness and size factor of the current task according to the task request, and calculate the function value of the current task. In the present invention, each task K is assumed to have a function value , Used to determine the priority of task caching.

[0060] Among them, the function value It is expressed as:

[0061]

[0062] in, Indicates that the popularity of a task is related to the number of times the task is requested. This means that the freshness of the task is related to the time when the task was generated. It is a value related to the amount of task data; , and Respectively represent different weight parameters. In the present invention, by calculating The value of is used to update the T-UAV cache space.

[0063] Regarding popularity:

[0064] Task request probability is an important concept, which indicates the possibility of a task being requested by a user. The higher the request probability, the more popular the task is. The request probability It can be defined as:

[0065]

[0066] in, Indicates the task The number of requests for task k is represented by a.

[0067] The task access time refers to the time from when the task is requested to when the task result is retrieved from the cache and sent to the user. The shorter the access time, the faster the task is retrieved. Since the position of the B-UAV is constantly changing, we use the average access time to represent the task. Visit time It can be expressed as:

[0068]

[0069] in, Indicates the task The time when the ath request is completed (i.e. the time when the task result is retrieved and returned to the user), It means the task The time when the ath request begins, the length of the access time is directly related to the user experience.

[0070] In task requests, the average time interval of task requests is also very important. If the time interval is short, it means that users request the task frequently. Average request interval It can be defined as:

[0071]

[0072] in, Indicates the task The most recent request time, For the first request time, so the popularity of the task It can be written as:

[0073]

[0074] 2) Regarding freshness:

[0075] The freshness of a task is the difference between the task cache time and the current time. Tasks are time-sensitive and freshness is taken into account. This feature can ensure that the tasks in the cache space are in a relatively new state.

[0076]

[0077] in, Indicates the current moment, Indicates the time when the task was generated. The smaller the difference, the fresher the task.

[0078] 3) For size factor:

[0079] The cached tasks are not only affected by popularity and freshness, but also by the size of the data associated with the task. By introducing the size factor, tasks can be cached more effectively based on storage resource requirements.

[0080] For example, if a task has a high popularity value but also requires a lot of processing resources due to its data size, its size factor will be lower than tasks with similar popularity values ​​but smaller data sizes.

[0081]

[0082] in, Indicates the size of the input data, The maximum task data size in the request set.

[0083] Step S2: Obtain the size of the processing result of the current task and the size of the available cache space of the cache drone, and update the available cache space of the cache drone based on the size of the processing result of the current task and the function value and the size of the available cache space of the cache drone.

[0084] Let ξ be the size of the total cache space, and is the available size of the cache space, is the size of the task result. Figure 3 , the following are the cache strategies for different situations, assuming that the available cache space .

[0085] when hour:

[0086] Step 1. If the task is not cached, calculate the task , the processing result of this task Add to the cache space and update the size of the cache space , and then compare the cache space of all tasks , find the minimum value and mark it as .

[0087] Step 2. If the task is already in the cache, update the task value, find out the .

[0088] when hour:

[0089] Step 3. Calculate the task .

[0090] Step 4. If , the cache space is not updated.

[0091] Step 5. If , then evict from cache space if Corresponding task results, update

[0092] Update available cache space size .

[0093] Step 6. If the updated cache space is not large enough to store , then jump back to and The comparison step.

[0094] In the present invention, a task cache mechanism is used to update the cache space according to the function value of the task. Since the cache space is limited, this mechanism takes into account the size, application frequency and freshness of the task, and can maximize the cache hit rate.

[0095] The goal of the system model is to minimize the cost of completing the task while satisfying the constraints of UAV flight and transmission power and the maximum delay of task processing. Specifically, the model of each module in the system model, the specification of the optimization problem and the description of the optimization method are as follows:

[0096] 1. Communication model:

[0097] 1. Communication model between user and B-UAV layer

[0098] Distance formula between user and B-UAV It is expressed as follows:

[0099]

[0100] Where: represents the distance between user n and B-UAVi at time t.

[0101]

[0102] in, is the power gain at a reference distance of 1 m; is the channel gain of user n and B-UVA i at time t, which follows the free space path loss model.

[0103] The transfer rate of tasks from the user to the B-UAV It is expressed as:

[0104]

[0105] in is the total bandwidth, is the number of users connected to drone i, is the transmission power of user n, is the power of Gaussian white noise.

[0106] 2. Communication model between B-UAV layer and T-UAV layer

[0107] B-UAV and T-UAV distance formula It is expressed as follows:

[0108]

[0109] in, Represents the distance between UAV i and UAV s.

[0110]

[0111] in, is the power gain at a reference distance of 1 m; represents the communication gain between B-UAVi and T-UAVs, which follows the free-space path loss model.

[0112] The transmission rate at which users send tasks to the T-UAV It is expressed as:

[0113]

[0114] in, represents the communication bandwidth between B-UAVi and T-UAVs, is the transmission power of UAV i at time t, is the power of Gaussian white noise.

[0115] 2. Task Offloading and Caching Model

[0116] In order to facilitate subsequent resource allocation, the present invention uses represents the joint decision of task offloading and cache matching, Indicates that the user assigns the task to the associated drone for processing. Indicates that the task is offloaded to T-UAV for processing. Indicates the selection of task cache.

[0117] In each time period, the drone needs to check the location of the users and divide them into I clusters. , represents the set of users served by drone i. User association can be obtained through the heuristic algorithm k-means, and the sum of squared errors SSE is calculated based on the user locations:

[0118]

[0119] in, is the centroid of the cluster. For collection The goal of the K-Means method is to minimize SSE so that users in the same cluster are as close as possible. To achieve this, first randomly select I centroids and assign each user to the cluster that is closest to them. Second, recalculate the centroid of each cluster and update the clustering state of the least squares; this process is iterated until the centroids of all clusters no longer change, and finally obtain the clustering result that minimizes SSE. We use a binary variable represents the connection between user n and drone i, Indicates that user n is connected to B-UAV, otherwise it is not connected. A user can only connect to one UAV at a time.

[0120] 1. The user offloads the task to B-UAV for calculation:

[0121] The transmission delay of user n transmitting computing task k to B-UAVi at time t for:

[0122]

[0123] Its calculation delay as follows:

[0124]

[0125] in, It represents the computing resources allocated by drone i to user n.

[0126] Then, the time when the task result is returned for:

[0127]

[0128] The total computation delay of B-UAVi for task k of user n for:

[0129]

[0130] Energy consumption calculated by users offloading tasks to B-UAV It can be expressed as:

[0131]

[0132] Where k is the effective switch capacitance coefficient, The user's transmission power is a fixed value, The transmission power of UAV i is a variable value.

[0133] 2. Users offload tasks to T-UAV through B-UAV:

[0134] The delay of B-UAV transmitting tasks to T-UAV It is expressed as:

[0135]

[0136] Where: is the transmission rate between UAV i and UAV s.

[0137] Its calculation delay It is expressed as:

[0138]

[0139] in, Represents the computing resources of T-UAV.

[0140] Total delay in returning task results for:

[0141]

[0142]

[0143] in, Indicates the delay of transmitting the task result from T-UAV to B-UAV.

[0144] Total latency calculated by T-UAV for:

[0145]

[0146] Therefore, the energy consumption generated by T-UAV calculation It can be expressed as:

[0147]

[0148] 3. Cache:

[0149] Different from task offloading, the latency required for caching consists of two parts: 1) the latency of reading the required task results from the cache space and 2) the latency of sending the task results to the user.

[0150]

[0151] The first part is the read cache time. The first part is the cache read speed of the top-level UAV, and the second part is the latency of transmitting the results back to the user through the B-UAV.

[0152] However, when the user chooses to process the task in a cached manner, there may be a situation where the requested content is not cached in T-UAV. At this time, the user chooses to send the task to T-UAV for calculation. The concept of cache hit rate is introduced here, and the time required for caching is expressed in the expected form. ,Right now:

[0153]

[0154] Cache hit rate The formula can be expressed as:

[0155]

[0156] The energy consumed by the cache is mainly the energy consumed by sending the results to the user. , which can be expressed as:

[0157]

[0158] In the same way as computing latency, the energy consumed by cache It can be expressed by the following formula:

[0159]

[0160] 3. Latency and Energy Consumption Model

[0161] 1. B-UAV flight energy consumption:

[0162] In the present invention, the B-UAV flies in the same posture and at a constant speed in each time slot, so its flight power consumption includes two parts: energy consumption for horizontal flight and energy consumption for overcoming air resistance.

[0163] Its horizontal flight power It can be expressed as:

[0164]

[0165] Where: M represents the total mass of the UAV, and A is the area of ​​the UAV's propeller blades.

[0166] The energy consumed to overcome air resistance It can be modeled as:

[0167]

[0168] in, The air resistance coefficient related to the propeller blades, A is the area of ​​the propeller blades of the UAV, and η represents the air density. can be written as:

[0169]

[0170] 2. Total delay model:

[0171] Total delay model It can be written as:

[0172]

[0173] 3. Total energy consumption model:

[0174] Total energy consumption model It can be written as:

[0175]

[0176] 4. Optimization Issues

[0177] In the present invention, the goal is to minimize the total cost of the system model consisting of total delay, total flight energy consumption and total energy consumption according to the environmental state data under the preset constraints. Then the minimized total cost of the system model is expressed as follows:

[0178]

[0179] Where: represents the weight value associated with the energy consumption of the system model, Represents the weight value associated with flight energy consumption, Indicates the weight value associated with the system model delay.

[0180] The preset constraints include:

[0181]

[0182] Where: C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV, Indicates the maximum speed of the drone; C3 and C4 indicate the horizontal flight boundary constraints of the drone. Indicates the maximum value of the drone on the x-axis, Indicates the maximum value of the drone on the y-axis; C5 indicates the flight altitude constraint of the drone. Indicates the maximum value of the drone on the z-axis; C6 indicates the constraint of the drone’s transmission power. represents the maximum transmission power of the UAV; c7 represents the constraint of the collision distance between two UAVs. represents the minimum collision distance between two UAVs; C8 represents the joint decision constraint of task offloading and cache matching; C9 represents the delay constraint of the system model processing task. Indicates the maximum delay value; C10 indicates the connection constraint between the user and the drone.

[0183] Step 103, modeling the optimization problem as a multi-agent Markov decision process, and solving it using the MATD3 algorithm to obtain the optimal task offloading and caching joint strategy.

[0184] In a drone-assisted edge computing system, the drone needs to determine its own position, transmit power, and task processing decisions for associated users to obtain the minimum total system cost. Considering that the drone's actions (drone movement, transmit power, and task processing decisions) may affect the environmental state, the total cost of the system is determined by the current state of the system and the joint actions of all drones. In addition, the previous state and the previous action together transform the system environment into a new state.

[0185] In this case, the optimization problem can be transformed into a multi-agent Markov decision process I is the set of agents, S is the set of all agents, is the action of agent i, P represents the state transition probability, is the reward of agent i, Represents the discount factor. In this paper, B-UAV is used as the intelligent agent, the position information of all drones in the B-UAV layer is used as the state, the flight trajectory, transmission power and task processing decision of the drone are used as the action of the intelligent agent, the minimum total cost under the preset constraints is set as the reward, and the violation of the drone collision constraint, the drone flying out of the boundary and the user not being served by the drone are set as the penalty.

[0186] The specific description of the multi-agent Markov decision process is as follows:

[0187] 1) Agent I: Each B-UAV is an agent that jointly learns its position, transmit power, and its joint decision on covered users to minimize the system cost.

[0188] 2) State Space : For the optimization problem, the state space of each time slot consists of the positions of all drones, i.e.

[0189] 3) Action Space : Each agent’s action consists of three parts: the flight trajectory of the drone, the launch power of the drone, and the joint decision, ,in represents the speed of agent i in the three directions of X, Y, and Z. denote the transmit power and joint decision respectively. For a discrete action, it can be made continuous in the form of probability.

[0190] 4) Reward Function : To solve the above optimization problem, I agents should collaboratively minimize the total system cost under certain constraints. If the constraints are not met, a corresponding penalty is added to the reward function. ,in Penalty for violating the drone collision constraint, Penalty for drones flying outside the borders, It means there is a penalty for users not being served by the drone.

[0191] in, represents the cost of each drone i, .

[0192] In order to solve the above multi-agent MDP problem, considering the high-dimensional continuous action space of the task offloading optimization problem, the MATD3 algorithm is proposed. Each UAV adopts the MATD3 algorithm, which includes a participant network and two critic networks. Through these two critic networks, each UAV can deal with the Q-value overestimation problem within a critic framework. In addition, in order to improve the learning stability, the target actor network and the target critic network are adopted. After each iteration, the cache space of the T-UAV is updated according to the optimal strategy of the iteration and the cache update mechanism mentioned above.

[0193] See also Figure 4 The second aspect of the present invention also provides a multi-UAV task offloading and cache optimization system based on edge computing, the system comprising:

[0194] The system model building unit 201 is used to build a system model consisting of a T-UAV layer, a B-UAV layer and a ground user layer; the T-UAV layer provides a cache drone for selectively storing the processing results of the task; the B-UAV layer provides multiple drones for providing mobile edge computing services for the ground user layer;

[0195] The optimization problem generating unit 202 is used to obtain the environmental state data of the system model based on the task request of the ground user layer, and obtain the optimization problem according to the environmental state data with the goal of minimizing the total cost of the system model consisting of total delay, total flight energy consumption and total energy consumption under the preset constraint conditions;

[0196] The optimal strategy generation unit 203 is used to model the optimization problem as a multi-agent Markov decision process and solve it using the MATD3 algorithm to obtain the optimal task offloading and caching joint strategy.

[0197] A third aspect of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements any of the steps of the above multi-UAV task offloading and cache optimization method.

[0198] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above multi-UAV task offloading and cache optimization methods.

[0199] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0200] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0201] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0202] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0203] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0204] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-UAV task offloading and cache optimization method based on edge computing, characterized in that: The method comprises: A system model consisting of a T-UAV layer, a B-UAV layer and a ground user layer is established; the T-UAV layer provides a cache UAV for selectively storing task processing results based on a task cache mechanism; the B-UAV layer provides multiple UAVs for providing mobile edge computing services for the ground user layer; Acquire environmental status data of the system model based on the task request of the ground user layer, and obtain an optimization problem based on the environmental status data with the goal of minimizing the total cost of the system model consisting of total delay, total flight energy consumption and total energy consumption under preset constraints; The optimization problem is modeled as a multi-agent Markov decision process and solved using the MATD3 algorithm to obtain the optimal task offloading and caching joint strategy.

2. The multi-UAV task offloading and cache optimization method according to claim 1 is characterized in that: In the T-UAV layer, the T-UAV is suspended at a fixed height, and its three-dimensional coordinates are expressed as ; In the B-UAV layer, there are I B-UAVs, where the B-UAV set is represented as , represents the 3D coordinates of the i-th B-UAV, represents the flight angle of the ith B-UAV, and the flight trajectory of the ith B-UAV is expressed as ; In the ground user layer, there are N users, where the user set is represented as , the three-dimensional coordinates of the nth ground user are expressed as , the speed of the nth ground user follows a uniform distribution .

3. The multi-UAV task offloading and cache optimization method according to claim 1 is characterized in that: The task caching mechanism includes: Determine the popularity, freshness and size factor of the current task according to the task request, and calculate the function value of the current task; The size of the processing result of the current task and the size of the available cache space of the cache drone are obtained, and the available cache space of the cache drone is updated based on the size of the processing result of the current task and the function value and the size of the available cache space of the cache drone.

4. The multi-UAV task offloading and cache optimization method according to claim 3 is characterized in that: The step of obtaining the size of the processing result of the current task and the size of the available cache space of the cache drone, and updating the available cache space of the cache drone based on the size of the processing result of the current task and the function value and the size of the available cache space of the cache drone, comprises: Obtain the size of the processing result of the current task and the size of the available cache space of the cache drone, and determine whether the size of the available cache space of the cache drone is greater than or equal to the size of the processing result of the current task; If yes, determine whether the current task has been cached; if yes, update the function value of the current task and mark the minimum value of the function values ​​of all tasks in the cache space as ; If not, add the current task to the cache space, take the next task as the current task and jump to the step of obtaining the processing result of the task, and determine whether the size of the available cache space of the cache drone is greater than or equal to the size of the processing result of the current task; If not, determine whether the minimum value of the function values ​​of all tasks in the cache space is greater than or equal to the size of the processing result of the current task; if so, do not update the available cache space size of the cache drone; if not, delete the task result corresponding to the minimum value from the cache space of the cache drone to update the available cache space size of the cache drone.

5. The multi-UAV task offloading and cache optimization method according to claim 2 is characterized in that: The minimized total cost of the system model is expressed as follows: Where: represents the transmission power of UAV i at time t, represents the joint decision of offloading and cache matching of task k in time slot t; represents the weight value associated with the energy consumption of the system model, A binary variable representing the connection relationship between user n and B-UAVi, Represents the total energy consumption of the system model; Represents the weight value associated with flight energy consumption, represents the flight energy consumption of B-UAVi; represents the weight value associated with the system model delay, Represents the total delay of the system model; The preset constraints include: Where: C1 represents the flight angle constraint of the UAV; C2 represents the speed constraint of the UAV, Indicates the maximum speed of the drone; C3 and C4 indicate the horizontal flight boundary constraints of the drone. Indicates the maximum value of the drone on the x-axis, Indicates the maximum value of the drone on the y-axis; C5 indicates the flight altitude constraint of the drone. Indicates the maximum value of the drone on the z-axis; C6 indicates the constraint of the drone’s transmission power. represents the maximum transmission power of the UAV; c7 represents the constraint of the collision distance between two UAVs. represents the minimum collision distance between two UAVs; C8 represents the joint decision constraint of task offloading and cache matching; C9 represents the delay constraint of the system model processing task. Indicates the maximum delay value; C10 indicates the connection constraint between the user and the drone.

6. The multi-UAV task offloading and cache optimization method according to claim 5 is characterized in that: The total delay of the system model It is expressed as: Where: represents the total computational delay of B-UAVi for task k of user n at time t, represents the total computational delay of B-UAVi offloading task k of user n to T-UAVs at time t, represents the expected time for user n to send task k to T-UAVs for calculation via B-UAVi; The total energy consumption of the system model It is expressed as: Where: represents the energy consumption of task k when user n offloads the task to B-UAVi. represents the energy consumption of task k calculated by user n by T-UAVi, represents the energy consumption of T-UAVs to cache the task results of user n.

7. The multi-UAV task offloading and cache optimization method according to claim 5 is characterized in that: The step of modeling the optimization problem as a multi-agent Markov decision process and solving it in combination with the MATD3 algorithm to obtain the optimal task offloading and caching joint strategy includes: The optimization problem is modeled as a multi-agent Markov decision process, with B-UAV as the agent, the location information of all drones in the B-UAV layer as the state, the flight trajectory, transmission power and task processing decision of the drone as the action of the agent, the minimization of the total cost under the preset constraints is set as the reward, and the violation of the drone collision constraint, the drone flying out of the boundary and the user not being served by the drone are set as the penalty. Each agent uses the MATD3 algorithm to solve and obtain the optimal task offloading and caching joint strategy.

8. A multi-UAV task offloading and cache optimization system based on edge computing, characterized in that: The system comprises: A system model building unit, used to build a system model consisting of a T-UAV layer, a B-UAV layer and a ground user layer; the T-UAV layer provides a cache drone for selectively storing task processing results; the B-UAV layer provides multiple drones for providing mobile edge computing services for the ground user layer; an optimization problem generating unit, configured to obtain environmental state data of a system model based on a task request of a ground user layer, and obtain an optimization problem according to the environmental state data with a goal of minimizing a total cost of the system model consisting of a total delay, a total flight energy consumption, and a total energy consumption under preset constraints; The optimal strategy generation unit is used to model the optimization problem as a multi-agent Markov decision process and solve it using the MATD3 algorithm to obtain the optimal task offloading and caching joint strategy.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the multi-UAV task offloading and cache optimization method as described in any one of claims 1-7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the steps of the multi-UAV task offloading and cache optimization method as described in any one of claims 1-7 are implemented.

Images