Edge cache based charging uav assisted mec system delay optimization method and device

Abstract

The application discloses an energy charging UAV auxiliary MEC system time delay optimization method and equipment based on edge caching, comprising the following steps: constructing a system model; building a time delay model for the whole system operation process, and dividing the system task execution process into multiple time slots; calculating the flight energy consumption of the UAV, the calculation energy consumption, the task offloading time delay of the user and the calculation time delay according to the system model and the time delay model, and constructing an optimization problem with the minimum system time delay as the target; modeling the system operation process into a Markov process according to the system model and the time delay model, and solving the optimal UAV trajectory and task offloading scheme, the UAV service caching scheme and the UAV charging scheme of the system by using a multi-agent deep reinforcement learning algorithm. The application considers the application scene of large time delay sensitivity tasks, models the system task execution process into a Markov process, and solves the optimal time delay and energy efficiency of the system by using a multi-agent deep reinforcement learning algorithm, thereby improving the utility of the system.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and specifically to a latency optimization method and device for a power-assisted MEC system based on edge buffering. Background Technology

[0002] With the development of modern mobile communication technology, emerging mobile network applications are constantly appearing, such as virtual reality (VR), autonomous driving, and live streaming. These computationally intensive and latency-sensitive applications exacerbate the energy and computing power issues of mobile user devices. To address these new challenges, Mobile Edge Computing (MEC) has emerged as a new network architecture. It offloads the execution of computing tasks from centralized processing to the network edge, effectively alleviating the computing pressure on the core network and accelerating the system's task processing to meet users' low-latency requirements. However, in the application and development of MEC, its limitations have also been exposed: insufficient deployment flexibility, geographical constraints, and inadequate coverage. UAVs, with their autonomy, mobility, flexibility, and low maintenance requirements, can become an excellent tool for solving the problems of traditional terrestrial MEC systems. Airborne MEC systems supporting UAVs can significantly reduce task offloading latency and energy consumption compared to terrestrial MEC. Specifically, by carrying mobile edge servers, UAVs can expand the coverage of MEC networks, providing an effective information propagation architecture for the high data traffic generated by user devices. At the same time, UAVs offer a new relay method: mobile relay. Drones can replace many fixed edge servers because of their rapid and efficient location switching capabilities. Compared to the static relays of traditional MEC systems, drone MEC systems can significantly improve response time and utility through proper trajectory planning. Furthermore, aerial MEC systems can minimize cellular link shadowing and congestion, enabling drones to establish LOS links with user equipment, thereby reducing the offloading power consumption of user equipment.

[0003] However, airborne MEC systems are still limited by the UAV's limited computing power, storage capacity, and battery life. To address this issue, joint optimization of communication, computing, and caching resources within the MEC system is necessary. Caching resources in the system often store input and output data and programs for computing tasks that can be reused by multiple users. Therefore, this caching can be divided into two categories: ① compute content caching, which caches the input or output data of computing tasks; ② compute service caching, which caches the programs or databases used to execute tasks. Optimizing the system's caching strategy can save computing and communication resources in the MEC network. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of the existing technology by proposing a latency optimization method for a powered UAV-assisted MEC system based on edge caching. This invention achieves control over the total system latency by jointly optimizing various decisions in this architecture, thereby improving the system's response speed and meeting the computing needs of users for latency-sensitive tasks.

[0005] The technical solution adopted by this invention to solve its technical problem is:

[0006] A latency optimization method for a rechargeable UAV-assisted MEC system based on edge caching is proposed, applied to scenarios where energy-constrained UAVs provide communication and computing services to ground user equipment, where the ground user equipment generates latency-sensitive tasks. In a system with M ground user equipment, N UAVs equipped with edge service caching and servers, one cloud server, and L ground charging stations: the locations of the user equipment, charging stations, and cloud server are fixed. The UAVs choose their own action plan based on their remaining battery power and the system status (the locations of other UAVs, user locations, and task distribution), either providing unloading services to ground users or flying to the charging station area for recharging. Therefore, the UAV's caching decision, its charging plan, and its flight plan (UAV trajectory) are mutually constrained, thus determining the system's task processing latency. By modeling the system's task execution process as a Markov process and using a multi-agent deep reinforcement learning algorithm to solve for the optimal actions of each agent (UAV) (the UAV's caching decision, its charging plan, and its flight plan, i.e., the UAV trajectory), the overall system latency is reduced, and the system energy efficiency is improved. The method includes the following steps:

[0007] Step 1: Construct a system model containing N drones, M user devices, and L ground wireless charging stations. Ground users will generate various types of computing tasks and offload these tasks to the drones for computation. The drones are equipped with edge servers and edge service caches. The content of the edge service cache includes business programs, databases, etc., required for computing various user tasks. That is, drones can only provide edge computing services to ground user devices if they have the relevant service cache. Furthermore, the drones calculate the service cache factor based on their own location, the location of the ground user devices, the type of task, and other information, which determines the service cache update priority.

[0008] Step 2: Build a latency model for the entire system operation process, divide the system task execution process into multiple time slots, and determine the type of time slot based on whether the drone flies to the charging station for charging. These time slots can be divided into charging time slots and working time slots. During the charging time slot, the drone will execute its own charging plan. During the working time slot, the drone will update its own service cache, execute a flight plan, and provide services to ground equipment.

[0009] Step 3: Calculate the UAV's flight energy consumption, computational energy consumption, and user's task offload latency and computational latency based on the system model and latency model, and construct an optimization problem with the goal of minimizing system latency and energy consumption;

[0010] Step 4: Based on the established system model and latency model, model the system operation process as a Markov process, and use deep reinforcement learning algorithms to solve for the optimal UAV trajectory and task offloading scheme, UAV service caching scheme, and UAV charging scheme.

[0011] Furthermore, in the system model, the horizontal plane where the ground equipment is located is divided into multiple identical sub-regions. The horizontal projections of the ground charging station and the drone are located at the center of one sub-region. The positions of the user equipment are fixed. Based on the distribution of the equipment, the equipment group is divided into L sub-clusters. The charging station is located in the central sub-region of the equipment cluster. The drone flies on a horizontal plane at a height of H. The positions of the drone and the ground equipment are respectively... The drone can execute a flight plan once in each work slot. This means hovering in the current sub-region, flying to an adjacent sub-region, or executing a charging plan. The charging decision factor is a binary variable; a value of 0 indicates that the charging plan has begun or the vehicle is en route to a charging station. To unload the correlation factor, a value of 1 indicates that in the time slot The ground-based device m offloads its computing tasks to the drone n. This is a service cache vector for the drone. Each element in the vector is a binary variable, and each element corresponds one-to-one with a specific type of service cache. A value of 1 indicates that the drone has that type of service cache in that time slot. Regarding task type... The service caching factor can be calculated using the following formula. A larger value indicates a higher priority for drones caching the corresponding service programs, as they are closer to the device that generated the task. If the value is 0, then the program update corresponding to task type z will not be considered.

[0012]

[0013] in, For slot i unloaded task type The number of user devices, For a specific unloaded task type in time slot i The distance between a user device and a drone.

[0014] Furthermore, the system's task execution process is divided into K unequal time slots. Before execution, the cloud server collects initial system information, including the location information of each drone and user device, as well as the task type of each user device. Based on this information, a deep reinforcement learning algorithm is executed to obtain the drone's optimal cache vector and its charging decision factor, flight plan (drone trajectory), and the user device's optimal unloading decision. Then, at the beginning of each time slot, each drone determines whether the time slot is a working time slot or a charging time slot based on the charging decision factor. If it is a working time slot, the drone updates its service cache according to its own service cache vector, flight plan, and unloading correlation factor and executes the task. The flight plan provides services to ground users, and the user equipment will offload the computing tasks to the corresponding drones according to the offloading scheme (the drone can handle a maximum of one task per time slot). Finally, the drone calculates and returns the results. If it is a charging time slot, the drone executes the charging scheme, which means flying to the nearest charging station and continuing to fly to the charging station in the subsequent time slots until charging is complete. However, in order to ensure that the time slots are not wasted, the drones need to synchronize with other drones in time. If other drones are all in charging time slots, each drone will fly to the adjacent sub-area once according to its own route. If there is a working time slot, it will fly to the nearest charging station within the time slot length.

[0015] Therefore, the length of each time slot is the maximum working time slot length or the time required to execute one flight plan. The working time slot of a drone consists of flight latency and drone service latency, and the charging time slot length is equal to the flight latency required to fly to the adjacent sub-region.

[0016] The flight delay of a drone can be calculated using the following formula, where d is the distance between adjacent sub-regions and v is the drone's flight speed. This is a charging arrival flag (a binary variable). A value of 1 indicates that the charging plan executed in the current time slot can reach the charging station. This refers to the time required to reach the charging station during this time slot.

[0017]

[0018] The service latency of a drone consists of three parts: service cache update latency, task unloading latency, and task computation latency, which can be calculated separately using the following formulas:

[0019]

[0020]

[0021] in, For the type of task; A set of task types; The number of service caches that need to be updated; The time required to update a service cache; The amount of data that needs to be unloaded for each user device; This indicates the ground equipment linked to time slot i and UAV n (given by the offloading association factor and the charging decision factor; if the UAV chooses to execute the charging scheme, it cannot provide services to the user equipment). This represents the communication rate between time slot i, UAV n, and the ground equipment it serves. Since communication between the UAV and user equipment can be considered as transmission over a LOS link, it can be calculated using the following formula.

[0022]

[0023] in To calculate 1-bit user equipment The number of CPU cycles required to unload data from a certain task. This refers to the CPU frequency of the drone.

[0024] Thus, the time slot length and the total latency of the system processing tasks within the time slot can be obtained as follows:

[0025]

[0026] The hovering time of the UAV in each time slot can be calculated from the time slot length.

[0027] .

[0028] Furthermore, the energy consumption of a drone includes flight energy consumption and computing energy consumption. The computing energy consumption of a drone is... Flight energy consumption includes hovering energy consumption and propulsion energy consumption. The power of both can be calculated using the following formula (when the speed is 0, the result is the hovering power):

[0029]

[0030] in This refers to the power consumed by the blade profile when the drone is hovering, also known as the profile power (drones need to rotate to maintain altitude, which depends on the drone's weight, air density, rotor speed, and rotor surface area). The speed at the tip of the rotor blade. The induced power is the power required to overcome the induced drag when the drone is hovering (the airflow over the rotor generates additional speed, i.e., induced speed, which will bring induced drag to the rotor rotation; induced power is the power required to overcome induced drag). This indicates the average inductive speed of the rotor during the hovering period; , and These represent the fuselage drag ratio, air density, and rotor solidity, respectively. Represents the area of ​​the rotor's rotating surface. The rotor radius is... The rotor angular velocity, This represents the rotor profile drag coefficient.

[0031] From this, we can obtain the calculation formulas for hovering energy consumption and propulsion energy consumption. (The time spent hovering within the time slot)

[0032]

[0033]

[0034] If the drone starts executing the charging scheme within the time slot, the binary variables This value will be set to 0. This variable is used to indicate the execution status of the charging scheme and will not be cleared until charging is complete. The remaining energy of the drone when charging is complete. It uses its maximum energy and records and updates the time slot when charging is complete. .

[0035] Based on hovering energy consumption, propulsion energy consumption, and charging scheme, the energy consumption of the drone in each time slot can be calculated. And the remaining energy at the start of the time slot, respectively

[0036]

[0037] In order for drones to complete missions independently without running out of energy, there are constraints.

[0038] .

[0039] Furthermore, based on the system model and the time slice model, the proposed optimization problem is as follows:

[0040]

[0041] Constraint C1 means that the charging scheme and the unloading plan cannot conflict. Let C1 be a binary variable, where 1 indicates that UAV n caches the service programs required by user equipment m in time slot i; constraint C2 indicates that the communication range limit must be met during task offloading; and constraint C3 indicates that the service programs cached by the UAV cannot exceed the size of the cache space. The size of a service cache, Let C be the size of the drone's cache space; constraint C4 indicates that the drone has an energy surplus after completing the mission; constraint C5 indicates that the drone cannot fly out of the service area; constraint C6 indicates that each decision must satisfy the requirements of its binary variables.

[0042] Furthermore, this method performs Markov modeling of the system operation process and builds the operating environment based on the multi-agent deep reinforcement learning algorithm, including agent space, state space, action space, and reward function.

[0043] Each drone in the intelligent agent space acts as an independent intelligent agent.

[0044] The state space is used to describe the current state of the entire system, including the number of user devices that have not performed task offloading, the location information of each drone, and the service cache factor.

[0045] The action space describes the decisions made by the system at each time slot that can change the system state, including task unloading correlation factors. Flight plan (drone trajectory) Charging decision factors Cache vector .

[0046] The reward function is used to correct the iteration direction of the algorithm. To minimize the total latency of serving all user devices, the reward can be expressed as... , This indicates the reward for drones updating their service cache according to the priority determined by the service cache factor. The higher the priority of the updated service cache (the larger the service cache factor), the higher the reward. This represents a penalty for wasting time slot resources. Due to unreasonable allocation of various decisions, the working time slot length of each drone in the same time slot may vary too much, resulting in the time slot being lengthened and wasting time slot resources. This indicates the penalty imposed for exceeding constraints C1-C6, etc.

[0047] Based on the above, a multi-agent environment can be built, and the original optimization problem can be solved using a multi-agent deep reinforcement learning algorithm to obtain the joint optimal solution for drone trajectory, charging scheme, offloading association, and service cache management.

[0048] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0049] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0050] In summary, this invention discloses a latency optimization method for a power-charging UAV-assisted MEC system based on edge caching, mainly including the following steps: Step 1, constructing a system model containing N UAVs, M user devices, and L ground wireless charging stations; Step 2, building a latency model for the entire system operation process, dividing the system task execution process into multiple time slots; Step 3, calculating the flight energy consumption and computation energy consumption of the UAVs, as well as the task offloading latency and computation latency of the users, based on the system model and latency model, constructing an optimization problem with the goal of minimizing system latency; Step 4, modeling the system operation process as a Markov process based on the constructed system model and latency model, and using a multi-agent deep reinforcement learning algorithm to solve for the optimal UAV trajectory, task offloading scheme, UAV service caching scheme, and UAV charging scheme. This invention considers the application scenario of large-scale latency-sensitive tasks, models the system task execution process as a Markov process, and uses a multi-agent deep reinforcement learning algorithm to solve for the optimal latency and energy efficiency of the system, thereby improving the system's utility.

[0051] This invention proposes a latency and energy efficiency optimization method for a powered UAV-assisted edge computing system based on edge service caching technology. This method constructs a system model including multiple UAVs, multiple user devices, and a few ground charging stations, and builds a time-slot model for the entire system operation process. In this architecture, ground devices generate various types of tasks and offload all tasks to the edge computing server mounted on the UAV. Each UAV downloads and updates the service cache on the edge computing server from the cloud server during its respective caching update time slot for task computation and then sends the computation results back to the ground devices. Simultaneously, influenced by task offloading decisions and flight plans in each time slot, UAVs generate varying computational and flight energy consumption. Therefore, it is necessary to rationally arrange these and the charging scheme to ensure that the UAV's energy is higher than the energy required to fly to the charging station for charging. This invention achieves control over the total system latency by jointly optimizing various decisions in this architecture, improving the system's response speed and thus meeting the computational needs of users for latency-sensitive tasks.

[0052] The beneficial effects of this invention are:

[0053] The latency optimization method for a powered UAV-assisted MEC system based on edge caching provided by this invention can overcome the limitations of UAV energy, effectively reduce the total system latency, and improve the system's energy efficiency for large-scale latency-sensitive tasks that require UAVs to operate for extended periods.

[0054] From the perspective of reducing the overall system latency, drones can update their own service cache from the cloud server in each time slot to adapt to the types of tasks generated by nearby ground users. Compared with fixed service cache, this can reduce the allocation of time slot resources in trajectory planning.

[0055] From the perspective of improving system energy efficiency, the dynamic update mechanism of service cache reduces the proportion of propulsion energy consumption by reducing the allocation of time slot resources in trajectory planning. The increased hovering energy consumption caused by cache updates (hovering and waiting is required when updating cache) is far less than the reduced propulsion energy consumption, thus improving system efficiency. Attached Figure Description

[0056] Figure 1 This is a system model diagram of the present invention.

[0057] Figure 2 This is a time-slot model diagram of the present invention.

[0058] Figure 3 This is a flowchart of the method of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] like Figure 1 As shown, this invention proposes a latency optimization method for a powered UAV-assisted MEC system based on edge buffering, including the following:

[0061] Suppose there are N drones, M ground devices, and L ground charging stations. The horizontal plane where the ground devices are located is divided into multiple identical sub-regions. The horizontal projection of each ground charging station and drone is located at the center of a sub-region. The positions of the user devices are fixed. Based on the distribution of the devices, the device group is divided into L sub-clusters. The charging stations are located in the central sub-region of the device clusters. The drones fly on a horizontal plane at a height of H. The positions of the drones and ground devices are respectively... The drone can execute a flight plan once in each work slot. This means hovering in the current sub-region, flying to an adjacent sub-region, or executing a charging plan. This is the service cache vector for the UAV. Each element in the vector is a binary variable, and each element corresponds one-to-one with a specific type of service cache. A value of 1 indicates that the UAV has that type of service cache in that time slot. The positions of the UAV and ground equipment are represented by... This indicates that during system operation, each ground user equipment generates a computing task, which can be described from three dimensions: User device task type, task offload data size (Unit: bit) Number of CPU cycles required to process a single bit of this type of task data Furthermore, the user device uninstallation association factor is determined by a binary variable. Given that 1 indicates that the ground device m offloads its computational tasks to the drone n, therefore we have

[0062]

[0063] The charging scheme for drones is determined by charging decision factors. The decision is a binary variable; a value of 0 indicates the start or ongoing execution of a charging plan, specifically flying to the nearest charging station and continuing to the station in subsequent time slots until charging is complete. The charging plan is constrained by the task offloading correlation factor; that is, if the drone needs to charge, it cannot provide offloading and computational tasks for the user equipment. The ground equipment (given by the offloading decision and charging scheme) connecting time slot i to UAV n is represented by the following relationship:

[0064]

[0065] in, Let be a binary variable, where 1 indicates that the drone n caches the business program required by the user equipment m in time slot i.

[0066] The priority of drone cache updates is determined by the service cache factor, depending on the task type. The service caching factor can be calculated using the following formula. A larger value indicates a higher priority for drones caching the corresponding service programs, as they are closer to the device that generated the task. If the value is 0, then the program update corresponding to task type z will not be considered.

[0067]

[0068] in, For slot i unloaded task type The number of user devices, For a specific unloaded task type in time slot i The distance between a user device and a drone.

[0069] Figure 2The system's task execution process is divided into K unequal time slots. Before execution, the cloud server collects initial system information, including the location information of each drone and user device, as well as the task type of each user device. Based on this information, a deep reinforcement learning algorithm is executed to obtain the drone's optimal cache vector and its charging decision factor, flight plan (drone trajectory), and the user device's optimal unloading decision. Then, at the beginning of each time slot, each drone determines whether the time slot is a working time slot or a charging time slot based on the charging decision factor. If it is a working time slot, the drone updates its service cache according to its own service cache vector, flight plan, and unloading correlation factor and executes the flight. The flight plan provides services to ground users, and user equipment will offload computing tasks to the corresponding drones according to the offloading scheme (each drone can handle a maximum of one task per time slot). Finally, the drone calculates and returns the results. If it is a charging time slot, the drone executes the charging scheme, which means flying to the nearest charging station and continuing to fly to the charging station in subsequent time slots until charging is complete. However, in order to ensure that the time slots are not wasted, the drones need to synchronize with other drones in time. If other drones are all in charging time slots, each drone will fly to the adjacent sub-area once according to its own route. If there is a working time slot, it will fly to the nearest charging station within the time slot length.

[0070] Therefore, the length of each time slot is the maximum working time slot length or the time required to execute one flight plan. The working time slot of a drone consists of flight latency and drone service latency, and the charging time slot length is equal to the flight latency required to fly to the adjacent sub-region.

[0071] The flight delay of a drone can be calculated using the following formula, where d is the distance between adjacent sub-regions and v is the drone's flight speed. This is a charging arrival flag (a binary variable). A value of 1 indicates that the charging plan executed in the current time slot can reach the charging station. This refers to the time required to reach the charging station during this time slot.

[0072]

[0073] The operating latency of a drone consists of three parts: service cache update latency, task unloading latency, and task computation latency, which can be calculated separately using the following formulas:

[0074]

[0075]

[0076] in, For the type of task; A set of task types; The number of service caches that need to be updated; The time required to update a service cache; The amount of data that needs to be unloaded for each user device; This indicates the ground equipment linked to time slot i and UAV n (given by the offloading association factor and the charging decision factor; if the UAV chooses to execute the charging scheme, it cannot provide services to the user equipment). This represents the communication rate between time slot i, UAV n, and the ground equipment it serves. Since communication between the UAV and user equipment can be considered as transmission over a LOS link, it can be calculated using the following formula.

[0077]

[0078] in To calculate 1-bit user equipment The number of CPU cycles required to unload data from a certain task. This refers to the CPU frequency of the drone.

[0079] Thus, the time slot length and the total latency of the system processing tasks within the time slot can be obtained as follows:

[0080]

[0081] The hovering time of the UAV in each time slot can be calculated from the time slot length.

[0082]

[0083] The energy consumption of a drone can be categorized into two types based on different decisions made during flight planning (whether the drone hovers in the current sub-region or flies to an adjacent sub-region): hovering energy consumption and propulsion energy consumption. The power consumption of both can be calculated using the following formula (when the speed is 0, the result is the hovering power):

[0084]

[0085] in This refers to the power consumed by the blade profile when the drone is hovering, also known as the profile power (drones need to rotate to maintain altitude, which depends on the drone's weight, air density, rotor speed, and rotor surface area). The speed at the tip of the rotor blade. The induced power is the power required to overcome the induced drag when the drone is hovering (the airflow over the rotor generates additional speed, i.e., induced speed, which will bring induced drag to the rotor rotation; induced power is the power required to overcome induced drag). This indicates the average inductive speed of the rotor during the hovering period; , and These represent the fuselage drag ratio, air density, and rotor solidity, respectively. Represents the area of ​​the rotor's rotating surface. The rotor radius is... The rotor angular velocity, This represents the rotor profile drag coefficient.

[0086] From this, we can obtain the formulas for calculating hovering energy consumption and propulsion energy consumption:

[0087]

[0088]

[0089] If the drone starts executing the charging scheme within the time slot, the binary variables This value will be set to 1. This variable indicates the execution status of the charging scheme and will not be reset to zero until charging is complete. The remaining energy of the drone when charging is complete. It uses its maximum energy and records and updates the time slot when charging is complete. .

[0090] Based on hovering energy consumption, propulsion energy consumption, and charging scheme, the energy consumption of the drone in each time slot can be calculated. And the remaining energy at the start of the time slot, respectively

[0091]

[0092] In order for drones to complete missions independently without running out of energy, there are constraints.

[0093]

[0094] In summary, the system latency minimization problem can be formulated as follows:

[0095]

[0096] Constraint C1 means that the charging scheme and the unloading plan cannot conflict. Let C1 be a binary variable, where 1 indicates that UAV n caches the service programs required by user equipment m in time slot i; constraint C2 indicates that the communication range limit must be met during task offloading; and constraint C3 indicates that the service programs cached by the UAV cannot exceed the size of the cache space. The size of a service cache, Let C be the size of the drone's cache space; constraint C4 indicates that the drone has an energy surplus after completing the mission; constraint C5 indicates that the drone cannot fly out of the service area; constraint C6 indicates that each decision must satisfy the requirements of its binary variables.

[0097] After posing the optimization problem, Markov modeling is performed on the system operation process, and the operating environment, agent space, state space, action space, and reward function are built based on the multi-agent deep reinforcement learning algorithm.

[0098] Each drone in the intelligent agent space acts as an independent intelligent agent.

[0099] The state space is used to describe the current state of the entire system, including the number of user devices that have not performed task offloading, the location information of each drone, and the service cache factor.

[0100] The action space describes the decisions made by the system at each time slot that can change the system state, including task unloading correlation factors. Flight plan (drone trajectory) Charging decision factors Cache vector .

[0101] The reward function is used to correct the iteration direction of the algorithm. To minimize the total latency of serving all user devices, the reward can be expressed as... , This indicates the reward for drones updating their service cache according to the priority determined by the service cache factor. The higher the priority of the updated service cache (the larger the service cache factor), the higher the reward. This represents a penalty for wasting time slot resources. Due to unreasonable allocation of various decisions, the working time slot length of each drone in the same time slot may vary too much, resulting in the time slot being lengthened and wasting time slot resources. This indicates the penalty imposed for exceeding constraints C1-C6, etc.

[0102] Based on the above, a multi-agent environment can be built, and the original optimization problem can be solved using a multi-agent deep reinforcement learning algorithm to obtain the joint optimal solution for drone trajectory, charging scheme, offloading association, and service cache management.

[0103] In summary, existing research on MEC edge caching focuses on fixed edge caching. Introducing UAV assistance can fully utilize the maneuverability of UAVs for effective collaborative work, enabling resource sharing and load balancing among UAVs, thereby reducing overall system latency. Building upon existing research, this invention considers the limited energy of UAVs and proposes a charging scheme for better energy-saving scheduling. The UAV needs to determine not only its flight path but also whether its remaining energy requires recharging at a charging station. Practical experience has shown that for large-scale, latency-sensitive tasks requiring long-term UAV operation, the proposed edge-caching-based UAV-assisted MEC system latency optimization method can overcome the energy limitations of UAVs, effectively reducing overall system latency and improving system energy efficiency.

[0104] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0105] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0106] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the methods described in the above embodiments.

[0107] It is understood that the systems, devices, and storage media provided in the embodiments of the present invention correspond to the methods provided in the embodiments of the present invention, and the explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0108] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0109] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0110] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0111] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A latency optimization method for a power-assisted UAV MEC system based on edge buffering, characterized in that, Includes the following steps, Step 1: Construct a system model containing N drones, M user devices, and L ground wireless charging stations. Ground users will generate various types of computing tasks and offload these tasks to the drones for computing. The drones are equipped with edge servers and edge service caches. The content of the edge service caches includes the business programs and databases required for computing various user tasks. Step 2: Build a latency model and divide the system task execution process into multiple time slots. The type of time slot is determined based on whether the drone flies to the charging station for charging. They are divided into charging time slots and working time slots. During the charging time slot, the drone will execute its own charging plan. During the working time slot, the drone will update its own service cache, execute a flight plan, and provide services to the ground equipment. Step 3: Calculate the UAV's flight energy consumption, computational energy consumption, and user's task offload latency and computational latency based on the system model and latency model, and construct an optimization problem with the goal of minimizing system latency and energy consumption; Step 4: Based on the system model and latency model constructed in Step 2 and Step 3, model the system operation process as a Markov process, and use deep reinforcement learning algorithms to solve for the optimal UAV trajectory and task offloading scheme, UAV service caching scheme, and UAV charging scheme. Step two includes dividing the system's task execution process into K unequal time slots. Before the process is executed, the cloud server collects the system's initial information, including the location information of each drone and user device, as well as the task type of each user device. Based on this information, a deep reinforcement learning algorithm is executed to obtain the drone's optimal cache vector and its charging decision factor, flight plan (drone trajectory), and the user device's optimal unloading decision. Then, at the beginning of each time slot, each drone determines whether the time slot is a working time slot or a charging time slot based on the charging decision factor. If it is a working time slot, the drone updates its own service cache based on its own service cache vector, flight plan, and unloading correlation factor and executes the flight plan to provide services to ground users. The user device will unload the computing task to the corresponding drone according to the unloading scheme. Each drone can process at most one task in each time slot. Finally, the drone calculates and returns the result. If it is a charging time slot, the drone will execute a charging plan, which means flying to the nearest charging station and continuing to fly to the charging station in subsequent time slots until charging is complete. However, in order to ensure that the time slots are not wasted, the drone needs to synchronize with other drones in time. If other drones are all in charging time slots, each drone will fly to the adjacent sub-area once according to its own route. If there is a working time slot, the drone will fly to the nearest charging station within the time slot length. Therefore, the length of each time slot is the maximum working time slot length or the time required to execute a flight plan. The working time slot of a drone consists of flight latency and drone service latency, and the charging time slot length is equal to the flight latency required to fly to the adjacent sub-region. The flight delay of the drone is calculated using the following formula, where d is the distance between adjacent sub-regions and v is the drone's flight speed. The charging arrival flag is a binary variable; a value of 1 indicates that the charging plan executed in the current time slot can reach the charging station. The time required to reach the charging station during this time slot; The service latency of a drone consists of three parts: service cache update latency, task unloading latency, and task computation latency, which are calculated using the following formulas: in, For the type of task; A set of task types; The number of service caches that need to be updated; The time required to update a service cache; The amount of data that needs to be unloaded for each user device; The ground equipment linked to time slot i and UAV n is given by the offloading association factor and the charging decision factor. If the UAV chooses to execute the charging scheme, it will be unable to provide services to the user equipment. Let represent the communication rate between time slot i, UAV n, and the ground equipment it serves. Communication between the UAV and user equipment can be considered as transmission over a LOS link, therefore it is calculated using the following formula. in To calculate 1-bit user equipment The number of CPU cycles required to unload data from a certain task. This refers to the CPU frequency of the drone. Thus, the time slot length and the total latency of the system processing tasks within the time slot are obtained as follows: The hovering time of the UAV in each time slot can be calculated from the time slot length. 。 2. The latency optimization method for a power-assisted MEC system based on edge buffering according to claim 1, characterized in that: Step one includes, In the system model, the horizontal plane where the ground equipment is located is divided into multiple identical sub-regions. The horizontal projections of the ground charging station and the drone are located at the center of one sub-region. The positions of the user equipment are fixed. Based on the distribution of the equipment, the equipment group is divided into L sub-clusters. The charging station is located in the central sub-region of the equipment cluster. The drone flies on a horizontal plane at a height of H. The positions of the drone and the ground equipment are respectively... The drone can execute a flight plan once in each work slot. This means hovering in the current sub-region, flying to an adjacent sub-region, or executing a charging plan. The charging decision factor is a binary variable; a value of 0 indicates that the charging plan has begun or the vehicle is en route to a charging station. To unload the correlation factor, a value of 1 indicates that in the time slot The ground-based device m offloads its computing tasks to the drone n. This is a service cache vector for the drone. Each element in the vector is a binary variable, and each element corresponds one-to-one with a specific type of service cache. A value of 1 indicates that the drone has that type of service cache in that time slot. Regarding task type... Service caching factor The following formula can be used to calculate the priority of the corresponding business program cached by drones that are closer to the device that generated the task. A larger value indicates a higher priority for the cached business program. If the value is 0, then the program update corresponding to task type z will not be considered; in, For slot i unloaded task type The number of user devices, For a specific unloaded task type in time slot i The distance between a user device and a drone.

3. The latency optimization method for a power-assisted MEC system based on edge caching according to claim 2, characterized in that: Step three includes the drone's energy consumption, which includes flight energy consumption and computing energy consumption. The drone's computing energy consumption is... Flight energy consumption includes hovering energy consumption and propulsion energy consumption. The power of both is calculated using the following formula. When the speed is 0, the calculation result is the hovering power: in This refers to the power consumed by the blade profile, or cross-sectional power, when the drone is hovering. The drone needs rotor rotation to maintain altitude, which depends on the drone's weight, air density, rotor speed, and rotor surface area. The speed at the tip of the rotor blade. The induced power is the power required to overcome the induced drag when the drone is hovering. Airflow over the rotor generates additional velocity, known as induced velocity, which introduces induced drag into the rotor's rotation. This indicates the average inductive speed of the rotor during the hovering period; , and These represent the fuselage drag ratio, air density, and rotor solidity, respectively. This represents the area of ​​the rotor's rotating surface. The rotor radius is... The rotor angular velocity, This represents the rotor profile drag coefficient; This gives us the formulas for calculating hovering energy consumption and propulsion energy consumption. The time spent hovering within the time slot: If the drone starts executing the charging scheme within the time slot, the binary variables This value will be set to 0. This variable is used to indicate the execution status of the charging scheme and will not be cleared until charging is complete. It represents the remaining energy of the drone when charging is complete. It uses its maximum energy and records and updates the time slot when charging is complete. ; Based on hovering energy consumption, propulsion energy consumption, and charging scheme, the energy consumption of the drone in each time slot is calculated. And the remaining energy at the start of the time slot, respectively In order for drones to complete tasks independently without running out of energy, there are constraints. 。 4. The latency optimization method for a power-assisted MEC system based on edge buffering according to claim 3, characterized in that: Step three also includes, Based on the system model and time slice model, the proposed optimization problem is: Constraint C1 means that the charging scheme and the unloading plan cannot conflict. Let C1 be a binary variable, where 1 indicates that UAV n caches the service programs required by user equipment m in time slot i; constraint C2 indicates that the communication range limit must be met during task offloading; and constraint C3 indicates that the service programs cached by the UAV cannot exceed the size of the cache space. The size of a service cache, Let C be the size of the drone's cache space; constraint C4 indicates that the drone has an energy surplus after completing the mission; constraint C5 indicates that the drone cannot fly out of the service area; constraint C6 indicates that each decision must satisfy the requirements of its binary variables.

5. The latency optimization method for a power-assisted MEC system based on edge caching according to claim 4, characterized in that: Step four includes building the operating environment based on the multi-agent deep reinforcement learning algorithm, including agent space, state space, action space, and reward function; Each drone within the intelligent agent space acts as an independent intelligent agent. The state space is used to describe the current state of the entire system, including the number of user devices that have not performed task offloading, the location information of each drone, and the service cache factor. The action space describes the decisions made by the system at each time slot that can change the system state, including task unloading correlation factors. Flight plan, i.e., drone trajectory Charging decision factors Cache vector ; The reward function is used to correct the iteration direction of the algorithm. To minimize the total latency of serving all user devices, the reward is expressed as... , This refers to the reward for drones updating their service cache according to the priority determined by the service cache factor. The higher the priority of the updated service cache, that is, the larger the service cache factor, the higher the reward. This represents a penalty for wasting time slot resources; This indicates the penalty for exceeding the C1-C6 constraints; Based on the above-mentioned multi-agent environment, the original optimization problem can be solved using a multi-agent deep reinforcement learning algorithm, resulting in a joint optimal solution for drone trajectory, charging scheme, offloading association, and service cache management.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, the processor performs the steps of the method as described in any one of claims 1 to 5.

Images